Skip to content

Use GitHub Releases for SDK runtime artifacts - #2505

Open
roji wants to merge 3 commits into
mainfrom
roji-unify-runtime-artifacts
Open

Use GitHub Releases for SDK runtime artifacts#2505
roji wants to merge 3 commits into
mainfrom
roji-unify-runtime-artifacts

Conversation

@roji

@roji roji commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Continues #2463, completing the transition away from consuming the @github/copilot npm package from the SDKs, instead getting the github-copilot-*.tgz asset from GitHub Releases, which already packages the needed SDK-facing runtime assets. This paves the way to removing the duplicated runtime.node from the npm package, which is getting dangerously big and close to the npmjs limit.

#2463 did this for the Node.js SDK, Rust and Java (partially); this completes the work for the other SDKs.

The language SDKs should obtain Copilot CLI runtime components from one canonical source instead of extracting them from @github/copilot npm packages. This keeps runtime acquisition consistent and avoids carrying SDK-facing runtime payloads in the CLI npm package.

This change moves Python, .NET, and Go to the checksum-verified github-copilot-<version>-<platform>.tgz assets published by github/copilot-cli. Java and Rust runtime packaging were already Release-backed after #2463; Java's remaining schema-codegen and smoke-test dependency on @github/copilot is also removed. Java codegen now extracts schemas from the same verified Release artifact, while smoke tests use the shared Release-backed CLI setup.

Runtime compatibility entrypoints continue to use copilot-runtime[.exe] beside runtime.node, preserving existing SDK behavior. Downloads are verified against the Release's SHA256SUMS.txt, and incomplete caches are not treated as valid.

Validation

  • Compared normalized runtime paths, modes, sizes, and SHA-256 hashes between npm and Release packages across all eight supported platforms; every runtime payload matched byte-for-byte.
  • Python downloader/client tests, Ruff, and ty passed.
  • Focused .NET MSBuild target tests and Go bundler/runtime tests passed.
  • Full Java Maven verification passed, and Release-sourced schemas regenerate identical Java output.

roji added 3 commits September 3, 2026 15:31
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: daec7b3b-799c-4396-b372-6eb638d0faf8
Remove Java and Rust changes because those SDKs were already release-backed, simplify Python's direct runtime staging, and drop the misleading .NET npm URL alias.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: daec7b3b-799c-4396-b372-6eb638d0faf8
Fetch Java codegen schemas from the checksum-verified CLI release artifact and use the shared release-backed CLI setup for smoke tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: daec7b3b-799c-4396-b372-6eb638d0faf8
@roji
roji requested a review from a team as a code owner September 3, 2026 15:58
Copilot AI balanced review requested due to automatic review settings September 3, 2026 15:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Runtime asset staging has unresolved cache-integrity, Go installation, and cross-platform test issues.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 1 High severity · 3 Medium severity

New issues introduced by this change (4)
Severity Finding
High severity dotnet/​src/​build/​GitHub.Copilot.SDK.targets — An incomplete cache is reacquired by overlaying the verified archive onto the existing directory:…
Medium severity go/​internal/​embeddedcli/​embeddedcli.go — When RuntimeExecutable/RuntimeAssets are configured without the optional in-process…
Medium severity go/​internal/​embeddedcli/​embeddedcli_test.go — This test is named for Path, but it calls RuntimePath, which goes through the pre-existing…
Medium severity python/​test_cli_download.py — This assertion hard-codes POSIX separators, but the Python test matrix includes Windows, where…
What changed in this PR

Moves Python, Go, and .NET runtime acquisition to checksum-verified Copilot CLI Release artifacts while removing Java’s remaining npm dependency.

Changes:

  • Adds unified Release-backed runtime staging and caching.
  • Fetches Java schemas from verified Release archives.
  • Updates tests, documentation, and CI workflows.
File Description
python/​test_cli_download.py Expands runtime provisioning tests.
python/​README.md Documents Release-based staging.
python/​copilot/​_ffi_runtime_host.py Updates layout terminology.
python/​copilot/​_cli_version.py Defines Release asset naming.
python/​copilot/​_cli_download.py Implements verified bundle staging.
java/​sdk/​src/​main/​java/​com/​github/​copilot/​ffi/​NativeRuntimeLoader.java Updates layout documentation.
java/​sdk/​pom.xml Removes npm schema-update profile.
java/​scripts/​codegen/​package.json Adds schema fetching and tests.
java/​scripts/​codegen/​package-lock.json Removes Copilot npm dependencies.
java/​scripts/​codegen/​java.ts Reads staged Release schemas.
java/​scripts/​codegen/​fetch-schemas.test.mjs Tests schema extraction.
java/​scripts/​codegen/​fetch-schemas.mjs Downloads and verifies schemas.
java/​pom.xml Removes the Java npm version pin.
java/​copilot-native/​scripts/​fetch-native.mjs Uses Release artifact terminology.
java/​copilot-native/​pom.xml Updates packaging documentation.
go/​README.md Documents Release-backed bundling.
go/​internal/​embeddedcli/​embeddedcli.go Installs the adjacent runtime pair.
go/​internal/​embeddedcli/​embeddedcli_test.go Adds runtime-pair coverage.
go/​cmd/​bundler/​main.go Downloads verified Release bundles.
go/​cmd/​bundler/​main_test.go Tests checksums and Release downloads.
dotnet/​test/​Unit/​MSBuildTargetsTests.cs Adds Release server integration tests.
dotnet/​src/​build/​GitHub.Copilot.SDK.targets Implements verified Release acquisition.
dotnet/​README.md Documents runtime acquisition options.
.github/​workflows/​update-copilot-dependency.yml Simplifies shared CLI updates.
.github/​workflows/​java-smoke-test.yml Uses shared Copilot setup.
.github/​workflows/​java-codegen-fix.md Updates agent codegen guidance.
.github/​workflows/​java-codegen-fix.lock.yml Regenerates workflow metadata.
.github/​workflows/​java-codegen-check.yml Tests and runs schema fetching.
.github/​workflows/​java-adapt-handwritten-code-to-accept-upgrade-changes.md Updates CLI-upgrade instructions.
.github/​workflows/​java-adapt-handwritten-code-to-accept-upgrade-changes.lock.yml Regenerates workflow metadata.
.github/​actions/​setup-copilot/​action.yml Exposes the JavaScript CLI path.
Files not reviewed (1)
  • java/scripts/codegen/package-lock.json: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

<!-- Delete archive if binary missing (handles partial/corrupted downloads) -->
<Delete Files="$(_CopilotArchivePath)" Condition="!Exists('$(_CopilotCliBinaryPath)') And Exists('$(_CopilotArchivePath)')" />
<!-- Write the completion marker only after the full archive is extracted and validated. -->
<Delete Files="$(_CopilotArchivePath);$(_CopilotChecksumPath);$(_CopilotRuntimeBundleCompleteMarker)" Condition="'$(_CopilotRuntimeBundleMissing)' == 'true'" />
Comment on lines +308 to +312
if config.RuntimeExecutable != nil {
path, err := installRuntimePair(installDir)
if err != nil {
return "", err
}
Comment on lines +280 to +281
runtimePath := RuntimePath()
installDir := filepath.Dir(runtimePath)
Comment on lines +258 to +261
assert wrapper.endswith(
f"prebuilds/{runtime_platform}/"
f"{'copilot-runtime.exe' if os.name == 'nt' else 'copilot-runtime'}"
)

public byte[] CreateReleaseArchive(string runtimeWrapperContents)
{
var sourceDir = Path.Combine(ProjectDir, "release-source");
public byte[] CreateReleaseArchive(string runtimeWrapperContents)
{
var sourceDir = Path.Combine(ProjectDir, "release-source");
var packageDir = Path.Combine(sourceDir, "package");
{
var sourceDir = Path.Combine(ProjectDir, "release-source");
var packageDir = Path.Combine(sourceDir, "package");
var prebuildDir = Path.Combine(packageDir, "prebuilds", GetReleasePlatform());
var packageDir = Path.Combine(sourceDir, "package");
var prebuildDir = Path.Combine(packageDir, "prebuilds", GetReleasePlatform());
Directory.CreateDirectory(prebuildDir);
File.WriteAllText(Path.Combine(prebuildDir, "runtime.node"), "runtime");
var prebuildDir = Path.Combine(packageDir, "prebuilds", GetReleasePlatform());
Directory.CreateDirectory(prebuildDir);
File.WriteAllText(Path.Combine(prebuildDir, "runtime.node"), "runtime");
File.WriteAllText(Path.Combine(prebuildDir, RuntimeWrapperName), runtimeWrapperContents);
File.WriteAllText(Path.Combine(prebuildDir, "runtime.node"), "runtime");
File.WriteAllText(Path.Combine(prebuildDir, RuntimeWrapperName), runtimeWrapperContents);

var archivePath = Path.Combine(ProjectDir, "release-asset.tgz");

var archivePath = Path.Combine(ProjectDir, "release-asset.tgz");
var tarPath = OperatingSystem.IsWindows()
? Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "System32", "tar.exe")
_cancellation.Cancel();
_listener.Stop();
try { _serverTask.GetAwaiter().GetResult(); }
catch (OperationCanceledException) { }
@SteveSandersonMS

Copy link
Copy Markdown
Contributor

Thanks for the migration work here — I manually validated two concrete behaviors:

  1. I ran Python's new release-backed runtime acquisition end-to-end against the real v1.0.83-3 GitHub Release asset, and it staged the expected runtime bundle layout successfully.

  2. I reproduced the stale-cache overlay risk in practice: when a verified tarball is extracted into an already-populated cache directory, unrelated pre-existing files remain on disk. For this PR, that means cache reacquisition should use fresh staging (or full clear-and-replace) rather than overlay extraction into an existing cache directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants