agentSdkDownloader.ts ×15

Frontier kind: Code frontier

unlabeled · c_b76d22bf24ae

230 tests · 21073 LOC · 69 files · introduces 0 tests · 266 LOC · 1 file

Introduces — evidence that enters the hierarchy at this concept

Code
15 ranges266 lines · 1 files
Tests
0 tests

Contains — complete concept membership

All code (extent)
1844 ranges21073 lines · 69 files · Browse complete extent
All tests (intent)
230 testsBrowse complete intent

Neighbourhood graph

The orange circle is the focus. Violet and green circles are every ancestor and descendant, broader and narrower, at any distance; blue squares and pink diamonds are the introduced files and exact introduced tests of every visible concept, not only the focus's. Arrows point from broader to narrower concepts and bridge only concepts omitted from this view. Undirected links show source or test introduction. Concept and file size follows LOC; exact test nodes use test-count units.

Introduced files, introduced tests, and structurally relevant concept specialization

In the embedded map, ordinary wheel input scrolls the page; use the visible controls to zoom and drag to pan. Open the full-screen map for canvas navigation: wheel pans, Ctrl/Command plus wheel zooms, and arrow keys pan when this region is focused. On touch screens, open the full-screen map to pan or pinch. If JavaScript or WebGL is unavailable, use the native relationship evidence on this page.

Graph controls are ready.

Interactive rendering requires JavaScript and WebGL. Use the native relationship evidence on this page while the interactive map is unavailable.

Native relationship evidence

Every exact file and test below is linked only from the concept that introduces it.

Introduced tests

Every collected test enters the hierarchy at exactly one concept.

No tests are introduced at this concept. Its intent tests are introduced by other concepts.

Introduced code

Every collected source range enters the hierarchy at exactly one concept.

1 file ranked by introduced lines: 266 introduced LOC across 15 ranges. Expand a file to inspect source; the > gutter marks introduced lines.

src/vs/platform/agentHost/node/agentSdkDownloader.ts 266 introduced LOC · 15 ranges

Open complete file

1 > /*--------------------------------------------------------------------------------------------- agentSdkDownloader.ts
2 > * Copyright (c) Microsoft Corporation. All rights reserved.
3 > * Licensed under the MIT License. See License.txt in the project root for license information.
4 > *--------------------------------------------------------------------------------------------*/
5 >
6 > import * as fs from 'fs';
7 > import * as tar from 'tar';
8 > import { VSBuffer } from '../../../base/common/buffer.js';
9 > import { CancellationToken } from '../../../base/common/cancellation.js';
10 > import { CancellationError } from '../../../base/common/errors.js';
11 > import { Emitter, Event } from '../../../base/common/event.js';
12 > import { Disposable } from '../../../base/common/lifecycle.js';
13 > import * as path from '../../../base/common/path.js';
14 > import { format2 } from '../../../base/common/strings.js';
15 > import { URI } from '../../../base/common/uri.js';
16 > import { generateUuid } from '../../../base/common/uuid.js';
17 > import { detectLibcSync, type LibcFamily } from '../../../base/node/libc.js';
18 > import { INativeEnvironmentService } from '../../environment/common/environment.js';
19 > import { FileOperationError, FileOperationResult, IFileService, toFileOperationResult } from '../../files/common/files.js';
20 > import { createDecorator } from '../../instantiation/common/instantiation.js';
21 > import { ILogService } from '../../log/common/log.js';
22 > import { IProductService } from '../../product/common/productService.js';
23 > import { IRequestService } from '../../request/common/request.js';
24 > import { IRequestContext } from '../../../base/parts/request/common/request.js';
25 >
26 > // #region Per-package strategy
27 >
28 > /**
29 > * One agent-SDK package the downloader can fetch. Holds the per-package
30 > * knowledge that varies between Claude, Codex, and any future provider —
31 > * the package id, the env var that acts as a dev override, and one
32 > * boolean covering the only mapping detail that differs between SDKs
33 > * today (Claude has separate `linux-*-musl` SKUs; Codex's Linux binary
34 > * is statically musl-linked and ships as a single `linux-*` SKU).
35 > *
36 > * The downloader itself is package-agnostic: it consumes this interface and
37 > * never branches on `id`. Concrete `IAgentSdkPackage` instances live in
38 > * their owning agent module (e.g. `ClaudeSdkPackage` in
39 > * `claude/claudeAgentSdkService.ts`, `CodexSdkPackage` in
40 > * `codex/codexAgent.ts`) so Claude-specific / Codex-specific knowledge
41 > * stays in those modules — the downloader doesn't name the providers it
42 > * serves.
43 > *
44 > * Each shipped `product.json` carries one `{version, urlTemplate}` per
45 > * SDK. The downloader substitutes `{sdkTarget}` (resolved via
46 > * `resolveSdkTarget(pkg)`) into the template to get the per-target
47 > * tarball URL. This shape supports macOS Universal builds, where the
48 > * same `product.json` is shared by arm64 and x64 launches.
49 > */
50 > export interface IAgentSdkPackage {
51 > /** Key under `product.agentSdks` — e.g. `'claude'`, `'codex'`. */
52 > readonly id: string;
53 > /**
54 > * Brand display name for user-facing progress, e.g. `'Claude'`, `'Codex'`.
55 > * The downloader puts this on {@link IAgentSdkDownloadProgress.displayName}
56 > * so clients can build a localized "Downloading {displayName} agent" label.
57 > */
58 > readonly displayName: string;
59 > /** Env var that, when set, becomes the SDK root and short-circuits the download. */
60 > readonly devOverrideEnvVar: string;
61 > /**
62 > * True iff this SDK publishes separate `linux-{x64,arm64}-musl`
63 > * packages alongside the glibc default. Claude does; Codex doesn't
64 > * (its Linux binary is statically musl-linked and runs on both).
65 > */
66 > readonly hasSeparateMuslLinuxPackage: boolean;
67 > }
68 >
69 > /**
70 > * Per-host info used by `resolveSdkTarget`. Defaulted from the running
71 > * process; tests inject synthetic values to exercise targets the test
72 > * host doesn't actually run on (Universal-launch case, musl, etc.).
73 > */
74 > export interface ISdkTargetHost {
75 > readonly platform: NodeJS.Platform;
76 > readonly arch: string;
77 > readonly libc: LibcFamily | undefined;
78 > }
79 >
80 > const SUPPORTED_PLATFORMS = new Set<NodeJS.Platform>(['linux', 'darwin', 'win32']);
81 > const SUPPORTED_ARCHES = new Set<string>(['x64', 'arm64']);
82 >
83 > /**
84 > * Resolves the build's `sdkTarget` suffix for the given host. Defaults
85 > * to the current Node process — production callers omit `host`; tests
86 > * pass a synthetic host to cover targets the test machine can't reach
87 > * (Universal launches from a single-arch host, musl Linux on macOS CI,
88 > * etc.).
89 > *
90 > * - claude on glibc Linux: `linux-x64` / `linux-arm64`
91 > * - claude on musl Linux: `linux-x64-musl` / `linux-arm64-musl`
92 > * - codex Linux (any libc): `linux-x64` / `linux-arm64`
93 > * - everywhere else: `<platform>-<arch>`
94 > *
95 > * Returns `undefined` when no SDK applies (`armhf`, web, etc.); the
96 > * downloader treats that the same as "no product config" and never
97 > * registers the provider.
98 > *
99 > * Mirror of the build pipeline's `getSdkTargetForBuild` (in
100 > * `build/agent-sdk/common.ts`) translated from build-time
101 > * `vscodePlatform` to runtime `process.platform` + libc detection.
102 > * Keep the two in sync when adding new target SKUs.
103 > */
104 > export function resolveSdkTarget(
105 pkg: Pick<IAgentSdkPackage, 'hasSeparateMuslLinuxPackage'>,
106 host: ISdkTargetHost = { platform: process.platform, arch: process.arch, libc: detectLibcSync() },
114 return `${host.platform}-${host.arch}`;
115 }
117 > // #endregion
118 >
119 > // #region Service decorator
120 >
121 > export const IAgentSdkDownloader = createDecorator<IAgentSdkDownloader>('agentSdkDownloader');
122 >
123 > /** Lifecycle phase of a single SDK download (downloader-internal). */
124 > export type AgentSdkDownloadPhase = 'started' | 'progress' | 'completed' | 'failed';
125 >
126 > /**
127 > * A process-global download-progress sample fired on
128 > * {@link IAgentSdkDownloader.onDidDownloadProgress}. The downloader owns the
129 > * lifecycle: one `started`, throttled `progress` frames, then exactly one
130 > * terminal `completed` / `failed` — all sharing a `downloadId`. Concurrent
131 > * `loadSdkRoot` callers for the same tarball are deduped, so they observe one
132 > * shared download (one `downloadId`).
133 > */
134 > export interface IAgentSdkDownloadProgress {
135 > /** Stable id for one download; coalesces frames and distinguishes concurrent fetches. */
136 > readonly downloadId: string;
137 > /** Package id, e.g. `'claude'` / `'codex'`. */
138 > readonly packageId: string;
139 > /** Brand display name, e.g. `'Claude'`. */
140 > readonly displayName: string;
141 > /** Lifecycle phase of this frame. */
142 > readonly phase: AgentSdkDownloadPhase;
143 > /** Bytes written so far. Monotonically non-decreasing within a `downloadId`. */
144 > readonly receivedBytes: number;
145 > /** Total bytes from `Content-Length`, or `undefined` when unknown (indeterminate). */
146 > readonly totalBytes: number | undefined;
147 > /** Short, non-localized failure reason; present only when `phase: 'failed'`. */
148 > readonly error?: string;
149 > }
150 >
151 > export interface IAgentSdkDownloader {
152 > readonly _serviceBrand: undefined;
153 >
154 > /**
155 > * Fires while a tarball is being fetched (cold cache only): one `started`,
156 > * throttled `progress` samples, then one terminal `completed` / `failed`.
157 > * Never fires for dev-override or cache-hit resolutions (no bytes move).
158 > * Process-global so a single subscriber (the protocol server) can forward
159 > * progress to clients regardless of which session triggered the fetch.
160 > */
161 > readonly onDidDownloadProgress: Event<IAgentSdkDownloadProgress>;
162 >
163 > /**
164 > * Returns the absolute path of the SDK root directory — the directory that
165 > * contains the package's `node_modules/` subtree. Callers resolve the
166 > * package-specific entrypoint from there themselves.
167 > *
168 > * Resolution order:
169 > * 1. dev-override env var (returned unchanged)
170 > * 2. on-disk cache hit (`.complete` sentinel present)
171 > * 3. download from `product.agentSdks?.[pkg.id]` with
172 > * `{sdkTarget}` substituted into the urlTemplate
173 > *
174 > * Repeated failures are latched for {@link LOAD_FAILURE_NEGATIVE_CACHE_MS}
175 > * so a misconfigured CDN doesn't get hammered on every SDK method call.
176 > */
177 > loadSdkRoot(pkg: IAgentSdkPackage, token: CancellationToken): Promise<string>;
178 >
179 > /**
180 > * Cheap, synchronous gate used at startup to decide whether to register
181 > * the corresponding agent provider. True iff the dev override is set, OR
182 > * (`product.agentSdks?.[pkg.id]` is populated AND `pkg.currentSdkTarget()`
183 > * resolves — i.e. an SDK exists for this host). Does NOT trigger a
184 > * download.
185 > */
186 > isAvailable(pkg: IAgentSdkPackage): boolean;
187 >
188 > /**
189 > * True iff {@link loadSdkRoot} would resolve WITHOUT a network download —
190 > * the dev override is set, or a completed cache for the configured version
191 > * already exists on disk. False when product config is present but the
192 > * cache is cold (a fetch would be required), and false when neither an
193 > * override nor product config is configured.
194 > *
195 > * Performs at most a single sentinel `exists` check and never downloads.
196 > * Eager / background callers (e.g. a provider listing its sessions at
197 > * startup) use this to avoid kicking off a multi-second cold download
198 > * before the user has asked for anything.
199 > */
200 > isSdkResolvableWithoutDownload(pkg: IAgentSdkPackage): Promise<boolean>;
201 > }
202 >
203 > // #endregion
204 >
205 > // #region Implementation
206 >
207 > /** How long a `loadSdkRoot` failure latches before we try again. */
208 > const LOAD_FAILURE_NEGATIVE_CACHE_MS = 30_000;
209 >
210 > /**
211 > * Minimum gap between download-progress samples. A 70-95MB tarball over a fast
212 > * link produces thousands of chunks; without throttling we'd flood the progress
213 > * channel. ~250ms keeps the percentage visibly moving without spamming.
214 > */
215 > const PROGRESS_EMIT_THROTTLE_MS = 250;
216 >
217 > /**
218 > * Parses a `Content-Length` header into a positive integer byte count, or
219 > * `undefined` when the header is absent, an array, or not a clean integer.
220 > */
221 function parseContentLength(header: string | string[] | undefined): number | undefined {
222 if (typeof header !== 'string' || !/^\d+$/.test(header)) {
226 return parsed > 0 ? parsed : undefined;
227 }
229 > export class AgentSdkDownloader extends Disposable implements IAgentSdkDownloader {
230 > declare readonly _serviceBrand: undefined;
231 >
232 > private readonly _onDidDownloadProgress = this._register(new Emitter<IAgentSdkDownloadProgress>());
233 > readonly onDidDownloadProgress: Event<IAgentSdkDownloadProgress> = this._onDidDownloadProgress.event;
234 >
235 > /**
236 > * In-flight downloads keyed by the destination `cacheDir` (which
237 > * already encodes `<pkg>/<sdkVersion>/<sdkTarget>`). Concurrent
238 > * `loadSdkRoot` calls in the same process share the same promise so
239 > * we never download the same tarball twice. Universal launches that
240 > * resolve to different targets get distinct entries because their
241 > * cacheDirs differ.
242 > */
243 > private readonly _pendingDownloads = new Map<string, Promise<string>>();
244 >
245 > /**
246 > * Negative cache: most recent failure per package id, with an expiry.
247 > * While within the window, `loadSdkRoot` re-throws the cached error
248 > * immediately instead of re-attempting the download. Without this, a
249 > * broken CDN causes every SDK method call (poll-driven UIs hit this
250 > * hard) to fire a fresh request.
251 > *
252 > * Keyed by `pkg.id` (not the finer cacheDir): CDN failures are
253 > * effectively global per SDK (DNS, proxy auth, 5xx) and per-target
254 > * latching wouldn't protect against the actual failure modes — the
255 > * broader latch is intentional.
256 > */
257 > private readonly _failureLatch = new Map<string, { error: Error; expiresAt: number }>();
258 >
259 > constructor(
260 @INativeEnvironmentService private readonly _environmentService: INativeEnvironmentService,
261 @IProductService private readonly _productService: IProductService,
266 super();
267 }
269 > isAvailable(pkg: IAgentSdkPackage): boolean {
270 if (process.env[pkg.devOverrideEnvVar]) {
271 return true;
273 return !!this._productService.agentSdks?.[pkg.id] && resolveSdkTarget(pkg) !== undefined;
274 }
276 > async isSdkResolvableWithoutDownload(pkg: IAgentSdkPackage): Promise<boolean> {
277 if (process.env[pkg.devOverrideEnvVar]) {
278 return true;
289 return this._fileService.exists(sentinel);
290 }
292 > async loadSdkRoot(pkg: IAgentSdkPackage, token: CancellationToken): Promise<string> {
293 // 1. Dev override.
294 const override = process.env[pkg.devOverrideEnvVar];
321 }
322 }
324 > private async _resolveOrDownload(pkg: IAgentSdkPackage, token: CancellationToken): Promise<string> {
325 const config = this._productService.agentSdks?.[pkg.id];
326 if (!config) {
374 return pending;
375 }
377 > private _cacheDir(packageId: string, sdkVersion: string, sdkTarget: string): string {
378 // `sdkTarget` is in the path so macOS Universal builds keep two
379 // independent caches — one per resolved target — instead of
388 );
389 }
391 > private async _download(
392 pkg: IAgentSdkPackage,
393 url: string,
473 }
474 }
476 > private _fireProgress(
477 pkg: IAgentSdkPackage,
478 downloadId: string,
492 });
493 }
495 > private async _handleRenameLoser(
496 err: unknown,
497 sentinel: URI,
511 return true;
512 }
514 > private async _fetch(
515 url: string,
516 dest: string,
605 });
606 }
608 > private async _extractTarGz(tarball: string, dest: string): Promise<void> {
609 // `tar` (node-tar) is pure JS — works on every platform the agent host
610 // runs on without depending on a system `tar` binary.
611 await tar.x({ file: tarball, cwd: dest });
612 }
614 > private async _delIgnoringMissing(uri: URI): Promise<void> {
615 try {
616 await this._fileService.del(uri, { recursive: true });