25 · Remote tasks on your computer

Ensemble on the web can assign work to a person's paired computer, which runs it with the same local runtime as the desktop app and CLI. The host stores assignments, progress, decisions, results and bounded review metadata; the computer enforces its own execution rules. 06 covers execution; 24 covers confinement.

1. Execution and ownership

WorkspaceJob.deviceId targets a paired computer. The host worker and its recovery path skip those jobs. The computer claims the assignment, creates a local job (local deviceId null) and runs the normal queue, agent and engine path.

When POST /api/devices/self/claim claims queued work, devices/routes.ts moves the agent-owned task to In progress and publishes both task and workspace events on the task's space channel. The open board, task details and run queue refresh without a page reload or tab switch. Work waiting for an offline computer or a queue slot stays in To do.

The host's spec carries task context and execution choices (and steer, the change for a run that continues one a node interrupted: 30, Agent sessions), never shell argv or a raw path on the computer; folder labels resolve against folders the computer shares. Hosted email verification and account quotas apply. An offline computer leaves its job queued; work never falls back to the server or another device. remote/client.ts, accept.ts, policy.ts, store.ts and devices/routes.ts implement this boundary.

Continue an earlier task. Assign's "Continue an earlier task" on a computer lists, from GET /api/agent/earlier-runs?deviceId=&taskId= (devices/earlier.ts), the code runs that ran (or are queued to run) on that computer, under its current pairing or an earlier one of it (§8): each chain of continued runs once, by its newest step, the task's own first (and chosen by default). Assigning checks the earlier run was on that computer; the computer maps it to its own run by the web job id (context.hostedJobId, remote/accept.ts) and works in that run's folder, uncommitted work included, after it finishes (the chain's resource key). The same agent resumes its own session there (30, Agent sessions). On a desktop with no computer chosen, GET /api/workspace/checkouts lists this server's own runs whose folder still exists.

2. Pairing and local state

Signing the desktop app in to an account (DESKTOP §1) or ensemble login pairs through a browser approval code (/link). Both use the same runtime and data folder, so a computer is one device whichever paired it: the second one keeps the existing pairing when it is with the same server. A desktop app kept local only never pairs, and ensemble login there refuses to (--mcp-only still works). Editor context access and permission to run tasks are separate. A web logout does not unpair a computer; CLI logout, unpair and revocation do.

The device token is an encrypted model_credentials row in the local database (remote/store.ts). Each pairing also makes the computer's sealing key pair, which the web encrypts secrets for: the private key in seal-key.json (0600, encrypted with the runtime's local key), the public key reported as capabilities.sealKey and pinned on the device row (§4.9, 35 §8.7). Unpairing deletes it. remote.json holds switches, labels, folder mappings and the installation's id (§8); api.json lets the CLI and desktop app find the loopback API. Never commit either file or expose the token. The runtime connects outward to ENSEMBLE_REMOTE_API; ENSEMBLE_SITE_URL sets the UI origin, not the remote API. Desktop and CLI share one runtime lock and discovery path, not two databases. Pairing does not copy local data to the account; a local-only app's one-time move does (DESKTOP §1.3).

3. Lease, heartbeat and interruption

devices/constants.ts sets a 120-second lease and a 90-second online window; the computer heartbeats every 30 s (remote/contract.ts). Heartbeats and progress extend only a held lease, and the scheduler sweeps expiry, so the interruption shown can lag the expiry by up to a minute.

Claims and writes are fenced by device and lease token: another device or a stale token cannot complete the job. A missed lease marks the attempt Interrupted, settles open run and decision state and keeps partial work. Sleep, app exit and network loss are interruptions, not completion.

The server cannot tell a computer that was switched off from one that only lost its connection, so the computer settles each run it held when it is back (reconcile in remote/loop.ts, devices/recover.ts, protocol 5):

A run is reattached or carried on only when its lease ran out (or the computer restarted before it did), the person has not stopped it, run it again or paused the agent, and only for the computer that ran it (or the same computer paired again, §8). Carry-ons stop after MAX_AUTO_RESUMES (2) per task chain and are not made for a run interrupted more than 24 hours ago; the next interruption stays for Run again. Runs in a shared space and chat messages are never carried on by themselves. A run the web no longer wants stops on the computer.

After an outage of the server itself (no heartbeat from any computer for a lease period), leases that ran out meanwhile get one more lease period when it starts. The claim refuses past the person's concurrent-run limit, so a run that ends wakes their computers that have queued work, and each heartbeat also claims.

The event stream only wakes the computer; persisted job and lease state is authoritative. File and code relays use scoped Redis keys with short lifetimes; assistant turns on a subscription live in the API process's memory (subscriptions/relay.ts). Decision waits also poll the database, because a second API process does not share the in-memory bus.

4. Hosted interfaces

4.1 Account and device routes

scope=device is not an account token; device routes have their own allowlist and authentication. Pairing does not allow minting other tokens.

4.2 Stored model and job data

The host stores device identity and capabilities, the task and job spec, lease state, decisions, progress, logs, evidence, the settings the agent reported (observed_settings) and result, including the worker's own report (workspace_jobs.report, §4.11). Ensemble on the web stores no file contents from a computer, with one exception: the files a run's worker attaches for its owner (§4.11), unless that computer turned it off. The files a run changed are listed by name and line counts, and their text is read from the computer when opened (§4.6). A desktop app or local server does keep changed-file copies; relays are transient everywhere.

Local secrets stay local. The exception runs the other way: with Settings → Models keysOnComputers on (the default), GET /api/devices/self/model-keys gives the computer the account's model keys for its own model calls, except keys with a custom base URL; the ledger records once a day which providers' keys a computer read.

4.3 Local routes

GET /api/remote, POST /api/remote/pair, /unpair, /folders, and DELETE /api/remote/folders/:label manage the runtime locally. A hosted job cannot set Code folders, trust or web-control settings. 26 lists the CLI commands.

4.4 Web UI

Assign offers available computers with their shared labels and capabilities. The assistant can queue the same job from chat ("do this on my machine") with hub_assign_to_agent after the person presses Apply; it goes through createJob like the dialog, so the computer's checks are unchanged (11 §8.3). Cards show last contact, queue/run/wait/interruption state, model and result. A run's review opens Code on the computer that ran it. Failure to load capabilities shows an error and grants nothing.

4.5 Attachments and task context

Claims carry bounded task, page, source and skill context; embedded owned diagrams come as source. The description also has a Context from Ensemble section (workContext in context/graph/packet.ts, at most 5,000 characters, inside the description's 20,000): the meeting the task came from with the words asked, the project, people and their roles, dependencies, related open work and how similar work went. While the job holds its lease, the run's ensemble_context and ensemble_search (30 §9) ask the web for more with POST /api/devices/self/jobs/:id/context ({name, args} → {text}; the device token and x-ensemble-job-lease), answered for the job's space only; a wrong or expired lease is a 409. remote/accept.ts downloads only the attached originals the current lease authorizes, at most 100 MB total, then remaps hosted page references to the local copies.

4.6 A task's files on the web

remote/files.ts sends changed-file text before completion: at most 100 files, 256 KB/file, 3 MB/run. The server rechecks paths and redaction and stores JobFile copies for at most 30 days, except Ensemble on the web, which keeps none (keepsRunFileText in lib/hosted-access.ts): its claim carries keepsFileText: false so a computer sends no text, and POST /api/devices/self/jobs/:id/files drops any that an older computer still sends (answer { stored: 0 }). There, GET /api/agent/jobs/:id/file always reads live. Metadata reports additions, deletions and line counts; renames can appear as delete + add. Unsafe cloned Git configuration is never run to inspect changes.

The changed files include those outside the run's folder: other folders the run was allowed to write, and repositories the folder's own snapshot does not cover (a nested repository, or any repository when the folder is not one). The first write in such a repository snapshots it, so only the run's changes are listed (workspace/outside-changes.ts); they are named ~/…. When the run folder itself has no changes, the review covers the repository it changed most. Such a file is read only because the run's own evidence lists that exact path, under the same file rules and the outside floor (24 §3; readChangedFile in workspace/folder-read.ts).

Live GET /api/agent/jobs/:id/tree and /file ask the computer for its current authorized folder through devices/file-relay.ts and time out rather than act as remote shell access. Text views are bounded and redacted; secrets, Git internals, links leading outside, hard links and unsafe path races are refused by workspace/folder-read.ts on both ends.

Only the person whose computer ran the job reads its file text; other shared-space members see metadata. The computer must still offer the task folder. browseFiles (Show task files on the web, ensemble browse on|off) is checked again locally; turning it off hides stored copies and refuses live reads.

4.7 The Code tab and terminal on the web

POST /api/devices/:id/code accepts only the calls listed in remote/contract.ts; remote/code.ts runs them through the local API. It is a browser-session route for the computer's owner, never for the assistant, a bridge key or an API token. Relays hold transient request, answer and output state, never a second checkout.

The local switches browseFiles, webEdit and webTerminal are on by default and checked on every call. Secret paths are refused, and redacted views cannot be written back. Binary and oversized files follow the view and refusal limits in 16. Code's allowed folders are set on the computer.

Ensemble Node's file workspace uses the same Code relay with codeApi: 3 (CLI/desktop 0.6.4+), through its own browser-only /api/node/files/open route. The inner GET/PUT /api/code/node-file resolves an original from a local opaque source ID or a held run's exact changed path, including non-Git folders. Saves require the current byte hash and preserve encoding and permissions; stale versions conflict rather than overwrite. Safe downloads use bounded chunks from a short-lived computer-side snapshot. node-file-sources.json stores paths and pending metadata only. The Node origin does not unlock the Hub's terminal or passkey endpoints. Full contract: 35 §8.18.

Runtime 0.6.5 reports canonical fileId and modifiedAt with each changed run file, and updatedAt per direct-file reference. These are metadata only. Node can resolve a quoted file to its last recorded editor across turns and sessions without confusing separate checkouts, and queue a question directly into the editing agent session. File reads also report modification time so an open binary preview refreshes after a change. Older runtimes remain compatible, but cannot supply provenance they did not record.

Runtime 0.6.6 applies file-scoped protection to provenance-backed original reads and saves. A parent folder too broad or sensitive to grant as a whole execution root does not block a safe generated file in it. The exact file's secret, runtime, symlink/hardlink and write-floor checks still apply; node tools retain their existing folder execution guards.

The web terminal needs a passkey unlock with user verification for the web's origin, ending after 30 minutes idle or 8 hours (devices/web-unlock.ts). Lock now ends the session and stops its commands. Commands follow workspace/terminal-exec.ts: allowed programs, arguments and folders, hardened Git, confirmation for destructive actions, OS confinement, time and output limits. It is not a raw shell or an assistant tool. See 16.

4.8 Protocol and compatibility

remote/contract.ts and the computer's receiver define the supported operations; each job records its protocol (changed-file text needs protocol 3, Code and terminal relays protocol 4, secret scopes, the worker's report, evidence and node tools protocol 6, held captures, a video's poster and preview, a node's browser and capture tools, evidence.request and the live view protocol 7, sign-ins (login in a browser answer, login.resolve, ROUTES.runLogin, and with logins: 2 ROUTES.loginState) and takeovers (takeover.*, ROUTES.takeoverFrame, ROUTES.takeoverState) protocol 8). An optional field an older server does not know is left out by it, so it needs no new protocol: observed on progress and complete is one (2 November 2026), a claim's mission another (§4.12), a claim's project (the brief of the task's Ensemble project for a run by the person's own agent, which the computer keeps in the project's folder: 30, Agent projects) a third, and related, sessionChoice and newSession (what the computer chooses the run's agent session with: 30, Choosing a session) more. Until a computer runs a release that sends it, its runs show their observed settings as "not reported", except the agent's model from its engine event. The server and UI check advertised capabilities and name the update an outdated computer needs: the coding agents a runtime can run are in capabilities.engines (deviceEngines in devices/labels.ts; a runtime without the list has Claude Code, Codex and Cursor's agent), so GitHub Copilot is refused for an older one. Existing jobs keep their own protocol and evidence.

4.9 Subscriptions and included models

Agent sessions: a computer with capabilities.sessions 1 runs a job that continues another in the agent's own session (Claude --resume, Codex thread resume, ACP session/load) and takes the claim's steer. A run that would start a new session may continue one of the computer's own: the computer decides when it takes the job, from the claim's related, sessionChoice and newSession, and reports the decision (POST /api/devices/self/jobs/:id/session-choice, under the job's lease), which links the run on the web too (30, Choosing a session). An older computer ignores those fields and starts a new session. With capabilities.sessionDelete 1 it deletes sessions on session.delete, which the web also sends on its own for sessions of retired work (30, Cleaning up sessions). Ensemble Node continues sessions only on such a computer, or one reporting 0.4.18 or later (35 §8.2).

Session deletion: capabilities.sessionDelete 1 handles owner-confirmed session.delete {id,jobIds} after the jobs stop. The runtime resolves its own job records and native session IDs, deletes the agent's saved history, and acknowledges only success. Project files and sign-ins stay; absent local records or native failures are explicit, not treated as a completed cloud-only delete. Native methods and retries: 35 §8.9.

Answers for the run: a computer with capabilities.runAnswers 1 takes a permission answered on the web or in the Node chat for this run (scope: "session"): it keeps the answer as a rule of its run, which covers the same command with other arguments, edits in the run's folder and the same site (30 §5). The web refuses that answer for an older computer with "Update Ensemble on <computer> …"; a question or a push is always answered once.

Secrets for runs (35 §8.7): a computer takes secret.save frames on its event stream, keeps the value encrypted in its own run_secrets with where it may be sent (allow), and answers on /api/devices/self/code-requests/:id with only whether it saved it ({saved, name, sealed, allow}). Whatever the computer sends back has every saved value hidden (hideSecretsDeep). Ensemble Node's secure box sends them; ensemble secrets manages them on the computer.

The web pins the first sealKey a computer reports (at pairing, or on its first report after updating) on its devices row and encrypts only for that key; a later different key changes nothing until the computer is paired again (credentials/device-key.ts). GET /api/devices returns each computer's pinned sealKey and sealKeyMismatch.

Which secrets a run gets. A claim may carry secrets (protocol 6): the names the run may have, possibly none. The computer keeps the list on the local job (context.secrets) and gives the run only those (secretScopeOf and runSecretEnv in lib/run-secrets.ts). Ensemble Node's runs always carry it, and so does a run whose run profile names secrets (35 §8.7). Without the list, the computer's mode decides (credentials/mode.ts): every environment secret in all, the compatibility mode and the default; none in named. A secret with allowed destinations is never put in an environment: a run uses it by reference, with secret_http or git (credentials/run-env.ts). A computer that reports capabilities.secretScope 1 applies the list; an older one ignores it and has neither evidence upload nor node tools.

The assistant can run turns on AI subscriptions signed in on a paired computer through subscriptions/ and the assistant-turn routes; this needs CLI 0.3.10+ or desktop app 0.3.7+ there (capabilities.assistantTurns 1 or more, ASSISTANT_TURNS_MIN). Offline sign-ins are not offered. This is separate from engine assignment and personal API keys (30).

A computer that reports assistantTurns: 2 (ASSISTANT_TURN_FILES) takes the files attached to the message: the turn's spec lists them (files), and the computer downloads each with GET /api/devices/self/assistant-turns/:id/files/:fileId (assistantFile in remote/client.ts, at most the listed size plus 64 KB, stopped with the turn). The route answers only the device the turn went to, only for a file on that turn's list (404 otherwise, or when the file is gone), and 410 once the turn ended; it sends the bytes with the file's media type, Cache-Control: no-store and X-Content-Type-Options: nosniff. A computer that reports 1 gets no file list; a note after the message names the files and says Ensemble on that computer must be updated to receive them. How each agent is given the files: 30 §8.

A computer that reports assistantTurns: 3 (ASSISTANT_TURN_SESSIONS) keeps one agent session per chat: the spec carries session: { key, history } (the conversation id and the chat so far), and the computer resumes that chat's session or starts one with the history (30 §8). Its done says whether the turn went on in the earlier session (result.resumed), which the web records with the turn's model.call metric (session: { key, resumed }). An older computer gets the history in front of the message instead.

A computer that reports assistantTurns: 4 (ASSISTANT_TURN_OWN_TOOLS, CLI and desktop 0.7.15) runs a chat, Ask Ensemble or a routine with the agent's own tools too (ownTools on the spec), each call judged by the hard floor in the turn's folder through POST /api/engine/assistant/ask on its own runtime (30 §8). An older one runs them with Ensemble's tools only.

A computer that reports assistantSessionDelete: 1 (ASSISTANT_SESSION_DELETE_API; CLI 0.7.10+, the desktop app from its next release) takes the frame assistant.session.delete {id, key} when the person deletes a chat (and 30 minutes after an inline Ask Ensemble's last answer, for that page's session: 30 §8), drops that chat's agent session (remote/assistant-session-delete.ts) and answers through POST /api/devices/self/code-requests/:id with { deleted: true, found }. Requests it missed while away are sent when its event stream (GET /api/devices/self/events) next opens (30 §8).

A computer that reports projectDelete: 1 (PROJECT_DELETE_API in remote/contract.ts) takes the frame project.delete {id, projectId} once an Ensemble project has been in Trash 10 minutes (workspace/project-delete.ts), removes the Codex projects Ensemble made for it there from the ChatGPT app's sidebar and from Codex (remote/project-delete.ts; 30, In the agents' own apps) and answers through POST /api/devices/self/code-requests/:id with { deleted: true, found, owed }. owed: the ChatGPT app is open, or Codex has not deleted one yet, so the computer keeps the removal and finishes it itself (every 2 minutes, and when its runtime starts). The web asks every computer paired to the project's account and every one that ran a Codex task of the project: an online one from the scheduler's minute tick, again at most every 5 minutes until it confirms, and one that was away when its event stream next opens. The requests wait per computer in the projects.codex-project-deletes preference for up to 90 days; those of a project back from Trash, or of a computer no longer paired, are dropped. The desktop app and the CLI runtime remove the Codex projects of their own deleted projects themselves, from the same minute tick.

A paired desktop app or CLI calls included models through /api/devices/self/included and /api/devices/self/included/turn. The device token identifies the account; Azure credentials stay on the host, and usage counts against that account's credits and the global ceiling (33).

4.10 A node's own tools on the computer

A node can run a command, take a screenshot of a terminal, run one approved command as the administrator, read git, call a development server on the computer, change or share a file, and browse and capture (§10) without starting an agent session (35 §8.13). Each call goes to the computer as a node.tool.request frame on its event stream, like code.request (askDeviceFor in devices/file-relay.ts). The computer judges it and answers once on /api/devices/self/code-requests/:id (device-tools/answer.ts). Only a computer that reports capabilities.nodeTools (NODE_TOOLS_API) gets these frames; for an older one the node is told "Update Ensemble on <computer>". Version 2 adds terminal, sudo, files share and calls without a folder; version 3 (NODE_TOOLS_RUN_STOP) adds run_stop. capabilities.nodeFiles 1 (NODE_FILES_API) adds a folder to files share (zipped here), any file up to 100 MB (NODE_FILE_MAX_BYTES), and files save. A computer that reports capabilities.live (LIVE_API) also takes live calls: terminals kept open between calls (§10.4).

Folders. A folder is one this computer shares (by its label) or a full path Ensemble may hand to an agent (resolveWorkFolder). A call that names none works in the node's own folder, <workspace>/nodes/<handle> (the node's nodeSlug, else the start of its id), made with mode 0700 on first use and refused when a link or a file is in its place (ownToolFolder). Ensemble's run folders (<workspace>/runs/<name>) and every node's folder (<workspace>/nodes/<handle>), and folders inside them, count as the node's own too (isEnsembleWorkFolder): the floor still applies there. A link, the home folder itself, a system folder, Ensemble's data folder and folders that hold keys, sign-ins, Ensemble's git helper or its code are refused (device-tools/folders.ts). The sandbox also denies the secret folders and files in the home folder (SECRET_PATHS in workspace/guard.ts, which includes SECRET_HOME_PATHS), Ensemble's data folder (with the run secrets, the key that encrypts them and the browser profile folder) and its git helper, inside any folder (nodeToolDenies).

Who decides (device-tools/policy.ts, on the computer). In order:

1. The hard floor: the folders above, and the list of dangerous commands (device-tools/dangerous.ts; disks, the system, the home folder and keys, power, accounts, the network, packages the system needs, fork bombs, and as the administrator open shells, piped downloads and keys) for run, terminal, sudo, the command of capture_app_start and each line a node submits in a live terminal. 403, and no answer opens it; the computer's ledger keeps node.tool.refused with the rule.
2. The node's own folder (and Ensemble's run and node folders), and public web pages (a name that is not local and resolves only to public addresses, publicWebAddress in device-tools/http.ts): allowed without asking. On this computer ensemble approvals websites ask (askBeforeBrowsing in remote.json) makes public pages ask too. Where the sandbox is weak, commands in the node's own folder still ask once per mission.
3. The owner's answers for this computer: rules in decision_rules with source node-tools, one per folder or web address (toolName folder or origin), for the mission (sessionId) or always.
4. Unattended (the node's run profile runs unattended, and the node does not ask first): allowed inside the sandbox.
5. Attended: 428 with body.ask (NodeToolAsk; capability run, git, files, read for sharing a file, http or browse). The server raises one card with Once, This mission and Always for this folder (or for this address). The answer goes back with the node's next call (grants), and the computer keeps This mission and Always as its own rules.

A sudo call is not judged per folder: its card is the owner's answer for that exact command.

The answers count only while the computer accepts approvals from the web (acceptWebApprovals); with it off, a call without a rule here is refused (409). Where the sandbox is weak (no OS sandbox, or capabilities.sandbox advisory or ask), the floor is best effort: Always does not cover commands, which ask once per mission instead.

Answers. 200 with the result (a NodeCaptureAnswer for terminal, whose result is {screen, exitCode, ended}, and for files share); 202 for a command still going; 403 for the hard floor; 404 for a missing path; 409 when switched off or not possible here. What a command prints is redacted for saved secret values and key shapes. An answer carries at most 64 KB (NODE_TOOL_INLINE_BYTES, the start and the end). Longer output goes in order, in chunks at line ends, to /api/devices/self/tool-runs/:runId/output, at most 1 MB per command. The server keeps it in Redis for 6 hours under the run's id, never in the database (devices/tool-runs.ts). A command still going after 60 seconds (NODE_TOOL_FOREGROUND_MS), or started with background, runs on. Its last chunk says how it ended (exitCode, and timedOut when it was killed at its limit), and the server wakes the node once (enqueueTurn, reason tool), except for a run the node stopped, a sign-in a newer attempt replaced, or a sign-in that ran out of time (35 §8.16.4). Stopping remote work stops these commands too.

Sign-ins a command needs (35 §8.16.4). While a run command runs, foreground or background, its output is watched for a sign-in code (device-tools/device-codes.ts: Microsoft's device login, GitHub's one-time code, AWS SSO, Google's device page, any "open ‹page› and enter the code ‹code›"); each code goes to POST /api/devices/self/nodes/device-codes as it appears, and again with status done (exit code 0) or expired when the command ends. On Linux and macOS the command runs with BROWSER, and xdg-open, sensible-browser, x-www-browser, www-browser and gio (Linux) or open (macOS) first on its PATH, set to helpers in <workspace>/.cache/tmp/ensemble-signin-<run>/bin: a helper writes an http(s) address to a file there and exits, and passes anything else to the real program. The runtime reads that file every 300 ms, opens each page (public https pages and this computer's own addresses only, five per command) in the node's own tab, raises a sign-in for it (source: "cli", done when a page reaches the command's own redirect_uri on this computer or the command ends well) and posts it at POST /api/devices/self/nodes/logins; declined or expired, the command is stopped. A command with a sign-in answers as soon as the owner is asked (deviceCodes, signInPages in the answer): in the foreground it goes on in the background then; a background one with a login word in it waits up to 8 seconds for its prompt first.

Ledger. Each call writes one entry on the computer (node.tool.run, node.tool.terminal, node.tool.terminal_session (open and close) and node.tool.terminal_line (each line's program, never its text), node.tool.sudo, node.tool.git, node.tool.http, node.tool.files.*, node.tool.browser, node.tool.capture, node.tool.ask and node.tool.refused) and one on the server (node.device_tool.<tool>). A command's sign-ins add node.cli_login (and node.cli_login.refused) on the computer and node.device_code on the server, with the site and status, never the code. They keep the program or operation and the folder or origin, never the output; node.tool.sudo keeps only the command's SHA-256, the folder and the exit code.

4.11 The worker's report and evidence

Report. A run's worker reports its own result with report_result, or .ensemble/result.json when its agent has no MCP (30 §9). The computer validates it as a StepReport (node/report.ts) and keeps it on its local job. complete sends it as report (protocol 6). The server checks the shape again, redacts it and keeps it on the job (workspace_jobs.report); an invalid report is dropped with a warning.

Evidence. Each file the worker attaches is uploaded before complete, while the run holds its lease (evidence/upload.ts, also during the run):

• POST /api/devices/self/jobs/:id/evidence, authenticated like progress: the device token and the job's lease in x-ensemble-job-lease. The body is the file's bytes (application/octet-stream). x-ensemble-evidence carries EvidenceMeta as base64url JSON: {id, kind, name, size, sha256, mime?, caption?, requirementId?, local}.
• The computer reads the file again from the run's evidence folder, under the same checks as when it was attached, and sends it only if it is unchanged. Text is redacted before it goes.
• The server checks it again: the size, the digest, credential names, and the type sniffed from the bytes (it never trusts the name or the computer). Text is redacted again. Then it stores the bytes where Files keeps uploads (32 §7), lists the file on the run's card in Ensemble Node and writes evidence.upload to the ledger. The same id again answers with the stored file.
• Answers: 201 (stored), 200 (already stored), 400 (bad description, size or digest), 413 (over 100 MB), 415 (type does not match the kind), 422 (credential file), 409 (lease, another run's id, or 100 files already), 404 (no such job).
• Every kind (screenshot, image, video, trace, report, log, file) is at most 100 MB (evidenceLimit in evidence/limits.ts).

Off on this computer. ensemble evidence off (evidenceUpload in remote.json, reported as capabilities.evidenceUpload) keeps the files here. Only their description goes up, with local: true and no bytes. The card shows them as "on this computer, not uploaded", and reading one answers 409. A server without the route (404) gets nothing more for that run.

Captures (protocol 7). What a run's browser and capture tools write is recorded as its evidence where it lies, with no attach step (evidence/captures.ts), and goes the same way. EvidenceMeta may also say:

A server before protocol 7 gets no held file and none of these fields.

Upload ids. A file the web asks for outside a run's lease goes to POST /api/devices/self/evidence-uploads/:uploadId, with the same headers and checks but no lease: the web makes the id (evidence/grants.ts, in Redis) for one device and account, and for one purpose: new (a node's tool call: files of that node operation, rows with no job, at most 100 per operation), release or preview (one file this computer keeps). 410 when nobody waits for it any more.

Files kept here (held, or upload off) are asked for with evidence.request {id, evidenceId, action, uploadId?} on the event stream, answered like a Code answer (evidence/requests.ts): preview sends the file under the upload id for its owner's eyes (the server keeps it five minutes in Redis, at most 32 MB); release sends it the same way and marks it released here (409 with evidence upload off); discard deletes it, its poster and preview, here. The file is read again by the same rules as any upload: unchanged since it was recorded, no saved secret value in it, text redacted.

4.12 Worktrees for parallel work

An Ensemble Node mission may build in parallel in one repository (35 §8.15). Its runs' claims carry mission (optional; older servers leave it out):

• Making one (prepareWorktree in workspace/worktrees.ts, from remote/accept.ts). With Ensemble's own git (hooks off), like a run's separate copy (24 §3). Only of a shared folder that is a git repository's own top folder, with a commit and a config that starts nothing. A folder shared as a repository gets a worktree instead of a fresh clone. The local job then works in place in the worktree, on its branch as it is (the web's branch, pull request and Git choice for this run do not move it), and may write the repository's own git folder, which every worktree shares, so its commits land. A check reuses the build's worktree as it is; a snapshot that is gone cannot be checked again. A continued run (a repair, a retest) stays in its run's folder.
• When it cannot be made (not a repository, no commit, a folder in the way), the run works in the folder itself. The ledger's remote.claim entry says why, and this computer's queue makes runs in one folder take turns.
• Removing. After an integration run succeeds, each worktree it brought in goes, only when clean and when nothing would be lost: a branch keeps its commits; a detached one must already be in the folder's history. Never forced. The rest stay for the person (git worktree list shows them).
• Capability. A computer that does this reports capabilities.worktrees 1. The web gives a parallel build on an older computer the folder itself, one step at a time, and its packet says to update Ensemble there. The new field and capability need no new protocol: an older computer ignores mission.
• Folder claims. Runs of one mission that meet in a folder outside their own take turns there without asking (context.mission.id; 24 §3).

5. Local execution and permissions

planLocalJob validates the hosted spec against shared folders, local settings, available engines and models, and code, sandbox and Git rules. Local conflict locks and budgets apply. A "trusted" flag from the server is not local trust.

5.1 Phone approvals

acceptWebApprovals (Accept approvals from the web and phone, ensemble approvals) is on by default. When on, the web's run settings apply and supported risky decisions can be confirmed remotely. When off, risky answers stay on the computer and web settings can only narrow local permissions. The computer checks its own pending decision and action hash; the first valid answer wins, and a timeout never allows.

A task from the web is limited in git only by the person's own choices: its Git choice (No limits by default), its branch (current, a new one by name, or an existing one), Use my git sign-in for private and SSH repositories (applied when the computer accepts web settings), and a pull request into any branch (24 §5). Pushes and gh use the computer's own sign-in; Ensemble puts no branch, push or merge rule of its own on it. Merging a pull request Ensemble opened waits for the person's Allow on Needs me unless the task said to merge when checks pass (24 §5.1).

A web confirmation is an authenticated session action, not a device-verified WebAuthn signature over each decision; the terminal passkey protects only the terminal.

5.2 Rate limits

Pairing, heartbeat and progress, file browsing, code, subscriptions and model calls have their own buckets in lib/rate-limit.ts and the relays. Account quotas and device/lease checks still apply.

6. Shared spaces

A run can belong to a shared space while running on a member's own computer. Members see permitted progress and results; execution decisions and file text belong to the person running it. Revocation, sharing access and the computer's switches are each checked (29).

7. What synchronizes

Assignments, bounded context and attachments, progress, decisions, results, usage and review metadata cross the protocol, plus model keys as described in §4.2. So do the worker's report and the files it attaches, unless the computer keeps them (§4.11), and the bounded answers of a node's tool calls (§4.10). The local database, arbitrary folder contents, the local ledger and other credentials do not.

8. Device lifetime and removal

Rename is owner-only and heartbeats keep the saved name. Revoke or unpair stops remote-originated work and turns and forgets the device token; local work follows its own lifecycle. An unreachable network is different from a 401/revocation.

A computer paired again is the same computer to the web when it is the same installation (capabilities.installId, kept in remote.json through unpairing); for pairings made before that, the newest computer with the same name and platform, or the only one on that platform (devices/successor.ts). Its earlier runs then show on it, open in Code and the file view, and run again there.

Packaged launchers set ENSEMBLE_INSTALL_PATH; on each heartbeat the runtime checks it, and once the executable has been missing on every check for 10 minutes it stops remote work, tries to revoke itself and drops the pairing even when the host is unreachable (remote/installed.ts, remote/loop.ts). The wait is for package upgrades, which remove the launcher for a moment (Homebrew relinks bin/ensemble, Scoop swaps current); before it, an upgrade with the runner running unpaired the computer. The web's online dot can stay until the heartbeat window passes.

9. Tests and limits

API, device and remote tests cover lease fencing, ownership, local-policy narrowing, decisions, attachments, file and code relays and revocation. Sleep/wake, physical passkeys, provider subscriptions and installed releases need their own environment.

Captures: src/evidence/captures.test.ts (tool results as evidence: kept, held, variants, a file with a saved secret deleted, one outside the work's folder refused), src/evidence/capture-run.integration.test.ts (a run's browser and capture tools through its MCP server) and src/node/capture.integration.test.ts (a node's, through node.tool.request against a simulated computer running the real tool layer; upload ids; held captures previewed, released and discarded through evidence.request; the live view).

Evidence and node tools: src/evidence/files.test.ts (a link to a key, a hard link, .env, a file swapped after the check, a file outside the evidence folder, a saved secret value, sizes and kinds), src/evidence/evidence.integration.test.ts (the upload route's lease, sizes, sniffed types, active formats only as downloads, each kind's type and disposition, redaction, the run card, owner-only reads, a computer that keeps its files, the blob store and its sweep), src/evidence/report.integration.test.ts (the MCP tools into job.report and the evidence folder; the .ensemble/result.json fallback), src/device-tools/device-tools.test.ts (the policy, the node's own folder, public web pages, the hard floor and its dangerous commands, loopback-only HTTP with redirects, read-only git, file changes and sharing, a folder shared as a zip with what it leaves out, archives refused by what they hold, the 100 MB cap, an attachment saved by its download id, long output and background commands, sign-in codes and pages a command opens, and the OS sandbox denying keys, Ensemble's data and the browser profile), src/device-tools/dangerous.test.ts (the list of dangerous commands), src/device-tools/terminal.test.ts (keys, the pseudo-terminal taking keys from a socket as Node passes them and giving back the program's exit code, a screen's colours and hidden secrets, and programs in a pseudo-terminal in the sandbox kept as PNG evidence; skipped without an OS sandbox, script or Chromium), src/device-tools/sudo.test.ts (sudo replaced: its arguments and input, the password hidden and wiped, refusals, sealed boxes and their reuse), src/workspace/sandbox/os-escape.test.ts (a process monitor lists the computer's processes but reads and signals none) and src/node/device-tools.integration.test.ts (the relay round trip with a fake computer, one Needs me card per folder, an older computer, owner-only tools, an attachment fetched once by that computer only and saved, hub_save_file {attachment}, a zipped folder in the reply's media, a long command waking its node once, a sign-in command's code on one owner-only card, and a node run getting only the secrets it names). macOS and Windows sandboxes are not exercised there.

Worktrees for parallel work: src/workspace/worktrees.test.ts (a build's worktree on its branch, its check in the same one, a snapshot, what is refused, and removing only what loses nothing, in a real repository) and src/node/parallel.integration.test.ts (the claim's mission, and two builds claimed at once).

There is no automatic move to another computer, full database sync, mid-command replay or inbound remote shell or tunnel. A compromised browser session can use whatever the computer's switches allow; local switches and OS rules are the boundary. Redaction is bounded and does not prove content has no sensitive data.

10. Browser and capture on the computer

The local runtime has a browser broker (apps/hub-api/src/browser/) and a capture service (apps/hub-api/src/capture/). Ensemble Node's device tools and a coding agent's run MCP server both call them (35 §8.16), through one tool layer: runBrowserTool and runCaptureTool validate a call and run it for one operation (a job attempt, or a node's tool operation), and browserToolDefinitions() and captureToolDefinitions() give each tool's JSON Schema. The server never runs a browser.

The desktop app's sidecar and the CLI's runner configure both when the hub starts (startCaptureRuntime in capture/runtime.ts, from index.ts on the computer only): the owner's privacy rules (the values of their saved secrets, read again at most every 5 seconds, and this computer's site lists, below), the OS sandbox for apps (§10.2) and logging. A runner (not a short-lived admin runtime for a CLI command) also cleans up after a crashed runtime and starts the janitors. When the runtime stops, every tab, app, display and recording ends.

10.1 Browser broker

Tools: browser_open (http and https only), browser_tabs (with the windows its pages opened: popup, openedBy), browser_close, browser_login_needed (hand the tab to the owner to sign in or to pass a human check, purpose, below), browser_login_check (check a waiting sign-in now, as when the owner says they did it), browser_snapshot (the accessibility tree with [ref=…] references, cut at 24,000 characters by default), browser_click, browser_type, browser_press, browser_wait (a selector, a text, a URL glob or a time), browser_screenshot (the viewport by default, one element, the full page when asked, another viewport size), browser_console, browser_network_errors, browser_record (a video, or a Playwright trace), browser_upload and browser_download. browser_open and browser_snapshot say when the page is a human check or blocks automated browsing (a visible captcha frame of reCAPTCHA, hCaptcha, Cloudflare, DataDome, PerimeterX or Arkose; "Robot or human?", Cloudflare's "Just a moment…", Google's unusual traffic, Amazon's robot check and its 503, Akamai's Access Denied; browser/human-check.ts), with humanCheck in the result, so the work hands the tab to its owner or uses another source. There is no script evaluation and no raw CDP, so no tool reads cookies or storage. An operation of work with the internet off (loopbackOnly) opens and loads only this computer's own addresses (localhost, 127.0.0.0/8, ::1); every other request of its tabs, and of capture's private tabs for it, is aborted. Recordings a closing tab finishes (a close, the work's end, an expired lease) are kept, held as they were watched, for the work to take (takeFinished).

Web tools (browser/web.ts, browser/shop/): one call each for what a small model would otherwise do in several. browser_view, browser_fetch and browser_extract with a url are judged like browser_open (public pages open without asking); browser_shop asks only for a shop named by an address on this computer's network.

Page hygiene (browser/hygiene.ts) runs after browser_open (unless clean: false), in every web tool, and before a screenshot (consent only, so a dialog the work opened stays): the buttons of OneTrust, Cookiebot, TrustArc (also in its frame), Quantcast, Didomi, Usercentrics (its shadow DOM), Sourcepoint (its frame), Google's Funding Choices and a dozen more; Google's and Yahoo's consent pages; Amazon's "Deliver to …" toaster; then, in dialogs and fixed overlays, a button whose whole text is an accept word inside a consent banner or a close word ("No thanks", "×"). Never a button that buys, subscribes, signs up or in, downloads or asks for notifications or a location. At most 2.5 seconds; the answer says what it closed. Pages at this computer's own addresses are apps under test and are left as they are.

The clean browser for read-only views (browser/clean-view.ts) is a CleanBrowser with its own profile beside each work profile, <data>/browser/<profile>-clean (agents cannot open profiles ending in -clean or -signin), kept warm three minutes after its last read. It only navigates, takes a screenshot and reads the HTML through the DOM domain; its screenshot is evidence like any capture, held when its text shows a saved secret or the site is preview-first. Work with the internet off never uses it.

A computer reports capabilities.browser: 2 for these tools; the server tells a node to update Ensemble on an older one.

What a capture never shows:

• Password, one-time-code and card fields, and elements marked data-ensemble-sensitive, are masked in screenshots, in every frame of the page (SENSITIVE_SELECTOR), and their values are hidden in snapshots, also when a placeholder puts the value on a line of its own.
• The owner's saved secret values (as written, and as escaped inside a quoted snapshot value) and key-shaped strings are hidden in snapshots, titles and console text. A screenshot, video or trace of a page that shows one comes back held: true with the reason: the owner previews it before it is sent.
• Page addresses and titles in tool results and the live view have secret query values (?token=, code= and the like), saved secrets and key shapes hidden.
• Sites the owner marks never captured are refused for screenshots, snapshots and the live view; a video that reaches one records none of its frames and is held. Sites marked preview-first are held. The two lists are this computer's own (captureSites in remote.json: ensemble capture sites, 26); a site is a host, covering its subdomains, or an exact origin.
• browser_type refuses a saved secret value. Uploads take only regular files with one name from the operation's evidence folder or folders the caller allows, never credential files (.env*, keys, certificates) or files containing a saved secret, up to 50 MB; the file is read by descriptor and checked again after opening. Downloads go into the evidence folder, up to 100 MB.
• A trace records every page of its browser, so it starts only when every page there is this work's and none is in a sign-in. A trace of the shared profile has screenshots only, no page source or network data; a private tab's trace has both, and is held when a page shows a filled password, code or card field.

Sensitive tabs. markSensitive(tab, on) is for a sign-in or a takeover, not for agents. While on, the tab refuses snapshots, screenshots, recordings, console and network reads and every action; browser_wait for a URL still works. A video drops its frames, a trace of its browser (whichever tab started it) is stopped and deleted, and console and network entries are not kept.

Live view. An operation's onLiveFrame receives a small JPEG of its most recently used tab about every 3 s (masked, quality lowered to keep it near 60 KB). It pauses, saying why, for a sensitive tab, a site the owner marked and a page that shows a secret. The broker keeps no frame. For a run from the web, the remote loop sends each frame to ROUTES.liveFrame under the hosted job and its lease (remote/live.ts; protocol 7, at most about 220 KB); a run started here, and a node's own browsing, have no live view (35 §8.16).

Sign-ins (design B11, 35 §8.16.1; logins/). After browser_open, a click, a key press or a submitted browser_type, the tool layer asks checkForLogin whether the page wants the owner's account (logins/detect.ts: one password field that is not a new-password form, a sign-in address with a user name or code field, a 401, or a known identity provider). On another site in the shared profile it raises a sign-in at once; on this computer's own addresses, in a private tab, or for a site already answered for this work it only adds a hint. browser_login_needed {tab, origin?, purpose?, signedIn?, reason?} raises one on purpose: purpose is sign_in (default) or human_check, and a page that shows a human check (browser/human-check.ts) with no sign-in form on it is raised as human_check whatever was asked. A raised sign-in (logins/requests.ts) marks the tab sensitive, holds it (two hours at most; above: a crash or a closed window opens it again rather than ending the sign-in, and only a tab that is gone ends it), brings a window already on the screen to the front, and puts login (with its purpose) in the tool's result: a node's answer carries it (NodeCaptureAnswer.login) and the server raises the card; a run's call waits in awaitRunLogin (logins/run.ts) after telling the server (ROUTES.runLogin, protocol 8), or asking on Needs me when no card is raised (withdrawn when the sign-in is seen done).

Seen done. While a sign-in waits, its page is looked at every 2 s and after each load (logins/watch.ts): the node's tab, or the clean browser while the owner uses one. A sign-in counts as done only by a change: the page asked first (a sign-in form, a code, an identity provider, another site, a challenge), and now it is the site (its origin or another address of the same site) with no password or code field, not a step of signing in (/forgot, /verify, consent, account choice, sign-out) and not a human check, loaded, with the site's sign-in state changed meanwhile (a cookie that is not analytics, consent or bot management, or a sign-in key in its local storage). A declared signedIn condition replaces the page rules and needs no storage change. A human check counts as passed when its challenge is gone and the page has loaded, seen twice in a row (in the node's tab, after the challenge was seen there). Done, the sign-in ends signed_in or passed, the tab is given back, and the computer tells the server at ROUTES.loginState (via: computer, window, takeover or check). browser_login_check {tab} and login.resolve check run the same check at once without the history (a human-check page is never signed in): {status: "signed_in" | "passed" | "waiting", message}.

The clean browser (logins/clean-signin.ts, browser/clean.ts). Signing in from the web (a takeover) and in a window on this computer happen in a Chromium Ensemble drives only with real input and the screen (Input.*, Page.startScreencast) and reads only through the browser's protocol (a DOM snapshot, Storage.getCookies, DOMStorage), never a page script, so Google lets the owner sign in and Cloudflare's checkbox works there. It is the full Chromium the broker uses (never the headless shell), with the profile's user agent, language and time zone and the broker's extra switches, in the profile's own folder <data>/browser/<profile>-signin (kept, so the owner stays signed in to identity providers there; a private tab gets a folder for the session only). It opens at the node tab's page, with the tab's cookies for the site (not an identity provider's), follows the windows a page opens (the newest is shown and takes input; when it closes, its opener again), and is watched as above. Done: its cookies for the site and for every host its pages went to (the identity providers), session ones too, move into the tab's browser (context.addCookies), with the site's local storage (a command's sign-in, source: "cli", ends there: the command got its answer on this computer); the tab opens where the clean browser ended (or reloads) and must then pass the check itself, else the sign-in keeps waiting with why (a human check ends passed, saying when the site still shows its check to the tab). A clean browser without a window waits 10 minutes for the next sign-in, then closes; a window closes when its sign-in is over. A warm one does not keep the runtime's process alive.

The owner's answer comes as login.resolve (logins/answer.ts):

The signed-in check is the work's condition (urlPattern, a glob matched without the query, or selector), else: on the site, with no password or code field, and no user name field on a sign-in address, and not a human check. Signed in or passed, the tab is unmarked and given back; ended any other way, it is first navigated to about:blank. The answer (LoginResolution) is only a status and a sentence; a server that does not send v: 2 hears signed_in for passed and failed for needs_code. The work's end (releaseOperation) and a stopped remote connection end its sign-ins (the second closes the clean browsers too).

Taking over (design B21, 35 §8.16.2; takeover/). takeover.start brings the channel's secret sealed for this computer: opened and checked like a secret box (this computer and key, the takeover as decision, the tab as name, a trusted site, unexpired) and taken once. The tab must be the waiting sign-in's, or the run's (its hosted job mapped to the local job that owns the tab); one that is opening again is waited for up to 20 s. The computer derives one AES-256-GCM key per direction (HKDF-SHA-256; takeover/channel.ts, the same messages and fixed vectors as the browser's @ensemble/shared-types takeover.ts) and marks the tab sensitive. A sign-in's takeover opens the clean browser (above) and streams its page shown (mode: "clean"); a run's, or a sign-in's when the clean browser cannot start (no Chromium with a window, a launch failure, or ENSEMBLE_CLEAN_SIGNIN=off), streams the tab itself (mode: "tab", with a note saying why). Both use Chromium's screencast (Page.startScreencast, JPEG at up to 1280 pixels). Each frame goes, encrypted with the next number, to ROUTES.takeoverFrame, one at a time, a newer one replacing any that waits; its header is {t: "frame", w, h, url, site, title, popup, mode} (url without its query; popup: a window the page opened). takeover.input is applied in order: in the clean browser at once, a message's commands sent together and then waited for (CleanSignIn.inputs), through Input.dispatchMouseEvent (with the buttons held and a click count), the wheel, keys as a keyboard sends them (takeover/keys.ts: a printable key or Enter is a keyDown with its text, Enter's being \r; Tab, Backspace, Delete, Escape, the arrows, Home, End, Page Up and Down without; Shift, Control, Alt and Meta held as modifiers; a character no US key makes, and text from the page's box or Paste, inserted with Input.insertText); in the tab through Playwright's mouse and keyboard. A message numbered at or below the last, or one that does not open, is refused and counted. It ends on takeover.end (I'm done runs the sign-in's check), the sign-in seen done (reason signed_in or passed, after its session moved into the tab), its time (endsAt, at most ENSEMBLE_TAKEOVER_SECONDS, 15 minutes; takeover.extend moves it later, never past an hour after the start), a minute without input or ping, the tab closing, the clean browser closing (closed, or not_carried when the site did not take the session in the tab), or stopRemoteWork; ending it itself, the computer says so at ROUTES.takeoverState. A tab it marked sensitive is given back; a sign-in's stays paused for its card, and its clean browser stays warm. The computer reports capabilities.takeover: 3.

Saved logins. <data>/logins.json, mode 0600: per site (an https origin, or this computer's own http address) the user name, password and an optional one-time-code seed, each encrypted with the runtime's local key (lib/secrets.ts), with when it was saved and last used. Kept apart from run secrets and never sent anywhere: the server hears only whether one is saved for the site of a card. ensemble browser logins, save-login, forget-login (26) use the local GET, PUT and DELETE /api/browser/logins (refused on the web). The computer reports capabilities.logins: 2.

Denied to runs. browserDeniedPaths() returns <data>/browser and <data>/capture: the profiles' cookies (with their session cookies' file and the clean browsers' folders) and capture's lease file. They are in Ensemble's own data, which the guard refuses and every OS sandbox backend denies wherever this runtime keeps it, the testing CLI's folder and ENSEMBLE_DATA_HOME's included (ensembleDataDenies, 24 §3).

10.2 Capture service

Virtual displays (Linux). Each app gets an Xvfb server that picks a free display (-displayfd), takes no TCP connections and accepts only its own cookie, kept in a private folder in $XDG_RUNTIME_DIR (else the temporary folder) so that an app in the OS sandbox can still read it; plus the first window manager installed (openbox, matchbox, fluxbox, icewm, jwm, xfwm4, metacity, twm). The app's environment has that display and no Wayland socket or session bus, so it never opens on the owner's screen.

Apps run in the OS sandbox. An app started for capture runs its own code, so it starts through the runtime's wrapCommand (configureCapture; captureAppWrapper in capture/sandbox.ts), which confines it as prepareSpawn does for agent commands (24): its folder and the cache writable, the agent environment (no credential variables, the cache's home) plus the variables the call gave it, Ensemble's data, keys and the browser profiles denied. On Linux the sandbox gives the app back what its private display needs and nothing more: that display's X socket, bound read-only at /tmp/.X11-unix/X<n> under the sandbox's own /tmp (sockets in SandboxPolicy, bubblewrap; Landlock shares /tmp), and its cookie's folder; another display, the owner's included, is out of reach. The app's internet follows the work's setting; an Electron app keeps the computer's network, because Playwright drives it over a local debugging port, and runs without Chromium's own sandbox, which cannot nest inside the OS sandbox. Electron goes through a small launcher script that does the same. The folder may not be the home folder, a system folder, or one that holds keys or Ensemble's data. Without a wrapper, capture_app_start is refused unless the call says the owner allowed this work to run apps unsandboxed (unsandboxedApps). Its window is found by process id (its own or a child's) through xdotool, wmctrl or xwininfo with xprop; else by title; else as the only window; else the whole display is used.

Frame-stepped video freezes the page's time and moves it one frame at a time: Playwright's clock for timers, animation frames, Date and performance.now(); the Web Animations API for CSS animations and transitions (those running when recording starts are seeked from their beginning); and window.__ensembleSeek(ms) for code with its own clock. Each frame is a masked screenshot, so frame n is at exactly n/fps seconds however busy the computer is. Real-time video takes the page's screencast and resamples it to a constant rate.

Media. MP4 (H.264) when a system ffmpeg has libx264 or libopenh264; otherwise WebM (VP9 or VP8) with Playwright's own ffmpeg. ENSEMBLE_FFMPEG names an ffmpeg to try first; each one found is asked what it can encode (capture/ffmpeg.ts). Each video has a poster (<name>.poster.jpg, or .png for a desktop app) and a preview no wider than 320 pixels (<name>.preview.jpg or .png). Videos stop at two minutes unless the caller raises the limit (maxVideoMs, the owner's setting). Size, length, frames and dimensions are read from the written file (capture/probe.ts). Files never overwrite each other.

Privacy. Web and Electron captures are masked and scanned as in §10.1; a real-time video of an Electron window records no frames of a site the owner never wants captured. A desktop app's window cannot be checked for secrets here, so its captures are always held. Transcripts and terminal screens are redacted before they are written (a line of a screen that shows a secret is drawn as plain text, the secret replaced), so they are not held.

Leases. Displays, window managers, apps and recordings are process groups under leases (capture/leases.ts, recorded in <data>/capture/leases.json), and each process carries its lease's id in ENSEMBLE_CAPTURE_LEASE. An app's lease runs 30 minutes from its last use. The janitor (every 10 s) ends leases that expired and leases whose process ended; an app's display goes with it. When the runtime exits it kills them all. At start-up (startCapture, or the first capture) it stops what a runtime that is no longer running left behind. A group is signalled only while its leader is the process recorded (by start time), or, once the leader is gone, while a member still carries the lease's id (read from /proc on Linux), so a reused process id is never hit.

Checks before sending (capture_check, in the Check shape of node/report.ts):

10.3 What a computer can capture

captureCapabilities() reports the browser and its revision, each ffmpeg and its encoders, the video format, the virtual display pieces, and for each target whether screenshots and video work, with the reason when not. On macOS it also reports the Screen Recording permission, which web pages and Electron apps do not need.

ensemble capture status prints this. A paired computer reports a summary on every heartbeat (capabilities.capture: browser (a Chromium for Ensemble is installed), video (mp4, webm or null), targets (each with screenshot, video and reason; terminal also tui: full-screen programs are captured as their screen, which needs Chromium and a pseudo-terminal, so not on Windows) and setup (whether ensemble capture setup or one system command would add something)), with capabilities.browser 1: the web offers a node's browser and capture calls only to such a computer, and says to update Ensemble, or to run ensemble capture setup, otherwise. ensemble capture setup installs Playwright's Chromium and ffmpeg into <data>/playwright with Playwright's own installer and prints the one command for system packages, for example sudo apt install xvfb ffmpeg openbox xdotool; it never runs sudo (26).

10.4 Live terminals and live views

A node keeps terminals open on this computer between its calls, and its owner watches them, and the node's own browser tabs, live from Node (35 §8.17). The computer reports capabilities.live: 1 (LIVE_API) and takeover: 3.

Terminal sessions (device-tools/terminal-sessions.ts). live {action: "terminal_open"} is judged like run in its folder (§4.10; the node's own folder by default) and starts a shell, bash (--noprofile --norc -i, no history file, no history expansion, EDITOR, VISUAL and FCEDIT set to false and its prompt variables read-only, all before its first prompt) or else sh -i, in a pseudo-terminal (script, as terminal starts one) under the OS sandbox: its folder writable, the denials of §4.10, the internet as the node's run profile says, no credentials. Without the OS sandbox or script (Windows) it is refused (409). Its output is replayed in a headless xterm with 1,000 lines of scrollback; output waiting to be drawn past 2 MB pauses the shell until the screen catches up. The prompt carries OSC 133 marks (133;B where typing starts, and from bash's PS0 133;C when a command starts), so the computer knows when the shell waits at its prompt and what the line typed there says.

A session ends on terminal_close, its shell exiting (exited, with its exit code), 30 minutes with nobody typing or watching (idle, LIVE_IDLE_MS), 4 hours (max, LIVE_MAX_MS), or stopRemoteWork (stopped); the computer then posts ROUTES.liveEnded. Another node's session is not found (404); one that has ended answers {ended: true}. The screen the node reads and the owner sees has saved secret values and key shapes hidden, line by line as capture/screen.ts hides them (screenRows, bufferLines). The computer's ledger keeps node.tool.terminal_session (open and close, with the folder and shell) and node.tool.terminal_line (each submitted line's program), never what was typed or shown.

Size. The terminal starts at columns × rows (120 × 32 from the node). A small helper started with the shell reads a new size from a pipe in the cache (<workspace>/.cache/live/<session>/size) and sets it on the terminal (stty … < /dev/tty), which tells the program in front; the owner's screen size (tty-size) goes there, and the node's size comes back when nobody watches.

Live views (takeover/terminal-view.ts, takeover/session.ts). takeover.start with kind: "terminal" names the session as tab and the node's operation; the start box is checked and taken once as for any takeover (§10.1), with the session's id as its name. Each frame is the whole screen, sent when it changed, at most 10 a second (TERMINAL_FPS; changes in between go in the next), up to two on their way at once: packMessage({t: "frame", kind: "terminal", cols, rows, cursor: {x, y}, title}, utf8 JSON {rows}), each row as runs {t, fg?, bg?, b?, i?, u?, dim?} (colours as #rrggbb; a row with a secret drawn plain). A frame refused while the web is still starting the takeover is sent again. Input is {t: "tty", text}, {t: "tty-key", key}, {t: "tty-size", cols, rows} and {t: "ping"} (a browser's text and key events are taken too), applied whether the owner watches or controls: it is their shell, and it is not judged. Control (takeover.control {on}) makes the node's terminal_send wait (409). With kind: "browser" and operation the tab must be the node's; view: "watch" streams it as a takeover does while the node keeps it (not sensitive, input ignored but pings), and control marks it controlled and sensitive (the node is refused on it, as in a sign-in) and applies input; handing it back gives the tab back. A window the shown page opens (page.on("popup"), a tab of the same work in the broker) is shown instead, its frames saying popup: true with its title and site, and its opener again when it closes; under control it is paused for the node too. A tab's frames come from Chromium's screencast at a level takeover/quality.ts sets (watching 1024 pixels wide at JPEG quality 45, control 1280 at 55; a step smaller while frames take over 250 ms to send, a step better after five seconds of quick ones), at most two on their way at once, none like the last. A view ends as a takeover does (Stop, its time, a minute without input or ping, stopRemoteWork) or when its session or tab ends (ended, tab_closed, closed_by_node when the node closed the tab); the session or tab goes on after a view ends.

10.5 Tests and limits

src/logins/logins.test.ts and src/node/logins.integration.test.ts cover sign-ins and saved logins, src/logins/signin.integration.test.ts sign-ins seen done, human checks, the clean browser (a popup followed, its session moved into the node's tab, a window on the computer) and one-time codes, src/takeover/takeover.test.ts and src/node/takeover.integration.test.ts taking over, src/device-tools/terminal-sessions.test.ts and src/node/live.integration.test.ts live terminals and live views, src/browser/clean.test.ts the clean browser (35 §11). src/browser/broker.integration.test.ts and src/capture/capture.integration.test.ts drive a headless Chromium against a local server, including a one-second CSS animation recorded at 30 fps and checked frame by frame in the encoded video; src/browser/browser.test.ts, src/capture/capture.test.ts and src/capture/leases.test.ts cover the rest without a browser; src/capture/runtime.test.ts the owner's privacy rules as the runtime gives them, and src/capture/sandbox.test.ts an app in bubblewrap reaching its own display's socket and cookie and not another display, Ensemble's data or a credential variable. The web tools: src/browser/web-tools.test.ts without a browser (the HTML reader, Markdown, products and prices, links, places, focus scoring, the interactive snapshot, every store recipe on its markup, ranking) and src/browser/web-tools.integration.test.ts with one (hygiene, focused screenshots, browser_view with a place, a clean-browser read and a human check, browser_fetch and browser_download of a PDF, browser_extract, and browser_shop over fixture stores). apps/hub-api/scripts/browser-live-check.mjs runs them against the real sites (not in CI). Parts that need Xvfb, Electron or an H.264 encoder skip, saying so, where those are missing.

There is no text recognition in images, so a desktop app's captures are always held. Other desktop apps on macOS, anything on Windows beyond web pages, and the real screen are not captured.