Competitive delivery work

Competitive delivery work

Baseline: NineDeploy 0.17.1, main abf5a39a. This is an implementation and acceptance tracker, not a statement of competitor parity.

Latest recovery work, 2026-11-03: the internal PostgreSQL restore/query adapter passed real hosted acceptance 41649048270 on frozen source 608fe3f094d952d0aff4b9073a340e2d708b2bee. Plain/custom dumps restored the known earlier data after the source changed; wrong data, invalid SQL and a missing extension were refused. Independent receipts confirm source/backup preservation and owned cleanup. Local covering checks passed 91 tests in four files, server typecheck and lint. Public drill integration, other engines, PostgreSQL PITR and final full-source certification remain required. See Isolated restore verification.

Latest Swarm evidence, 2026-11-06: the Swarm support matrix (DEPLOYMENTS.md §7) passed hosted acceptance 42238384185 on abd8eb45, a real two-task Swarm service, with main CI 42238365115 green on the same commit. Logs came from both tasks in time order; Stop → 0/0 and Restart → 2/2 with the route serving again; a service scaled to 0 outside the panel was brought back to 2/2 by the runtime patrol; inspect and the File Browser reached a task on the panel host; stats showed the per-task 128 MiB limit (tasks averaged); a 404 health path read healthy; Container running readiness deployed a service whose port answers nothing; a Swarm service removed behind the panel's back was marked error. Volume attachment (422 swarm_unsupported), TCP readiness (400), sleep (400 cannot_sleep) and autoscaling (no metric) were refused. This closes the R2 "Remaining Swarm support" gate. Still out of scope: metrics of tasks on worker nodes (the panel host's tasks only), distributed workers and Swarm autoscaling, which is not offered.

Earlier Swarm evidence, 2026-11-03: the single-manager artifact/restart cohort passed on frozen source 1e2214e3ae765e1d414f89149649abb9c38bfee1. Independent verification counted 13,174 exact application HTTP 200 responses with no errors/502 across Docker-to-Swarm migration, pinned promotion after repository HEAD advanced, readiness-held panel restart recovery and recorded rollback. Native/image/commit identities and owned cleanup passed. Its dedicated integration gate, distributed workers, other failure timings and final release certification remain open; see Swarm runtime acceptance. PR #25 remains the only open integration PR while its full source/image CI finishes.

Latest node evidence, 2026-11-03: run 41607197631 passed automatic panel retirement with 1,099 verified waking pages and no 502/connection errors; its fixture seeds dormant state. Subsequent run 41616036959 passed real five-minute idle per service, concurrent Stop, held strict HTTP readiness, new-process recovery, guarded demand and Stop during wake: 2,328 valid responses and owned cleanup. PRs #15–#18 and #22 are merged; #22 passed full source/runtime CI. The next public manual-wake comparison reproduced wrong-host Docker mutation before the fix, then passed with the same-name panel decoy untouched and 675 valid responses, no 502/connection errors. Its API/worker are real; sleep admission is a private real-observation factory. PR #23 passed full CI and merged to main as 9e79b931. The later real lost-routing-acknowledgement cohort restored the healthy application before any Stop, with 640 valid responses, no 502/connection errors and owned cleanup. PR #24 also passed full CI and merged. The later public automatic cycle passed normal API/plugin timers, a genuine traffic reset, automatic sleep/wake, readiness hold and panel restart with 791 valid responses and settled audit events. That public candidate passed all required source/runtime and serving-traffic checks on 65d15f69f71eb0e63e59dd1600e7455089e331a8 and merged through PR #25 as 977390d7279268b3ab8dffd0dde133fa40668c01; it remains unreleased. Never-bound orphan collection, other fault timings and the wider R2–R6/final release scope remain open; see NODE_SLEEP_WAKE.md for frozen sources and exact validation boundaries.

Development checkpoint, 2026-11-03: node RPS autoscaling has hosted 1→2→1 and source-loss acceptance (CI 41464316996); see NODE_RPS_AUTOSCALING.md for the exact source and scope. Node sleep/wake remains incomplete: agent idle watches and the wake catcher exist, and the panel's read-only 15-second scheduler, bound transport and conservative continuous idle proof passed source/runtime CI 41524700096 before PR #11 merged. The durable sleep/wake authority foundation passed full source/runtime CI 41538310932 before PR #12 merged. Fallback routing and permission-bound publication passed CI 41538382781 before PR #13 merged. The internal wake controller passed CI 41538384727 before PR #14 merged. Subsequent sleep, polling and maintenance recovery remain under feature acceptance; automatic node sleep stays disabled. Public capability/status reporting, non-disruptive catcher replacement and hosted sleep/wake traffic acceptance remain open. These development results do not certify the full R2–R6 scope or a new release.

Current published release: 0.18.21 (d5664149699a8cfc614ca3f7303950a9b6af900e), which completes R3. R4 and R5 shipped in 0.18.15, and R6's desktop source is in that release too; its installers are not published yet. Next feature release: 0.19.0 (R2: remaining runtime support). Validated maintenance fixes keep shipping as patch releases between feature releases. The paragraphs below about 0.18.16 and earlier record those releases as they were.

Published increment 0.18.16 fixes local Docker primary-network readiness and includes seven additional version-pinned Hub blueprints. Resolve the configured primary network by name/full ID, preserve legacy default/bridge compatibility, refuse missing or invalid addresses, pin explicit HTTP/TCP probes to that address and require the address to remain unchanged afterward. Command readiness inherits the corrected TCP gate. Running-mode policy and node-agent RPC operands are unchanged. This increment does not complete R2–R6; desktop installers remain a separate unsigned acceptance track and signed macOS credential persistence is not certified.

The 0.18.16 release workflow completed successfully on attempt 1: full release gate, backup integration, signed multi-architecture image, all seven versioned-image smoke cohorts, promotion and all five npm publications. Exact-source CI passed 17,788 tests in 1,125 files plus Docker/runtime and real Docker/PostgreSQL integration. The release gate also ran all eight test tasks; four build dependencies were cached. The source certificate applies to the frozen tag, not subsequent main development.

Independent verification downloaded all five GitHub assets, checked API digests and signed checksums, verified the archive commit and its installer/changelog contents, and confirmed both Linux architectures carry the tagged revision. Exact-tag image signature and SLSA provenance passed. The version image retained its independently verified candidate index sha256:6107911e5001651ef78675296de07178da6f778dd44f86845147c56341ed8910; latest resolves to that same index. All five public npm tarballs passed identity, version and integrity checks. Normal npm execution outside the repository with a new cache returned 0.18.16; its temporary directory was removed. Evidence: .temp_files/release-01816-6fa1046b/artifact-verification-state.json.

The versioned-image normal-deploy artifact independently retained 1,894 HTTP 200 responses, zero unexpected errors/502, across HTTP (503), strict HTTP (342), TCP (343), running (376) and command (330). Each trace contains responses from both the previous and replacement generation; owned-fixture cleanup was confirmed. Its source CI counterpart retained 1,882 HTTP 200 responses. Evidence: .temp_files/release-01816-6fa1046b/independent-release-runtime-verification.json and the CI counterpart.

The primary-network regression also passed on the signed versioned image: five topologies and all 15 readiness decisions agreed with real Traefik. Healthy controls returned 60 HTTP 200 responses; detached-primary and private-listener fault controls deliberately returned 40 proxy 502 responses and were refused by readiness. In comparison 41301478855, the unchanged digest-pinned 0.18.15 image falsely reported ready five times across those two unreachable topologies; the candidate refused all of them. Both comparison fixtures and versioned-image fixtures were removed. Four Go-template cases and four address regressions failed before the first fix; selected-address/stability checks failed before the second. The covering eight-file suite passed 228 tests. These direct admission and isolated traffic checks do not certify a production DNS/TLS cutover or complete the wider R2 matrix.

The signed versioned-image PostgreSQL autoscaling trace independently retained 2,407 HTTP 200 responses, all carrying SQL value 42. The service scaled 1→2→1 and both new immutable replicas served. Owned-fixture cleanup was confirmed. This accepts the stated single-node memory metric and PostgreSQL configuration, not RPS scaling or every runtime topology. Evidence: .temp_files/release-01816-6fa1046b/release-autoscale-database/independent-verification.json.

After the frozen tag, PR #6 fixed a packaged-desktop smoke race: a Chromium target may appear before its launcher preload is ready. The smoke now waits for both form and isolated bridge on first launch and restart. All 27 local desktop tests and typecheck passed; three-platform packaged/installed acceptance and full source/runtime CI passed on 2a37c05e before merge. This harness change is outside v0.18.16. The tag's original macOS smoke passed failed-job retry after the initial preload-timing failure; no numbered tag or image was replaced.

The 0.18.15 release workflow completed successfully, including all seven published-image smoke cohorts, promotion and all five npm publications. Its initial image job passed the source and backup gates; a stalled workflow was cancelled and resumed with failed-job retry, preserving the already verified candidate index sha256:4237e24eeb01ec5d5fe3055fe7b1e03efd53f2221f521a5f20c4e5c848e33173. The successful source CI executed 17,769 tests in 1,123 files; all eight test tasks ran, while four build dependencies were cached. The merged main tree matched the tag exactly; subsequent development commits are outside this certificate.

Independent publication verification downloaded all five GitHub assets, checked their digests and signed checksums, verified the archive commit and its installer/changelog contents, and confirmed both Linux image architectures carry the tagged revision. Image signature and SLSA provenance passed with the exact tag/source identity; latest resolves to the same index. All five public npm tarballs passed integrity/version checks. The CLI initially returned a transient registry metadata 404 after publication; ordinary metadata then became available and a fresh npm execution outside the repository, with a separate empty cache, returned 0.18.15. The temporary execution directory was removed. Evidence: .temp_files/release-01815-f2c282d1/artifact-verification-state.json and its downloaded assets/logs.

The published-image normal-deploy acceptance independently retained 1,945 HTTP 200 responses across HTTP, strict HTTP, TCP, running and command readiness, with no unexpected error/502 and confirmed owned-fixture cleanup. The equivalent source CI additionally passed all 11 canary cases (10,052 requests; zero unexpected errors/502, with 354 deliberately injected business-error 503 responses) and PostgreSQL autoscaling (2,685 successful responses carrying SQL value 42; both new immutable replicas served). These are the stated isolated scenarios, not a production DNS/TLS cutover or completion of R2's wider runtime matrix.

The v0.18.14 tag remains on eddf9a64281cb71c991a88951b57e3023396e9ef. Release run 41254561302 passed the full source gate (12 tasks, four cached build dependencies; all eight test tasks ran), backup integration and signed multi-architecture image publication. Six smoke cohorts passed. Public canary reached the owner-disabled refusal, which correctly failed the deployment and retained the stable service, but its traffic collector had fewer than 20 samples when stopped. That assertion blocked promotion and npm publication. The tag is not a GitHub Release. The normal panel health artifact independently contains 1,888 HTTP 200 responses across all five modes and confirms owned-fixture cleanup. Evidence: .temp_files/release-01814-eddf9a64/.

The 0.18.15 smoke harness waits for a marker written after 20 actual collected outcomes before submitting any canary deployment. A 30-second baseline deadline refuses to proceed without collection; early 502 responses remain in the final trace and still fail acceptance. Two regression assertions failed before the patch; three focused collector tests now pass, including the deadline control. The covering five-file suite passed 51 tests and server typecheck passed. The hosted CI and published-image confirmation are recorded above; the prior fast-refusal case collected 63 samples in CI without starting a candidate. Normal CI now also runs the public canary lifecycle, retaining synthetic traffic traces; final release diagnostics retain those traces on failure.

R2 node autoscaling now prepares immutable serving and spare containers during eligible deployments, confirms node readiness and routing, and fences scale decisions against deployment, lifecycle and database attachment changes. On development commit c08fde7edff19f4613240ff65856997dbf3b03f5, GitHub Actions run 41129661473 passed the isolated public-API memory autoscaling smoke: one serving replica scaled to two and back to one, both immutable replicas served through Traefik, and all 2,550 monitored requests returned HTTP 200. The runner's owned containers, volumes and network were removed. The same run passed real Docker/PostgreSQL integration, but its overall CI failed on two migration-test lint errors; the subsequent correction and CI run are separate evidence. This focused smoke does not cover production DNS/TLS, node autoscaling with a database, RPS, sleep/wake, fan-out or all R2 acceptance. Artifact: ci-node-autoscale-41129661473 (local copy: .temp_files/gh-node-autoscale-41129661473/).

The v0.18.12 release points to 757fc8edd38a9ea94e54a64a1e6d4173ffdba94f. Its uncached Linux release:check passed 17,179 tests, exact-commit GitHub CI passed, and all seven published-image smoke groups passed, including upgrade, node resource limits, database pooling and Swarm routing/file-secret updates. The completed release workflow published the GitHub assets and five npm packages. Independent checks verified the source archive commit, asset checksums and signature, signed amd64/arm64 image identities, npm tarball integrity and the normally downloaded CLI reporting 0.18.12. A cached npm tarball 404 cleared after its 300-second cache lifetime. Evidence: .temp_files/release-01812-757fc8ed/. Public DNS/TLS, production traffic and the wider R2 acceptance remain separate work.

R2 development has a separate sealed docker.startSet capability for starting pre-existing managed replicas by immutable ID under the shared journal revision. It preflights every target, confirms the resulting state and refuses an old request after a newer fence. A local compiled image passed 11 private-Docker scenarios through independent sealed agent processes, including Start receipt replay, a newer Stop, and same-name replacement safety. The smoke fixture verifies its random owner label and immutable container identity before host-resource cleanup. Evidence: .temp_files/startset-real-20261030/. This was agent-protocol evidence only; the newer public autoscaling proof is recorded above.

On source commit ae921a0d, an isolated local image passed 13 private-Docker agent scenarios, including sealed inventory watermark enforcement and ordered proxy-write fencing. A separate controller fixture on the same image prepared a running replica and two stopped spares, recovered a lost Start receipt, scaled down, and rejected the stale Start using real Docker and SQLite. Its routing and readiness callbacks were test doubles; it did not exercise a public panel or sealed HTTP. Both accepted runs reported no remaining labelled test containers or volumes. Evidence: .temp_files/autoscale-agent-ae921a0d-20261030/ and .temp_files/autoscale-controller-proof-ae921a0d-20261030/. The later public proof above covers its stated readiness and routing gaps only for the tested configuration.

The v0.18.11 release points to 1b69b62d0ecda7269c81039fc1c14c8f9706bf53. Its clean Linux release:check, exact-commit GitHub CI, release workflow and six published-image smoke groups passed, including two initial-deploy Stops followed by a successful deploy. The release archive and installer matched SHA256SUMS, and all five npm packages became available. The public smoke uses an isolated HTTP fixture; production DNS/TLS and broader R2 acceptance remain separate work.

After v0.18.11, initial Stop and completed-Stop recovery preflight every participant and identify agents older than 0.18.11 before attempting a handoff. The Stop remains held with an update reason. A local public API/worker smoke paired the candidate panel with the actual v0.18.10 agent image and confirmed the candidate container was not stopped; a second smoke passed two Stop/redeploy cycles using the updated agent. Evidence: .temp_files/public-old-agent-permanent-20261030/ and .temp_files/public-initial-after-mixed-20261030/. The v0.18.12 release workflow includes this mixed-agent scenario; production DNS/TLS remains unverified.

The v0.18.0 tag and image remain available as an unpublished candidate. The hosted smoke caught a setup race: readiness was changed during primary retirement, so the safety journal retained both generations and held the next deployment. The corrected smoke waits for settlement before the next readiness case. The 0.18.1 tag was also withheld after a repeated real canary smoke exposed agent operation capacity exhaustion; its workflow was cancelled before producing a registry image. Commit d8615abc separates bounded operation capacity from the invalid-request rejection budget. Published 0.18.2 preserves both earlier Git tags and the v0.18.0 image.

Normal panel deploy healthcheck traffic acceptance — 2026-10-31

Full source checkpoint — 2026-11-01

Source de1b512dcc2c6fd085e56c669cb27ce85f07b137 passed uncached Windows release:check: all 12 tasks, 17,764 passed tests in 1,122 files, two platform skips, and unchanged coverage floors. Vitest ran with two workers after an earlier parallel run exhausted memory. Server coverage: 93.70% statements, 88.54% branches, 92.19% functions and 96.58% lines. Log: .temp_files/release-check-de1b512d.log. Its exact-commit hosted CI also passed every job and published the development image under main-de1b512dcc2c6fd085e56c669cb27ce85f07b137. This is not a new numbered release and does not certify subsequent desktop workflow changes or the entire roadmap.

Candidate CI now additionally rehearses an in-place upgrade from the latest published release to the freshly built image. Its first hosted run 41249288176, on 1c98d7f2ec7cb0e914e6718018d720ca836742d2, passed. The actual published v0.18.13 image seeded 111 migrations; the candidate recorded all 118 once, kept the operator login, service, domain, deployment history and production environment, recovered the interrupted deployment as failed, and completed a new deployment. Application containers continued running across the panel swap; the upgrade itself kept the existing Traefik container and configuration. PostgreSQL import, terminal sessions, access grants and traffic analytics also passed their stated smoke assertions. Artifact: ci-candidate-upgrade-41249288176, local copy .temp_files/gh-candidate-upgrade-41249288176/. The smoke fails if its owned resource removal cannot be confirmed; the job exited successfully.

Both images in that first rehearsal reported 0.18.13: it proved a published-to-development image upgrade, not a new numbered release. Subsequent runs also retain the two image IDs, available registry digests, source commit and run ID alongside the log so matching version strings cannot obscure the tested identities. Identity output was verified in CI run 41289836391 for the 0.18.13 to 0.18.15 upgrade; its merge tree matched the tagged source tree. The final versioned-candidate matrix and signed-artifact verification for 0.18.15 are recorded above; production cutover remains separate.

The same run's normal-deploy traffic artifact was independently counted: HTTP 495, HTTP strict 350, TCP 337, running 363 and command 330; all 1,875 responses were HTTP 200. Its cleanup receipt confirms removal of the three owned fixture containers, both volumes and the network. Artifact: ci-panel-health-modes-41249288176, local copy .temp_files/gh-panel-health-modes-41249288176/. This repeats the delayed-listener acceptance on the upgrade-gated source; it does not prove all production failure timings.

The isolated public-API normal Docker deploy smoke reproduced 189 HTTP 502 responses in command mode on 8fe77f4f7b551f55d0a550566efcc9691a3bc94b (run 41207708389). A command returning zero admitted the candidate before its application port opened. HTTP, HTTP strict, TCP and running controls passed that same delayed-start fixture.

On 3a24af190a685e4e16f85436e9c24253e96ef011, command readiness also waits for the configured application port within the original deadline. The unchanged real panel, Docker and Traefik smoke passed all five modes (run 41210193261). Independently counted traffic: HTTP 501, HTTP strict 342, TCP 363, running 382, command 330; all 1,918 requests returned HTTP 200. The fixture delays application listening by 15 seconds for each replacement. Evidence: ci-panel-health-modes-41210193261, local copy .temp_files/gh-panel-health-modes-41210193261/; owned-runner cleanup was recorded. The same Docker job passed native-health controls, loopback-only refusal and node autoscaling. Production traffic, DNS/TLS and every possible failure timing remain separate acceptance work.

The comprehensive CI job on that commit failed MCP audit/import-cycle checks and two Swarm recovery tests. These failures remain explicit blockers to full certification. The published-image release matrix now requires the normal panel health-mode traffic smoke and retains its traces; that added release gate has not yet run on a new published candidate.

The two Swarm failures subsequently reproduced in a focused run (29 passed, two failed). Its Docker fixture lacked the newly required native-health/network inspection and TCP helper response. After modelling those boundaries only for known immutable Docker identities, all 31 Swarm recovery tests and 65 covering release/recovery checks passed. This focused correction does not replace full CI; the MCP audit/import-cycle failures remain open.

Node autoscaling with PostgreSQL acceptance — 2026-10-31

Subsequent verification: all three mandatory jobs of run 41141188617 completed successfully on 9cb09bae2d341f689b378deb61f059622df1e1df, including the comprehensive test job. The published-image release matrix now requires the same PostgreSQL autoscaling smoke before release promotion and retains its evidence on both success and failure. This workflow addition still needs validation on a future release candidate.

Development commit 9cb09bae2d341f689b378deb61f059622df1e1df passed the isolated public-API PostgreSQL autoscaling smoke in GitHub Actions run 41141188617. The real panel, sealed bridge agent, private PostgreSQL database and Traefik exercised memory-based scaling from one serving replica to two and back to one. Both immutable replicas joined the private database network, the spare began stopped, and both passed SQL readiness after scale-up. Independent inspection of the downloaded traffic trace counted 1,838 HTTP 200 responses, each containing the SQL result 42; both new replicas served requests. The runner's owned containers, volumes and network were removed. Evidence: artifact ci-node-autoscale-database-41141188617, local copy .temp_files/gh-node-autoscale-database-41141188617/.

The same run passed the ordinary node autoscaling smoke and real Docker/PostgreSQL integration. Its comprehensive test job was still running when this checkpoint was recorded. The previous commit 5d37f954110c55e52c3448d493405582a58c6127 passed an uncached Windows release:check (11/11 tasks; server: 12,515 passed, two platform skips) and hosted CI; that full local gate predates the new SQL smoke. This checkpoint accepts only the tested single-node memory autoscaling configuration. Public DNS/TLS, database topology changes during real scaling, RPS, fan-out, sleep/wake and wider R2 acceptance remain open. No release was published by this checkpoint.

0.18.5 publication attempt — 2026-10-27

The v0.18.5 release workflow completed with failure; no GitHub Release was published. Its tag points to 054570746eb42c93df6b85b1da5b153fe6181f77, and its signed version image exists at digest sha256:bbecbae08e25b37859c5967b606fd6bee24b19b6e803ab57ea5661833bd0bbbf. Both are preserved. Published v0.18.4 remains the latest release.

The exact candidate passed all 11 uncached local release tasks (15,597 tests in 925 files) and its main-branch hosted CI. The hosted image build and a local 0.18.4-to-0.18.5 upgrade passed. Publication was held by smoke failures: independent runners did not pull the image before inspection, the user-journey expected an outdated secret-provider list, and two node scenarios timed out waiting for cancelled or stopped candidate containers to be removed. The first two smoke-tooling changes are committed locally for the 0.18.6 candidate; the node cleanup failures still require diagnosis and safe verification. A local rerun of the user journey passed the provider step but later timed out waiting for a canary, so the complete journey is not yet verified.

The replica-database Stop failure also reproduced locally against the published v0.18.5 image: SQL readiness and rejection of an unhealthy candidate passed, then Stop completed while both replacement containers were still running and the former generation was stopped. The test exited 1 after the 90-second candidate-retirement deadline. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/replica-stop-repro/. The selected-runtime Stop controller discovers the selected generation, while the superseded deploy continuation loses its owner; this is a suspected cleanup gap, not yet a verified complete root cause or fix.

The independent public-canary Stop smoke also reproduced locally against the unchanged v0.18.5 registry image. Ten preceding scenarios passed, including healthy and measured rollouts, rollback, cancellation and panel crash recovery. Stop then timed out after 60 seconds waiting for its candidate to be removed; an inspected candidate remained running while the previously selected generation was stopped. The smoke exited 1. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/public-canary-stop-repro/.

An isolated native SQLite characterization confirmed the protocol boundary: the existing selected-runtime lifecycle fence still permits a subsequently admitted legacy creation, while the newer ordered runtime fence refuses unversioned creation altogether, including a legitimate future legacy deploy. Both observed-behavior controls passed. This is not a complete race reproduction or a fix; producer adoption and mixed-version handling must be verified before switching selected Stop to the ordered fence. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/node-stop-fence-characterization/.

Do not infer publication or runtime safety from the tag, image, or the earlier green gates. A future candidate still needs a fresh full gate, hosted release smokes, and verification of the actual GitHub Release and published artifacts.

0.18.4 publication acceptance — 2026-10-26

Published v0.18.4 from 44d1eb1fff996126a1cfe9cecf1810a56c766e0e. All release workflow jobs passed, including image publication/signing, every published-image smoke, GitHub Release promotion and the reusable npm publication job. The immutable local candidate passed 15,139 tests in 903 files, all 11 tasks uncached, followed by the repository tag gate.

Downloaded all five GitHub assets and verified the archive/install checksums, checksum signature and exact source archive commit. The signed version image and latest resolve to sha256:1cbafcc764598c28cf781a312086d6c51fa8dfde0fa74c29b3fe44c2329c6518; both linux/amd64 and linux/arm64 OCI revisions match the source commit. All five public npm tarballs passed SHA-512 integrity and package identity checks, and the installed public CLI returned 0.18.4. Evidence: .temp_files/release-0.18.4-20261026-cf626eda/artifact-verification-state.json.

The v0.18.3 tag and version image remain an unpublished candidate: its final public lifecycle smoke used the old 0.19.0 capability threshold instead of the maintenance support floor. The corrected policy passed twelve production-contract parity tests and seven actual public lifecycle scenarios on the same image before 0.18.4 preparation. Earlier candidate tags/images are preserved. Development protocol commit ec10b5ce and the future parallel smoke configuration remain outside the immutable 0.18.4 release; production deploy/scale/wake coordination and R2 acceptance are still open.

User-journey authentication checkpoint — 2026-10-27

After a temporary permission correction in the disposable v0.18.5 panel container, the long user journey passed canary, autoscaling and wake checks, then failed with an approved deployment reported as missing. The fixture ran for 20 minutes 34 seconds; the default access token lasts 15 minutes. Expiration is consistent with the observed failure, but that run did not record its HTTP response, so the cause is not established. A separate fresh-session approval journey against the patched test image passed approval, deployment and webhook rejection.

The user-journey fixture now sets a two-hour access-token lifetime for its isolated panel, and its deployment polling reports a non-200 HTTP status immediately instead of misclassifying it as a missing deployment. An executable before-proof reproduced a 401 as missing; the updated poller reports HTTP 401 and still recognizes a running deployment. A real fixture login confirmed a 7,200-second access token. A subsequent full updated journey was interrupted when its runner and result writer stopped during a daemon restart, after canary and autoscale scale-up had passed. Its owned Docker resources were removed. The restarted complete journey against the same patched v0.18.5 test image exited 0: normal and webhook deploys, Compose, databases, canary promotion/cancellation, autoscale up/down, sleep/wake, protected-lane approval, webhook rejection and fork-preview checks passed, followed by teardown. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/journey-after-restart/. This focused patched-image run is not certification of the final 0.18.6 release image.

Local retirement permission fix — 2026-10-27

A non-root panel could not retire its previous local Docker generation because the shared agent operation wrote its lifecycle journal under the read-only application directory. In the v0.18.5 image, the real user journey left a prior transition in retiring with an EACCES mkdir error and the next deploy queued. Making that directory writable inside that disposable test container allowed retirement, canary promotion and canary cancellation to proceed.

The panel now supplies a journal path under its writable data directory for local primary and previous-Docker retirement; remote agent operations retain their existing journal location. The regression failed before the change and passed afterwards with real SQLite, and the unchanged agent-path control passed. Five focused agent/recovery files passed 36 tests, primary recovery and canary passed 58 tests, and server typecheck passed. A wider five-file run timed out and is not counted as passing.

A derivative of the v0.18.5 image containing the three freshly compiled modified server modules passed an isolated real-Docker initial deploy, redeploy, 50% canary promotion (30/30 requests, zero 5xx), and canary cancellation (5/5 requests to the retained generation), exit 0. This is a focused patched-image proof, not a full release-image certification. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/local-retire-fixed-image/. The independent node Stop candidate-cleanup failure remains open.

Patched-image upgrade checkpoint — 2026-10-27

The published v0.18.4 image was upgraded to the patched v0.18.5 test image containing the local-retirement fix. The real isolated Docker upgrade smoke exited 0: existing users, service, domain, history and running application survived; the interrupted deployment recovered; a new deploy succeeded; and all 104 recorded migrations were applied once. Additional Traefik, terminal, data-access and settings checks passed on the upgraded state. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/upgrade-local-retire-fix/. This proves the tested derivative, not the final v0.18.6 artifact or the unresolved node Stop case.

0.18.6 preliminary source gate — 2026-10-28

The immutable committed snapshot d863daed9a648ec05c4312be1ae5b47b20f69cb3 passed the complete uncached local pnpm release:check: 11 of 11 tasks, 15,601 tests in 927 files, exit 0. Server coverage was 93.26% statements, 87.90% branches, 91.54% functions and 96.37% lines; web coverage was 98.16%, 93.63%, 97.52% and 98.70% respectively. Evidence: .temp_files/preflight-0186-20261028/.

This snapshot predates the subsequent node environment receipt and fan-out ordering changes. It is a preliminary local result only: the final commit needs a new full gate, the node Stop failure remains open, and no GitHub Release was published.

A later immutable snapshot 160e1be2f459bce37c7628bbda636b8e05a9543b, which includes the environment receipt change, failed its full local gate: 17 server tests failed in two fixture files, while 10,904 server tests passed. The four fan-out routing failures and thirteen pipeline failures came from mocks that did not return the agent’s actual environment-file receipt. The mock corrections passed all 24 fan-out routing tests and all 259 pipeline tests in isolated runs; the complete gate on the corrected final source remains outstanding. Evidence: .temp_files/preflight-0186-node-env-20261028/ and .temp_files/node-receipt-pipeline-tests-20261028/.

Node environment receipt checkpoint — 2026-10-28

A real builder call with a mocked node response showed that missing, duplicated or wrong-file environment receipts still led to container startup. The same wrong-file response also allowed fan-out and a managed node database to start. A host-port fan-out additionally retired its previous generation before validating the replacement environment receipt. Five new cases failed before the narrow changes and passed afterwards; a sixth host-port case failed before the ordering change and passed afterwards.

Node deploy, fan-out and managed node database creation now require exactly one successful receipt for the requested environment file. Host-port fan-out validates the receipt before retiring its previous generation. A file from an unconfirmed write is not deleted when its ownership is uncertain. Existing receipt shapes were checked in the v0.10.42, v0.16.0, v0.17.1 and v0.18.4 agent sources. Eight affected test files passed 176 tests; server typecheck and lint exited 0.

A derivative test image containing the receipt-validation changes (before the later fan-out host-port ordering adjustment) passed a focused real sealed-agent, worker and Docker check: a node PostgreSQL database and two application replicas started, and both replicas passed SQL readiness. This did not exercise fan-out, public DNS/TLS or the unresolved Stop cleanup. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/node-env-receipt-fixed-image/. This is not final release-image certification.

Two existing fan-out success fixtures were also strengthened: after the receipt requirement, their old mock responses caused the target to fail before the behavior under test while the original assertions still passed. The unchanged success assertions failed against those old mock responses, then passed with the real agent receipt shape; all 33 fan-out and readiness tests passed.

Node file mount receipt checkpoint — 2026-10-28

Node file mounts previously accepted a missing, duplicate or different deployment file receipt and could continue with the fallback or reported path. Three before-tests failed while the exact-path control passed. The panel now requires a successful, single receipt for the requested service, deployment and file before passing the mount to the node runtime. Agent sources from v0.16.5, v0.17.1 and v0.18.4 were checked for the same receipt format.

The affected node environment, mount, fan-out, database and agent suites passed 262 tests in 11 files; after adding an additional container-start refusal control, the mount/volume/agent neighborhood passed 86 tests in three files. Server typecheck and lint exited 0. A derivative image with these changes passed a focused real panel, worker, sealed-agent and Docker test: PostgreSQL and two replicas started, both passed SQL readiness and both contained the requested mounted file. The test exited 0 and removed its owned resources. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/node-mount-receipt-fixed-image/. Public DNS/TLS, request continuity and the unresolved active-deployment Stop case were not tested by this focused smoke.

R2 completed node preparation receipt replay — 2026-10-27

A completed ordered node preparation could not replay its receipt after its temporary environment file was deleted. The agent now records a hash of the normalized request shape alongside its journal head in the same transaction. If an environment file is absent, it returns only a previously completed receipt whose protocol version, request identity, shape, remaining environment fingerprints and receipt identities still match. A missing file on a new or incomplete request cannot create a runtime; an unknown request cannot create an empty journal. A legacy receipt without shape evidence remains held unless the original environment is still available to establish that evidence.

The receipt describes the earlier completed operation; it does not attest current readiness or recover missing environment contents. The producer must freeze its inputs and recheck readiness. Production deploy/scale/wake has not adopted the ordered protocol, and the remaining physical producer, recovery and mixed-agent acceptance gates for 0.19.0 remain open.

A real SQLite before-proof failed on the removed file while an existing replay control passed. The unchanged strengthened smoke reproduced the failure against the older built agent, then passed 10 real Docker, native SQLite and sealed HTTP scenarios across independent agent processes with the fixed artifact; it also checked that the temporary environment file was absent. All 112 focused agent/client/creation tests passed. The frozen 15e73d42 source snapshot plus the two agent files passed the complete uncached gate: 15,419 tests in 912 files, all 11 tasks, exit 0. The separately frozen GitHub candidate e1cd6b79 passed its complete web build/lint/typecheck and 2,440 tests; the final pipeline fixture passed all 259 tests. These are separate snapshots, and later uncommitted UI work was excluded. The exact agent output matched the native smoke; a clean, socket-free UID 999 startup returned HTTP 200 and db: ok. Evidence: .temp_files/r2-prepare-receipt-replay-20261027-6096c41a/verification-state.json.

One prior artifact test failure is reproducible without DNS: its GitHub fixture now uses a deterministic public address while the real egress policy remains active. Both artifact cases passed in the network-isolated control and the full pipeline suite. This does not establish the cause of the other earlier artifact failure, which remains recorded in the preceding checkpoint.

R2 superseded node continuation checkpoint — 2026-10-27

The persisted deployment stamp was checked at admission but not by later continuations. Real migrated SQLite with mocked agent responses reproduced two failures: an obsolete rejection changed a newer running deployment's service to idle, or overwrote a service moved to another node with the old runtime and error status. A third before-proof showed a successful old image-pull reply attempting to write its environment after Stop and a newer deployment had taken authority.

The production pipeline now checks the captured private stamp, shared deploy epoch/nonce, deployment/service identity and original plain-Docker node placement at subsequent boundaries. Primary builder agent calls check before dispatch and after either success or rejection. Readiness and routing checkpoints use the same authority, while publication/failure SQL predicates recheck it at statement execution. An obsolete continuation retains effects for the current owner, avoids independent failure cleanup and suppresses duplicate outcome callbacks. Cancellation under the same owner still permits authorized cleanup. Final recovery completion also requires a current authority check; unresolved recovery records remain held.

A concurrent-read before-proof also showed an earlier SQL response being accepted after a different checkpoint had already observed ownership loss. That loss is now terminal across in-flight reads. The final focused set passed 42 tests, with 255 pipeline cases not selected. It covers unchanged-owner failure settlement, delayed success/rejection, Stop winning after a checkpoint, changed placement/nonce/epoch/kind, missing history and unavailable authority reads.

The final Linux/Node 26.12.1 snapshot passed TURBO_FORCE=true pnpm release:check: 15,393 tests in 911 files, all 11 tasks uncached, exit 0, including all 297 pipeline/owner cases. The server passed 10,770 tests in 600 files with unchanged coverage floors: 93.26% statements, 88.00% branches, 91.48% functions and 96.39% lines. Database, schemas, SDK and plugin SDK retained 100%. All 27 affected and inherited source inputs matched the successful container and frozen workspace hashes; 2,756 compiled/runtime files were hashed. A socket-free cold start of that exact output with a clean database as UID 999 returned HTTP 200 with status: ok, db: ok and version 0.18.4. Its owned container and volume were removed. The three unrelated UI files remained unchanged and were excluded from certification. These final documentation notes were added after the source gate.

The first full snapshot passed 10,767 server tests and failed two artifact fixture assertions. Both assertions passed in an isolated coverage replay and in the final complete run without changing them. The first failure logs are retained and their cause has not been established; no artifact-production fix is claimed. Evidence: .temp_files/r2-deployment-owner-boundaries-20261027-eb044d45/verification-state.json.

These are panel checkpoints and SQL fences, not adoption of the node's ordered mutation protocol. A legacy RPC already admitted on a node can still finish after a panel check. Owner-specific immutable environment files, durable phase requests, typed prepare/network/retire integration and remaining recovery/controller/fan-out boundaries must be completed together before physical producer acceptance. Scale/wake, the public mixed-agent matrix and R2/0.19.0 release remain open.

R2 node deployment authority checkpoint — 2026-10-27

Primary plain-Docker node deployments now persist a private per-deployment owner record during actual admission. The conditional deployment claim, configuration snapshot, shared service epoch/nonce, captured node identity and deploying status commit in one SQLite transaction. Queued successors do not reserve authority or invalidate an active predecessor. A resumed eligible deployment reuses its original stamp even when configSnapshot is rewritten; a Stop or newer owner prevents that old attempt from reserving authority again.

Migration 0104 adds an empty private journal without guessing ownership for existing work or changing deployment payload columns, configuration or current service authority. Removing a node retains the original stamp rather than permitting silent reassignment. Deployment/service deletion still prunes the corresponding journal through foreign keys. The journal is separate from list/detail API payloads.

The valid before-proof reached the real pipeline and migrated SQLite, then failed on missing deploy authority at the mocked node boundary while two eligible-run controls passed. Ownership tests cover queued predecessors/successors, immutable retry identity, Stop and newer-deploy authority, removed/moved nodes, rejected states, transaction rollback, exhausted revisions and foreign/missing deployments. The database package passed all 68 tests in 23 files at unchanged 100% coverage; schema generation reported no changes. The earlier complete pipeline/owner neighborhood passed 303 tests in three files, and the final focused set passed 42 tests, including Stop winning before pipeline admission. The final complete TURBO_FORCE=true pnpm release:check passed 15,369 tests in 911 files, all 11 tasks uncached, exit 0. Server coverage was 93.27% statements, 87.99% branches, 91.47% functions and 96.40% lines; database and schemas retained 100% at unchanged floors. The first full attempt caught the missing mandatory retention declaration; the journal catalog now records its existing deployment/service cascade policy, all five retention checks passed, and the corrected full gate passed. All 27 affected and inherited source inputs matched the successful container and frozen workspace hashes; 2,756 compiled/runtime files were hashed. A socket-free cold start of that exact output with a clean database as UID 999 returned HTTP 200 with status: ok, db: ok and version 0.18.4. Its owned container and volume were removed. The three unrelated UI files remained unchanged and were excluded from certification. These final documentation notes were added after the source gate. Evidence: .temp_files/r2-deployment-owner-20261027-751d7f3a/verification-state.json.

This increment does not adopt the node mutation protocol or claim physical producer acceptance. Displaced or superseded active work is refused rather than retargeted; fenced recovery and settlement remain required. Next, bind the persisted stamp to owner-specific environment files, prepare/network/retire requests and every publication, cleanup and recovery boundary. Keep existing unresolved-transition holds and capability floors until the real public worker/agent late-request, Stop and legitimate-new-deployment matrix passes. Scale/wake and the remaining R2/0.19.0 gates remain open.

R2 deployment admission checkpoint — 2026-10-27

Production pipeline entry previously checked only that the deployment existed. A stale invocation for a completed, superseded, actively deploying or gated row could still reach the builder. The conditional queued/building claim also returned zero rows without stopping a continuation whose status changed after entry. Real migrated SQLite reproduced both boundaries: seven initial-state failures plus four passing controls, then seven lost-claim/node-boundary failures. Builder and proxy effects were mocked; this is not a real Docker deployment proof.

Entry now accepts only queued/building work before starting a pipeline log writer, hooks or lifecycle events. A failed conditional claim ends a stale attempt before runtime effects and suppresses its duplicate outcome notification. Explicit cancellation keeps its existing cleanup semantics. The worker still owns its separate writer mark; an ignored pipeline invocation does not close another run's log. The existing event fixture now starts queued rather than replaying an already-running row, with its original event assertions unchanged.

All 18 real-SQLite admission/Stop controls and 23 focused admission/kernel-event tests passed. The initial broader pipeline diagnostic passed 246 cases and failed the one event fixture corrected above. The final Linux/Node 26.12.1 snapshot passed TURBO_FORCE=true pnpm release:check: 15,349 tests in 909 files, all 11 tasks uncached, exit 0, including all 254 pipeline cases. Server coverage remained above the unchanged floors at 93.26% statements, 87.98% branches, 91.46% functions and 96.40% lines; database and schemas retained 100%. Both changed files and the 19 inherited certified inputs copied from the successful stopped container matched their frozen/workspace hashes. The 2,750 retained compiled/runtime files were hashed. A socket-free cold start of the exact output as UID 999 returned HTTP 200 with status: ok, db: ok and version 0.18.4; its owned container and volume were removed. The three unrelated working-tree UI files remain unchanged and were excluded from the frozen input. These final documentation notes were added after the source gate. Evidence: .temp_files/r2-deployment-admission-20261026-1c76d305/verification-state.json.

This closes a prerequisite before durable deployment ownership is activated. The independent persisted owner stamp is covered by the subsequent authority checkpoint above. Production preparation/network/retirement wiring, scale/wake and the remaining R2 acceptance matrix are still open; this checkpoint does not publish 0.19.0.

R2 generation-independent Stop checkpoint — 2026-10-26

The initial public API reproduction returned 404 while a first node deployment was still building, with one expected failure and 35 passing controls. The development implementation now accepts a durable 202 Stop with nullable runtime/deployment anchors, atomically cancels queued/building/deploying/CI/approval work and advances the existing shared service authority. The never-deployed idle case retains its 404. Migration 0103 preserves existing command data, foreign keys and the AUTOINCREMENT high-water mark even when old history has been pruned; null anchors are allowed only as a complete pair on Stop.

The controller fences every accepted participant node before inventory, captures actual immutable IDs across managed generations, confirms route withdrawal on every node and stops the captured set before completing. Equal Docker IDs on different hosts remain separate targets. Successful completion also clears an additional node's pending deployment status when it had no selected runtime. Routing loss retains the captured set for an explicit retry; selection of a newer runtime prevents the old Stop from resuming. Unsupported/offline agents remain visibly pending. The new operations require the sealed, owner-disableable docker.mutations capability at the existing 0.19.0 support floor; the published 0.18.4 agent is not represented as supporting this development path.

The service detail panel offers Stop for all five cancellable first-deployment states and reports a pending request rather than completed physical effects. Local Docker, Swarm and idle-node controls retain their prior availability. Stop refreshes deployment history after cancellation. With no selected node runtime, the panel offers Deploy instead of an unavailable Start action. The five missing-button cases failed before the UI change, then passed in all 138 final service-detail tests. The additional-node status defect likewise failed before its narrow completion fix. API tests use real migrated SQLite with the agent/Docker boundary mocked; real sealed-agent/native-Docker acceptance is tracked separately below.

The extended native protocol command passed all nine real Docker/native SQLite/sealed-HTTP scenarios in independent agent processes, including mixed-generation Stop, empty-node completion and protection of a newer typed deployment from stale Stop. Owned containers and volumes were removed. The final UI package passed typecheck, lint, build and all 2,422 tests in 171 files at unchanged coverage thresholds. An initial concurrent UI diagnostic failed one unchanged Login error-card assertion; the final serialized run passed all 24 Login tests without changing their source or timeout. The first uncached project gate passed all 10,697 server tests but failed its unchanged statement threshold (93.24% against 93.25%). Fourteen additional safety and registered-operation tests now cover incomplete Docker inventory, corrupt/unadopted journal state, physical Stop failure/retry, missing captured IDs and actual machine-frame dispatch; all 67 tests in those two files passed. The final combined Linux/Node 26.12.1 snapshot passed TURBO_FORCE=true pnpm release:check: 15,333 tests in 909 files, all 11 tasks uncached, exit 0. Server coverage is 93.26% statements, 87.97% branches, 91.46% functions and 96.40% lines; database and schemas retain 100%. All 19 owned source files copied from the successful stopped container matched both the frozen input hashes and working tree. The 2,750 retained runtime files were hashed. A socket-free cold start of that exact compiled output as UID 999 returned HTTP 200 with status: ok, db: ok and version 0.18.4; its owned container and volume were removed. The three unrelated working-tree UI files were excluded from the frozen input and their original hashes are unchanged. These final documentation notes were added after the source gate. Hosted CI run 40404297162 subsequently passed for 1c76d305, including full verification, Docker image build, real Docker/PostgreSQL integration tests and the edge-publication job. This development CI result does not publish a versioned 0.19.0 release. Evidence: .temp_files/r2-unselected-stop-20261026-67ad3b93/verification-state.json. This is not R2 acceptance or a new release.

Next, connect production deployment preparation, database-network changes and retirement, followed by scale and wake, to the same durable service owner. Preserve worker FIFO: queued jobs must not invalidate an active predecessor, and accepted Stop must cancel waiting jobs atomically with ownership transfer. Keep an independent persisted deployment owner across configSnapshot rewrites and use owner-specific environment paths. New public deployment after Stop and unresolved fan-out handoff cancellation still require real end-to-end acceptance; keep existing unresolved-transition holds until those proofs pass. Node autoscaling/sleep/wake, remaining Swarm/mixed-agent acceptance and final uncached 0.19.0 upgrade/promotion/package gates follow. R3–R6 remain separate feature releases.

0.18.2 publication acceptance — 2026-10-25

Published v0.18.2 from c097f3a3f29feb3521b6ad22b3d09181acfcbc9f after every release workflow job passed. The local immutable-artifact canary smoke passed all 11 cases with 10,703 monitored requests, 364 deliberately injected HTTP 503 responses and no unexpected 502. Upgrade from 0.17.1, node operations, Docker readiness, recorded Compose recovery and the full user journey passed. Real Chrome checks at 1280px and 375px saved settings, observed pending promotion and aborted it with no console/runtime errors. The pushed registry image separately passed all 11 cases on GitHub with 10,473 monitored requests and no unexpected 502. These are isolated fixtures; operator production traffic and public DNS/TLS were not exercised.

Both linux/amd64 and linux/arm64 registry manifest/config identities were verified; the published amd64 image also passed a non-root cold-start health and exact-file smoke. Image digest: sha256:68f92786868364ad5b7b89f8579eb5ada37978af3706a1e6302519c8fdfe11e6. Image and checksum signatures, GitHub build provenance, source archive/install script checksums, five npm provenance bundles and the installed public CLI were verified. Existing v0.18.0/v0.18.1 tags and the v0.18.0 image were preserved. Earlier failed candidates remain unpublished. Local evidence: .temp_files/release-0.18.2-20261025-c097f3a3/.

Release smoke concurrency — 2026-10-26

The next release workflow partitions the existing fourteen published-image smoke commands into six isolated cohorts: upgrade/journey/Compose, agent runtime, panel lifecycle, public canary, public lifecycle and replica databases. At most three runners execute concurrently. Every cohort remains required for promotion; fail-fast: false lets the other cohorts finish their evidence when one fails, while the aggregate matrix still blocks latest, GitHub Release and npm promotion on any failure. Previous-published-release discovery runs only in the upgrade cohort. Existing commands, timeout budgets and the requested tag checkout are retained.

All 43 focused tests in three files passed, including complete command coverage with and without a previous release, one cohort per command, read-only smoke permissions and mandatory promotion dependencies. The official checksum-verified actionlint 1.7.12 returned zero. The four new matrix tests failed against the original serialized configuration before passing against the final workflow. Hosted parallel execution and its actual duration are NOT RUN / NOT MEASURED until a subsequent release uses this configuration. The completed 0.18.4 workflow was pinned to 44d1eb1f and used its original serial smokes. Evidence: .temp_files/r2-runtime-mutation-admission-20261026-35eb0b01/release-matrix-proof/.

R2 ordered mutation protocol checkpoint — 2026-10-26

The development protocol adds sealed, owner-disableable preparation, fencing, network membership and retirement under the shared per-service native SQLite journal. Preparation binds every serving replica and stopped spare to one pinned image, service/deployment identity, durable nonce and normalized specification fingerprint. Protocol adoption persists across later lifecycle commands so delayed unversioned creation and cleanup cannot bypass the current owner. Receipt retries preserve captured immutable IDs rather than following a reused container name. The browser-neutral schemas use the existing UTF-8 length helper to keep the complete preparation request within the sealed transport budget.

The frozen protocol passed schema build, server typecheck and server/schema lint. All 678 schema tests in 32 files passed at the unchanged 100% coverage floors; 200 node tests in six files and 42 selected-node client behavior tests passed. The client verifies the durable request identity, complete generation coverage and requested serving/spare state, and repeats SQL authority, cancellation and deadline checks around capability discovery and execution.

The permanent pnpm smoke:runtime-mutations --image=<built image> command passed six actual Docker/native SQLite scenarios through sealed HTTP in independent agent processes. It verified stopped-capacity preparation and database-network membership, delayed creation after Stop, replacement-safe retirement replay, stale network/legacy cleanup refusal, hard-agent-crash replay using the original created ID, and a fence waiting for an admitted Docker Start in another process. The original scratch proof and the exact portable command both passed; their owned containers and volumes were removed. Evidence: .temp_files/r2-runtime-mutation-admission-20261026-35eb0b01/verification-state.json.

The first complete protocol diagnostic failed four Docker-boundary fixture assertions in two files while 10,656 server tests passed. The fixtures now return the production metadata shape and assert that a persisted replay performs no additional inspection or effect; fresh requests still execute exactly one removal. All 46 tests in those two files passed after the fixture updates. The corrected frozen snapshot subsequently passed its complete final gate, as recorded below.

This is a development checkpoint, not R2 acceptance. Production deploy/scale/wake producers are not yet wired to the new owner-bound client; existing unresolved-handoff holds remain required. The corrected package passed TURBO_FORCE=true pnpm release:check on Linux/Node 26.12.1: 15,278 tests in 907 files, no failures or skips, all 11 tasks uncached. Coverage floors were unchanged; server coverage was 93.26% statements, 87.97% branches, 91.44% functions and 96.39% lines, while schemas and database remained at 100%. All 27 owned source files copied from the successful stopped container matched both the frozen inputs and working tree. The retained output contains 2,559 compiled files, 180 migration files and nine manifests. A socket-free cold start of that exact compiled output as UID 999 returned HTTP 200 with status: ok, db: ok and version 0.18.4; its owned container and volume were removed. These documentation acceptance notes were added after the source gate. Full public handoff, node autoscaling/sleep, mixed-agent and remaining Swarm acceptance stay open. The protocol and client are excluded from the immutable 0.18.4 maintenance candidate at 44d1eb1f.

R2 implementation checkpoint — 2026-10-25

The development agent now exposes sealed, owner-disableable docker.metrics sampling. Each request binds one service/deployment to up to ten immutable Docker IDs and expected names. It verifies running state, management labels, restart counts and actual configured CPU/memory limits before and after bounded Docker stats. Missing, duplicate, malformed, replaced or late evidence is refused; unlimited memory remains explicitly zero rather than inheriting the host RAM denominator. The panel helper selects an explicit server and repeats ownership/cancellation/deadline checks around capability discovery and collection. Older agents are refused with a 0.19.0 update hint. This development capability is absent from the published 0.18.2 agent.

Focused acceptance: 277 covering server tests in nine files, all 591 schema tests in 29 files with unchanged 100% coverage floors, and all 11 workspace typecheck tasks on Linux/Node 26.12.1. A compiled-image smoke used an isolated Docker daemon with no host socket and a same-name panel decoy. All six cases passed: actual node CPU/memory limits, foreign-deployment refusal, owner-disabled refusal, stopped-target refusal, stale-ID refusal after same-name replacement, and a fresh sample using the replacement's new ID. The two initial fixture failures (dind's private /tmp mount and test environment setup) are retained separately and do not count as product RED proofs. All owned runtime containers/volumes were removed. Evidence: .temp_files/r2-metrics-20261025-501ad82a/.

Node autoscaling and sleep remain held until these controller gates pass. The read-only metric foundation alone does not enable either feature. R3–R6 remain unchanged.

The next R2 prerequisite corrects explicit service lifecycle replica selection (e895fe04). Stop and node teardown cover recorded serving members plus configured stopped autoscale capacity; Start and Restart use the recorded serving count, including after autoscaling is removed or the desired manual count changes. Local Restart quiesces spare capacity before restarting serving members. Every count remains bounded by the ten-replica ceiling. The preserved original sources failed all 11 new API behavior cases while 19 existing controls passed. The corrected covering suite passed 207 tests in seven files. An actual API/migrated-SQLite/Docker proof on an isolated daemon stopped all four containers, started only three serving replicas, quiesced the spare on Restart, preserved the serving shape after configuration changed and removed all containers on Delete. Evidence: .temp_files/r2-lifecycle-20261025-aa55e2c7/. These checks do not accept concurrent node controllers, immutable lifecycle effects, readiness on Start or agent-side fencing. Those R2 gates remain open.

The complete server suite on the immutable aa55e2c7 metrics snapshot passed 10,231 tests in 578 files, no failures or skips, with unchanged coverage floors (93.36% statements, 88.02% branches, 91.52% functions and 96.38% lines). The earlier diagnostic used an unsuitable network-none/container-host fixture; it was stopped after observed DNS and install-mode failures, retained, and replaced with the same Linux/CI host context used for prior acceptance. The successful complete suite predates e895fe04; its result does not certify the later lifecycle changes or the entire 0.19.0 release. Their focused and actual-Docker evidence is recorded separately above. Hosted CI for the latest code remains a separate gate.

Ordered node lifecycle checkpoint — 2026-10-25

The development agent now implements sealed, owner-disableable docker.lifecycle Start/Stop by immutable container ID. A separate native SQLite journal per service commits the increasing revision and semantic intent before effects. The effect transaction serializes competing agent processes; an interrupted operation retains the accepted revision. Exact retries are idempotent, conflicting revisions and superseded work are refused, and bounded timed-out callers cannot let subsequent commands overtake an undrained child. No public lifecycle caller or node controller is enabled by this foundation alone.

The first contract proof failed 14 cases while its existing identity control passed. Windows and Linux focused checks cover 34 lifecycle cases; the Linux covering suite passed 304 tests in nine files. All 609 schema tests passed with unchanged 100% floors; workspace typecheck and lint passed. Seven actual-Docker scenarios passed on each supported runtime (Node 22.23.4, 24.22.1 and 26.12.1), including independent processes and a hard kill after Start but before its receipt. The exact generated agent mount layout initially exposed an anonymous-volume mistake: an older Start was accepted after container replacement. The journal now lives under the persistent agent working directory. The permanent location regression failed on that original source while three controls passed, then passed after correction. The final uncached TURBO_FORCE=true pnpm release:check passed 14,785 tests in 880 files, no failures or skips, all 11 tasks uncached, with unchanged coverage floors. Server coverage was 93.36% statements, 88.02% branches, 91.51% functions and 96.38% lines. This frozen snapshot predates the parallel config.ts / updateCheck.test.ts edits; those edits were preserved and are outside this checkpoint. Older partial snapshots were withdrawn after the location correction and do not count as completed gates. Evidence: .temp_files/r2-lifecycle-fence-20261025-af416f9c/.

Required next work is shared ordering for manual/deploy/scale/wake effects, complete replica preparation, public lifecycle acceptance and the remaining R2 public-surface and Swarm criteria. R3–R6 remain open.

Durable panel lifecycle dispatcher checkpoint — 2026-10-25

Migration 0101 adds durable node Start/Stop intents. Each service has at most one unresolved command. AUTOINCREMENT IDs allocate distinct fence/effect revisions, including after old history is deleted. Acceptance records the selected node, deployment, serving shape and health configuration. Stop atomically records the stopped desired state and cancels older queued/building/deploying work. Newer commands supersede previous commands; expiring leases permit recovery while stale callbacks cannot publish running or clear a new owner's lease. Changed Docker/Compose/Swarm placement is refused at acceptance and every ownership checkpoint.

The internal dispatcher persists immutable target IDs before effects. Start keeps its previous public state through three consecutive configured readiness rounds, confirms routing and then records running. Stop sends a durable fence-only receipt before discovery, confirms route removal, drains and stops immutable IDs. A missing, malformed or mismatched receipt preserves the unfinished phase. Recovery repeats readiness or routing/drain as required without allocating another command revision or adopting a same-name replacement. Older or owner-disabled agents are held without legacy fallback. Hourly housekeeping removes terminal history older than 30 days while retaining pending commands, owner/lease-bearing records and each service's latest receipt.

Focused checks cover ownership, replay, receipt mismatches, readiness failures, route-confirmation failures, changed placement, expired leases and retained history. The 0100 → 0101 upgrade preserves existing services and deployment history; tests also exercise request/pending uniqueness, foreign keys, cascading deletion and the revision high-water mark. All 610 schema tests passed at the existing 100% coverage floors. The first complete diagnostic identified the missing runtime-reader classification and retention registration; both were addressed with placement behavior checks and a real terminal-history sweep. The final frozen code checkpoint, committed as 626374a67d2ca7a1636a74ab2acf80356870a24b, passed TURBO_FORCE=true pnpm release:check: 14,839 tests in 883 files, no failures or skips, all 11 test/build tasks uncached, with unchanged coverage floors. Server coverage was 93.36% statements, 88.03% branches, 91.54% functions and 96.39% lines. This snapshot excludes parallel shared-checkout work; it certifies this code increment, not an entire 0.19.0 release. The earlier diagnostic remains a failed run.

A compiled-image fixture exercised real migrated SQLite, sealed agent HTTP, two serving Docker replicas, untouched stopped capacity and actual Traefik routing acknowledgements: three-round HTTP readiness, route removal/drain before Stop, HTTP 404 refusal, a hard process kill after Start, lost routing replies and a fresh-process retry after the real persisted backoff. All six dispatcher cases and all eight agent ordering cases passed on the final code under Node 22.23.4, 24.22.1 and 26.12.1: 42 real-Docker cases. The permanent command is pnpm smoke:node-lifecycle --image=<candidate> and the release workflow includes it. This checks the dispatcher directly, not the public Start/Stop endpoints or automatic recovery scheduling. The initial routing fixture used a Fetch Host override that Node 26 ignored; a separate header-echo control reproduced that fixture error, and the corrected fixture uses http.request. The first fixture failure is not a product RED proof. Hosted CI for 626374a6 subsequently completed successfully, including the full checks, real Docker/PostgreSQL integration, image build and edge publication. Evidence: .temp_files/r2-panel-lifecycle-20261025-bd9f1513/.

This earlier dispatcher checkpoint left public lifecycle integration open. The subsequent primary-node implementation is recorded below. Node autoscaling and sleep remain held; R2 and 0.19.0 certification remain open.

Public node lifecycle and manual replicas checkpoint — 2026-10-25

Primary-node Docker Start/Stop/Restart now acknowledge durable acceptance with HTTP 202. A boot recovery worker executes the original command through immutable identities, readiness and confirmed routing. Restart reserves separate Stop and Start revisions; delayed acknowledgements remain bound to their accepted command, and expired-lease retries cannot be overwritten by an abandoned callback. Automatic primary-node revival uses the same durable Start path. Stop records desired state even when a node cannot be reached; its pending reason remains visible, and unsupported or owner-disabled agents never receive a legacy fallback. Compose, local Docker and Swarm retain their separate paths.

The public API, OpenAPI, SDK and CLI expose command status and same-command retry. The panel shows pending readiness/routing, the held reason and permission-appropriate retry controls. Late action or retry responses cannot replace another service's state, a newer command or a terminal receipt. Routing updates carry cancellation and bounded network budgets. These development features are absent from the published 0.18.2 agent and target 0.19.0.

The first real two-replica deploy exposed that the node builder created only its primary. Manual replicas now use one immutable image, retain managed service/deployment labels and resource settings, and omit the primary's host-port bindings from siblings. Every serving replica must pass readiness. Preparation fails rather than silently reporting fewer replicas; cleanup retires acknowledged immutable IDs and retains unconfirmed creations for recovery. Candidate failure and artifact-identity mismatch cover the whole acknowledged set. Previous managed replicas are rediscovered and retired after the desired count changes. This accepts the isolated HTTP replica flow; database/network attachments and stopped autoscale capacity remain separate gates.

Agent compatibility correction 45fe4902 restores full-ID legacy references used by recovery while preserving generation fences and rename/refusal checks. Its hosted CI passed. Modern primary recovery retires by immutable ID instead of issuing an additional fenced legacy Stop. The configured retirement grace is carried through the agent child and bounded transport budget; older agents retain the previous grace-preserving path.

The frozen code snapshot based on 45fe4902, excluding unrelated shared-checkout changes, passed TURBO_FORCE=true pnpm release:check: 14,931 tests in 889 files, no failures or skips, all 11 tasks uncached. Existing coverage floors were preserved. Server coverage was 93.27% statements, 87.99% branches, 91.40% functions and 96.36% lines. Drizzle reported no migration diff. Earlier diagnostics remain failed runs: one snapshot accidentally included a parallel Doctor reader classification without that module, and CLI registration assertions needed the new command and handler wiring.

The real public API/worker fixture passed seven cases on each of Node 22.23.4, 24.22.1 and 26.12.1: 21 public lifecycle cases. Actual deploy creates both replicas; readiness holds, spare-capacity Restart, newer Stop, hard panel termination, offline/owner-disabled handling and same-command retry are exercised. A ready primary with an HTTP-503 sibling keeps the previous generation serving until the entire candidate is ready, routing is confirmed and the old immutable generation is retired. The agent fixture separately passed 39 cases across the same runtimes, including cross-process ordering, hard termination, immutable-ID compatibility and first-adoption fencing. The permanent commands are pnpm smoke:node-canary --image=<candidate> --lifecycle-only and pnpm smoke:container-lifecycle --image=<candidate>; the release workflow includes both. Chrome exercised the pending/refusal card at 1280px and 375px without overflow or console/runtime errors. These are isolated Linux amd64 and browser fixtures, not production traffic, public DNS/TLS, or publication proof. Evidence: .temp_files/r2-node-lifecycle-public-20261025-4bf7d5a5/.

All 2,251 compiled files matched the checked build and a clean runtime fixture byte for byte. The clean image cold-started as UID 999 and repeated the authenticated browser acceptance at both widths without console/runtime errors. The initial layered fixture retained 57 unreferenced older web assets; the fixture build now removes inherited output directories before copying the checked artifacts. That diagnostic is retained and is not a product RED proof or a published-image certification.

Required next acceptance remains explicit: shared ordering for fan-out/deploy/scale/wake effects, complete node capacity and database/network attachment preparation, node autoscaling and sleep/wake, abortable runtime patrol shutdown and held-recovery alerts, the remaining actual-task Swarm matrix, and final upgrade/mixed-agent/promotion/package/publication gates. This checkpoint does not close R2, publish 0.19.0, or accept R3–R6.

Release gate snapshot

Updated 2026-11-09. R1 and R3–R5 are published (0.18.2, 0.18.21 and 0.18.15). R6 ships unsigned installers with every release since 0.18.25. R2 is the open package, targeted at 0.19.0. The rows below are an inventory of the packages, not a completion percentage.

Latest published certification: 14,634 tests in 874 files, all 11 tasks uncached with unchanged coverage floors, on c097f3a3f29feb3521b6ad22b3d09181acfcbc9f. Hosted CI passed its full matrix and all 10 real integration tests. The 2,412 compiled/runtime files matched both the local candidate and published Linux amd64 image exactly. Publication, signatures, source assets and all five npm packages were independently verified.

Incremental release plan — 2026-10-24

Source review baseline: 39830616, version 0.17.1. R1–R6 remain the agreed delivery program, split into independently accepted releases. The versions after 0.18.2 are planning targets, not publication dates or reasons to hold an already accepted release. Release 0.18.x patches may ship between feature milestones for validated fixes, compatibility and maintenance. R2 introduces new runtime features and targets 0.19.0.

R1 is published in 0.18.2 with public API/worker and browser acceptance; R7 certification is complete for this release. The real lifecycle covers healthy readiness holding, measured business-5xx rollback, full-step promotion, changed generations, unconfirmed rollback, lost source, cancellation during readiness and observation, owner-disabled refusal, hard panel restart and explicit Stop. All 11 scenarios also passed in one registered smoke on the final 0.18.0 candidate: 10,750 monitored requests, 360 deliberately injected HTTP 503 responses and no unexpected errors. The 0.18.2 artifact completed the publication gates. Real candidate browser checks at 1280px/375px saved settings, observed pending promotion and aborted it successfully with no console/runtime errors. Five rendered layout mutation controls reproduced clipping only when the width fix was removed. A responsive-layout mutation reproduced clipped availability before the fix and verified its visible right edge after restoration. Earlier R1 compatibility runs on Node 22.23.4 and 24.22.1 each passed 102 focused tests. Full certification for 0.18.2 is recorded above; those focused runs remain historical evidence. Evidence: .temp_files/r1-release-20261024-ad6bffb9/.

The acceptance criteria below apply to each package's own release. The initial effort estimates describe package work, not time remaining until 0.18.2.

The former 27–46 developer-day estimate covered the entire program with one final certification pass. It is not a 0.18.2 estimate. R1 and R7 are complete for published 0.18.2. R2 is the active feature package for 0.19.0. Re-estimate its remaining gates as their real-system results arrive; do not convert development-day estimates into an autonomous-session duration or promise a publication date.

Execution rules:

1. R1 and the 0.18.2 R7 certification/publication are complete. Deliver R2–R6 in order, with fresh R7 certification and publication after each accepted increment.
2. Keep each release's scope fixed. Unfinished later packages do not block the current release. Reproduced defects that invalidate current-release acceptance remain blockers; validated maintenance fixes may ship as interim patches.
3. Finish each coherent increment with focused regression/behavior checks and a commit. Close a package only with its API/UI, applicable SDK/CLI, documentation, migration and real-system evidence.
4. Run the full uncached release matrix at each release's code freeze. Any subsequent code change invalidates the relevant frozen certification and requires a fresh final matrix. Retain all existing coverage floors.
5. Report the current release, completed gate IDs, remaining current-release gates, commit IDs and actual blockers. Show the later backlog separately. Distinguish focused checks, full local certification, hosted checks and published/live proof; do not substitute test counts for delivery progress.

The active work is R2 for 0.19.0: node replica capacity, fenced scaling and node sleep/wake (the Swarm support matrix was accepted 2026-11-06). R2 must pass its real-system and public-surface criteria before a fresh R7 code freeze. R3–R6 remain subsequent releases.

Program backlog and accepted foundations

The package descriptions below span multiple releases. Their release allocation is defined above; pending future features did not block published 0.18.2. All dated acceptance entries that follow are historical evidence, not current scope decisions.

1. Observability — published in 0.18.2. Configurable resource-history retention with bounded chart queries and rollups; authenticated Prometheus exposition and OTLP/HTTP JSON metrics; encrypted per-destination log queues, persisted cursors, bounded backpressure and operator queue visibility. Verify restart/retry, destination changes, tenant boundaries and real collector delivery.
2. Runtime parity — in progress. Existing autoscaler safety, truthful telemetry/error states, panel-host and routed primary-node HTTP/TLS routing-load acknowledgement, a nonterminal deployment phase until routing confirmation, per-target fan-out readiness/confirmed routing transitions and durable routed fan-out transition recovery are implemented, validated and published in 0.18.2. Recorded Docker artifact promotion and rollback to fan-out targets are now locally validated. Durable HTTP/TLS primary Docker recovery is now locally validated, including stopped autoscale capacity and authoritative Stop/cancellation. Swarm preload preparation now preserves current and rollback placement and is locally validated. Recorded source artifacts, immutable promotion/rollback on Swarm and durable native Swarm update recovery are now locally validated. Explicit-host node container inspection, Compose manifests and file operations are now locally validated. Docker ↔ Swarm migration, published-0.17.1 runtime compatibility and crash recovery are now locally validated. Explicit-host node-volume file tools are now locally validated. Node database client terminals, seven-engine private Node Studio and database-scoped node telemetry are locally validated. Agent backpressure, readiness deadlines, canary proxy health propagation and independent measured local-canary file windows are locally validated too. Sealed bounded node traffic reservations with request-sequence coverage and expected HTTP/TLS generation checks are locally validated; opaque run ownership, primary-node scoped rendering and measured node-mode promotion are locally validated too; shared primary worker traffic binding, confirmed rollback retirement and batched node readiness are locally validated; public node canary implementation has R1 acceptance and is published in 0.18.2. Node autoscaling/sleep and remaining Swarm parity remain R2 work. Default node readiness remains container-state based; application HTTP readiness requires explicit configuration. Keep capability checks and application/data compatibility constraints explicit.
3. Recovery — done on main (2026-11-05). Backups of every managed engine restore into isolated real databases, where restored data is queried and the recovery duration recorded. Opt-in PostgreSQL continuous archiving/PITR has tested recovery, retention and refusal of missing or damaged WAL. Hosted acceptance is run 42029003391.
4. Remote MCP — implemented, release verification pending. Streamable HTTP, OAuth, revocable scoped grants and per-call resource authorization. Retain stdio compatibility and exercise real clients.
5. CI and runners — implemented, release verification pending. Project workflows, runner enrollment/isolation, secrets, logs, approvals, capacity and durable execution/recovery (CI_RUNNERS.md). Do not equate waiting for GitHub checks with an execution engine.
6. Desktop — implemented, verification pending. A packaged desktop application using the same API, explicit instance selection, secure credential storage, restricted navigation/IPC and platform installers (DESKTOP.md). Exercised on Windows and Debian; built and launched on macOS CI. Sign-in on a real Mac, signing and the release process are not verified yet.

Each package includes API/schema, SDK/CLI where applicable, user-facing controls, documentation and meaningful behavior tests. Validate migrations and upgrade behavior before publication. A local test, a published release and verification on an operator's production host are separate outcomes.

Local acceptance — 2026-10-22

The observability package passed pnpm release:check on Linux/Node 26.12.1: 13,318 tests across the eight packages, no failures or skips, with coverage thresholds unchanged (7 of 11 Turbo tasks cached). Windows web coverage also passed all 2,346 tests. Actual Prometheus/OTLP receiver validation, Docker log delivery after source removal and a process restart, migration 0095 → 0096 preservation, and real-API browser flows at 1280px/375px passed. The Linux test container models the bare-metal CI host; container-specific tests retain their explicit markers.

Release promotion/upgrade smokes, publication and verification on an operator production host remain separate gates. Runtime parity is in progress; its remaining capabilities and packages 3–6 are pending. See Observability for configuration and capture/retention limits.

The subsequent autoscaler/telemetry hardening round passed pnpm release:check on Linux/Node 26.12.1: 13,338 tests in 797 files, no failures or skips (8 of 11 Turbo tasks cached; server and web tests freshly executed). Coverage thresholds stayed unchanged. Actual Docker/Traefik testing kept an HTTP-503 candidate unrouted, admitted it after readiness, preserved capacity after a failed route write and drained scale-down: 429 requests, all HTTP 200. Docker Desktop watcher events were exercised in an owned Linux volume using the exact generated routing output. Real-API browser checks at 1280px/375px covered unavailable status, retry, holding reasons and missing telemetry without fabricated zero usage. This is local acceptance, not publication or production-host verification. At that checkpoint, route writes used the documented two-second watcher interval, not a reload acknowledgement.

The following routing-acknowledgement round passed the exact pnpm release:check on Linux/Node 26.12.1: 13,352 tests in 798 files, no failures or skips (3 of 11 Turbo tasks cached; server and web tests freshly executed). Typecheck, lint, build and existing coverage thresholds passed. All 1,917 frozen source files matched the tested snapshot; 14 changed files were copied back from the test container and hash-checked against the workspace.

An actual Traefik six-second reload throttle reproduced the retirement race with the prior write-and-wait behavior: 77 requests included one HTTP 502 and one timeout. With HTTP/TLS handler-generation acknowledgement, all 194 requests returned HTTP 200; the route write waited 5.6 seconds before the request drain and retirement. An isolated real API/worker/Docker/Traefik deployment exercise then covered delayed HTTP readiness, permanent HTTP 503 rejection and deliberately blocked routing acknowledgement: 1,100 requests, zero errors. When both the transition and rollback could not be confirmed, the deployment failed and both healthy runtime generations remained running. Real Traefik checks also covered weighted/sticky canary definitions, external and spoofed-forwarded-header denial of the internal probe, and its exact API path restriction. Canary measurement starts after the loaded generation is confirmed.

Evidence and cleanup receipts are under .temp_files/proxy-ack-20261022-3c9deecf/verification-state.json. These are owned local fixtures, not an operator production host or public DNS/TLS validation. Retained candidate cleanup remains an operator task identified in the deployment log; durable reconciliation is still pending. At that checkpoint, node writes retained their transport acknowledgement/grace without confirming the loaded HTTP/TLS generation. Node canary and node/Swarm autoscaling remain unsupported. The acceptance text here was added after source validation; no tested implementation changed. Publication and upgrade validation remain separate gates.

Primary-node routing and deployment state acceptance — 2026-10-23

The exact pnpm release:check passed on Linux/Node 26.12.1: 13,382 tests in 800 files, no failures or skips, with existing coverage thresholds unchanged. Nine of eleven Turbo test tasks were cached; server and MCP tests executed freshly. Typecheck, lint and build passed. All 1,920 frozen source files matched the workspace; the 26 files changed in this round were copied from the successful test container and matched both the frozen snapshot and workspace. Existing unrelated dirty work was preserved.

Routed primary-node deployments now require a sealed proxy.routingAck capability and the exact loaded HTTP/TLS generation receipt before final success and retirement. An old agent is refused with an update hint. A real API/worker/sealed-agent exercise used separate panel and node Docker daemons and actual Traefik v3.1: 1,559 routed requests, all HTTP 200. It covered delayed HTTP readiness, permanent HTTP 503 rejection, an unsupported agent, blocked routing acknowledgement and rollback, cancellation during transition, and an explicit Stop command that remained stopped. Traffic measurement ended before the intentional Stop. Unconfirmed rollback retained both healthy runtime generations; manual cleanup remains required.

The same exercise proved that pending routing stays deploying with no finishedAt, preserves the previous deployment history until confirmation, remains visible in the queue and deduplicates another trigger. Real login and panel queue checks at 1280px and 375px showed the pending status/spinner with no browser exceptions, console errors or page overflow. Both screenshots were visually inspected. Regression proofs also cover the actual worker selection and atomic claim guards for a transitioning service. Separate actual-Traefik checks passed HTTP/TLS private-probe confirmation, weighted/sticky route matching, external/spoofed-forwarded-header denial and API-path restriction.

Evidence, RED/GREEN proofs, source hashes and cleanup receipts are under .temp_files/node-routing-ack-20261022-a4d14344/verification-state.json. All owned test containers, volumes and networks were removed after evidence capture. This acceptance update is documentation added after source validation. Publication, upgrade smokes, public DNS/TLS and verification on the operator production host remain separate gates. This round does not add node canary, node/Swarm autoscaling, fan-out promotion parity or automatic retained-candidate reconciliation. Default node readiness remains container-state based; this runtime proof explicitly enabled HTTP readiness.

Fan-out readiness and routing acceptance — 2026-10-23

The exact pnpm release:check passed on Linux/Node 26.12.1: 13,399 tests in 802 files, no failures or skips, with unchanged coverage thresholds. All eight packages executed their test tasks freshly; the three cached Turbo tasks were dependency builds. Typecheck, lint and build passed. All 1,922 frozen source files matched the workspace; the ten files changed in this round were copied from the successful test container and hash-checked against both the snapshot and workspace. The acceptance update here is the only documentation change after validation.

Fan-out now carries explicit HTTP/TCP readiness, the resolved source-build port and the configured deadline to each target. Routed target replacement requires sealed proxy.routingAck, a guarded target-row transition and loaded-generation confirmation before the two-second drain and old-runtime retirement. Failed preparation cannot overwrite a replaced target; pending transitions cannot publish over a deleted/replaced target, a stopped service or a newer primary runtime. Confirmed rollback permits candidate cleanup; unconfirmed rollback retains both possible serving generations. Routed fan-out with published host ports is refused before replacing the target. Unrouted workers preserve their existing compatibility behavior.

The permanent readiness/routing cases failed with 12 assertion failures and three passing controls on the preserved original fan-out source, then all 15 passed on the patch. Both verifiers passed three consecutive runs. The covering suite passed 320 tests in eight files. An actual public-API/worker/sealed-agent exercise used independent panel and target Docker daemons and real Traefik: 894 routed requests, all HTTP 200. An old target agent was refused before candidate start while the primary succeeded; an open TCP listener returning HTTP 503 stayed unrouted until ready; permanent HTTP 503 removed only the failed candidate; blocked transition and rollback acknowledgements kept both healthy generations. The pending target record was observed as idle, with no false running status.

Evidence and cleanup receipts are under .temp_files/fanout-transition-20261023-359ab181/verification-state.json; the audit history is .temp_files/ledger_apps-server-src-engine-fanout.md. All owned runtime and matrix containers, volumes and networks were removed. The local proof image and source/proof records remain for review. Dependency junctions in the preserved RED snapshot must not be recursively cleaned. This is local acceptance, not publication, public DNS/TLS validation or production-host verification. Crash/restart reconciliation, automatic retained-generation cleanup, node canary/autoscaling and recorded-artifact fan-out promotion remain pending. No schema migration or default-readiness change was introduced.

Durable fan-out recovery acceptance — 2026-10-23

The exact pnpm release:check passed on Linux/Node 26.12.1: 13,438 tests in 807 files, no failures or skips, with unchanged coverage thresholds. All eight packages executed their test tasks freshly; three cached Turbo tasks were dependency builds. Typecheck, lint and build passed. Server coverage was 93.63% statements, 88.12% branches, 91.66% functions and 96.23% lines. All 1,933 frozen files matched the workspace; the 40 files changed in this round were copied from the successful test container and hash-checked after completion. Existing unrelated dirty work and staging were preserved.

Routed fan-out now journals preparation before creating a candidate, records immutable Docker identities and recovers interrupted transitions after boot. A renewable lease, SQL ownership fences and a persisted retirement commitment prevent stale workers from publishing or restoring an obsolete generation. Recovery repeats configured readiness and loaded-generation confirmation, drains requests and retires only a known obsolete container ID. Unknown ownership, stopped services, unavailable nodes and failed routing keep the transition pending. Operators can see its reason and next retry in Settings, or retry through the API/SDK without stealing a live deployment lease. Target deletion preserves the journal until routing removal and known-generation cleanup are confirmed. Migration 0097 enforces one unresolved record per service/node; service/node deletion cascades the journal.

The covering server suite passed 160 tests in eight files, the authorization matrix passed 801 tests, OpenAPI passed 21 tests, and the migration test exercised 0096 → 0097 preservation, uniqueness, target-removal retention and parent-deletion cleanup. The first complete run exposed missing retention/runtime-reader classifications; both project guards now pass without weakening their rules. The monotonic-retirement regression has preserved RED/GREEN evidence.

An actual public-API/worker/sealed-agent exercise used independent panel and target Docker daemons and real Traefik. The panel was hard-killed while candidate routing was awaiting acknowledgement, then restarted on the same database and keys. The journal and both runtime generations survived; an operator retry could not steal the active lease. A failed acknowledgement retained both containers and a visible reason. Once acknowledgement was restored, automatic recovery finished without another deploy, kept the candidate running and removed the old immutable container ID. 2,579 routed requests all returned HTTP 200. Real login and panel checks at 1280px/375px covered persisted pending state, disabled retry during a live lease and the actual retry request with an honest holding reason; no browser exceptions, console errors or page overflow occurred. All three screenshots were visually inspected.

Evidence, failure/correction logs, source hashes and cleanup receipts are under .temp_files/fanout-recovery-20261023-16798027/verification-state.json. All owned runtime containers, volumes, networks and browser profiles were removed. The final release-check container was removed after evidence capture. Only test policy/coverage assertions changed after the real runtime proof; no product source changed. This acceptance text and the explicit pre-journal-container limitation in Node operations are documentation added after validation. Publication, release promotion/upgrade smokes, public DNS/TLS and an operator production host remain separate gates. Primary-runtime retained-container cleanup, older unjournalled containers, recorded-artifact fan-out promotion, Swarm recovery, node canary and node/Swarm autoscaling remain separate work. Default node readiness is still container-state based unless HTTP/TCP readiness is explicitly selected.

Recorded artifact fan-out acceptance — 2026-10-23

Release target remains 0.18.0, after the requested remaining work and final publication gates; no intermediate release was published. The checkout version is still 0.17.1. Unreleased routing/recovery capability update hints now name 0.18.0; live capability negotiation remains authoritative.

The exact pnpm release:check passed on Linux/Node 26.12.1: 13,446 tests in 807 files, no failures or skips. All eight packages executed their test tasks freshly; three cached Turbo tasks were dependency builds. Typecheck, lint and build passed with coverage thresholds unchanged. Server coverage was 93.63% statements, 88.13% branches, 91.66% functions and 96.24% lines. All 1,933 frozen files matched the workspace, and the twelve changed files were copied from the successful test container and hash-checked after completion. Existing unrelated dirty work and staging were preserved. This acceptance update is the only documentation change after validation.

Artifact promotion and artifact rollback now admit configured fan-out targets. The pipeline prepares the recorded immutable image for each target from its retained panel copy or bound registry digest, checks identity, and enters the existing readiness/routing/recovery transition without source checkout, buildpack discovery or rebuild. The target service's current runtime environment and resolved port apply; a Nixpacks PORT default is added only when not explicitly set. Unsupported agents, transfer failures and image-identity mismatches preserve the previous target and leave the primary live. Swarm and source-dependent host hooks remain refused; protected-source authority, current-target permissions, required CI and approvals remain in force. No API/SDK call shape changed.

The preserved original sources failed all eight new scenarios while two unaffected controls passed; an initial isolated-copy dependency-resolution error ran no tests and was not counted as RED. The normal suite now covers these cases, including a node-hosted primary, environment/PORT handling and preparation failures. Covering artifact/runtime suites passed 138 tests in six files; UI promotion tests passed 17. The first complete run exposed one test still expecting the superseded 0.17.2 update hint; the exact 0.18.0 expectation now passes without weakening the refusal or no-side-effect assertions.

An actual public-API/worker/sealed-agent exercise used independent panel and target Docker daemons, real Traefik and an owned Git HTTP server. Production first built an old image; staging then built a different image. Git was stopped before promotion. The real panel queued artifact promotion, the additional node received the exact staged image, and a TCP-open HTTP-503 candidate remained unrouted until ready. Artifact rollback then restored the original image on both primary and additional node while Git stayed unavailable. The target kept production environment values rather than staging values; obsolete containers were removed after confirmation and the transition journal ended empty. 1,208 routed requests all returned HTTP 200. This live proof used stream transfer; existing registry-digest preparation was covered by the artifact registry suite.

Real login and artifact confirmation passed at 1280px/375px, pinned the displayed image and source deployment, explained additional-node behavior and sent the actual promotion request. No browser exceptions, console errors or horizontal page overflow occurred. Both captures were inspected; the desktop frame contains the dialog's enter transition and the mobile capture shows its steady state.

Evidence, original-source RED/control results, source hashes, failure/correction logs and cleanup receipts are under .temp_files/artifact-fanout-20261023-fa802bcd/verification-state.json. All owned runtime/test containers, data/agent volumes, networks and the browser profile were removed; the local proof image and evidence snapshots are retained. Only the update-hint test expectation changed after the real runtime proof; product sources stayed identical. Publication, real release upgrade/promotion smokes, public DNS/TLS and an operator production host remain separate gates. Primary/Swarm transition recovery, Swarm artifact promotion and the other pending delivery packages are still open.

Durable primary Docker recovery acceptance — 2026-10-23

The exact pnpm release:check passed on Linux/Node 26.12.1: 13,497 tests in 813 files, no failures or skips, with all eight package test tasks executed freshly and only three dependency builds cached. Typecheck, lint, build and existing coverage thresholds passed. Server coverage was 93.53% statements, 87.96% branches, 91.55% functions and 96.23% lines. All 1,944 frozen source files matched the tested filesystem; 42 changed files were copied from the successful test container and hash-checked against the frozen snapshot and workspace. HEAD, staging and unrelated dirty work were preserved. The acceptance updates here and in Roadmap are documentation added after validation.

Routed plain Docker primary replacements now journal immutable serving-replica and stopped-capacity identities after preparation and before canary/full traffic publication. A renewable lease, SQL ownership fences and a persisted retirement commitment protect both generations across panel restart. The worker hands interrupted jobs to recovery instead of deleting possibly routed candidates or rebuilding them as stale work. Pending recovery blocks only its own service queue; other services keep their deployment capacity. Autoscaling and sleep cannot change a protected runtime. Recovery repeats serving readiness, confirms loaded HTTP/TLS routing, drains requests and retires only obsolete known IDs. Stopped autoscale capacity is never a readiness backend: selected capacity is kept, obsolete capacity is removed, and unrelated containers at spare names are refused.

Migration 0098 preserves existing service state on 0097 → 0098 upgrades, enforces one transition per service and cascades with service/node deletion. Settings, viewer GET/operator POST /v1/services/:id/runtime-recovery and SDK methods expose pending reasons and lease-safe retries without deletion IDs or owner tokens. Stop and cancellation remain authoritative; canary replicas are checked throughout observation and again before full cutover. Interrupted canaries restore the previous generation and preserve its deployment history. Unknown identities, changed owners/placement/replica shape and unconfirmed routing retain the generations for review. Restricted TCP sidecars still need separate routing confirmation; direct published host ports have no overlap guarantee.

Permanent RED/GREEN proofs reproduced unsafe boot deletion, Stop during readiness and immediately after claim, stale canary readiness, missing local ownership labels, premature history demotion, automatic scale/sleep interference and leaked obsolete stopped spares. The Stop claim proof uses a real SQLite AFTER UPDATE trigger, not a mocked ownership query. The final suite includes 26 direct recovery cases, seven SQLite canary integration cases, four real worker cases, the upgrade test, access control and UI contracts. Earlier source snapshots were interrupted after repairs. One complete diagnostic run failed only an old exact docker.runEnv fixture missing new service/deployment operands; its strict expectation was updated, all 45 targeted cases passed, and the final complete matrix passed. No guard or coverage threshold was weakened.

Actual public-API/worker/Docker/Traefik scenarios used independent panel/node daemons and a sealed agent. Four hard panel crashes covered local cutover, node cutover, active canary routing and autoscale capacity. HTTP-503 candidates remained unrouted before readiness; journals and both generations survived restart, live leases were not stolen, and automatic recovery settled the original jobs without redeploying. The canary returned to the previous live deployment; its traffic trace contains requests to both generations before recovery. The autoscale case removed two obsolete stopped spares and retained the selected stopped spares. 5,697 routed requests all returned HTTP 200. The last product change only fenced the initial deploying-status write against Stop; these runtime scenarios do not issue Stop at that statement, and the affected race was separately proved with real SQLite. Source compatibility is recorded explicitly.

Real login and Settings recovery flows passed at 1280px/375px: persisted pending state, disabled retry during a live lease, an actual retry and the honest unconfirmed-routing reason. No browser exception, console error or horizontal overflow occurred. Three screenshots were visually inspected, and all 69 files of the browser bundle matched the current build. All owned runtime containers, volumes, networks and browser profiles were removed.

Evidence, failed/corrected fixture runs, source hashes and cleanup receipts are under .temp_files/primary-recovery-20261023-0bb5d890/verification-state.json. The current checkout remains 0.17.1; 0.18.0 is the unpublished target after the remaining work and final release gates. Recovery does not replay builds, lifecycle hooks, canary observation or additional-node fan-out; it does not add Compose/Swarm recovery, adoption of historical unjournalled containers or database/volume restore. Default node readiness remains container-state based unless HTTP/TCP is explicitly selected. Swarm artifact/recovery work and the other delivery packages remain open. Release upgrade/promotion smokes, publication, public DNS/TLS and an operator production host remain separate gates.

Swarm preload placement safety acceptance — 2026-10-23

Changing the old per-service node label while preparing a different image evicted the serving task before stack submission. A real Docker/Swarm before-proof went from one running replica to 0/1, with the old task rejected because its node no longer met the constraint. Preloading now selects an unreferenced one of three fixed slots, keeps legacy labels unchanged and uses the full image content address. Current and previous placement references across every service are protected, including mixed-case constraint prefixes while label names remain case sensitive. Unreadable/incomplete inventory, ambiguous service IDs, more than 4,096 services or exhausted slots fail preparation before preload label changes.

The permanent builder regression failed on the original behavior; a second RED proof caught the mixed-case boundary before acceptance. All 141 covering tests in eight files passed on the final source. The exact Linux/Node 26.12.1 pnpm release:check then passed 13,518 tests in 814 files, no failures or skips. All eight package test tasks executed freshly; three dependency builds were cached. Typecheck, lint, build and existing coverage thresholds passed. Server coverage was 93.53% statements, 87.99% branches, 91.55% functions and 96.22% lines. All 1,946 frozen files matched the successful tested filesystem; the nine files changed before acceptance were copied from that container and matched the frozen snapshot and workspace. These acceptance edits and the Roadmap update are documentation added after validation.

A real single-manager Swarm proof invoked the compiled source selector and exercised four rolling image updates, native rollback, mixed-case current/previous constraints and an additional service holding the final free slot. Every preparation preserved the exact serving task IDs and the legacy label. Exhausted slots were refused without changing node labels or serving tasks. Repeated updates used only three slot keys plus the retained legacy key. 2,127 HTTP requests all returned 200. This proof covers native manager placement and rolling/rollback behavior through fixture ingress. Full public-API/two-node deployment, additional application-readiness modes, public DNS/TLS and an operator production host were outside this proof. No UI, API, SDK or database contract changed.

Evidence, RED/GREEN results, discarded source snapshots, failed/corrected fixtures, source identities and cleanup receipts are under .temp_files/swarm-artifact-20261023-4a890b2f/verification-state.json; the folder name retains the initial artifact-work intent, while this accepted scope is preload safety only. Runtime containers were removed; proof logs, source snapshots and the local runtime image are retained. The checkout remains 0.17.1 and 0.18.0 remains unpublished. Swarm artifact promotion/rollback, durable interrupted Swarm update recovery and the other pending delivery packages remain open. This fix protects the preparation window; it does not add HTTP/TCP readiness or a general Swarm availability guarantee. Real release upgrade/promotion smokes and final publication gates remain required.

Swarm artifact and native update recovery acceptance — 2026-10-23

At the Swarm checkpoint, the exact Linux/Node 26.12.1 pnpm release:check passed 13,644 tests in 824 files, no failures or skips. All eight packages executed their test tasks freshly; three dependency builds were cached. Typecheck, lint, build and the existing coverage thresholds passed. Server coverage was 93.30% statements, 87.74% branches, 91.37% functions and 96.18% lines. All 1,965 frozen files matched the successful tested filesystem, and the 53 files owned by this package were copied from that image and hash-checked against the frozen manifest. Subsequent node-container-tools work advances the checkout separately; this checkpoint does not certify that evolving package or the final release.

Swarm source builds record their immutable artifact before native submission. Promotion and artifact rollback preload and verify the recorded image on every required managed member, retain target runtime settings and do not fetch Git or build source. Actual immutable task/container identities are checked on the panel and sealed worker agents. Unknown identity retains the native service for review. Native image releases pin a manifest digest before submission; real Docker pinning and repository-matching/no-mutable-fallback cases passed.

Migration 0099 journals the exact candidate and previous native specs encrypted at rest, with a renewable lease, manager identity, full service ID and version-fenced Engine updates. A first service starts as a zero-replica reservation. Recovery changes a service-level owner epoch without replacing task templates, waits for the old lease, repeats task image/health checks and loaded routing confirmation, and settles the original deployment. Stop/cancellation remain authoritative. Failed/ambiguous worker health output is refused; pending recovery keeps history/logs and does not emit a completion hook. Unknown spec, manager, identity or routing stays pending. Settings, viewer GET/operator POST /v1/services/:id/swarm/recovery and SDK methods expose reasons and lease-safe retries without spec/env values or owner tokens.

An actual public-API/worker exercise used two independent Docker daemons, a sealed agent, real Traefik and an encrypted overlay. Source deployment recorded its image; Git was stopped before real-panel artifact promotion and rollback. Four candidate containers returned HTTP 503 before their command readiness gates opened. A hard panel kill during promotion preserved the journal and live lease; boot retained the original job, and automatic recovery completed it without another checkout/build. Rollback restored the exact original image and production environment. 3,424 routed requests all returned HTTP 200. A real browser verified promotion and persisted recovery at 1280px/375px with no exception, console error or horizontal overflow. A separate real API/UI fixture checked the final placement copy, disabled retry during a live lease and an actual operator retry that kept an unreadable seeded journal pending. Its seeded state is separate from the real native crash proof. Screenshots were inspected.

The first diagnostic matrix exposed a fake DB query chain and a success-hook fixture still returning building; the service-unique pending query now has the same bounded shape as primary recovery, and the hook fixture returns the persisted success state while asserting success=true. The final complete matrix passed. The discarded runtime attempts preserve the agent-health protocol mismatch and the fixture's stale ephemeral API port after Docker restart. Runtime containers, volumes, networks, prototype containers and matrix resources have cleanup receipts. Browser profile deletion was blocked by automatic approval review (blocked by policy), so those ignored proof profiles remain.

Evidence is under .temp_files/swarm-promotion-20261023-22b66f5f/verification-state.json. This accepts native Swarm artifact/update recovery only: it does not accept live cross-orchestrator migration, historical unjournalled-runtime adoption, HTTP/TCP Swarm readiness, public DNS/TLS, a production host or final release upgrade/publication. Cross-orchestrator migration compatibility remains required before 0.18.0 readiness. The checkout remains 0.17.1 and the release target remains unpublished.

Explicit-host node container tools acceptance — 2026-10-23

The frozen node-tools checkpoint passed the exact pnpm release:check on Linux/Node 26.12.1: 13,716 tests in 826 files, no failures or skips. All eight package test tasks executed freshly; the three cached Turbo tasks were dependency builds. Typecheck, lint, build and existing coverage thresholds passed. All 1,969 frozen source files matched the successful test filesystem; the 39 files differing from the previous frozen checkpoint were copied into the evidence directory and hash-checked. Three of those files also carry the preceding Swarm acceptance documentation. HEAD and the staging area stayed unchanged.

Actual public-API/sealed-agent tests used two independent Docker daemons with identically named containers carrying different identities and content. Inspection, generated Compose, directory operations, binary reads/writes, exact and whitespace-wrapped 1 MiB content, oversized refusals and the owner kill switch passed without local fallback. The real production CLI entrypoints for service inspect/compose used an isolated production dependency closure and returned the remote identity/environment. This local CLI proof image is not a publishable release artifact.

The same proof reproduced and corrected missing empty files, trimmed filename whitespace and unreadable files incorrectly returned as empty content. Permanent RED/GREEN controls cover the filesystem capture, host wiring and availability states. Real login and Manifest/Files flows passed at 1280px/375px, including an actual remote file edit, disabled-agent errors and operator retry after re-enrollment. All six screenshots were visually inspected; no browser exceptions, console errors or page overflow were observed.

Evidence is under .temp_files/node-container-tools-20261023-1ab75596/verification-state.json. Owned runtime and matrix resources were removed after validation; ignored browser-profile evidence remains. The acceptance documentation and subsequent cross-orchestrator changes were made after this frozen checkpoint and are not certified by its matrix. The cross-orchestrator runtime proof separately reproduced a refused Docker → Swarm migration and a Swarm → Docker deployment that left the old native replica running; that next package must complete its own recovery/upgrade tests and fresh release checks.

Node database Studio, node-volume browsing, node canary/autoscaling, restore/PITR, remote MCP, runner execution, desktop and final upgrade/artifact/publication gates remain open. Version remains 0.17.1; 0.18.0 is not ready or published. See node container tools for API, limits and exclusions.

Docker and Swarm migration acceptance — 2026-10-23

The final frozen checkpoint passed the exact pnpm release:check on Linux/Node 26.12.1: 13,744 tests in 827 files, no failures or skips, with unchanged coverage thresholds. All eight package tests executed freshly; the three cached Turbo tasks were dependency builds. Typecheck, lint and build passed. All 1,972 frozen files matched the successful test filesystem, and 18 changed files were copied into evidence and hash-checked. HEAD/index stayed unchanged. This checkpoint also includes the separately proved empty-volume-file fix; the next node-volume feature implementation began after its freeze.

The actual API/worker/Docker/Traefik proof reproduced a refused Docker → Swarm migration and a Swarm → Docker deployment leaving its old native replica running. Both directions now preserve readiness/routing gates and retire only known old generations. Docker captures serving/spare immutable IDs in the encrypted Swarm journal. Native previous-service provenance/fingerprints are encrypted and rotated with the instance key. Native retirement uses an observed-version update to zero replicas and confirms task drain; it does not perform an unfenced service delete. A same-service return to Swarm reuses the owned zero-replica archive.

Actual-image checks passed both directions (1,786 requests, all HTTP 200), an in-place upgrade from the published ghcr.io/ninedeploy/ninedeploy:v0.17.1 with active runtimes/history/keys preserved (1,964 requests, all HTTP 200), and that upgrade plus Swarm → Docker → Swarm on the same service (5,168 requests, all HTTP 200). The old 0.17 Docker service/deployment labels are accepted without requiring the new managed label. Separate hard-kill checks recovered both durably selected transitions on their original deployment IDs without stealing a live lease: 16,604 requests, all HTTP 200. A deterministic pre-selection crash kept the previous native service running, failed the interrupted job and removed its unselected candidate: 9,326 requests, all HTTP 200. Recovery journals settled empty.

Real login and the pending-recovery views passed at 1280px/375px, with protected retry while the lease was live. The four final screenshots were visually inspected; no browser exceptions, console errors or page overflow were observed. Permanent controls cover native revision conflicts, changed identity/manager/spec, failed drain, key rotation, previous Docker ownership/spares, readiness failure, cancellation, Stop and unconfirmed routing. The first full diagnostic failed because two new runtime readers were missing from the architectural inventory; their classifications were corrected without weakening the guard, and the full matrix was rerun. Failed/calibrated fixture attempts remain recorded separately from product evidence.

Evidence and cleanup receipts: .temp_files/cross-runtime-20261023-2339ddbc/verification-state.json. All owned runtime and matrix containers, nested data, named volumes/networks and the temporary source-verifier image were removed. Ignored browser-profile evidence remains. This accepts panel-host Docker/Swarm migration only; app/data compatibility, external storage drivers, explicit HTTP/TCP Swarm readiness, remote runtime parity and the remaining delivery packages retain their documented boundaries. Later node-volume source changes are not certified by this older checkpoint. Final release-wide upgrade/promotion smokes, publishable artifacts, publication and an operator production host remain separate gates. Version is still 0.17.1; 0.18.0 remains unpublished and unready.

Local acceptance — node volume files (2026-10-23)

The node-volume package passed the exact Linux/Node 26.12.1 pnpm release:check: 13,857 tests in 832 files, zero failures or skips. All eight package test tasks ran freshly; three dependency builds were cached. Existing coverage thresholds passed unchanged; server coverage was 93.27% statements, 87.76% branches, 91.32% functions and 96.20% lines. All 1,983 frozen source files matched the tested filesystem; 40 changed files were copied from the successful container and hash-checked. HEAD and staging were preserved. This certifies the frozen volume checkpoint; later node database-client changes in the shared checkout are explicitly outside it.

Operators can now list, read, write, create directories and delete paths on an explicit node through sealed volume.files / volume.fileTool, with an owner kill switch and the /v1/servers/:serverId/volumes/:name/files namespace. API, SDK, production CLI, the Volumes page and service volume browser retain the recorded host and never fall back locally. Browsing requires an existing managed Linux local-driver volume without external driver options; missing volumes are refused without auto-creation. Helpers pin a live bind and attest the current device/inode through isolated read-only probes before/after file operations, in addition to daemon/volume metadata. Commands, decoded file size and helper lifetime are bounded; cleanup requires nonce/immutable-ID ownership confirmation.

Actual two-daemon API/SDK/CLI checks preserved same-named panel/node data, exact binary bytes, wrapped 1 MiB content, empty files and whitespace in names. Oversized reads/writes, escaping paths, unconfirmed storage and the published 0.17.1 agent were refused. Real audit rows recorded actor, node and path without file content. A node owner switch removed support and prevented operations; re-enabling it restored the operator retry. Three physical Docker delete/recreate controls deliberately held inspect metadata identical through a shim: statuses were 409/502/409, the replacement data stayed unchanged and no stale read was returned. A separate RED/GREEN proof fixed deleting the last symlink through an internal directory alias.

Real login, node GET/PUT, refusal and retry passed at 1280px/375px. The four final screenshots were visually inspected, with no browser exceptions, console errors, horizontal overflow or menu occlusion. The mobile check found the file dialog trapped beneath the navigation by the page animation stack; a body portal and permanent regression now protect it. Host changes remount the editor and isolate caches. Service volume fallback now uses the builder's actual nd-svc-<slug>-data name. Permanent pnpm smoke:volume-files --image=<candidate> exercises helper CRUD, binary/size boundaries, symlink behavior, missing-volume refusal and cleanup on an owned Docker daemon.

Two failed diagnostic full runs remain recorded separately: the audit inventory did not recognize a generic helper, then two UI tests still expected the former unsupported node browser and incorrect fallback name. The routes now call audit directly; no exemption was added. UI assertions now exercise actual host/file-browser behavior, and the complete matrix was rerun successfully.

Evidence: .temp_files/node-volume-tools-20261023-a7c7549a/verification-state.json. Owned runtime/matrix containers, nested data, named volumes/networks and the temporary source-verifier image were removed; ignored browser profiles remain. The accepted local runtime image is ninedeploy-local:node-volume-files-accepted-20261023; it is not a published release artifact. This package adds no database Studio/client acceptance, node canary/autoscaling, Swarm task selection or external storage-driver support. Node database clients are now in progress with their own pending validation; Studio, remaining runtime parity, restore/PITR, remote MCP, runner, desktop and final release gates remain open. The checkout remains 0.17.1, with 0.18.0 unpublished and unready.

Node Studio groundwork — 2026-10-24 (in progress)

At that initial checkpoint, the actual API still refused node Studio with 422 remote_database; an owned two-daemon PostgreSQL fixture confirmed this while its authenticated node client queries, SDK, ticket and immutable-container controls passed. Studio is not delivered yet.

The initial encrypted duplex HTTP foundation has a separate key domain, strict bounded metadata, exact request/response byte counts and hashes, consumption-based flow control, cancellation and time limits. Real HTTP/WebSocket tests cover multipart requests, multi-window binary responses, duplicate cookies, credential-header filtering, redirects that are not followed, truncated responses and upstream socket closure on cancellation. That initial checkpoint passed 79 tests in four files on Node 22.23.4, 24.22.1 and 26.12.1. The extra Windows runtimes matched official Node archive checksums. A manual-read credit stall and a separate flowing-pipe stall were reproduced and fixed; regression assertions retain exact byte equality without expensive object-by-object Buffer comparison.

Evidence: .temp_files/node-studio-20261023-3fbebb9e/verification-state.json. At that initial checkpoint, helper lifecycle and agent registration still needed implementation. Panel API/SDK/UI integration, database mutation cleanup, authenticated SQL/KV Studio workflows and a fresh full matrix remain required. These later Studio files are excluded from the separately frozen 1,994-file database-client matrix. The later agent lifecycle/channel work below now advertises sealed db.studio and registers the node channel; panel-facing Studio integration remains pending. No version bump or publication occurred.

Node database client acceptance — 2026-10-24

The frozen client package passed the exact Linux/Node 26.12.1 pnpm release:check with TURBO_FORCE=true: 13,943 tests in 837 files, zero failures or skips; all 11 tasks fresh, none cached. The tested filesystem matched all 1,994 frozen hashes; 35 changed files were copied from it and hash-checked again. Server coverage was 93.26% statements, 87.78% branches, 91.33% functions and 96.20% lines; thresholds were unchanged. The first full attempt stopped only at a whitespace-sensitive legacy wiring assertion; the retained shell call was reformatted without weakening any test, then the entire uncached matrix passed.

Actual Docker queries passed for PostgreSQL, MySQL, MariaDB, Redis, Valkey, KeyDB and Dragonfly default images. Controls include password refusal, independent same-named local/node databases, immutable-ID replacement refusal, single-use tickets, owner disable/reenable, stored-credential handling and the production SDK. Real panel login, SQL queries, owner refusal and reconnection passed at 1280px and 375px; all four screenshots were inspected, with no JS exceptions, console errors, overflow or occluded controls. Mobile header/toolbar layout and terminal resizing defects have permanent regressions. pnpm smoke:database-clients --image=<candidate> retains the seven-engine runtime smoke.

Evidence: .temp_files/node-db-client-20261023-22c3e02c/verification-state.json. Owned runtime/matrix resources and the temporary source-verifier image were removed; ignored browser profiles remain. Retained local proof image: ninedeploy-local:node-database-clients-accepted-20261024 (not a release artifact). HEAD and staging were preserved. Later Studio files and this acceptance documentation are outside that frozen checkpoint. Node Studio, remaining runtime parity, recovery/PITR, remote MCP/OAuth, runner, desktop and final release gates remain open; version is still 0.17.1.

Node Studio agent lifecycle and channel proof — 2026-10-24 (in progress)

The agent now implements sealed db.studio.start, db.studio.stop and db.studio.open, with a single-use encrypted /agent/studio channel. Helpers use the database private bridge and resolved image ID, expose no host port and bind the recorded database/container/network identities. A fresh identity check after TCP connect precedes HTTP headers or body. Preparation, owner switches, expiry, cleanup and the connected-target gate have regression coverage; late readiness success is refused. Migration 0100 adds nullable, credential-free node Studio metadata while preserving existing local ports, placement and credentials.

Actual PostgreSQL/Adminer and Redis/Redis Commander roots passed over the registered sealed agent operations and actual agent WebSocket. Controls covered idempotent start, a running identically labelled replacement after reservation, refusal without deleting that replacement, restored access, owner disable/reenable, continued advertisement of ordinary database clients, and confirmed cleanup. Independent panel/node Docker daemons proved no local helper fallback. These checks cover Studio root pages; browser login, SQL/KV data actions and operator-session revocation through the panel proxy still need acceptance.

Current focused results: 219 server tests in eight files, 17 migration/schema-drift tests in three files and 24 schema contract tests. Earlier local Studio controls also passed. Evidence: .temp_files/node-studio-20261023-3fbebb9e/verification-state.json and runtime-agent-2/studios-result.json. All owned runtime containers, volumes and networks were removed. The first agent fixture expected a sealed exit-code refusal for a disabled operation; it was corrected to require the actual HTTP 400 bad_params response, exact disabled owner flag and absence of a helper. No product check was weakened. At that checkpoint, public panel integration and the full Studio gate were still pending; 0.18.0 remained unready.

Node Studio panel integration — 2026-10-24 (acceptance in progress)

The panel now dispatches Studio start/stop and HTTP proxy requests to the recorded node over sealed operations and a distinct encrypted WebSocket channel. PostgreSQL/MySQL/MariaDB and Redis/Valkey/KeyDB/Dragonfly are the supported image/driver set. No node host port is published. Private preparing/ready receipts survive lost replies; shared process/on-disk exclusion prevents database stop, restart, recreate or delete from racing a preparation. Unconfirmed cleanup preserves its receipt, and an explicit forced record deletion reports the possible orphan. API/SDK report a nullable port; the panel displays the recorded node and recovery controls.

Studio cookies are bound to the live authenticated API key or login session in addition to the operator, token version and instance epoch. Real auth-route proofs reproduced continued access after key deletion, scope narrowing and individual session revocation; the updated signing domain and parent-grant checks now end those grants while preserving unaffected sessions. Existing Studio tabs must be reopened after upgrade. Late target changes, cross-resource previous metadata, binary/multipart integrity and browser cancellation have runnable regression evidence.

Focused panel/auth/relay checks passed (181 tests at that checkpoint); SDK checks passed 127 tests and panel checks passed 34 tests. Server and web production builds passed, and all eight workspace lint tasks ran uncached successfully. A subsequent covering server run found one source-wiring regex that still expected the old start signature; it now requires the request cancellation signal. The full current Studio matrix and actual SQL/KV browser/data workflows remain pending. Actual two-daemon API startup has already created both private helpers; the first browser fixture stopped at a selector-escaping error and was corrected without changing product checks. Proofs are retained under .temp_files/node-studio-20261023-3fbebb9e. This is not full package acceptance or release readiness.

Actual Chrome acceptance subsequently completed PostgreSQL/Adminer login, the prefilled PostgreSQL driver and a node-only computed data query, plus 1280px/375px panel captures. The KV browser path exposed ignored REDIS_URL configuration (localhost instead of the managed database). Both node and local authenticated Redis Commander now use node-config JSON; credentials do not become helper entrypoint arguments, and old owned configuration is replaced before reuse. Fresh KV configuration/lifecycle/relay checks passed 225 tests; real KV data acceptance remains in progress. Studio HTTP requests use the sealed open operation as the authoritative per-request owner check, retaining fresh negotiation for startup and refused operations; this removes an extra ping for each asset that exhausted the agent request budget.

The first full Studio matrix stopped at an existing web test that still required the old disabled node Studio card (2,382 web tests passed, one old-contract expectation failed). That test now requires the implemented node Studio controls, recorded node and absence of a host port; unrelated PgBouncer/public-access refusals remain. The live KV run also reproduced the Studio WebSocket 60/minute cap rejecting otherwise valid sealed tickets from the 120/minute issuer. The route now matches the issuer budget while retaining single-use authentication, 16-channel capacity and all byte/lifetime bounds. A reserved transport failure now returns typed HTTP 502; the corresponding two RED proofs now pass with their covering controls (107 tests). Updated panel controls pass 39 tests. Native independent readback confirmed one actual UI-submitted KV write; complete clean KV browser SET/GET and a new full matrix are still pending.

A clean two-daemon/real-Chrome run now passes actual panel login, separate noopener Studio tabs, prefilled PostgreSQL driver, a computed node-only SQL result and Redis Commander SET/GET. It reports no JavaScript, console or unexpected HTTP failures. Explicit Studio stop and database-stop cleanup were confirmed through node inspection; all owned runtime resources were removed. The panel controls fit and remain clickable at CSS widths 1280/375. Third-party Studio pages retain wider desktop layouts in a narrow physical viewport (reported layout widths 959/981); this is not a claim of fully responsive vendor GUIs. Evidence: runtime-panel-6/runtime-result.json, browser-result.json, input-identity.json and runtime-cleanup.json under the Studio proof directory.

The same 266 Studio-core tests now pass separately on verified Node 22.23.4, 24.22.1 and 26.12.1 runtimes. The first Node 22 run reached its assertions but hung during fixture teardown after an aborted fetch; explicit closure of fixture-owned idle connections fixed it without increasing timeouts. The second full Linux matrix reached 9,791 server tests: 9,788 passed, three contract checks failed (new owner flag omitted from .env.example, migration 0100 column omitted from the manual database drift list, and an old Redis-family expectation requiring ignored REDIS_URL). The environment example and strict fixtures now match the implemented contracts; all 23 covering checks pass. Another exact uncached full matrix is required. Current live Studio data acceptance covers PostgreSQL and Redis; MySQL/MariaDB and the other three KV engines still need their Studio-specific data workflows. Node canary/autoscaling/sleep/Swarm parity, recovery/PITR, remote MCP, runners, desktop and final release gates remain open.

Node Studio default-bootstrap and full local matrix — 2026-10-24 (browser acceptance pending)

The exact uncached Linux/Node 26.12.1 matrix now passes: 14,230 tests in 851 files, zero failures or skips; 11/11 Turbo test tasks, zero cached. Typecheck, lint and build also passed uncached. Server coverage is 93.26% statements, 87.80% branches, 91.34% functions and 96.25% lines with the existing floors unchanged. All 2,020 frozen source files matched both the tested capsule and workspace. The 60 files changed since the database-client checkpoint were copied from the tested capsule and hash-checked. The retained local test capsule is ninedeploy-check:node-studio-20261024; it is not a published application artifact. These acceptance notes were added after source verification.

The standard agent bootstrap uses Docker default bridge networking. A real production-path probe failed to reach the private Studio helper from that network, while host and same-private-network controls passed. Studio now attaches only the verified agent container to the database bridge, with a strictly lower gateway priority; it never attaches the helper to the default bridge or publishes its port. Exact managed endpoint cleanup follows confirmed helper removal. The same lifecycle proof fails on the previous image and passes on the fix, including unchanged default routes and owned endpoint removal. Automatic containerized attachment requires Engine API 1.48+; see Docker API version history. Manual/pre-existing network connections remain untouched. The updated 296 core tests in 13 files pass separately on Node 22.23.4, 24.22.1 and 26.12.1.

An actual shared-bootstrap-builder fixture passed authenticated SQL workflows for PostgreSQL/MySQL/MariaDB and browser SET/GET for Redis/Valkey/KeyDB/Dragonfly. Independent native clients confirmed the browser writes for all four KV engines. All seven helpers were verified on a single private network each, with no published ports; agent gateway priorities remained below its original bridge. The overall browser acceptance still failed on one JavaScript exception. The first observer retained only Uncaught, so an instrumented rerun is collecting the full exception and phase instead of ignoring it. Full Studio acceptance remains open. The actual page capture also exposed an existing node-database resource card reading the deliberately local-only stats endpoint and displaying a false start instruction/live badge; target-scoped telemetry and truthful unavailable states are required before release.

Evidence and cleanup receipts are under .temp_files/node-studio-20261023-3fbebb9e/verification-state.json. Failed scopes remain recorded; owned matrix containers were removed after copying evidence. Node canary/autoscaling/sleep/Swarm parity, recovery/PITR, remote MCP/OAuth, runners, desktop, final upgrade/promotion and publication remain separate unfinished work. Version remains 0.17.1 and 0.18.0 remains unready.

The instrumented default-bootstrap rerun (runtime-default-bridge-3) subsequently passed all seven browser data workflows with zero JavaScript exceptions, console errors or unexpected HTTP failures. Independent native readback passed for all four KV engines. Explicit helper/database-stop cleanup returned the agent to its original bridge, preserved the original default route, and left the published agent reachable. All owned fixture resources were removed. This is a clean functional replay; the earlier opaque exception was not reproduced and its cause is still unresolved, not claimed fixed. Node database telemetry remains the next functional correction.

Node database telemetry acceptance — 2026-10-24

Database-only nodes now participate in collection. Database-scoped stats return only the authorized database and its recorded runtime source, with explicit live/pending/unavailable/stale/stopped receipts. Missing or stale node samples never become zero metrics or a false live badge. The SDK rejects old panels that ignore the database filter instead of displaying a local resource. An actual v0.17.1 panel reproduced that ignored filter and the new SDK refused its unconfirmed response. Real default-bootstrap node interruption/reconnection, local/node name isolation, database stop and 1280px/375px browser controls passed. The concurrent seven-engine Studio replay passed with eight browser targets observed and zero exceptions; the earlier opaque Studio exception remains unreproduced, with no root-cause fix claimed. Vendor Studio pages retain their existing narrow-screen layout limitations.

The frozen telemetry/SDK checkpoint passed 14,259 tests in 855 files and all 11 uncached tasks on Linux/Node 26.12.1; all 2,024 source hashes matched. Evidence: .temp_files/node-database-metrics-20261024-072948bb/verification-state.json. Later backpressure changes are covered by the next checkpoint.

Agent backpressure and readiness deadline acceptance — 2026-10-24

A real standard-bootstrap deployment exposed failed-candidate leakage when the sealed agent rate limit returned HTTP 429. The client now retries only a valid, bounded Retry-After within the original deadline, reusing the sealed request identity; cancellation, retry count and replay refusal remain enforced. Remote readiness inspection, probes and diagnostic logs receive the remaining readiness budget, and late successes are refused. The packaged Docker healthcheck selects the configured agent ping or panel health endpoint according to its role.

The same controlled Docker/Traefik scenario failed on the previous image with the rejected candidate still running after 90 seconds. The current image removed it and preserved the healthy previous runtime. All 1,596 routed requests returned HTTP 200. Actual HTTP 429 with Retry-After 57 seconds did not extend the 16-second readiness budget: failure was observed within 15,231ms. The standard-bootstrap agent reported Docker health healthy. These are owned-daemon HTTP Host routing checks; public DNS/TLS and older-agent recovery were not run in this scope.

The exact fresh Linux/Node 26.12.1 TURBO_FORCE=true pnpm release:check passed 14,279 tests in 857 files, all 11 tasks successful with zero cached tasks. Server coverage is 93.28% statements, 87.85% branches, 91.36% functions and 96.27% lines; existing floors are unchanged. All 2,026 frozen source files matched the tested capsule and workspace; the 11 changed files were copied from the capsule and rechecked. All 1,818 compiled server files matched the real traffic-test image. Owned matrix and runtime resources were removed after retaining evidence.

Evidence: .temp_files/agent-backpressure-20261024-1fdbe651/verification-state.json. Retained local test capsule: ninedeploy-check:agent-backpressure-deadline-20261024; this is not a published application artifact. These notes were added after frozen source verification. Node canary/autoscaling/sleep/Swarm parity, recovery/PITR, remote MCP/OAuth, runners, desktop, final upgrade/promotion, hosted gates and publication remain open. Version remains 0.17.1; 0.18.0 is not ready.

Canary health and exact health endpoint acceptance — 2026-10-24

Explicit HTTP-strict canary releases now have health probes on both leaves, including single-container releases, and the weighted parent follows both leaves' health changes. Probes use a 5-second interval, 3-second timeout and no redirect following. Other readiness modes retain their existing replica semantics; the parent subscribes only when both leaves have probes. Node canary remains gated. Health paths that satisfy the API contract retain query strings and percent encoding; invalid legacy rows retain the previous sanitization.

The same real production-renderer/Traefik v3.1 scenario sent 50 of 100 post-failure requests to the unhealthy candidate on the old image. Both old and new images passed the healthy start control. The corrected image passed plain, sticky-cookie, 1.5-second healthy-endpoint and query-endpoint scenarios. A separate ordinary-replica query scenario reproduced 100/100 failures on the old image with direct readiness controls returning 200, then passed on the fix. Across the five final green scenarios, all 1,500 routed requests returned HTTP 200, covering healthy start, post-detection fallback and recovery. The interval before health detection is not asserted error-free.

A final fresh Linux/Node 26.12.1 TURBO_FORCE=true pnpm release:check passed 14,296 tests in 857 files, zero failures/skips, all 11 tasks successful and zero cached tasks. Server coverage is 93.28% statements, 87.86% branches, 91.36% functions and 96.27% lines; thresholds are unchanged. All 2,026 frozen source hashes matched the tested capsule and workspace; all four changed files were copied from the tested capsule and rechecked. All 1,818 compiled server files matched the final real-traffic image. The focused 53 tests in four files also passed separately on Node 22.23.4, 24.22.1 and 26.12.1; the covering suite passed 216 tests in 11 files.

Two earlier matrix snapshots were stopped after review found an overly short proxy timeout and health-query stripping; neither is counted as acceptance. Twenty-five owned runtime scopes and the final matrix container were confirmed cleaned up; evidence and local images remain. A separate actual JSON-log characterization confirmed leaf attribution and omission of path/query/header data, but used an explicit test CLI override. Product node traffic transport and complete observation-window receipts are not implemented by that experiment.

Evidence: .temp_files/canary-health-20261024-563c831b/verification-state.json. Retained test capsule: ninedeploy-check:canary-health-query-20261024, not a published application artifact. This acceptance entry and related release/operations notes were added after frozen source verification. Version remains 0.17.1; no commit, tag or publication occurred. Node canary/autoscaling/sleep and remaining Swarm parity, isolated recovery/PITR, remote MCP/OAuth, runners, packaged desktop and final upgrade/promotion/artifact/hosted/publication gates remain unfinished.

Independent local canary observation — 2026-10-24

Measured local steps now open independent read-only JSON access-log windows after loaded routing and source opening, poll those windows and drain the final bytes before a verdict. They count completed records by byte position, independently of the analytics tailer's schedule and request timestamps. The local adapter checks fresh service placement/runtime ownership, running proxy ID/restart identity, desired static-config fingerprint and the exact writable log mount. Node and Swarm sources are refused before local observation. Missing/replaced/truncated files, invalid/partial final records, limit violations and unavailable source/settings reads cannot become zero traffic. Late results cannot clear a newer canary owner or write an old rollback.

Local owner rotation is held while a bounded window reads, then forced with prior invalidation above the normal 64 MiB trigger plus a 32 MiB allowance. Purge/truncation also invalidates windows. Limits are 32 MiB and 100,000 records per window, 16 KiB per record and 16 concurrent windows. A completed window describes logged responses, not a producer-level no-loss guarantee. Node transport, producer heartbeat/sequence receipts, loaded step-generation binding and remote reservation/restart lifecycle remain required before node canary can be enabled. The node guard stays in place.

The same actual canary engine, migrated database, production routing writer and owned Docker/Traefik experiment passed a healthy control on both images. With the analytics tailer deliberately paused and readiness always HTTP 200, the old image still completed after 89 HTTP 503 responses in the late-failure phase. The new image measured 82 failures among 682 canary responses (12.02%), rejected the candidate and confirmed routing back to the old runtime. Unexpected log replacement also refused the verdict. The new healthy control served 1,451 requests with zero errors; 100 post-rollback requests all returned the old runtime's HTTP 200. Failure-phase 503s were deliberately injected and are not claimed error-free. The local proxy used production tag traefik:3, pull digest sha256:c8a121e4bb1347e8fd52657fab12f2cb70b68f8a018895e2068f92c9e519b9c2; this was not a node-agent or full public-API/worker/browser experiment.

The exact uncached Linux/Node 26.12.1 TURBO_FORCE=true pnpm release:check passed 14,347 tests in 859 files, zero failures/skips, all 11 tasks successful with zero cached tasks. Server coverage is 93.29% statements, 87.87% branches, 91.38% functions and 96.29% lines; thresholds are unchanged. All 2,030 frozen source hashes matched the tested capsule and workspace. All 17 changed files were copied from the capsule and rechecked; all 1,826 compiled server files matched the real experiment image. The 100 core tests in four files also passed separately on Node 22.23.4 and 24.22.1. Focused coverage passed at 95.96% statements/94.11% branches, and the final classified covering suite passed 118 tests in six files.

The first full attempt had 9,926 passing server tests and one failure: its strict Swarm runtime-reader inventory did not classify the new observer. The local-only observer was classified with its existing explicit Swarm refusal and behavior tests; existing assertions and coverage floors were preserved, then the entire fresh matrix passed. Owned runtime containers, nested data, volume/network and both matrix containers were removed after evidence capture. Evidence: .temp_files/canary-observation-20261024-83c1744b/verification-state.json. Retained local test capsule: ninedeploy-check:canary-observation-20261024, not a published application artifact. These acceptance/deployment/Unreleased notes were added after frozen source verification. Version remains 0.17.1, with no commit, tag or publication. Node runtime parity and the recovery/PITR, remote MCP/OAuth, runner, desktop and final release gates above remain unfinished.

Sealed node traffic observation acceptance — 2026-10-24

The node owner can opt in to the sealed proxy.traffic reservation lifecycle with NINEDEPLOY_AGENT_PROXY_TRAFFIC=on; unset/off remains disabled. The controller and agent refuse plaintext fallback. Observations pin the running proxy/log source and managed-service runtime IDs, bind node/service/deployment/step and target receipts, verify current controller placement/checkpoints, and use existing default common access logs without changing logger configuration or recreating ingress to open a window. Raw client addresses, request paths, headers and access records stay on the node.

Unrouted loopback start/end markers and Traefik request counters attest contiguous coverage of a bounded HTTP interval. Out-of-order completions are handled; a final drain can accept a late response but cannot turn filtered/missing records, missing markers or still-pending requests into zero traffic. Long-lived requests and WebSockets can make a window unavailable. Limits are 16 active windows, 32 MiB per stream, 100,000 records and 16 KiB per line; cached results/tombstones are bounded to 128 entries and up to 60 seconds. Completed results remain retryable after the active-window deadline, without reopening a stream. This protocol does not certify connection failures before the HTTP handler or add producer-loss detection to local JSON file windows.

An actual migrated controller database and the production controller adapter called a real sealed agent bootstrapped by the standard Docker command on its default bridge, with the explicit owner opt-in added. The production static renderer and proxy write/ensure operations ran Traefik v3.1. Weighted routing was a controlled fixture, not the production node canary renderer, public API/worker or browser rollout. The corrected candidate passed all 11 phases: published-0.17.1 agent refusal, owner-off refusal, healthy and 5xx traffic, completed-result retry after the active deadline, late completion, pending-request refusal, proxy restart, agent restart, filtered 5xx and missing heartbeat. Healthy traffic was 80/80 HTTP 200; the failure control attributed all 40 injected HTTP 503 responses to the 40 canary requests while all 40 stable requests succeeded. A two-second response drained; an eight-second response refused the result. These were deliberate failure controls, not an error-free rollout claim.

The exact Linux/Node 26.12.1 TURBO_FORCE=true pnpm release:check passed 14,459 tests in 865 files across eight packages: zero failures/skips, all 11 tasks successful and zero cached tasks, in 17m7.391s. Server coverage is 93.30% statements, 87.89% branches, 91.42% functions and 96.31% lines; all existing thresholds are unchanged. The 548 schema tests retain 100% coverage. The 87 focused observation tests also passed separately on Node 22.23.4, 24.22.1 and 26.12.1. An expanded node compatibility/refusal preflight passed 385 tests in 11 files.

All 2,042 frozen sources matched the successful test capsule and workspace. All 26 changed files were copied back from the capsule and rechecked. All 1,986 compiled server/schema files matched both the successful Linux capsule and the corrected actual-node candidate. Two earlier snapshots were not accepted: the first was stopped after a completed-receipt lifetime defect was reproduced and fixed; the second exposed a missing new-capability label in the old-agent refusal fixture. Its existing assertions were preserved, the fixture was completed, the expanded preflight passed and the entire fresh matrix was rerun.

Evidence: .temp_files/node-proxy-traffic-20261024-c4851a03/verification-state.json. The retained local test capsule is ninedeploy-check:node-proxy-traffic-20261024; it is not a published application artifact. Owned runtime scopes and all three matrix containers were removed after evidence capture. Only this acceptance entry, the deployment guide and Unreleased notes changed after frozen-source validation. Git HEAD/index and unrelated dirty work were preserved. Version remains 0.17.1, with no commit, tag or publication.

Node canary remains disabled: loaded weighted-step generation binding, deployment lease integration, manual early-promotion behavior and actual API/worker restart, Stop/cancellation and unconfirmed-rollback acceptance remain required. Node autoscaling/sleep and remaining Swarm parity, isolated restore/query/RTO and PITR, remote MCP/OAuth, runners, packaged desktop, and final upgrade/promotion/artifact/hosted/publication gates remain unfinished.

Routing-bound node observation checkpoints — 2026-10-24

Observation requests now include the exact expected routing content, bounded to 1 MiB of UTF-8. The sealed receipt binds its SHA-256 routing generation together with node/service/deployment/step, immutable runtimes and observation identity. The agent validates both private HTTP and TLS handler generations and enabled definitions using the pinned proxy container ID, then rechecks its restart identity. Opening, polling and finishing perform these checks. A verification failure invalidates the reservation; recovery of the source cannot revive that window. Expected private routing content is discarded on completion/closure instead of retained in the bounded retry cache. The shared routing-reply validator retains the existing primary deployment acknowledgement behavior.

Five executable before-cases showed that the earlier inspector accepted a wrong generation header, changed weighted definition, unavailable TLS handler, malformed private response or restart during confirmation. The fix refuses all five. The 157 covering tests in 13 files passed, including existing routing acknowledgement, node proxy, Swarm inventory and cycle checks. The 548 schema tests retain 100% coverage. The 98 core tests in six files passed separately on supported Node 22.23.4 and 24.22.1.

A real migrated controller database, sealed agent on the standard bootstrap default bridge and Traefik v3.1 passed 15 scenarios. The four new controls refused a wrong loaded generation before opening, a generation/weight change during polling, a change before finishing and a missing private TLS handler while ordinary HTTP still returned 200. The existing healthy, 5xx, late/pending response, proxy/agent restart, filtered-log, missing-heartbeat, old-agent, owner-off and completed-receipt retry controls stayed green. Healthy traffic remained 80/80 HTTP 200; all 40 deliberately injected canary HTTP 503 responses were attributed correctly and all 40 stable requests succeeded. This uses controlled weighted routing and the production proxy bootstrap/writers; it is not the production node canary API/worker/browser flow.

The exact Linux/Node 26.12.1 TURBO_FORCE=true pnpm release:check passed 14,470 tests in 866 files, zero failures/skips, all 11 tasks successful and zero cached tasks, in 19m5.897s. Server coverage remains 93.30% statements, 87.89% branches, 91.42% functions and 96.31% lines; existing thresholds are unchanged. All 2,043 frozen source files matched the test capsule and workspace. All 16 changed files were copied back and rechecked. All 1,986 compiled server/schema files matched the successful Linux capsule and the actual-node image.

These checks attest routing at observation boundaries, not continuous per-request generation identity or deployment-lease ownership. Production primary-node weighted rendering/controller integration, manual early promotion, concurrent configuration-write/capacity behavior and actual API/worker restart, Stop/cancellation and unknown-rollback acceptance remain required before enabling node canary. The remaining packages in the release gate snapshot above are unchanged; no release date or weighted completion percentage is claimed.

Evidence: .temp_files/node-traffic-generation-20261024-c35f9a26/verification-state.json. The retained local test capsule is ninedeploy-check:node-traffic-generation-20261024. Owned runtime containers/data/volume/network and the matrix container were removed after evidence capture. Only the tracker, deployment guide and Unreleased notes changed after frozen-source verification. Git HEAD/index and unrelated dirty work were preserved. Version remains 0.17.1; no commit, tag or publication occurred.

Canary ownership, scoped rendering and readiness rechecks — 2026-10-24

Canary runs now receive an opaque incarnation owner, separate from the deployment ID. The engine validates its deployment checkpoint before creating state, checks that owner across routing/source/probe/close awaits and refuses stale success after asynchronous close. Parent pipeline cleanup captures the incarnation and cannot clear a newer run, including a retry of the same deployment. An old rollback is not retried after a new canary takes ownership. Scoped rendering requires the primary host and stable-runtime anchor to match; an ordinary fan-out target does not receive the primary split. Production worker node enablement remains guarded.

The engine supplies readonly deployment/node/stable-runtime/step/weight/duration context to observation opening. Internal node-mode manual promotion waits for the current full measured step, readiness and configured verdict, then skips remaining steps. Local manual promotion retains its existing early behavior. Initial candidate qualification and the remote builder’s three stable samples remain intact. Periodic canary verification now has up to ten seconds, bounded by the configured service deadline; a check that already finished after the step boundary is not repeated before final observation drain.

The previous accepted local image passed the healthy control but allowed a controlled newer-owner replacement during routing acknowledgement to be overwritten by the old continuation. After a subsequent ordinary production-renderer refresh, 50 of 100 requests returned deliberately injected HTTP 503 responses. The same-deployment retry reproduced the same 50/100 failure. Native readiness returned HTTP 200. The corrected image preserved the new owner in both cases: 100/100 post-refresh requests returned HTTP 200 and 99 reached the newer runtime; its healthy control also passed. This owner replay uses a controlled engine clock with real Docker/Traefik routing and HTTP traffic, not a full worker or wall-clock observation timing claim.

A separate real-clock component flow then combined a migrated controller database, standard default-bridge sealed agent, production primary-node renderer/routing acknowledgement, real primary transition lease, remote readiness builder and sealed sequence-complete observation windows. The single-replica healthy node step served 1,164 requests with zero errors. A deliberate root-path 5xx phase retained HTTP-200 readiness, measured 668 failures among 668 canary responses, rejected the candidate, confirmed the previous routing and passed 100/100 post-rollback old-runtime requests. The client saw 684 deliberately injected failures across the whole failure-phase traffic interval; observation is only its bounded window. An early promotion request waited for the current step and completed in 38.214 seconds, with 1,420 requests and zero errors; it did not start the remaining step.

The first corrected runtime attempt omitted the initial full readiness gate and did not pass node acceptance. Adding that precondition made healthy and measured rollback pass, but a later periodic check still timed out while the container reported running/healthy. Three stable samples with two-second poll spacing leave too little room for node round trips inside five seconds. The deterministic before-cases failed for the default/seven-second periodic budgets and duplicated final probe, while the explicit five-second service-limit control remained valid. The final replay preserved the full readiness policy with the corrected bounded budget and completed all six scenarios. Both failed runtime attempts are retained and are not counted as acceptance.

The final covering run passed 473 tests in 13 files; the 67 canary core/renderer tests passed separately on Node 22.23.4 and 24.22.1. The exact uncached Linux/Node 26.12.1 TURBO_FORCE=true pnpm release:check passed 14,492 tests in 866 files across eight packages, zero failures/skips, all 11 tasks successful with zero cached tasks, in 18m25.432s. Server coverage is 93.30% statements, 87.90% branches, 91.44% functions and 96.31% lines; thresholds are unchanged. All 2,043 frozen source files matched the successful capsule and workspace; all ten changed files were copied back and rechecked. All 1,986 compiled server/schema files matched the Linux capsule and the final real-traffic image.

Evidence: .temp_files/canary-owner-placement-20261024-b86f2878/verification-state.json. Retained local test capsule: ninedeploy-check:canary-owner-placement-20261024. Four owned runtime scopes and the matrix container were removed after evidence capture; logs and local images remain. Only the tracker, deployment guide and Unreleased notes changed after frozen-source verification. HEAD/index and unrelated dirty work were preserved. Version remains 0.17.1; there is no commit, tag or publication.

This validates canary components, not public node rollout support. Worker capability/source selection, actual API/worker restart, Stop/cancellation and unknown-rollback acceptance, concurrent configuration writes and replica-group/agent-rate budgets remain gates before enabling node canary. The remaining autoscaling/sleep/Swarm, recovery/PITR, MCP/OAuth, runner, desktop and final release packages in the snapshot remain unfinished.

Shared primary canary worker traffic and rollback retirement — 2026-10-24

The exact TURBO_FORCE=true pnpm release:check passed on Linux/Node 26.12.1: 14,515 tests in 867 files, no failures or skips, with all 11 tasks uncached across eight packages. Typecheck, lint, build and existing coverage thresholds passed. Server coverage was 93.33% statements, 87.99% branches, 91.47% functions and 96.33% lines. All 2,045 frozen source files matched the tested capsule and workspace; the five files changed in this round were collected from the successful capsule and hash-checked. All 1,990 compiled server/schema files matched the Linux capsule, workspace and real-runtime candidate. HEAD and the Git index were preserved.

The primary worker now selects canary traffic through one shared binding. Local canary routing and rollback use the durable primary lease's confirmation method, so a confirmed proxy rollback is also recorded before immutable candidate retirement. Replaying the prior accepted pipeline with only import relocation reproduced a rejected candidate remaining present; the neighboring no-canary control passed. The unchanged regression assertions now pass through the real canary engine, pipeline and recovery controller: the old generation remains, the rejected candidate is removed, and subsequent status-change reconciliation closes the journal. The permanent binding suite covers missing identities, capabilities, changed steps, cancellation, lease replacement and spare-capacity exclusion. The final covering run passed 496 tests in 14 files. Node 22.23.4 and 24.22.1 each passed 120 binding/core tests plus the two focused pipeline regression/control cases (226 unrelated pipeline cases filtered in those focused runs).

The internal node branch binds observation to the journal's captured serving container IDs and ports, the selected node's generated routes and sealed traffic/routing capabilities. It requires the node's loaded HTTP/TLS generation before opening measurement, rechecks ownership/step context and never substitutes panel traffic. An actual Docker/Traefik/sealed-agent replay invoked this exact shared binding: a healthy step served 1,138 requests with zero errors; a deliberately failing candidate produced 569 measured 5xx responses out of 569 canary requests, was rejected, and the following 100 requests all reached the old healthy generation. A promote requested after five seconds waited for the full measured step and passed after 36.696 seconds, serving 1,224 requests without errors.

A separate real HTTP-readiness failure returned 503 on the candidate's configured readiness URL. The shared binding confirmed rollback before retiring the captured candidate ID, preserved the running old container and completed the journal. All 701 requests during that transition and 100 following retirement returned HTTP 200. Both owned runtime scopes were cleaned after evidence capture. These are single-replica direct component replays, not a public-API/worker/browser node-canary acceptance or multi-replica capacity proof. The actual runtime image still refuses public node canaries. Capability preflight before candidate creation, replica/request-rate capacity, concurrent node refreshes and full API/worker Stop/cancellation/restart scenarios remain required before enabling the public feature.

Evidence, prior-source RED/control results, incomplete/superseded attempts, final checks, source/artifact identities and cleanup receipts are under .temp_files/primary-canary-worker-20261024-a380ec8c/verification-state.json. Earlier freezes were superseded after adding the permanent regression and required Swarm reader classification; they are not accepted full checks. Only CHANGELOG.md, docs/DEPLOYMENTS.md and this tracker changed after final source validation. Version remains 0.17.1; no commit, tag or publication was made. Runtime parity, packages 3–6 and final release/upgrade/publication gates remain open for 0.18.0.

Batched node readiness and image health compatibility — 2026-10-24

The exact TURBO_FORCE=true pnpm release:check passed on Linux/Node 26.12.1: 14,574 tests in 870 files, no failures or skips, across eight packages with all 11 tasks uncached. Typecheck, lint, build and existing coverage thresholds passed. Server coverage was 93.33% statements, 87.99% branches, 91.49% functions and 96.34% lines. All 2,052 frozen source files matched the tested capsule and workspace; all 21 changed files were collected from that capsule and hash-checked. All 2,002 compiled server/schema files matched the capsule, workspace and corrected real-runtime candidate. Git HEAD and index were preserved.

Updated agents advertise the sealed docker.readiness operation. Primary candidate verification negotiates it once per transition and checks up to ten journal-captured serving identities in shared rounds. Every replica still needs three consecutive passing samples; already-qualified replicas remain sampled, and Docker identity/health/restarts are checked again after application probes. Rounds share their original deadline, at most two groups run on an agent, and started probes/cleanup attempts settle before capacity is released. Older agents keep the existing path; a selected batch protocol never downgrades after unavailable or changed receipts.

An actual ten-container replay used the standard bridge bootstrap and sealed transport. All containers were initially healthy. The previous per-container path exhausted the 10-second budget after 48 agent requests; the batch passed in 6.412 seconds with three requests. A tenth-replica HTTP 503 failed within the original deadline; repairing it passed in 6.504 seconds with three requests. Losing batch support mid-transition failed without fallback. No HTTP 429 occurred in these owned cases. These timings are fixture measurements, not a general capacity guarantee or node replica-creation support.

A real normal-node API/worker run exposed a formatter bug that unit Docker stubs did not reproduce: images without Docker HEALTHCHECK omit State.Health, and direct field access made the new batch unavailable. The corrected formatter handles the missing key while still performing HTTP/TCP readiness. A permanent real-Docker smoke passed its health-enabled control and failed the missing-health case on the prior candidate; both cases pass on the corrected candidate. The script is registered as smoke:container-readiness and wired into the published-image release job. That hosted job has not run in this round.

With the corrected image, actual normal-node API/worker deployment, explicit HTTP-503 holding, hard panel restart, lease preservation and automatic confirmed recovery passed using independent panel/node daemons and the standard bridge agent: 1,497 requests, zero errors. The exact image also repeated the internal canary component replay: 1,172 healthy requests without errors; all 563 measured candidate 5xx responses caused rejection, followed by 100 old-generation requests without errors; a promotion requested after five seconds waited for the measured step, returning after 33.615 seconds with 1,223 error-free requests. This does not certify public node canary API/worker/browser behavior; its guard remains closed.

The covering run passed 639 tests in 18 files. The final capability/refusal/readiness selection passed 173 tests in five files; Windows/Node.js 22.23.4 and 24.22.1 each passed 125 tests in four files. The strict capability-description fixture was completed without weakening refusal expectations. Evidence, RED/control and corrected Docker smoke results, unaccepted attempts, runtime receipts and cleanup records are under .temp_files/canary-preflight-20261024-b9871777/verification-state.json. All owned runtime scopes were cleaned; proof images remain. Only CHANGELOG.md, docs/DEPLOYMENTS.md and this tracker changed after final source validation. Version remains 0.17.1; no Git commit, tag or publication was made. Node-canary preflight and public lifecycle acceptance, node scaling/sleep, remaining Swarm parity, packages 3–6 and final release gates remain open for 0.18.0.

Node patrol cancellation checkpoint — 2026-10-25

Node patrol shutdown now cancels the actual agent connection during sealed-transport negotiation, container inspection and lifecycle capability discovery. One serial reconciler owns the pass and its cancellation signal; closing it prevents a new pass and awaits the cancelled active pass. Each patrol RPC has a 15-second budget. The operation budget now includes sealed negotiation, and caller cancellation propagates through the negotiation request and response body. Shutdown cancellation does not declare a healthy node unreachable, mark its service down or enqueue a recovery command. Local Docker subprocess and PM2 shutdown remain open; this checkpoint does not certify their cancellation.

The original production-plugin HTTP proof failed both held-handshake and held-inspect cases while its healthy control passed. The corrected test also holds the real lifecycle capability request and requires the socket to close, rather than merely abandoning its Promise. All 142 covering tests passed on Windows/Node 26.12.1 and on Linux/Node 22.23.4 and 24.22.1. Server typecheck and lint passed. A frozen uncached full release check is still running and is not counted as a pass. Its first diagnostic stopped at SDK coverage: the already-committed bulk deployment removal method lacked a contract test. Commit ee48acea adds explicit-ID/status selection, authentication, response preservation and refusal coverage; all 366 SDK tests now pass at the existing 100% floors. Evidence: .temp_files/r2-patrol-shutdown-20261025-41014f1a/.

The preceding public lifecycle commit 41014f1a also completed all hosted CI jobs successfully. These checkpoints are development increments, not 0.19.0 certification. Held-recovery alerts, shared fan-out/controller ordering, capacity/network attachments, autoscale, wake and the Swarm acceptance matrix remain open.

Held node recovery notifications checkpoint — 2026-10-25

The production lifecycle worker now emits alert.service_recovery_held when the accepted command is still current, has a recorded hold reason and has no live owner. Automatic patrol also warns when agent version, owner policy, sealed transport or managed-identity requirements prevent accepting a recovery command. Both paths preserve desired runtime state and never introduce a legacy Start fallback. Each owner throttles repeated warnings for the same service/recovery identity for ten minutes; a new command can warn immediately. Restarting the panel may re-alert a hold. Cancellation, deleted/replaced/moved services, Swarm/Compose placement, pending handoffs and normal deployment/supersession conflicts do not produce these warnings.

Notification metadata contains only service/node identity, command/action/phase or an allow-listed refusal code. Private agent error output stays in the protected command status view. The event reaches normal service alert subscriptions and maps to a typed warning event for plugins; it does not announce a dead-service transition. A real migrated-SQLite audit/subscription fixture records successful delivery to a mocked recipient, with no external message sent.

Two independent behavior proofs failed on the original paths: an automatic command held by a missing routing acknowledgement, and an old-agent refusal before command acceptance. Their 30 and 31 existing lifecycle controls respectively passed after isolating the refusal fixture. The corrected frozen Linux snapshot passed 245 covering tests in seven files. The new notifier separately passed all 69 selected lifecycle/alert tests with 100% statement, branch, function and line coverage for that module; these are focused results, not global server coverage. Linux server typecheck, lint and build passed. Evidence: .temp_files/r2-held-recovery-alerts-20261025-4f519516/.

The earlier full patrol matrix failed 31 tests in three host-install files because its check container retained /.dockerenv; the unchanged source passed all 162 tests in those files when run in the intended Linux host context. That failed matrix remains a diagnostic and is not counted as release certification. An uncached TURBO_FORCE=true pnpm release:check is now running on the combined frozen patrol/notification sources in that corrected context. Its terminal result is still required. R2, 0.19.0 publication, local Docker/PM2 cancellation and the remaining controller/Swarm gates are open.

Replica database connectivity and asynchronous HTTP readiness — 2026-10-25

Every acknowledged node replica now joins every attached same-node database bridge before readiness begins. Multi-replica joins use immutable creation IDs; an empty, duplicate, malformed or oversized batch is refused before any node effect. The existing single-container compatibility path remains available. Each join has a maximum 15-second RPC budget within a 60-second stage budget. Deployment cancellation is checked before and after each join, so a late response cannot continue attaching another replica or publish a cancelled candidate. The valid pipeline reproduction originally joined only the reusable primary name while both replicas existed; its three existing attachment controls passed. An earlier fixture that omitted the database capability advertisement is retained as a setup diagnostic, not a product RED.

The real SQL fixture also exposed an independent readiness defect: the raw netcat input pipeline closed its socket before an asynchronous HTTP handler answered. Both synchronous controls passed while both 50 ms delayed controls failed, with and without an image HEALTHCHECK. Commit c2ea148b keeps the socket bidirectional until the bounded status-line read completes. An isolated transport experiment verified HTTP 200, 302 and 503 responses and a non-answering endpoint that stopped after approximately 3.4 seconds; the unsuccessful file-descriptor experiment is retained separately. All four native production-module cases passed on Node 22.23.4, 24.22.1 and 26.12.1: 12 real-Docker checks. The 27 covering protocol/identity tests also passed.

An API-created node PostgreSQL database backs a two-replica HTTP application using a pinned SQL test client. Each replica executes SELECT 42 for readiness. On the final frozen 532d4904 baseline plus this patch, all three real panel/worker/sealed-agent/Traefik scenarios passed: both replicas reach the private database bridge and return SQL value 42; a missing-table query refuses the complete new generation while the previous generation keeps serving and rejected immutable candidates retire; explicit Stop during SQL readiness cancels and retires the candidate set, stops the old serving set, confirms route removal and does not resurrect it. The fixture monitored 568 HTTP requests with no unexpected 502. The preceding 5696862f snapshot independently passed the same three cases with 584 requests. These are isolated Host-header HTTP fixtures, not production traffic or public DNS/TLS tests. They accept database connectivity and SQL-backed HTTP-strict readiness; automatic Docker-state readiness and R3 restore/PITR are separate requirements. The permanent --replica-databases-only command is now a release smoke gate.

The complete related pipeline/database/cancellation/worker suite passed 407 tests in eight files on Linux, including the newly committed node operation cancellation path. The new worker Stop test checks the completion audit and kernel event after immutable effects. Windows initially terminated the full lifecycle file in the native worker; its isolated Stop test passed. Test teardown now closes apps/workers before closing their native SQLite connection, after which all 33 lifecycle tests passed. The final check image was installed cleanly from the frozen lockfile and built all eight workspace tasks without cached build results. Server typecheck/lint/build and web lint passed. The candidate cold-started as UID 999, and all 2,263 compiled server/web/package files matched the checked build byte-for-byte with no missing or extra files.

The earlier uncached matrix passed all 10,467 server tests but missed the unchanged statement floor (93.24% versus 93.25%). A later diagnostic passed all 2,408 web tests but missed the unchanged function floor (97.48% versus 97.5%). Neither run is counted as a completed release gate. Bulk cleanup coverage now exercises cancellation without removing history, a refused request without clearing the selected log, and clearing a selection only after a successful removal receipt. The complete web coverage rerun passed 2,410 tests in 170 files at 98.09% statements, 94.08% branches, 97.50% functions and 98.64% lines, with every existing floor unchanged. The final combined TURBO_FORCE=true pnpm release:check passed 15,028 tests in 896 files, no failures or skips, all 11 tasks uncached. Server coverage was 93.26% statements, 87.99% branches, 91.32% functions and 96.36% lines. This certifies the frozen 532d4904 baseline plus the committed c2ea148b / 16d1fa59 / 41ee49e6 changes; unrelated in-progress shared-checkout edits are excluded. Hosted CI and feature-release publication remain separate gates. Evidence: .temp_files/r2-replica-database-20261025-be095021/.

R2 and 0.19.0 remain open. Shared ordering for fan-out/deploy/scale/wake, stopped autoscale capacity, node autoscale and sleep/wake, remaining network/mount acceptance, the actual Swarm support matrix, mixed-agent upgrades and final publication gates are not accepted by this checkpoint. R3–R6 remain planned separately. The published stable release remains 0.18.2.

Multi-node lifecycle and target routing checkpoint — 2026-10-25

The frozen 41ee49e6 behavior reproduction accepted public Stop with HTTP 202, stopped the primary immutable replicas and reported completed while a capable fan-out node remained running without any ordered lifecycle request. The new assertion failed while all 33 existing lifecycle controls passed. That exact regression now runs in the committed public lifecycle suite.

Ordered Start, Stop and Restart now snapshot every configured additional-node placement together with the primary at acceptance. Each participant keeps its own runtime/deployment generation and immutable container IDs; the primary's existing fence identity remains compatible. Every asynchronous boundary renews ownership and checks the accepted target membership/runtime. Each participating agent must advertise sealed lifecycle, identity/recovery, readiness and routing-ack support before effects begin. A changed placement or newer command supersedes stale work. Older primary-only journals hold when fan-out runtimes exist, instead of silently completing primary-only work. Idle configured placements without a runtime remain in the snapshot without inventing containers.

Stop fences every participating agent, confirms route withdrawal on every node, drains and stops the captured IDs before completing. Start/Restart require three stable readiness samples per serving node and loaded-routing acknowledgement from every participant. Lost effects replay the same request, revision and immutable targets through the agent journal; lost routing repeats readiness and routing without replaying Start. Offline nodes and malformed/foreign receipts hold the original command and never announce completion. Target status updates occur in the owned completion transaction. These changes apply to node-primary plain Docker services. An unresolved fan-out transition holds effects because it may own another generation outside the serving row; coordinating Stop with those pending generations remains open.

A second reproduction found that the renderer copied the primary replica count into fan-out routing even though the fan-out builder creates one container per target. Three new cases failed for achieved counts 2, 10 and an unrecorded count; four neighboring controls passed. Commit 5ab886ef renders only the target's actual single container and preserves the primary's achieved replica set.

The related Windows run passed 278 tests in nine files before five additional refusal-boundary cases were added. The final four-file lifecycle/renderer suite passed all 113 tests independently on Node 22.23.4 and 24.22.1, including those new boundaries. A real fixture used migrated SQLite, two independent private Docker daemons, sealed HTTP agents and two Traefik instances. All seven scenarios passed: complete multi-node Start; ordered Restart; primary-ready/child-HTTP-404 refusal and recovery; offline-child Stop and same-command recovery; a lost child effect receipt replayed against exact immutable identities; a newer Stop superseding a real child Start callback; and a lost child routing response requiring fresh readiness on both nodes without replaying Start. It invokes the production controller/runtime, not public lifecycle HTTP endpoints, and advances retry scheduling explicitly rather than claiming real backoff elapsed. The public route is covered separately by the permanent API regression. No public DNS/TLS or production traffic was tested.

The fixture's first attempts were setup diagnostics: a literally escaped YAML newline, a check image without the Docker CLI, and an insufficient five-second fixture readiness budget. The accepted fixture uses a fifteen-second service budget without changing production readiness or sample requirements. The refusal test's wording assertion was corrected to the actual unencrypted-transport explanation. None of these diagnostics is counted as a product RED or acceptance. All owned runtime containers, networks and volumes were removed. The clean runtime has exactly the same 2,535 compiled files as the checked build, with no changed, missing or extra files, and cold-starts as UID 999 with healthy application/database status. An earlier COPY overlay retained 57 old web assets; the clean image explicitly removes its old compiled directories before copying the checked build.

The first complete check stopped at web function coverage: all web tests passed, but 97.48% missed the unchanged 97.5% floor. Commit 4e69ebcd adds a behavioral routing refresh test: a refused observation remains visible, refresh stays disabled while pending, and the resulting container/domain/IPv6 allow-list route replaces the error. Its three focused tests pass. The final frozen TURBO_FORCE=true pnpm release:check passed 15,066 tests in 899 files, no failures or skips, all 11 final tasks uncached. Typecheck, lint, build and every existing coverage floor passed. Server coverage was 93.25% statements, 87.94% branches, 91.35% functions and 96.35% lines; web coverage was 98.09%, 94.09%, 97.50% and 98.64% respectively. This accepts source 899fa966 plus 5ab886ef / 4e69ebcd / de3ec37d, with all twelve frozen owned inputs matching their committed normalized blobs. The disposable check container removes only its own /.dockerenv marker to exercise existing native-host installation assumptions; no source, threshold or test selection is weakened. All 2,535 artifacts rebuilt during the successful gate also match the clean runtime byte for byte. Hosted CI remains a separate gate. Evidence: .temp_files/r2-fanout-ordering-20261025-41ee49e6/verification-state.json.

R2/0.19.0 remain open. Next is Stop during an unresolved fan-out generation handoff, followed by shared deploy/scale/wake ordering, local-primary fan-out, stopped autoscale capacity and network/mount acceptance, node autoscale/sleep/wake, the actual Swarm support matrix, mixed-agent upgrades and final publication gates. R3–R6 remain separate future releases. The published stable remains 0.18.2; these development commits do not publish 0.19.0.

Stop across retained generations: agent protocol and builder ordering — 2026-10-26

Commit 66eee027 adds the development agent capability docker.lifecycle.generations. An explicit Stop batch supplies the expected deployment label for every immutable container ID/name. The agent validates the complete batch before any Docker effect, checks it again afterward and persists one service revision and replayable receipt for every generation together. Explicit Stop batches are bounded at thirty targets; implicit legacy batches retain their ten-target limit. Start rejects explicit generation annotations. The existing NINEDEPLOY_AGENT_CONTAINER_LIFECYCLE switch disables both capabilities. Legacy Stop cannot mutate any generation adopted in that receipt, while the existing unrelated older-generation retirement control still passes. The panel also refuses target generation annotations not present in its request.

The original agent proof failed the new mixed-generation case while its 31 existing controls passed. A separate real-Docker reproduction confirmed that both generation-9 and generation-10 containers and a foreign-service decoy were running before the old protocol refused the Stop. On the corrected source, all eight real-Docker cases passed independently on Node 22.23.4, 24.22.1 and 26.12.1: 24 runtime checks. They cover one-revision Stop of both generations without stopping the foreign service, a fresh-process durable retry after an obsolete target is removed, same-name replacement protection, generation/service-label refusal, older or explicitly mixed Start refusal, legacy Stop fences for both adopted generations, and owner-disabled refusal before journal or Docker effects. These invoke the production registered agent operation with a sealed context, not actual sealed HTTP or the public panel controller. All owned daemons, runners and journal volumes were removed. The permanent smoke is included in the release workflow.

All 626 schema tests passed with unchanged 100% statement, branch, function and line floors. The related Linux suite passed 224 tests in nine files. The initial receipt fixture used a Start response, which the new protocol correctly rejects before the panel comparison; the corrected Stop fixture exercises that comparison itself. A full diagnostic then exposed the missing human-readable feature entry in the old-agent refusal test registry: 10,526 server tests passed and that new generated case failed. Commit 1766fe86 completes the exhaustive fixture; all 51 refusal tests pass, with the original version/message/only-ping assertions preserved. No threshold or test selection was weakened.

The final acceptance snapshot includes the concurrently committed Doctor changes in 819bd5bd and matches committed source 1766fe86. Its combined capability, lifecycle, Doctor and OpenAPI checks passed 220 tests in seven files. The final frozen TURBO_FORCE=true pnpm release:check passed 15,083 tests in 899 files, no failures or skips, all 11 final tasks uncached, on committed source 1766fe86. Typecheck, lint, build and all existing floors passed. Server coverage was 93.27% statements, 87.98% branches, 91.38% functions and 96.36% lines; web coverage was 98.09%, 94.09%, 97.50% and 98.64%. The disposable check container removes only its own /.dockerenv marker for existing native-host installation assumptions; source, thresholds and selection remain unchanged. The immutable checked-build image and complete runtime match in all 2,535 compiled files. Post-run container collection was unavailable because the successful terminal container was no longer present; no post-run artifact equivalence is claimed. Hosted CI and publication remain separate gates. The final complete runtime copies every checked package, including web assets changed by the shared capability enum; earlier server/schema-only packaging left 115 web differences and is not accepted as complete artifact identity. The combined runtime separately repeats all eight Docker cases and a UID-999 healthy cold start. Evidence: .temp_files/r2-generation-stop-20261026-6456a11e/verification-state.json.

Panel integration remains open because a second proof reproduces a real ordering defect. A multi-network docker.runSpec operation creates a container and pauses before its raw Docker Start. An ordered Stop then acknowledges that captured ID as stopped; resuming the builder starts it afterward. The isolated production-operation/SQLite fixture fails that assertion while its two neighboring controls pass. A separate real-Docker, independent-process replay confirms created -> complete Stop receipt -> running after builder resume; a subsequent ordinary Stop works as its positive control. The wrapper pauses only immediately before the real Docker Start. It changes no product module. Neither proof is counted as a fixed defect, and neither is inserted as a failing test into the passing main suite. Its evidence and cleaned runtime receipts are retained under late-start-proof/ and late-start-real/ in the same evidence directory.

The next required implementation is one persistent mutation owner shared by deployment creation, network attachment, Start, Stop, scale and wake. Stop must fence and drain older builder work before discovering identities or completing, and a legitimate newer deployment must retain its authorized creation path. Then the parent controller can capture every retained generation, withdraw and acknowledge routing on every node, stop the complete immutable set and settle its handoff journal. Keep the current unresolved-transition hold until both the real late-Start proof and the public panel cancellation/recovery matrix pass. This agent increment does not close R2, publish 0.19.0, or accept R3–R6. The published stable remains 0.18.2.

Shared runtime authority foundation — 2026-10-26

Migration 0102 adds a private, persistent service runtime owner shared by the declared deploy, lifecycle, scale, sleep/wake and remove mutation kinds. Reservations compare the captured owner and advance one bounded clock; duplicate reservations replay without advancing it, and a stale job cannot reclaim a newer owner. The clock survives lifecycle-history deletion and is retained until the service itself is deleted. The migration preserves the legacy AUTOINCREMENT high-water mark and pending command metadata and keeps already-issued wire revisions unchanged. Malformed snapshots and corrupt or exhausted ownership clocks are retained for review instead of being silently reset.

Node Start/Stop/Restart now reserve that owner in the same transaction as their command. Lease checkpoints, retries and public current/retryable status verify it. A different mutation reservation invalidates the older lifecycle continuation even when placement and desired status did not change. Start/Restart also refuse an unresolved additional-node handoff before taking its authority. The reservation helper alone changes no service status, placement or container state. Production deployment creation, scale, sleep/wake and removal have not yet adopted this authority; their declared kinds and tests are a foundation, not accepted end-to-end integration. The proven late raw runSpec Start remains unfixed and the unresolved-transition hold remains required.

The related Linux suite passed 232 tests in eight files, including 31 new owner boundary/compatibility cases. All 65 database tests in 21 files passed at unchanged 100% statement, branch, function and line floors; this includes a real 0101-to-0102 migration with pending commands, deleted history, malformed JSON, replay and cascading deletion. Schema generation reported no additional migration. A Windows diagnostic hit V8 allocation failures and is not counted as acceptance. The first full Linux gate stopped at the new migration test cleanup guard in finally; validation now occurs before the protected cleanup, with no lint suppression or weaker assertion.

The exact production runtime passed 20 real Docker scenarios on Node 26.12.1: six direct dispatcher cases through sealed HTTP/SQLite/Traefik, seven two-daemon controller cases, and seven public API/worker lifecycle cases. These include actual readiness refusal, immutable effects, route confirmation/drain, hard termination, lost replies, same-command recovery and redeploy after manual lifecycle commands. The direct dispatcher waits for its persisted retry backoff; the multi-node fixture advances retry scheduling explicitly. Public DNS/TLS, production traffic and the delayed raw builder race are not accepted by those smokes. All 2,539 compiled files and 180 migration assets match the checked build byte for byte. The runtime cold-started as UID 999 with HTTP 200 and healthy application/database status; its first unconfigured JWT attempt is retained as a setup diagnostic.

The final frozen TURBO_FORCE=true pnpm release:check passed 15,115 tests in 901 files, no failures or skips, all 11 final tasks uncached, certifying committed code 1b9d0370. Typecheck, lint, build and all existing coverage floors passed. Server coverage was 93.27% statements, 88.01% branches, 91.39% functions and 96.36% lines; web coverage was 98.09%, 94.08%, 97.50% and 98.64%. The disposable check container removes only its own /.dockerenv marker for existing native-host installation assumptions; no source, threshold or test selection was weakened. All 2,539 compiled files and 180 migration assets copied from the successfully terminated gate also match the production runtime with zero changed, missing or extra files. Hosted CI for 1b9d0370 had passed Docker/PostgreSQL integration and image build when local acceptance completed; its full hosted gate was still running then. Hosted completion and publication remain separate requirements.

Next: add a persistent agent mutation guard covering the complete create/network/Start and retirement sequence; bind primary and additional-node builders to the persisted authority without denying a legitimate newer deployment; make both retained late-Start proofs pass; then integrate complete-generation Stop and settle the parent handoff under that same authority. Remaining R2 autoscale/sleep/wake, local-primary fan-out, network/mount, Swarm, mixed-agent and publication gates stay open. R3–R6 remain separate future releases; the published stable remains 0.18.2. Evidence: .temp_files/r2-runtime-owners-20261026-4959edfa/.

Admitted creation drain and child closure — 2026-10-26

Commit 35eb0b01 puts already-admitted managed runSpec and runEnv work under the same per-service native SQLite guard as ordered lifecycle effects. The guard spans create, network attachment, Start and failure cleanup. A timed-out caller keeps its queued work and lock until the underlying child closes. The process helper now waits for close after timeout or a post-spawn error, preserves exit 124 for a timed-out child even if its later exit code is zero, and leaves SIGKILL escalation armed. True spawn failures still return 127. Other service keys remain independent and legitimate newer deployments retain their compatibility path.

The original creation source failed all three new ordering cases while four controls passed. The original timeout helper failed three closure tests while fifteen controls passed. An isolated real Node child ignored SIGTERM: the original helper returned timeout and allowed its successor while that child was still alive. The corrected helper waits for actual closure after SIGKILL. The covering suite passed 206 tests in nine files independently on Windows and Linux. These are related checks, not global coverage.

The permanent creation smoke passed three real-Docker/native-SQLite scenarios through independent agent processes: multi-network runSpec, single-network runSpec and runEnv. Each pauses immediately before a real Docker mutator, proves that the fence waits for the admitted creator, discovers its immutable identity after the fence, completes Stop without resurrection, and starts a legitimate newer deployment. The same fixture failed against the prior runtime because its fence acknowledged the paused creator immediately. A fourth case uses the production process helper with an isolated executable substitution and a real SIGTERM-resistant Node child; it is process evidence, not an additional Docker service scenario. The smoke is now a release gate. These invoke registered production operations with a sealed context; they do not exercise sealed HTTP or public lifecycle endpoints themselves.

Separate exact-runtime fixtures passed seven public API/worker lifecycle cases and seven two-node sealed-HTTP/SQLite/Traefik controller cases. The initial host drivers disappeared before completion; their partial outputs are retained as diagnostics and not accepted. After owned-resource cleanup, independent tracked drivers completed with exit zero and persisted their receipts. Public DNS/TLS and production traffic were not tested. The production runtime cold-started as UID 999 with HTTP 200 and healthy application/database status. All 2,539 compiled files and 180 migration assets copied from the successfully terminated full gate match that runtime byte for byte.

The uncached frozen TURBO_FORCE=true pnpm release:check on committed code 35eb0b01 passed 15,127 tests in 902 files, no failures or skips, all 11 final tasks uncached. Typecheck, lint, build and all unchanged coverage floors passed. Server coverage was 93.28% statements, 88.01% branches, 91.42% functions and 96.37% lines; web coverage was 98.09%, 94.08%, 97.50% and 98.64%. The disposable gate removes only its own /.dockerenv marker for existing host-install assumptions; no source, threshold or selection was weakened. Its original host log stopped before the final result; the terminal Docker container exited zero and supplied the recovered complete log. No suite was restarted to recover observation. Hosted CI for 35eb0b01 also completed successfully, including real Docker/PostgreSQL integration and image build.

The remaining ordering boundary is a stale request that reaches the agent only after the newer Stop has completed. A creation lock cannot distinguish that request from a legitimate newer deployment without an explicit persisted authority. Next is typed admission for immutable batch preparation, fencing and retirement, binding all production callers to the shared owner, and full retained-generation handoff settlement. Keep the current unresolved-transition hold until those delayed-request and public recovery cases pass. Remaining R2 autoscale/sleep/wake, local-primary fan-out, networks/mounts, Swarm, mixed-agent and publication gates stay open. R3–R6 remain separate future releases; stable remains 0.18.2. Evidence: .temp_files/r2-agent-creation-drain-20261026-31b578b8/verification-state.json.

Agent absence evidence: real runtime check — 2026-10-28

A disposable derivative of ninedeploy-proof:r2-prepare-replay-final-20261027, with the compiled agent absence and mutation modules from 9aeb7230, completed smoke:runtime-mutations with exit 0. All ten scenarios passed using an isolated Docker daemon, native SQLite, sealed HTTP and independent agent processes. They include a delayed older preparation refused after Stop, a later typed deployment accepted, Stop over multiple generations and the fence draining an admitted Start. Evidence: .temp_files/agent-absence-proof/runtime/cases.json; derivative image ID: sha256:d187f6b749afed80a95656adcf74e2d8c67d7b63c37431288d51455bcf835ec6.

This is agent protocol evidence, not a full build of the current source, a published image, public-panel integration or production traffic. Normal deploy and fan-out still require the ordered producer integration; the previously reproduced Stop race remains open. The uncached release check on 73157601 failed its unchanged server statement coverage floor (93.19% against 93.25%) despite 11,032 passing server tests; the later agent changes were not included in that run.

Readiness deadline and receipt validation — 2026-10-28

Commit 5327d775 rejects readiness received after the deadline or after ownership is lost, and requires an unambiguous successful node probe for the requested protocol and HTTP status. The local Docker and node/fan-out regression suites passed, including the complete 259-test pipeline file and 44 Swarm/native recovery tests after correcting their simulated Docker missing-container responses. The permanent smoke:local-docker-readiness exercises fast and slow HTTP responses for both explicit and default panel readiness and an ownership change; the release workflow runs it against its pushed image.

A disposable derivative image (sha256:4a33bdd0d366b004f175a296c0f801d670a9776843822be96b91a486844d22c4) with the updated server and corresponding schema/database outputs completed the real-Docker user journey with exit 0. Its canary held at 50% while 30/30 requests were served with no 5xx, promoted successfully, then retained the previous release through a cancelled canary with 5/5 requests served. The journey also verified scale-up/down, and intentionally received a 503 for a wake request whose client must retry. Evidence: .temp_files/agent-absence-proof/readiness-user-journey-v2.log. An earlier partial overlay failed to boot due to mismatched schema exports; it was corrected before this run and is not accepted as proof.

These checks are for a derivative fixture, not the final release image, public DNS/TLS or production traffic. The real-Docker agent readiness smoke also passed four cases on a separate derivative. Default node readiness without an explicit HTTP/TCP mode remains container-state based. The node deploy/Stop ordering race and the final exact-commit release and hosted publication checks remain open.

Stop during an unselected node deployment — 2026-10-29

The v0.18.9 tag points to 5e9a329dcdf697cdece7b7db2e2e4734e017970e; its published candidate image has index digest sha256:c1c0b0c427b004abfbe129f03bd1deb1de1816639125cd3bdcc868c150068b20. This is an image candidate, not a GitHub Release. Release workflow 40939573409 failed its public-canary and replica-database Stop scenarios and skipped promotion and npm publication. Upgrade, user-journey, Compose restore, agent and panel lifecycle cohorts passed in that run.

Both failures were reproduced on the same candidate image in isolated local Docker daemons. In the replica case the Stop command completed and the selected old replicas stopped, while both cancelled candidate replicas remained running. The cancelled deployment log states that node deployment authority changed and runtime effects were retained for the new owner. In the public canary case the stopped candidate likewise remained running. Local evidence is retained under .temp_files/replica-v0189-diagnostics-20261029/ and .temp_files/canary-v0189-repro-20261029/; these are isolated fixtures, not production traffic.

The unfinished regression apps/server/test/lib/nodeLifecycleStopCandidate.test.ts reproduces the selected-runtime case through the durable controller with two candidate replicas and a foreign-service control. On the current source it fails because both candidates remain running after Stop reports completion. On Node 26.12.1, the existing lifecycle intent, controller and runtime mutation suites passed 137 tests; the new regression was the one failure. This is a focused result, not a release gate.

Stop must durably take ownership of all affected immutable generations, drain or fence admitted creation, remove the cancelled candidates without mutating a foreign service or a newer generation, and retain enough state for retry after a crash. Production deploy creation must use the ordered mutation protocol before its fence can safely reject delayed old requests without rejecting future valid deploys. Verify the unchanged regression and both real-Docker public scenarios, including a delayed old create and a later legitimate deploy, before claiming this race fixed or promoting another release.

The development safeguard records interrupted deployment IDs in the Stop snapshot and holds a selected Stop after routing removal and selected-generation shutdown, rather than declaring completion before candidate cleanup. Repeated Stop retains those IDs; Start and later deployment admission wait for the unresolved handoff. A local derivative of the candidate image containing the four changed server modules confirmed the Stop remains in applying after the real PostgreSQL replica scenario, while both candidate replicas still run. The unchanged smoke failed because completion did not occur; it did not pass or prove cleanup. Evidence: .temp_files/replica-stop-hold-proof-20261029/. This safeguard cannot settle the handoff by itself and is not an acceptable release replacement for ordered candidate retirement and subsequent legitimate deploy acceptance.

The agent now records the highest deployment identity covered by an accepted lifecycle Stop under the same durable service lock used for managed creation. Later legacy creation at or below that identity is refused, even after a newer Start changes the lifecycle head; a later deployment identity remains admissible. A stale lifecycle request cannot advance the recorded boundary. Against the original v0.18.9 image the extended real-Docker smoke failed because a delayed stopped generation started; a derivative with the three modified agent modules passed all three creation modes, using independent agent processes and native SQLite, and admitted a later generation. The existing drain and process-timeout controls also passed. Evidence: .temp_files/agent-stale-create-red-20261029/ and .temp_files/agent-stale-create-green-20261029/. These are local agent proofs, not sealed HTTP, public-panel acceptance, an upgrade matrix or release artifacts. The panel still needs to fence the complete interrupted generation range and retire its candidates before completing Stop; the held-handoff guard remains in place.

For a selected Stop that interrupts a deployment, the panel now persists the deployment sequence high-water mark, including IDs whose rows were already deleted, and sends that boundary in the initial lifecycle fence. It requires the explicitly advertised docker.lifecycle.creationFence capability before the handoff; an old agent keeps the command held. Ordinary selected Stop without an in-flight deployment retains its existing agent compatibility. This is panel/agent contract coverage only: panel-side retirement and automatic settlement are not implemented, and no full public Stop or release acceptance is claimed.

The development agent also exposes a sealed docker.lifecycleRetire operation bound to an acknowledged Stop revision and immutable target IDs. Its durable retirement intent refuses changed target sets and a later lifecycle owner; removal is checked before and after the Docker effect and an exact retry tolerates a previously removed ID. A real-Docker RED on the original candidate image reported the operation unavailable; a derivative containing the changed agent/schema modules passed the expanded generation smoke, including foreign-container protection, process-independent retry, a same-name replacement and rejection after a newer revision. Evidence: .temp_files/agent-lifecycle-retire-red-20261029/ and .temp_files/agent-lifecycle-retire-green-final-20261029/. This proves the agent operation, not panel integration: the public Stop still holds until the controller discovers, stops and retires candidate identities, and no release has been promoted.

The selected-runtime Stop controller now freezes candidate names and placements, fences the pre-Stop deployment range, confirms all immutable identities, stops selected and candidate generations together and retires only candidates under the same acknowledged Stop. A lost or invalid retirement receipt holds the command for an idempotent retry. The prior failing controller regression passes, including lost-reply and unavailable-agent cases; two-node selected and unselected target shapes have focused controller tests. In a local derivative image, the public panel/worker/agent canary smoke passed eleven cases, including explicit Stop and panel crash, with 10,509 sampled requests and no unexpected 502. An independent PostgreSQL replica smoke passed Stop during readiness, removed both candidates, then deployed two new replicas and served 30 successful requests. Evidence: .temp_files/public-stop-handoff-final-20261029/ and .temp_files/replica-stop-redeploy-20261029/. The image is a derivative of the tagged candidate and does not certify the final commit, hosted release, public DNS/TLS or the two-node physical path. Initial deployment before any selected runtime and wider node mutation ordering remain separate acceptance work.