Dynamic Execution Sources, Model Discovery, Multi-Account & Security (D-053, #116) #124
No reviewers
Labels
No labels
accessibility
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
P1
P2
P3
question
wontfix
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Creatidy/scarcity-router!124
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "program/dynamic-execution-sources"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Implements program #116 (children #117–#123) — architecture decision D-053 in
docs/decisions.md.What this does
The user configures an ExecutionSource once (kind
codex_subscription, label, owning worker, entitlement, adoption policy — never a model slug). The worker discovers what its controlled runtime can execute and reports a bounded ModelInventory (worker-protocol v2). The server classifies against the reviewedmodel-tracks.json, adopts conservatively (floor-level capability inheritance only), materializes exact derived resources (<source_id>:<slug>), and the unchanged routing core selects and pins exactly as before.params {}onaccount/readand the firstmodel/listpage (method-specific, evidenced by the real 0.155 runtime); discriminating-32600test coverage; the fake App Server now mirrors the real strictness.codex:<source_id>), per-source controlled homes (0o700), one worker process, strict isolation, default per-source quota pools (sharing only via explicit pool id),--codex-source+ officialcodex-logincommand.ultraeffort.worker authenticated (pairing)vssource authterminology, durable UX review gate.security_critical→ independentgpt-daybreak-blue-latestreview gate withSECURITY_REVIEW_UNAVAILABLEsemantics; no silent substitutes.Contract reconciliations (explicit, in D-053)
quota_pool_ids(pools compose from registrations; observations can never alter pool membership).Verification
make checkgreen (2022 tests + basedpyright 0/0/0),make package-checkgreen,make guardrailsgreen,git diff --checkclean.codex-cli 0.155.0-alpha.16.3answersaccount/read {}→ honestauth_required(pre-fix:-32600→unknown).Open external gates (per program contract)
gpt-daybreak-blue-latestsecurity review, orSECURITY_REVIEW_UNAVAILABLE.Frozen v1 surfaces untouched (additive effort vocabulary only). No merges, no tags, no release.
codex-cli 0.155 rejects an absent params member on these methods with JSON-RPC -32600 (live evidence 2026-09-24), degrading every probe to auth_unverified/unknown health on a logged-in controlled home. The adapter now sends the evidenced method-specific shapes: account/read and the first model/list page carry params {}, later pages carry {"cursor": ...}. No global params-forcing. The fake App Server now mirrors the real 0.155 strictness, so a wire-shape regression fails the suite; new tests pin the exact first-page, later-page and -32600 shapes.Sources page (/admin/sources, /control/sources GET/POST/DELETE): add a Codex source by label + owning worker — the source_id derives from the label and the user never configures a model slug; each row shows the source auth state, detected models with per-model state, the routable/restricted/error counts, and the ONE login command to run on the worker host. The page states explicitly that 'source auth' is the PROVIDER account sign-in, distinct from the worker pairing. Worker label-first everywhere: the workers page and the resource-add dropdown show the friendly label before the technical id, and the doctor renders 'worker <label> (<id>)'. The resources ladder renders 'worker authenticated (pairing)' so worker pairing is never mistaken for provider authentication. Governance (durable): AGENTS.md and docs/llm-operating-policy.md make UX a mandatory review dimension ('UX impact: none' or a ten-point assessment; a material UX regression is CHANGES_REQUESTED) and freeze the security-critical review gate: security_critical -> independent gpt-daybreak-blue-latest review; SECURITY_REVIEW_UNAVAILABLE when the restricted access cannot be exercised; no substitute model closes the gate; the multi-stage review itself stays outside the Routing Core.Independent review (GLM-5.3-Flash, thinking=MAX) — remediation recorded
Verdict was
CHANGES_REQUESTEDwith 2 merge blockers + 8 defers. One remediation pass landed in786e616; the reviewed head is now786e616.Blocker 1 (slug-bound poisoning + session death) — fixed at all three layers:
SLUG_MAX_LENGTH = 40): a listing carrying an over-long slug is structural drift and fails the WHOLE observation closed at the worker (unverified, no models) — discriminatingly tested.adoptedbefore the derived id exists.ERR_INTERNALframe exactly like the resource section — the session thread can no longer die unhandled; test added asserting session survival and continued answerability.Blocker 2 (models never appear after login) — the sources page and
sources_viewnow surface BOTH worker-side actions (codex-login --source <id>ANDrun --codex-source <id>), in the row and the page copy.Defers fixed in the same pass:
:rejected insource_id; derived-namespace guard covers every configured resource; max 8 sources per worker (protocol inventory cap);codex_subscriptionentitlement pinned tosubscription_included; strict canonical-timestamp regex; deadCODEX_SURFACE_CAPABILITIESduplicate deleted; ASCII-folding for label→id derivation; new discriminating tests throughout.Defers recorded (open, non-blocking): cross-section atomicity (inventory applied before resource-section rejection — within the source grant, protocol-doc wording updated);
worker→v1-only-servercovered at thenegotiate_versionunit level only.Post-remediation gates:
make checkgreen (2026 tests; one load-sensitivetest_e2e_executionscenario-11 cancellation flake observed once under full-suite pressure — module and coordinator untouched by this branch, 9/9 green in isolation and suite reruns; no retry added, recorded per the no-flake policy), basedpyright 0/0/0, package-check re-run in progress,git diff --checkclean.UX impact: assessed per the new mandatory gate — the reviewer's material UX finding (silent stall after login) is fixed with the two-command remediation; label-first and auth-terminology items pass.
Continuation: develop merge + SSH-safe source authentication + live acceptance evidence
Canonical merge.
origin/develop@cebd1d64(PR #125, operational-property governance) merged intoprogram/dynamic-execution-sources(18370ee). Documentation conflicts auto-resolved: AGENTS.md anddocs/llm-operating-policy.mdnow carry BOTH #125's preserve-proven-operational-properties principle / operational-UX-and-compatibility review AND #124's mandatory UX review dimension + security-critical Daybreak gate — no duplication. Post-mergemake checkgreen (basedpyright 0/0/0).SSH-safe authentication correction (implemented,
46d02df). The exact supported invocation was VERIFIED from the installed runtime, not assumed:codex-cli 0.155.0-alpha.16.3 login --helpadvertises--device-auth, and the flow was exercised live (printshttps://auth.openai.com/codex/device+ a one-time code, polls — no localhost callback, no browser on the host).run_official_codex_loginnow: creates/validates the SOURCE's own controlled home (codex-sources/<source_id>/, 0o700); probes the installed CLI for the SSH-safe mode (fail-closed if absent — the browser/localhost login is NEVER attempted); runslogin --device-authwithCODEX_HOMEbound to that home; and on any refusal leaves the sourceauth_requiredwith an actionable message. No fallback to the legacycodexhome, none to~/.codex, and no credential is ever copied/imported/inspected between homes. Discriminating tests pin the exact two-invocation shape (probe → device login), the no-fallback gate, per-source isolation, env naming only the source home, no--with-api-key/--with-access-tokenargv, refused-login closure, and independent logins per source.Live acceptance evidence (real runtime, real SSH flow, twice). Fresh source home; one explicit
scarcity-router-worker codex-login --source personal-openai: the device code printed on the SSH terminal; the code expired without the browser step (owner unavailable) and the CLI timed out — the fail-closed contract held both times: exit nonzero, the actionable "source stays closed (auth_required)" remediation printed, no credential material in the home, and the product-path probe afterwards reportsauth_required/ zero models / empty derived resources. What remains for full authenticated acceptance is exactly one owner browser step (the printed URL + code) followed by product-path discovery and one real turn — the flow is proven up to that step and fails closed without it.PR left open; not merged. Stopping here for fresh independent review of the new head
18370ee.Fresh independent review (GLM-5.3-Flash, thinking=MAX) of continuation head
18370eeVERDICT: APPROVE (5 non-blocking DEFERs).
Key confirmations:
codexhome, or~/.codex; no credential material is ever read/copied/parsed between homes. The reviewer independently re-verified the installed CLI advertises--device-auth.auth_required, zero models) — proven by a real adapter observation over the refused-login state, not a mock.46d02dfpreserved every proven operational property of the SSH working path and repaired the one violated property (headless auth) with the vendor's supported mode, verified against the installed binary.Recorded DEFERs (non-blocking): (1)
CodexIneligiblefrom a corrupt home surfaces as a traceback instead of a clean CLI message; (2) capability probe discards stderr and has no timeout (safe direction, but a hung binary would hang the probe); (3) the per-worker source cap hardcodes8instead of importingMAX_INVENTORIES_PER_REPORT; (4) missing tests for the probe-failure branch and hostile inheritedCODEX_HOMEoverride; (5) mid-loop sink failure wording overstates per-report atomicity (per-inventory validation still holds, retry converges).Acceptance status: SSH-safe flow verified live end to end up to the owner's one browser step (two live device-code cycles; timeout fail-closed proven twice; no credential residue; product-path probe reports
auth_required). The authenticated turn remains gated on that single owner step. PR #124 left open for owner merge — not merged.Authenticated acceptance attempt — blocked at the one owner step; source correctly stays closed
Diagnosis of the completed login. The login the owner completed at 20:37 CEST landed in
~/.codex/auth.json(a default-environmentcodex login), NOT in the controlled source home. Per the program's hard rules (no~/.codexfallback, no credential copy/import/inspection between homes), Scarcity Router must not adopt it — and did not.codex login statuswithCODEX_HOMEbound to the source home reports Not logged in; the product-path probe reportsauth_required, zero models, zero derived resources. Fail-closed verified.Three controlled-home device codes were issued and expired unused (
W3XH-LR8CI,W5QP-1SVNH,XO4R-8ZO8N— each a fresh, correctly-boundcodex-login --source personal-openairun). Every timeout exercised the fail-closed contract live: exit nonzero, actionable remediation printed, no credential residue in the home.Consequence per the owner's own gating: the authenticated live turn cannot run, therefore the
gpt-daybreak-blue-latestreview cannot run either (stillSECURITY_REVIEW_UNAVAILABLE), therefore the final gates→merge condition is NOT met. PR #124 stays open and unmerged.Exact resume path (two steps, ~2 minutes while a code is live):
scarcity-router-worker codex-login --source personal-openai— a freshhttps://auth.openai.com/codex/deviceURL + code prints (do NOT run barecodex login; that writes~/.codex, which the router will never read).scarcity-router-worker run --codex-source personal-openai.When that login completes, the remaining acceptance (product-path discovery → one real authenticated turn → Daybreak review attempt → final gates → owner-authorized merge) can proceed immediately.
Authenticated live turn: PASS. Daybreak review: FAIL (5 blockers) — remediation cycle starting
Authenticated live turn (real, end to end): PASS. With the owner-completed device login in the controlled home: product-path discovery reported
authenticatedwith the live inventory (5 routable: gpt-5.6-luna/sol, gpt-6-luna/sol/astra; 1 restricted: gpt-daybreak-blue-latest; 2 discovered: gpt-5.5, gpt-5.6-terra). One real request through the composed server + real worker + real Codex:POST /v1/chat/completionspinnedsr-pin:personal-openai:gpt-6-sol/openai/gpt-6-sol/high→ 200, contentLIVE-ACCEPTANCE-OK, real provider usage (prompt 18333 / completion 11 tokens), auditexecuted_target== the pin exactly (personal-openai:gpt-6-sol,gpt-6-sol,high).Daybreak review: RAN (access confirmed:
gpt-daybreak-blue-latestexecuted in the controlled home) and returnedDAYBREAK-VERDICT: FAILwith 5 reproduced blockers — the gate caught real defects:highpin with requestreasoning_effort=lowdispatches low while audit records high (coordinator dispatches the request effort, not the selected identity's variant).retiredgrows without a lifetime cap; observations for retired resources persist.Remediation cycle starting now (authorized: concrete merge blockers found). PR stays open and unmerged until all five are fixed, gates re-run, and the Daybreak review re-passes.
Daybreak security review round 2 — executed in 3 segmented passes (gpt-daybreak-blue-latest, real controlled home). VERDICT: FAIL
The model executed successfully through the adapter's managed session path (network disabled, scratch-cwd isolation). Findings, verbatim scope per segment:
Segment 1 (fix verification + isolation + source-id + spoofing):
_config_is_containedaccepts a notice followed by[mcp_servers.evil]; project-trust containment uses string-prefix matching (scratch-elsewherepasses); tree creation/validation follow symlinks, so one source dir pointing at another collapses both CODEX_HOMEs (worker_codex_adapter.py:550-578, 614-632; worker_identity_store.py:562-575).SLUG_MAX_LENGTHomits the effort suffix:<source>:<slug>:mediumcan exceed the 64-char safe-id bound; adoption mutatesstate.adoptedbefore id validation (model_inventory.py:68-97; execution_sources.py:253-270); worker preflight checks only source+slug.display_nameis a 128-char free-text carrier that can hold emails/account-ids/token fragments; the official producer omits it but the wire contract does not enforce that (model_inventory.py:176-215, 230-237).Segment 2 (quota pools, ownership, metadata, audit, TLS):
quota_pool_ids: observation matching excludes pools (by design) but the stored/serialized snapshot retains the worker-supplied values, producing contradictory registry documents (resource_state.py:682-704, 1142-1155, 1596-1612).Segment 3 (exact execution, auth honesty, SSH login):
Noneand the turn omits effort entirely (runtime default executes) while audit records the pinned variant; legacy codex mode never binds the selected variant; a compromised coordinator could send variant=high + effort=low — the source branch fills absent efforts but never rejects present mismatches; audit copies selected→executed before dispatch (gateway_coordinator.py:696-720, 711-724; worker_codex_adapter.py:1749-1752, 1779-1790, 2068-2069).[mcp_servers.*]and prefix-trick paths; the rewrite path preserves arbitrary pre-project content (worker_codex_adapter.py:515-520, 568-578, 580-612).Status
The full reviewer output is preserved at
/tmp/daybreak-review/segmented-output.mdand summarized here as the durable record. Next step (awaiting owner direction given scope): a dedicated remediation cycle for the eight blockers — config-containment hardening (real TOML allowlist + symlink-resistant validation), combined-id length accounting, eager-materialization bounds, cross-report duplicate handling, quota-pool observation stripping, display_name removal or strict allowlist, end-to-end effort binding (including legacy mode and audit truthfulness), and snapshot/inventory ordering (refresh before collection).Authorized security-remediation pass complete — head
9789c67; Daybreak verification verdict and final statusAll eight blocker classes from the durable Daybreak record were remediated, each with a discriminating regression that fails on the reviewed head
4d2d238and passes after:tomllib): only the generated notice plus[projects]tables survive; every project path must RESOLVE under the source's own scratch root (prefix tricks rejected); project tables accept onlytrust_level(nested[...mcp_servers.*]injection rejected — found by the verification pass and closed in the same pass); homes/scratch/config arelstat-validated so symlink aliasing cannot collapse two sources' homes. Foreign content resets the config to the pristine notice; credentials are never read or rewritten.SLUG_MAX_LENGTH=35(20+1+35+1+7=64 with the effort suffix); every per-effort id is validated BEFORE adoption mutatesadopted/routablestate.quota_pool_idscan never survive into serialized/read-model state.display_nameremoved from the inventory wire contract; unknown keys fail closed, so no free-text carrier exists.effort_conflicts_with_target) BEFORE dispatch; the codex adapter binds/rejects likewise; audit selected==executed. Variant-less resources (loopback Ollama) keep carried-control semantics — preset wire mapping stays authoritative (verified: the Ollama preset has no evidenced mapping formax).reported_atis captured after the refresh (a report can never pair fresh-healthy snapshots with auth loss, nor carry a future-dated observation); the endpoint additionally drops source-derived snapshots whose source inventory is unauthenticated in the same report.Gates at
9789c67:make checkgreen (2,044 tests, basedpyright 0/0/0),make package-checkgreen,make guardrailsgreen,git diff --checkclean. One pre-existing load-sensitive cancellation flake (scenario_11) appeared once under full-suite pressure — 3/3 green in isolation, module untouched, no retry added.Daybreak narrow verification — executed but environmentally crippled.
gpt-daybreak-blue-latestran (adapter-session path), verified blockers 2 and 3 VERIFIED, then its own bubblewrap sandbox failed on the WSLg host-mount (/mnt/wslg/distro— the environment artifact excluded from remediation), after which it repeated stale pre-remediation findings with old line numbers rather than inspecting the current code. Its one genuine new technical point — nested TOML tables under contained paths — was real and is now closed (see fix 1 + regressions).Per the owner's instruction, I am stopping here and escalating: the single authorized verification pass was performed but could not complete honestly on this host due to the sandbox environment artifact, so a clean verdict could not be recorded. PR #124 stays open and unmerged at
9789c67. Options for the owner: (a) run the same narrow verification charter (preserved in this comment history //tmp/daybreak-review/charter.md) on a host without the WSLg bubblewrap issue; or (b) accept the documented evidence (all eight blocker classes closed with discriminating tests, full gates green, one environment-blocked verification) and decide the merge directly.MERGED — owner-authorized. PR #124 merged into
developas merge commita4370f0from head9789c67(exactly the authorized head).Preconditions verified at merge time, in a clean worktree at
9789c67:make check: 2,044 tests, OK (the local checkout's failing pre-merge run was caused by uncommitted owner WIP present in the working tree — D-054 effort-restriction edits, intentionally left untouched and excluded from this merge; the clean worktree proved the committed head is green).make package-check: PASS.make guardrails: PASS. basedpyright: 0 errors / 0 warnings.git diff --check: clean.Recorded per the owner's exception: the final Daybreak remediation verification was incomplete due to the known WSLg/bubblewrap reviewer-environment failure — this is an owner-approved waiver and is NOT recorded as a Daybreak PASS. All eight blocker classes were closed with discriminating regressions and bounded evidence (owner-accepted).
No merge to
main, no release, no deployment. Program #116 children (#117–#123) can be closed at the owner's leisure.