Gateway effective limits + Codex output-token semantics (D-058, #136) #152
No reviewers
Labels
No labels
accessibility
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
P1
P2
P3
question
wontfix
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Creatidy/scarcity-router!152
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "gateway/effective-limits"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #136 (child D of program #132) when merged. Implements the D-056 effective-capability intersection end to end and records its settlement as D-058. (Decision id was originally recorded as D-057; renumbered to D-058 after PR #149 merged D-057 for the reasoning-effort serialization contract — see the rebase note in the comments.)
What changed
resource_state.py,gateway_coordinator.py):effective = min(known of: model hard, channel ceiling, administrator allowance); UNKNOWN when no capability input is known — an administrator allowance alone never certifies a capability. New channel facts:ExecutionCapabilities.output_limit_tokensandoutput_limit_control.context_length_exceeded/output_limit_exceeded, never clipping). The administrator global pre-check stays authoritative; capability shortfalls remain the routing core'sno_eligible_target(routing core untouched, per the frozen complexity boundary).worker_codex_adapter.pybackstop + coordinator rule): re-evidenced the CURRENT runtime first (2026-09-28,codex-cli 0.155.0-alpha.16.3): the app-server turn contract has no output-token control (stable or experimental — live probes: candidate fields are silently ignored, genuinely gated fields error positively). Consequence per the frozen scope: a requested explicit limit on that channel is normalized away ONLY when non-binding (≥ the model's provenhard_properties.output_tokens, auditedoutput_limit_normalized); a binding request is rejectedoutput_limit_unenforceable; UNKNOWN hard max is never normalized;generation_paramsstay refuse-not-drop.model/listhas no context members;modelProvider/capabilities/readis feature booleans; today's docs publish no number) and the original M06 citation never contained 272000 — provenance gap documented. The 272000 fact is RETAINED as the owner-reviewed registration fact; client-visible effective context is always the intersection.GET /v1/modelsadvertises the STRONGEST bound route's effective ceiling as the headline (effective_context_limit_tokens,max_output_tokens) plus an additivex_scarcity_router.routesper-route detail array (nullwhen unknown) — never weakest-route constrained, never strongest-only claimed, never UNKNOWN as a number. Supersedes D-055's interimmax_output_tokensreading as that decision anticipated. Child H consumesroutes.gateway_contracts.py): 4 MiB body / 272000 input context / 131072 output / 1200 s execution — evidence-documented in the class docstring and D-058; export honesty verified (non-default only; an explicit legacy16384stays explicit).Tests
New
tests/test_effective_limits.pypins the issue's test list at the HTTP boundary: default supported-model accepted; explicit lower administrator ceilings enforced (context + output); large-but-bounded accepted; over-limit rejections unchanged; channel < model → effective enforced and advertised (incl. per-route metadata); non-binding normalization above proven hard max (normalized + audited); binding-below-hard-max rejected closed; UNKNOWN hard max never normalized; body/limit boundaries; export honesty. Child-A output-limit pin (test_representative_client.py) flipped to the new contract; codex adapter pin updated to the typed backstop; security 413 boundary moved to the new default.Docs
docs/decisions.md: D-058 (intersection, enforcement, normalization, aggregation, defaults, alternatives) — after the merged D-057.docs/codex-adapter-stage1-evidence.md: 2026-09-28 re-evidence section (runtime version, probes, findings, provenance note).docs/execution-surface.md: contract rows → Implemented; effective-capability section states the implemented rules.Gate
Rebased onto
develop@81b8e63(PR #149's merge); head9eec9a8;origin/develop..HEADis exactly this one commit, no merge commit. Full gate green on the rebased head:make check(full suite, including #149's tests),uv run basedpyright(0 diagnostics),git diff --checkclean. One full-suite failure observed during earlier development was the known pre-existing #143 flake (test_scenario_11_client_disconnect_propagates_cancel, cancel-assert race under load): passes in isolation, path untouched by this diff.UX assessment: material positive — the supported workload's requests (explicit 128k output ceilings, agent-sized conversations, long reasoning turns) now succeed by default; rejections carry actionable typed codes; discovery shows honest per-model and per-route numbers. No removed limits; no unbounded values.
Non-goals honored: nothing removed, no unbounded values, no tool-call work (#137), no admin UI.
Independent review round (bounded review lifecycle; reviewed head frozen at
cdf4ac6):VERDICT: APPROVED — no MERGE_BLOCKER findings. Reviewer model: GLM-5.3-Flash (independent session). Full gate re-verified green on the frozen head (2115 tests, basedpyright 0 diagnostics,
git diff --checkclean,origin/develop..HEAD= exactly the one scoped commit).UX impact: assessed, no unnecessary friction — new deployments admit the supported workload out of the box; in-limit requests unchanged; two intentional typed-error surface changes (binding limit on a no-control channel → 400
output_limit_unenforceable; above-calibrated-output → 503no_eligible_target), both test-pinned; upgrade is safe (no limits block → larger documented defaults; an explicit legacy16384stays enforced and exported).DEFER findings (recorded for the owner; none block merge):
sr-pin:. Fail-closed and honest; child H / a future routing-core change should own route-steering._model_hard_limitstakes the min across all calibrated variants of the identity rather than the exact selected variant's entry, so the normalization threshold uses the weakest variant's proven maximum — conservative direction only (can over-reject, never over-permit); fixture and real catalogs carry uniform hard properties across variants today. A tighter reading of "the exact selected model" would use the selected variant's entry.output_limit_normalizedaudit note rides only the RESULT_COMPLETED record; if a normalized request subsequently fails/cancels/times out, that attempt's record does not carry the note. Minor audit-fidelity gap on a path the D-057 rule does not pin.Non-findings confirmed by the reviewer:
gateway_openai.py/gateway_server.pytouches are required by scope item 5 and additive; the untrackedlitellm/directory in the local worktree is unrelated to this commit and is not part of the PR.Per the bounded lifecycle: no blockers → no remediation round; final verification = the gate above; the PR stays open at the human merge gate.
cdf4ac6aab9eec9a8601Gateway effective limits + Codex output-token semantics (D-057, #136)to Gateway effective limits + Codex output-token semantics (D-058, #136)Rebase + decision renumber (owner-requested conflict fix).
PR #149 (reasoning-effort serialization, #148) merged its own D-057 to
developwhile this branch was in flight, so this PR's decision collided with it. Resolution, per the repository's no-sync-merge rule:git merge origin/developwas aborted (no synchronization merge commit enters this PR) and the single commit was rebased ontodevelop@81b8e63with--force-with-lease.docs/decisions.md, where both branches appended their "D-057". Resolved by keeping the merged D-057 verbatim and renumbering this PR's decision to D-058.docs/execution-surface.md(4),docs/codex-adapter-stage1-evidence.md(1),gateway_openai.py(2),gateway_contracts.py(1),gateway_coordinator.py(4),tests/test_effective_limits.py(2). The D-057 references inselector.py,docs/machine-interfaces.md,docs/selection-policy.md,docs/architecture.mdand their tests belong to the merged #148 contract and were left untouched.9eec9a8(content identical to the reviewedcdf4ac6except the rebase, the renumbering and the commit message). Full gate re-run green on the new head:make check(full suite incl. #149's tests),basedpyright0 diagnostics,git diff --checkclean;origin/develop..HEADis exactly the one scoped commit, no merge commit. The earlier APPROVED review verdict and its three DEFERs carry over — the rebase introduced no logic changes.Fresh independent exact-head review (reviewed head
bc96c10vs develop81b8e63) — VERDICT: CHANGES_REQUESTEDReviewer: GLM-5.3-Flash (independent session). The prior APPROVED was treated as invalid (head/base had changed). The reviewer verified all four earlier remediations against the code (pre-ranking output eligibility incl. the weak/strong steering scenario, exact-variant hard properties, defaults-vs-channel-capability separation, audit provenance across all six terminal states, D-057 untouched, D-056 exact-execution invariants, all gates green) and raised:
output_limit_control: falseCodex surface, an unpinned request with an explicit binding sub-maximum limit (e.g.max_completion_tokens: 64000of an advertised 128000) is excluded at every route pre-ranking and surfaced as generic503 no_eligible_target— contradicting this branch's own D-058 point 4 / execution-surface claims, and non-actionable (503 suggests transient).exposed_logical_modelscould hide a bound routable model in a mixed reviewed/floor-variant corner case, and its "Unreachable" comment gave wrong reasoning.Remediation (head
35cc72b, per the bounded single-remediation lifecycle)_no_eligible_target_errornow inspects the decision's target exclusions — when routes were excluded on the OUTPUT dimension, the client gets the actionable typed 400 (priorityoutput_limit_insufficient>output_limit_unenforceable>output_limit_unknown, matching the pinned path); any other exclusion shape keeps the honest 503. The headline wording in D-058 pt 4 and execution-surface now states the no-control boundary explicitly (an explicit binding sub-max limit on such channels finds no eligible route; omit the limit or request at/above the maximum).output_limit_unenforceable(not 503); unpinned non-binding 128000 normalizes and executes with the audit note; omitting the limit executes; the weak-only-world and representative 128001 pins moved from 503 to their typed 400s.Narrow final verification (blocker + obvious regressions only, per the lifecycle)
Full suite 2147 tests OK (the #143 flake did not fire),
basedpyright0 diagnostics,git diff --checkclean, guardrails 42 OK.origin/develop..HEAD= exactly three scoped commits, no merge commit; D-057 untouched; D-058 unchanged in number.Status: READY_FOR_REVIEW — PR open, unmerged, at the human gate. The reviewer's DEFER is resolved in the same commit; per the bounded budget no second full review was run after this contained remediation — the human gate holds the full record (verdict + remediation + verification) and decides.
Review round 2 (head
35cc72bvs develop81b8e63) — REQUEST_CHANGES (2 contained blockers) → remediated inf1f191b→ fresh narrow verification: both VERIFIEDBlocker 1 — D-058 capability facts erased by administrator mutations.
control_api._registration_copy()hand-copiedExecutionCapabilitiesfield-by-field and droppedoutput_limit_tokens/output_limit_controlon every mutation that rebuilds the resource record (enable/disable via_with_enabled, provider removal / endpoint detachment via_with_endpoint). Fix: the copy now uses the validated serializer round-tripExecutionCapabilities.from_dict(capabilities.to_dict())— resistant to the next additive capability field by construction. A repo-wide audit found this was the ONLY field-by-field capability copy site (all other paths already usefrom_dict). Regressions (test_d058_capability_facts_survive_admin_mutations): disable → re-enable preserves the facts; provider removal (endpoint detached) preserves them; the persisted/exported configuration document is value-equivalent.Blocker 2 — mixed-cause no-target misclassification.
_no_eligible_target_errorreturned a typed output 400 whenever ANY exclusion carried an output code; its contract reserves the mapping for decisions caused PURELY by output semantics. Fix: the 400 now requires a non-empty exclusion set where EVERY exclusion's reason codes are purely output codes; any availability/authorization/binding/other-feature exclusion keeps the ordinary503 no_eligible_target(the request could succeed unchanged once that cause clears). Regression (test_mixed_output_and_availability_stays_503): available 32k route (output-insufficient) + never-observed 128k route, request 64k → 503 with zero dispatches; after the strong route recovers, the same request executes there unchanged. The intended pure cases remain pinned: single insufficient route → 400output_limit_insufficient; Codex-onlycontrol=Falsebinding sub-max → 400output_limit_unenforceable; all-UNKNOWN → 400output_limit_unknown.Fresh narrow verification (independent GLM-5.3-Flash session)
Base re-verified unmoved at
81b8e63. Scope: exactly35cc72b..f1f191b(4 files). Fix 1 VERIFIED (copy path, no other hand-rolled copy, regression exercises both mutation paths through_registration_copyand would fail pre-fix, export equivalence). Fix 2 VERIFIED (pure-cause logic incl. priority order, all three intended 400 scenarios and the mixed 503 scenario classified as specified, regression includes the recovery half and fails on the pre-fix base). No findings.basedpyright0 diagnostics,git diff --checkclean, targeted suites 77 OK; full suite on this exact head: 2149 tests OK (the #143 flake did not fire); guardrails 42 OK.Status: READY_FOR_REVIEW — PR #152 open, unmerged, at the human gate. D-058 unchanged (develop did not move); D-057 untouched; #137 not started.
f1f191b45ee2f1624c66Rebase onto the advanced merge gate — develop
e25de16(merged PR #153) — fresh exact-head integration review: APPROVEDPR #152's branch was rebased onto the new canonical develop (reset-aware quota state + Z.ai reset plausibility, #153/#151). No merge commit; no new PR; conflicts resolved semantically.
Conflicts encountered: none textual — the rebase auto-merged because the two PRs occupy disjoint axes of the overlapping files. The reviewer then verified the merge is semantically sound, not just textual:
422b04d..e2f1624onscarcity_router/resource_state.pyis purely additive (0 deletions): #153's reset-aware freshness (_window_reset_instants,_reset_crossed,classify_freshness,refresh_due) is byte-identical at HEAD, and D-058's capability axis (output_limit_tokens/output_limit_control,route_output_code,OUTPUT_LIMIT_*) is intact — exactly one definition of each symbol, no dropped or duplicated hunks.quota_factswindow instants and sets freshness/refresh-due;route_output_codetakes only capability/variant/request facts; gate ordering (binding → authorization → availability → compatibility) is untouched except for argument threading, so a route made stale across a quota-window reset still fails at the availability stage with its registration-owned capabilities untouched, and the output-eligibility check is never consulted for it.docs/codex-adapter-stage1-evidence.md: both the #150 Revalidation block and the #136 2026-09-28 re-evidence section coexist, non-contradictory.control_api._registration_copystill uses the additive-safe serializer round-trip and preserves every field (checked post-rebase; #153 added no capability fields).docs/decisions.mdat basee25de16ends at D-057; #153's merge added no decision record; exactly one### D-058heading at HEAD.Gates on the frozen HEAD
e2f1624: full suite 2173 tests OK (now including #153'stest_resource_state/test_zai_parsersuites alongside all D-058 suites; the #143 flake did not fire),basedpyright0 diagnostics,git diff --checkclean, guardrails 42 OK.Integration review verdict (independent GLM-5.3-Flash session): APPROVED — "the textual clean rebase is also semantically sound… integration approved." One DEFER (non-blocking, left for the human gate): D-058's provenance line could optionally note the #153 in-flight merge and this rebase; the two decisions are provably orthogonal (capability facts vs freshness semantics, zero shared state).
Status: READY_FOR_REVIEW — PR #152 open, unmerged, at the human gate. D-058 unchanged; D-057 untouched; #137 not started.
e2f1624c66fb5da15171Rebase onto develop
781bace(merged PR #155 / D-059) — fresh exact-head integration review (D-058 × D-059): APPROVEDThe branch was rebased onto the advanced gate; no merge commit, no new issue/PR.
Conflicts and resolution: exactly one file conflicted —
docs/decisions.md(both sides append at the tail). Resolved semantically, ledger order now D-057 → D-058 → D-059 with D-058 and D-059 both verbatim and unrenumbered (D-058 reserved for #152 as instructed; the merged D-059 untouched). A range-diff proves the approved PR #152 patch series survived byte-identically outsidedocs/decisions.md.Semantic integration (verified by the independent reviewer with file:line evidence, not assumed from the auto-merge):
route_output_codein the compatibility stage) qualifies resources first; the narrowed catalog view contains only qualified identities; D-059 blackout/happy-hour policy runs insideselect_modelover survivors only.eligible=Falseearly-return); happy-hour is ranking-only and validation-enforced to apply to eligible candidates only; no policy path adds candidates or reopens gates.TargetExclusioncodes (output_limit_*); D-059 = selector-levelexclusion_stage="policy_blackout"+policy_blocked+BlackoutDecisionin the embedded selection decision — disjoint vocabularies/layers, both serialized.fb5da15): D-059 blackout exclusions never reach the gate-level exclusion records, so a purely-output gate set could coexist with a blackout on a route that would satisfy the request._no_eligible_target_errornow also requires the embedded selection decision to show no blocked blackout candidate; any such candidate downgrades to the ordinary503 no_eligible_target. New regression: available 32k route (output-insufficient) + 128k-capable route blocked by an active campaign blackout, request 64k → 503 with zero dispatches; control world without the blackout → 200 on the strong route. Blackout-only worlds stay 503 by the same mechanism.e25de16..781baceon the D-058 machinery files is empty).control_apiserializer round-trip copy intact post-rebase.Gates on the frozen HEAD
fb5da15: full suite 2190 tests OK (incl. all #153/#155 suites; #143 did not fire),basedpyright0 diagnostics,git diff --checkclean, guardrails 42 OK; no merge commits inorigin/develop..HEAD; working tree clean.Findings: none. One non-finding observation recorded (the
closest_candidateshalf of the downgrade scan is redundant withexcluded— harmless and defensive).Status: READY_FOR_REVIEW — PR #152 open, unmerged, at the human gate. D-058 and D-059 both correctly numbered; D-057 untouched; #137 not started.
kilo referenced this pull request2026-09-28 15:26:18 +00:00