Public contract: serialized candidates carry their catalog reasoning effort (D-057, #148) #149

Merged

adrian.tkacz merged 1 commit from contract/reasoning-effort-serialization into develop

2026-09-28 07:28:04 +00:00

kilo commented

2026-09-28 00:08:40 +00:00

Member

Summary

Extends the public recommendation contract (issue #148, decision D-057): every serialized CandidateEvaluation now carries

reasoning_effort: string | null

Always present in selected, alternatives, excluded, closest_candidates and recoverable_candidates — one representation, no special selected shape.
Catalog-sourced: carried verbatim from the same ModelCatalogEntry that participated in ranking, as a required construction argument (a candidate evaluation cannot exist without its catalog provenance). Never reconstructed from ModelIdentity.variant or any other opaque identity; no new inference rule.
null ≠ "none": null means unconfigured; "none" is the real configured effort.
Versioning: additive backwards-compatible machine-interface v1 member per docs/machine-interfaces.md (D-039 eligibility precedent). Envelope schema_version stays 1; no compatibility serializer.
One-core parity: propagates through CLI JSON, loopback REST /v1/select, MCP scarcity_select, authenticated control /v1/select and the remote bridge with zero transport-adapter logic.

Motivating consumer evidence: Creatidy Kernel K8 issue #11 — its public-contract gate failed because /v1/select could not convey the exact execution configuration without forbidden inference (in the shipped catalog, identity.variant names coincide with effort names — a calibration coincidence, not a contract). Recorded as evidence only; repositories are not coupled operationally.

Changes

scarcity_router/selector.py — CandidateEvaluation.reasoning_effort (required, construction-validated against the frozen REASONING_EFFORTS vocabulary), always-present to_dict() member, and reasoning_effort=entry.reasoning_effort at all seven _evaluate_candidate construction sites. Nothing else.
tests/test_selector.py — new ReasoningEffortSerializationTests (9 tests): decision-wide catalog provenance walk, independent alternative efforts, capacity-excluded/recoverable and closest-candidate preservation, explicit-null for unconfigured, "none" serializes as "none", variant≠effort proves no inference, ranking still consumes the catalog effort, constructor rejects out-of-vocabulary values. Updated the one assertion that pinned the old contract (assertNotIn("reasoning_effort", ...)).
tests/test_interfaces_guardrails.py — test_select_reasoning_effort_parity (direct == CLI == REST == MCP, key present on every candidate list, values match the catalog) and test_remote_select_passes_reasoning_effort_through_verbatim (bridge neither strips nor invents).
tests/test_control_api.py — test_authenticated_select_decision_carries_reasoning_effort (real authenticated server, bearer client key).
Docs: docs/machine-interfaces.md (additive v1 note on /v1/select), docs/selection-policy.md (supersedes the D-032 "public explicit effort output is deferred" sentence), docs/architecture.md, and the D-057 record in docs/decisions.md.

Not changed

Selector ranking, capability ratings, model identity semantics, execution routing/gateway behavior, provider telemetry, request contracts. No account/access-mode/business-model fields; no Kernel-specific code.

Validation

uv sync --only-dev, make check (2112 tests OK), uv run basedpyright (0 errors, 0 warnings, 0 notes), git diff --check — all green on the reviewed head.
Scope check: origin/develop..HEAD contains exactly this change (single commit, no merge commits); touched files in scope.

Notes for review

One full-suite run on pristine develop during investigation and one on this branch each hit test_e2e_execution…test_scenario_11_cancel_reaches_exactly_the_owning_worker under load; it passes 8/8 in isolation on both — the known load-sensitive race family tracked in issue #143, unrelated to this diff.
Classification: not security_critical (no credential, network-exposure, auth or storage change) — no daybreak gate required.
UX impact: none on human surfaces (JSON members only); positive for machine consumers executing the recommended configuration.

Reviewed head: 2ffc224 — frozen for review; no pushes during the review window. PR left open for human merge gate; workers never merge.

## Summary Extends the public recommendation contract (issue #148, decision **D-057**): every serialized `CandidateEvaluation` now carries ``` reasoning_effort: string | null ``` - **Always present** in `selected`, `alternatives`, `excluded`, `closest_candidates` and `recoverable_candidates` — one representation, no special `selected` shape. - **Catalog-sourced**: carried verbatim from the same `ModelCatalogEntry` that participated in ranking, as a *required* construction argument (a candidate evaluation cannot exist without its catalog provenance). Never reconstructed from `ModelIdentity.variant` or any other opaque identity; no new inference rule. - **`null` ≠ `"none"`**: `null` means unconfigured; `"none"` is the real configured effort. - **Versioning**: additive backwards-compatible machine-interface **v1** member per `docs/machine-interfaces.md` (D-039 `eligibility` precedent). Envelope `schema_version` stays `1`; no compatibility serializer. - **One-core parity**: propagates through CLI JSON, loopback REST `/v1/select`, MCP `scarcity_select`, authenticated control `/v1/select` and the remote bridge with zero transport-adapter logic. Motivating consumer evidence: **Creatidy Kernel K8 issue #11** — its public-contract gate failed because `/v1/select` could not convey the exact execution configuration without forbidden inference (in the shipped catalog, `identity.variant` names coincide with effort names — a calibration coincidence, not a contract). Recorded as evidence only; repositories are not coupled operationally. ## Changes - `scarcity_router/selector.py` — `CandidateEvaluation.reasoning_effort` (required, construction-validated against the frozen `REASONING_EFFORTS` vocabulary), always-present `to_dict()` member, and `reasoning_effort=entry.reasoning_effort` at all seven `_evaluate_candidate` construction sites. Nothing else. - `tests/test_selector.py` — new `ReasoningEffortSerializationTests` (9 tests): decision-wide catalog provenance walk, independent alternative efforts, capacity-excluded/recoverable and closest-candidate preservation, explicit-null for unconfigured, `"none"` serializes as `"none"`, variant≠effort proves no inference, ranking still consumes the catalog effort, constructor rejects out-of-vocabulary values. Updated the one assertion that pinned the old contract (`assertNotIn("reasoning_effort", ...)`). - `tests/test_interfaces_guardrails.py` — `test_select_reasoning_effort_parity` (direct == CLI == REST == MCP, key present on every candidate list, values match the catalog) and `test_remote_select_passes_reasoning_effort_through_verbatim` (bridge neither strips nor invents). - `tests/test_control_api.py` — `test_authenticated_select_decision_carries_reasoning_effort` (real authenticated server, bearer client key). - Docs: `docs/machine-interfaces.md` (additive v1 note on `/v1/select`), `docs/selection-policy.md` (supersedes the D-032 "public explicit effort output is deferred" sentence), `docs/architecture.md`, and the **D-057** record in `docs/decisions.md`. ## Not changed Selector ranking, capability ratings, model identity semantics, execution routing/gateway behavior, provider telemetry, request contracts. No account/access-mode/business-model fields; no Kernel-specific code. ## Validation - `uv sync --only-dev`, `make check` (2112 tests OK), `uv run basedpyright` (0 errors, 0 warnings, 0 notes), `git diff --check` — all green on the reviewed head. - Scope check: `origin/develop..HEAD` contains exactly this change (single commit, no merge commits); touched files in scope. ## Notes for review - One full-suite run on pristine `develop` during investigation and one on this branch each hit `test_e2e_execution…test_scenario_11_cancel_reaches_exactly_the_owning_worker` under load; it passes 8/8 in isolation on both — the known load-sensitive race family tracked in issue #143, unrelated to this diff. - Classification: not `security_critical` (no credential, network-exposure, auth or storage change) — no daybreak gate required. - UX impact: none on human surfaces (JSON members only); positive for machine consumers executing the recommended configuration. **Reviewed head:** `2ffc224` — frozen for review; no pushes during the review window. PR left **open** for human merge gate; workers never merge.

kilo added 1 commit

2026-09-28 00:08:40 +00:00

feat: serialized candidates carry their catalog reasoning effort (D-057, #148 )

ci / check (pull_request) Successful in 7m49s

Details

2ffc224a28

Every serialized CandidateEvaluation now carries reasoning_effort — the
exact configured effort of the catalog entry that participated in ranking,
or an explicit null when unconfigured (distinct from the real effort
"none"). The member is always present in selected, alternatives,
excluded, closest_candidates and recoverable_candidates, propagates
additively through machine-interface v1 on every recommendation surface
(CLI JSON, loopback REST /v1/select, MCP scarcity_select, authenticated
control /v1/select, remote bridge), and is never derived from the opaque
identity. Ranking, capability, identity and execution semantics are
unchanged.

Motivating consumer evidence: Creatidy Kernel K8 issue #11 (public-contract
gate could not obtain the exact execution configuration without forbidden
inference). Decision record: docs/decisions.md D-057.

kilo reviewed

2026-09-28 00:14:39 +00:00

kilo left a comment

Independent review record (bounded review lifecycle)

Reviewed head (frozen): 2ffc224 (origin/develop..HEAD = exactly one non-merge commit; no pushes during review)
Reviewer: GLM-5.3-Flash, independent subagent context (the repository-required independent review; not classified security_critical, so no daybreak gate is required — recorded per docs/llm-operating-policy.md)
Scope frozen: issue #148 / D-057; threat model: output-only serialization extension, no credential/network/auth/storage change; complexity boundary ~500 lines / 8 files, core delta ~30 lines — matched.

VERDICT: APPROVED

MERGE_BLOCKER: none.

Verified against the code (not the PR text):

reasoning_effort is an unconditional member of CandidateEvaluation.to_dict(); null = unconfigured, distinct from "none".
Required construction argument (no default), validated against the frozen REASONING_EFFORTS vocabulary; all 7 _evaluate_candidate sites pass entry.reasoning_effort (the only construction sites in the codebase) — catalog provenance guaranteed, no variant reconstruction path exists.
No ranking change: _ranking_key, select_model, effort_by_identity untouched; the effort tie-break still sources from catalog entries. Full tests.test_selector module (89 tests) green.
Tests are discriminating: variant≠effort, explicit-null, "none"-vs-null, and whole-document parity across direct/CLI/REST/MCP/remote-bridge each fail under their target regression.
Docs consistent: D-057 record, additive-v1 note (D-039 precedent), superseded D-032 deferral sentence; remaining "variant identifies effort" phrasing exists only in append-only historical records.
Security: output-only; no secrets in fixtures. UX impact: none (additive JSON member; machine consumers gain exact configured effort).

DEFER (awareness only, no action required for this diff):

tests/test_interfaces_guardrails.py — the closest_candidates/recoverable_candidates walk is vacuous in the deep_coding parity scenario; real coverage comes from the core-level tests plus full-document equality on every surface.
The shipped model-catalog.json has no unconfigured or "none"-effort entries; the null vs "none" distinction is exercised via synthetic fixtures (correct vehicle per repo fixture policy).
reasoning_effort mid-field-list insertion changes the frozen dataclass's positional construction signature; all production and test sites use keyword arguments (verified) — internal-API only.

Gate evidence at reviewed head: make check 2112 tests OK; uv run basedpyright 0 errors/0 warnings/0 notes; git diff --check clean. One full-suite flake observation (test_scenario_11_cancel_reaches_exactly_the_owning_worker, passes 8/8 isolated on branch and pristine develop) recorded on issue #143.

No remediation round required. PR left open, unmerged for the human merge gate; workers and reviewers never merge.

## Independent review record (bounded review lifecycle) - **Reviewed head (frozen):** `2ffc224` (`origin/develop..HEAD` = exactly one non-merge commit; no pushes during review) - **Reviewer:** GLM-5.3-Flash, independent subagent context (the repository-required independent review; not classified `security_critical`, so no daybreak gate is required — recorded per `docs/llm-operating-policy.md`) - **Scope frozen:** issue #148 / D-057; threat model: output-only serialization extension, no credential/network/auth/storage change; complexity boundary ~500 lines / 8 files, core delta ~30 lines — matched. ### VERDICT: APPROVED **MERGE_BLOCKER: none.** **Verified against the code (not the PR text):** - `reasoning_effort` is an unconditional member of `CandidateEvaluation.to_dict()`; `null` = unconfigured, distinct from `"none"`. - Required construction argument (no default), validated against the frozen `REASONING_EFFORTS` vocabulary; all 7 `_evaluate_candidate` sites pass `entry.reasoning_effort` (the only construction sites in the codebase) — catalog provenance guaranteed, no variant reconstruction path exists. - No ranking change: `_ranking_key`, `select_model`, `effort_by_identity` untouched; the effort tie-break still sources from catalog entries. Full `tests.test_selector` module (89 tests) green. - Tests are discriminating: variant≠effort, explicit-null, `"none"`-vs-null, and whole-document parity across direct/CLI/REST/MCP/remote-bridge each fail under their target regression. - Docs consistent: D-057 record, additive-v1 note (D-039 precedent), superseded D-032 deferral sentence; remaining "variant identifies effort" phrasing exists only in append-only historical records. - Security: output-only; no secrets in fixtures. UX impact: **none** (additive JSON member; machine consumers gain exact configured effort). **DEFER (awareness only, no action required for this diff):** 1. `tests/test_interfaces_guardrails.py` — the `closest_candidates`/`recoverable_candidates` walk is vacuous in the `deep_coding` parity scenario; real coverage comes from the core-level tests plus full-document equality on every surface. 2. The shipped `model-catalog.json` has no unconfigured or `"none"`-effort entries; the `null` vs `"none"` distinction is exercised via synthetic fixtures (correct vehicle per repo fixture policy). 3. `reasoning_effort` mid-field-list insertion changes the frozen dataclass's positional construction signature; all production and test sites use keyword arguments (verified) — internal-API only. **Gate evidence at reviewed head:** `make check` 2112 tests OK; `uv run basedpyright` 0 errors/0 warnings/0 notes; `git diff --check` clean. One full-suite flake observation (`test_scenario_11_cancel_reaches_exactly_the_owning_worker`, passes 8/8 isolated on branch and pristine develop) recorded on issue #143. No remediation round required. PR left **open, unmerged** for the human merge gate; workers and reviewers never merge.

kilo referenced this pull request

2026-09-28 00:14:49 +00:00

Flaky under load: test_scenario_11 client-disconnect cancel can assert before the cancel reaches the worker reader #143

adrian.tkacz approved these changes

2026-09-28 07:27:58 +00:00

adrian.tkacz merged commit 81b8e6393a into develop

2026-09-28 07:28:04 +00:00

adrian.tkacz deleted branch contract/reasoning-effort-serialization

2026-09-28 07:28:04 +00:00

adrian.tkacz referenced this pull request from a commit

2026-09-28 07:28:04 +00:00

Merge pull request 'Public contract: serialized candidates carry their catalog reasoning effort (D-057, #148)' (#149) from contract/reasoning-effort-serialization into develop