Public recommendation contract: expose catalog reasoning_effort on serialized CandidateEvaluation #148

Closed
opened 2026-09-27 23:33:56 +00:00 by kilo · 1 comment
Member

Goal

Expose the reasoning effort belonging to every serialized recommendation candidate through the stable public recommendation contract, so independent consumers (e.g. Creatidy Kernel) can execute the exact recommended configuration without parsing opaque identifiers.

This closes the contract gap discovered by Creatidy Kernel K8 (Kernel issue #11): the Kernel public-contract gate failed because /v1/select could not convey the exact execution configuration without forbidden inference. This is an upstream Scarcity Router contract correction, not a Kernel-specific compatibility hack.

Required semantics

Extend the public CandidateEvaluation serialization with:

reasoning_effort: string | null
  • The member MUST always be present in serialized candidate evaluations.
  • A non-null string is the exact normalized reasoning effort configured for that candidate (the frozen REASONING_EFFORTS vocabulary).
  • null means the candidate has no configured reasoning effort.
  • null is distinct from the real configured effort value "none".
  • Consumers must never derive reasoning effort from variant, model name, resource id, source id, display name or other opaque identity; no new mapping or inference rule is introduced.

Because SelectionDecision reuses CandidateEvaluation, the same field and semantics must appear consistently in selected, alternatives, excluded, closest_candidates and recoverable_candidates. No special selected.reasoning_effort representation.

Public surfaces (one-core parity)

The field must propagate consistently through all existing recommendation surfaces that serialize SelectionDecision:

  • CLI JSON output
  • loopback REST /v1/select
  • MCP scarcity_select
  • authenticated machine/control /v1/select
  • remote bridge envelope validation/tests where relevant

No selection logic in any transport adapter.

Versioning

Verified: docs/machine-interfaces.md (Versioning section) states additive backwards-compatible domain fields may flow through machine-interface v1 as long as existing v1 clients remain valid (the D-039 eligibility precedent). This change keeps envelope/schema version 1 and is documented as an additive-compatible extension. No version bump, no compatibility serializer.

Provenance and source of truth

reasoning_effort must come from the same catalog entry that participates in the authoritative selector ranking (the ranking already consumes the catalog effort as the D-032 effort tie-break). It must NOT be reconstructed after selection from ModelIdentity.variant. The serialized field reflects the exact candidate catalog configuration used by that decision.

Required regressions

Discriminating tests proving at minimum:

  1. selected candidate serializes its configured reasoning effort;
  2. alternatives serialize their own effort independently;
  3. excluded / closest / recoverable candidates preserve the same field semantics where present;
  4. unconfigured effort serializes as explicit null (key present);
  5. configured "none" serializes as "none", not null;
  6. a candidate whose opaque variant differs from its reasoning effort still serializes the catalog reasoning effort (no inference from variant);
  7. REST, MCP and authenticated control surfaces preserve parity;
  8. remote bridge accepts the additive field without inventing or stripping semantics;
  9. existing selection/ranking behavior is unchanged.

Scope constraints (do NOT)

  • change selector ranking, capability ratings or model identity semantics;
  • make variant equal effort;
  • expose internal resource/source ids;
  • change execution routing or execution gateway behavior;
  • add Kernel-specific code or account/access-mode/business-model fields;
  • broaden into the Upwork/business-model redesign;
  • alter provider telemetry;
  • modify main, release or deployment.

This issue only makes an already-existing authoritative selection input visible in the public decision output. Documentation updates: docs/machine-interfaces.md (additive v1 note), docs/selection-policy.md (records the D-032 deferral of public explicit effort output — superseded by this change), docs/architecture.md, and a decision record in docs/decisions.md (serialized public contract change requires an explicit decision per repository policy).

Cross-project evidence

Motivating external-consumer proof: Creatidy Kernel K8 issue #11 — public-contract gate failed because /v1/select could not convey the exact execution configuration without forbidden inference. Recorded as evidence only; the repositories are not coupled operationally.

Workflow

Branch from develop, PR targets develop, PR left open (unmerged) for human review.

## Goal Expose the reasoning effort belonging to every serialized recommendation candidate through the stable public recommendation contract, so independent consumers (e.g. Creatidy Kernel) can execute the exact recommended configuration without parsing opaque identifiers. This closes the contract gap discovered by Creatidy Kernel K8 (Kernel issue #11): the Kernel public-contract gate failed because `/v1/select` could not convey the exact execution configuration without forbidden inference. This is an upstream Scarcity Router contract correction, not a Kernel-specific compatibility hack. ## Required semantics Extend the public `CandidateEvaluation` serialization with: ``` reasoning_effort: string | null ``` - The member MUST always be present in serialized candidate evaluations. - A non-null string is the exact normalized reasoning effort configured for that candidate (the frozen `REASONING_EFFORTS` vocabulary). - `null` means the candidate has no configured reasoning effort. - `null` is distinct from the real configured effort value `"none"`. - Consumers must never derive reasoning effort from `variant`, model name, resource id, source id, display name or other opaque identity; no new mapping or inference rule is introduced. Because `SelectionDecision` reuses `CandidateEvaluation`, the same field and semantics must appear consistently in `selected`, `alternatives`, `excluded`, `closest_candidates` and `recoverable_candidates`. No special `selected.reasoning_effort` representation. ## Public surfaces (one-core parity) The field must propagate consistently through all existing recommendation surfaces that serialize `SelectionDecision`: - CLI JSON output - loopback REST `/v1/select` - MCP `scarcity_select` - authenticated machine/control `/v1/select` - remote bridge envelope validation/tests where relevant No selection logic in any transport adapter. ## Versioning Verified: `docs/machine-interfaces.md` (Versioning section) states additive backwards-compatible domain fields may flow through machine-interface v1 as long as existing v1 clients remain valid (the D-039 `eligibility` precedent). This change keeps envelope/schema version 1 and is documented as an additive-compatible extension. No version bump, no compatibility serializer. ## Provenance and source of truth `reasoning_effort` must come from the same catalog entry that participates in the authoritative selector ranking (the ranking already consumes the catalog effort as the D-032 effort tie-break). It must NOT be reconstructed after selection from `ModelIdentity.variant`. The serialized field reflects the exact candidate catalog configuration used by that decision. ## Required regressions Discriminating tests proving at minimum: 1. selected candidate serializes its configured reasoning effort; 2. alternatives serialize their own effort independently; 3. excluded / closest / recoverable candidates preserve the same field semantics where present; 4. unconfigured effort serializes as explicit `null` (key present); 5. configured `"none"` serializes as `"none"`, not `null`; 6. a candidate whose opaque `variant` differs from its reasoning effort still serializes the catalog reasoning effort (no inference from variant); 7. REST, MCP and authenticated control surfaces preserve parity; 8. remote bridge accepts the additive field without inventing or stripping semantics; 9. existing selection/ranking behavior is unchanged. ## Scope constraints (do NOT) - change selector ranking, capability ratings or model identity semantics; - make variant equal effort; - expose internal resource/source ids; - change execution routing or execution gateway behavior; - add Kernel-specific code or account/access-mode/business-model fields; - broaden into the Upwork/business-model redesign; - alter provider telemetry; - modify `main`, release or deployment. This issue only makes an already-existing authoritative selection input visible in the public decision output. Documentation updates: `docs/machine-interfaces.md` (additive v1 note), `docs/selection-policy.md` (records the D-032 deferral of public explicit effort output — superseded by this change), `docs/architecture.md`, and a decision record in `docs/decisions.md` (serialized public contract change requires an explicit decision per repository policy). ## Cross-project evidence Motivating external-consumer proof: Creatidy Kernel K8 issue #11 — public-contract gate failed because `/v1/select` could not convey the exact execution configuration without forbidden inference. Recorded as evidence only; the repositories are not coupled operationally. ## Workflow Branch from `develop`, PR targets `develop`, PR left open (unmerged) for human review.
Author
Member

ALREADY_SATISFIED — closed; current develop (17f4918) fully implements this contract. Disposition pass 2026-09-30; no code changes made.

The work landed via PR #149 (merged as 81b8e63, D-057 in docs/decisions.md) and is an ancestor of current origin/develop. Evidence against the acceptance criteria:

Contract & provenance

  • CandidateEvaluation.reasoning_effort: str | None is a required construction argument (no default) — scarcity_router/selector.py:976 — so a candidate cannot exist without its catalog provenance; the docstring pins the no-inference rule.
  • Every one of the seven construction sites (all exclusion stages — execution, policy_blackout, hard-constraint, capability, reservation — plus the eligible path) passes reasoning_effort=entry.reasoning_effort verbatim from the ranked ModelCatalogEntry (selector.py:1247–1408). Nothing reconstructs it from variant post-selection.
  • to_dict() always emits the key (selector.py:1162): unconfigured serializes as explicit null, configured "none" as "none" — materially distinct.
  • Values are validated against the frozen REASONING_EFFORTS vocabulary (selection_types.py:73; enforced at selector.py:1007). Because SelectionDecision reuses CandidateEvaluation, selected, alternatives, excluded, closest_candidates and recoverable_candidates all carry it with one representation.

Surfaces (one-core parity) — tests/test_interfaces_guardrails.py:912 test_select_reasoning_effort_parity asserts byte-equal payloads across direct core, CLI --json, loopback REST /v1/select and MCP scarcity_select, walking all five candidate lists and comparing each value to the ranked catalog entry; tests/test_control_api.py:250 covers the authenticated control /v1/select; test_remote_select_passes_reasoning_effort_through_verbatim (test_interfaces_guardrails.py:1336) proves the bridge neither strips nor invents the member.

Discriminating regressions (issue points 1–9) — all present and passing: test_decision_walk_serializes_catalog_effort_everywhere, test_alternatives_serialize_their_own_efforts_independently, test_capacity_excluded_and_recoverable_candidates_preserve_the_field, test_closest_candidates_preserve_the_field, test_unconfigured_effort_serializes_explicit_null, test_configured_none_serializes_as_none_not_null, test_effort_is_never_inferred_from_variant (variant "medium"/catalog "high", and variant "high"/unconfigured → null), test_ranking_still_consumes_the_catalog_effort (non-effort variant names, frozen tie-break unchanged), test_constructor_rejects_effort_outside_the_vocabulary, plus vocabulary round-trip/boundary tests in test_selection_types.py.

Versioning & docs — envelope schema_version stays 1; additive v1 note in docs/machine-interfaces.md (~line 246); the D-032 deferral supersession recorded in docs/selection-policy.md (~line 138); docs/architecture.md (~line 213); decision record D-057 in docs/decisions.md.

Commands run (all green): uv run python -m unittest tests.test_selector -k effort (15 tests), tests.test_selection_types -k effort (3), tests.test_interfaces_guardrails -k reasoning_effort (2), tests.test_control_api -k reasoning_effort (1).

**ALREADY_SATISFIED — closed; current `develop` (`17f4918`) fully implements this contract.** Disposition pass 2026-09-30; no code changes made. The work landed via PR #149 (merged as `81b8e63`, D-057 in `docs/decisions.md`) and is an ancestor of current `origin/develop`. Evidence against the acceptance criteria: **Contract & provenance** - `CandidateEvaluation.reasoning_effort: str | None` is a **required construction argument** (no default) — `scarcity_router/selector.py:976` — so a candidate cannot exist without its catalog provenance; the docstring pins the no-inference rule. - Every one of the seven construction sites (all exclusion stages — execution, `policy_blackout`, hard-constraint, capability, reservation — plus the eligible path) passes `reasoning_effort=entry.reasoning_effort` verbatim from the ranked `ModelCatalogEntry` (`selector.py:1247`–`1408`). Nothing reconstructs it from `variant` post-selection. - `to_dict()` always emits the key (`selector.py:1162`): unconfigured serializes as explicit `null`, configured `"none"` as `"none"` — materially distinct. - Values are validated against the frozen `REASONING_EFFORTS` vocabulary (`selection_types.py:73`; enforced at `selector.py:1007`). Because `SelectionDecision` reuses `CandidateEvaluation`, `selected`, `alternatives`, `excluded`, `closest_candidates` and `recoverable_candidates` all carry it with one representation. **Surfaces (one-core parity)** — `tests/test_interfaces_guardrails.py:912` `test_select_reasoning_effort_parity` asserts byte-equal payloads across direct core, CLI `--json`, loopback REST `/v1/select` and MCP `scarcity_select`, walking all five candidate lists and comparing each value to the ranked catalog entry; `tests/test_control_api.py:250` covers the authenticated control `/v1/select`; `test_remote_select_passes_reasoning_effort_through_verbatim` (`test_interfaces_guardrails.py:1336`) proves the bridge neither strips nor invents the member. **Discriminating regressions** (issue points 1–9) — all present and passing: `test_decision_walk_serializes_catalog_effort_everywhere`, `test_alternatives_serialize_their_own_efforts_independently`, `test_capacity_excluded_and_recoverable_candidates_preserve_the_field`, `test_closest_candidates_preserve_the_field`, `test_unconfigured_effort_serializes_explicit_null`, `test_configured_none_serializes_as_none_not_null`, `test_effort_is_never_inferred_from_variant` (variant `"medium"`/catalog `"high"`, and variant `"high"`/unconfigured → `null`), `test_ranking_still_consumes_the_catalog_effort` (non-effort variant names, frozen tie-break unchanged), `test_constructor_rejects_effort_outside_the_vocabulary`, plus vocabulary round-trip/boundary tests in `test_selection_types.py`. **Versioning & docs** — envelope `schema_version` stays 1; additive v1 note in `docs/machine-interfaces.md` (~line 246); the D-032 deferral supersession recorded in `docs/selection-policy.md` (~line 138); `docs/architecture.md` (~line 213); decision record D-057 in `docs/decisions.md`. **Commands run** (all green): `uv run python -m unittest tests.test_selector -k effort` (15 tests), `tests.test_selection_types -k effort` (3), `tests.test_interfaces_guardrails -k reasoning_effort` (2), `tests.test_control_api -k reasoning_effort` (1).
kilo closed this issue 2026-09-30 19:33:08 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Creatidy/scarcity-router#148
No description provided.