D-054: max-only reasoning effort for light model families (#126) #127

Merged

adrian.tkacz merged 2 commits from calibration/max-only-light-families into develop

2026-09-26 18:16:30 +00:00

kilo commented

2026-09-26 17:57:41 +00:00

Member

Closes #126 (stays open for human review; do not merge without owner authorization).

What this does

Implements D-054 — the owner-approved estimation-mechanism change: a light model family (weak models: GPT Luna, GLM Flash) is represented and floor-expanded at max reasoning effort only.

model-tracks.json: openai/luna gains effort_restriction: "max_only" (rationale in the track notes). GLM Flash is already max-only in the catalog (ratified, no zai track exists); sol/astra/daybreak and GLM-5.3 base stay unrestricted — GLM-5.3 base must stay multi-effort (m31-repo-review-low is calibrated on glm-5.3/low).
model_tracks.py: optional effort_restriction on ModelTrack (closed vocabulary; rejected on restricted tracks), restrict_floor_efforts() clamp — a max_only track expands at max only and only when the runtime itself reports a max effort (never invented; ultra is a distinct effort and does not qualify) — plus the fail-closed validate_catalog_effort_restriction() cross-artifact conformance check.
execution_sources.py: derived_catalog_entries() validates the base catalog against the rule on every composition and expands max_only tracks at max only.
model-catalog.json v5: gpt-5.6-luna/medium removed; gpt-6-luna/high re-labelled gpt-6-luna/max with the track-floor vector unchanged (3/3/2/4/4/3, confidence low — the owner chose re-label over deletion to keep GPT-6 Luna selector-visible). Per-dimension assessed_on values unchanged.
model-policy.json v9: the rule recorded in reasoning_effort_policy.light_family_restriction (schema stays v1).
No selector-engine changes. Ranking order, minima evaluation and margin semantics untouched; only the calibrated input set changed.

Verified outcome deltas

Deterministic selector runs (user policy; 4 synthetic capacity scenarios × 9 profiles):

editorial: Luna Max in all scenarios (was Luna Medium) — now matching the owner's EDITORIAL_AUTHOR role that the selector previously could not see.
routine_coding: changes only in the OpenAI-lighter branch (Luna Medium → Luna Max); glm-5.3/low still wins the other scenarios.
mechanical: one runner-up changes in one scenario (re-labelled GPT-6 Luna loses the effort tie-break).
All other profiles select identically.

Governance

Decision D-054 in docs/decisions.md (context, rules, alternatives, boundary); docs/model-calibration.md gains a catalog-v5 section with re-pinned eligible sets; the D-032 historical record is preserved untouched.
Catalog change provenance: version bump 4 → 5, updated_on 2026-09-26, human-reviewable diff (6 added lines for the re-label; one entry removed).

Tests

Calibration pins reworked to the 11-entry catalog (excluded-count and exact-ranking updates; the capability-failure/effort-gradient test now uses the GLM-5.3 effort identities to preserve the lower-effort-loser semantics).
New LightFamilyRestrictionTests (shipped-artifact conformance, non-max rejection, unclassified passthrough), track-level validation/clamp/round-trip tests, and derived-catalog clamp tests (max-only expansion; no invented entry without a reported max; unrestricted track unchanged).

Branch state

Contains the newest develop (@ a4370f0, PR #124 merged) via merge commit 0ccf349; the merge base is the base branch tip, so the PR diff shows exactly the one D-054 commit (a3aafe5). Originally created stacked on #124 before it merged; retargeted to develop immediately after.

Validation

On the merged head (0ccf349): uv sync --only-dev, make check (2013 tests + typecheck) green; uv run basedpyright 0 errors / 0 warnings / 0 notes; git diff --check clean.

Review

UX impact: positive — the editorial recommendation stops contradicting the owner's accepted workflow roles; CLI/REST/MCP surfaces unchanged (parity re-verified by the interface tests). Review scope: one independent review of this single-commit calibration change; not security-critical (no Daybreak gate).

Closes #126 (stays open for human review; **do not merge without owner authorization**). ## What this does Implements **D-054** — the owner-approved estimation-mechanism change: a **light model family** (weak models: GPT Luna, GLM Flash) is represented and floor-expanded at **`max` reasoning effort only**. - **`model-tracks.json`:** `openai/luna` gains `effort_restriction: "max_only"` (rationale in the track notes). GLM Flash is already max-only in the catalog (ratified, no zai track exists); `sol`/`astra`/`daybreak` and GLM-5.3 base stay unrestricted — GLM-5.3 base must stay multi-effort (`m31-repo-review-low` is calibrated on `glm-5.3/low`). - **`model_tracks.py`:** optional `effort_restriction` on `ModelTrack` (closed vocabulary; rejected on `restricted` tracks), `restrict_floor_efforts()` clamp — a `max_only` track expands at `max` only and **only when the runtime itself reports a `max` effort** (never invented; `ultra` is a distinct effort and does not qualify) — plus the fail-closed `validate_catalog_effort_restriction()` cross-artifact conformance check. - **`execution_sources.py`:** `derived_catalog_entries()` validates the base catalog against the rule on every composition and expands `max_only` tracks at `max` only. - **`model-catalog.json` v5:** `gpt-5.6-luna/medium` **removed**; `gpt-6-luna/high` **re-labelled `gpt-6-luna/max`** with the track-floor vector unchanged (3/3/2/4/4/3, confidence low — the owner chose re-label over deletion to keep GPT-6 Luna selector-visible). Per-dimension `assessed_on` values unchanged. - **`model-policy.json` v9:** the rule recorded in `reasoning_effort_policy.light_family_restriction` (schema stays v1). - **No selector-engine changes.** Ranking order, minima evaluation and margin semantics untouched; only the calibrated input set changed. ## Verified outcome deltas Deterministic selector runs (user policy; 4 synthetic capacity scenarios × 9 profiles): - `editorial`: **Luna Max** in all scenarios (was Luna Medium) — now matching the owner's `EDITORIAL_AUTHOR` role that the selector previously could not see. - `routine_coding`: changes only in the OpenAI-lighter branch (Luna Medium → Luna Max); `glm-5.3/low` still wins the other scenarios. - `mechanical`: one runner-up changes in one scenario (re-labelled GPT-6 Luna loses the effort tie-break). - All other profiles select identically. ## Governance - Decision **D-054** in `docs/decisions.md` (context, rules, alternatives, boundary); `docs/model-calibration.md` gains a catalog-v5 section with re-pinned eligible sets; the D-032 historical record is preserved untouched. - Catalog change provenance: version bump 4 → 5, `updated_on` 2026-09-26, human-reviewable diff (6 added lines for the re-label; one entry removed). ## Tests - Calibration pins reworked to the 11-entry catalog (excluded-count and exact-ranking updates; the capability-failure/effort-gradient test now uses the GLM-5.3 effort identities to preserve the lower-effort-loser semantics). - New `LightFamilyRestrictionTests` (shipped-artifact conformance, non-max rejection, unclassified passthrough), track-level validation/clamp/round-trip tests, and derived-catalog clamp tests (max-only expansion; no invented entry without a reported `max`; unrestricted track unchanged). ## Branch state Contains the newest `develop` (@ `a4370f0`, PR #124 merged) via merge commit `0ccf349`; the merge base is the base branch tip, so the PR diff shows exactly the one D-054 commit (`a3aafe5`). Originally created stacked on #124 before it merged; retargeted to `develop` immediately after. ## Validation - On the merged head (`0ccf349`): `uv sync --only-dev`, `make check` (2013 tests + typecheck) green; `uv run basedpyright` 0 errors / 0 warnings / 0 notes; `git diff --check` clean. ## Review `UX impact: positive` — the editorial recommendation stops contradicting the owner's accepted workflow roles; CLI/REST/MCP surfaces unchanged (parity re-verified by the interface tests). Review scope: one independent review of this single-commit calibration change; not security-critical (no Daybreak gate).

kilo added 1 commit

2026-09-26 17:57:41 +00:00

feat: max-only reasoning effort for light model families (D-054, #126 ) a3aafe5880

Designate light families in model-tracks.json via effort_restriction
max_only and restrict their catalog representation and floor expansion
to the max effort. Owner-approved estimation-mechanism change after the
selector investigation showed Luna Medium winning editorial/
routine_coding by the minimal-margin rule against the owner's accepted
workflow roles.

- model-tracks.json: openai/luna designated max_only (rationale in notes)
- model_tracks.py: optional effort_restriction on ModelTrack (closed
  vocabulary, rejected on restricted tracks), restrict_floor_efforts
  clamp (max only, never invented), and the fail-closed
  validate_catalog_effort_restriction cross-artifact check
- execution_sources.py: derived_catalog_entries validates the base
  catalog against the rule and expands max_only tracks at max only
- model-catalog.json v5: gpt-5.6-luna/medium removed; gpt-6-luna/high
  re-labelled gpt-6-luna/max with the track-floor vector unchanged
- model-policy.json v9: light_family_restriction recorded in
  reasoning_effort_policy
- docs: D-054 decision record; model-calibration.md catalog-v5 section
  with re-pinned eligible sets and verified outcome deltas
- tests: calibration pins reworked (11 entries, new winners), new
  light-family enforcement tests (artifact conformance, rejection,
  floor clamp, round-trip)

Editorial now selects Luna Max in every capacity scenario, matching the
owner's EDITORIAL_AUTHOR role; all other profile outcomes unchanged.

Gate: make check (2013 tests), basedpyright 0/0/0, git diff --check.

kilo changed target branch from program/dynamic-execution-sources to develop

2026-09-26 17:58:47 +00:00

adrian.tkacz approved these changes

2026-09-26 18:02:05 +00:00

adrian.tkacz added 1 commit

2026-09-26 18:02:14 +00:00

Merge branch 'develop' into calibration/max-only-light-families

ci / check (pull_request) Successful in 7m42s

Details

0ccf3494c5

adrian.tkacz merged commit 14979613dd into develop

2026-09-26 18:16:30 +00:00