D-054: max-only reasoning effort for light model families (Luna, GLM Flash) #126

New issue

Closed

opened 2026-09-26 16:52:54 +00:00 by kilo · 0 comments

kilo commented

2026-09-26 16:52:54 +00:00

Member

Scope (frozen)

Change the capability-estimation mechanism so that a designated light model family (weak models: GPT Luna, GLM Flash) is represented and floor-expanded at max reasoning effort only. Owner-approved 2026-09-26 after a selector investigation showed Luna Medium winning editorial/routine_coding by the minimal-margin rule while contradicting the owner's accepted workflow roles.

Rules (owner-accepted)

Designation: light families are explicit per-track in model-tracks.json (new optional effort_restriction: "max_only" on ModelTrack), never inferred. openai/luna is light. GLM Flash is already max-only in the catalog (no zai tracks exist; ratified as-is). sol, astra, daybreak and GLM-5.3 base are NOT light (GLM-5.3 base must stay multi-effort: m31-repo-review-low is calibrated against glm-5.3/low).
Static catalog: light families carry max-effort entries only. gpt-5.6-luna/medium is removed; gpt-6-luna/high is re-labelled gpt-6-luna/max with the vector unchanged (3/3/2/4/4/3 = the luna track floor, carried at the family's single allowed effort; owner chose re-label over deletion). Catalog v4 → v5.
Dynamic floor expansion: ExecutionSources.derived_catalog_entries() clamps to max for max_only tracks — a floor entry is generated at max only, and only if the runtime reports a max effort.
Policy artifact: model-policy.json reasoning_effort_policy gains the light-family rule; policy_version 8 → 9.
No selector-engine changes: ranking, minima evaluation and margin semantics untouched.
Governance: decision D-054 in docs/decisions.md; docs/model-calibration.md updated; calibration tests re-pinned; new enforcement tests (non-max light entry rejected; floor expansion clamps).

Verified outcome deltas (selector run, 4 capacity scenarios × 9 profiles)

editorial: Luna Medium → Luna Max in all scenarios (now matches the owner's EDITORIAL_AUTHOR role).
routine_coding: winner changes only in the OpenAI-lighter branch (Luna Medium → Luna Max); glm-5.3/low still wins elsewhere.
mechanical: one runner-up changes in one scenario (gpt-6-luna loses the effort tie-break after re-label).
All other profiles select identically.

Dependency (not bypassed — stacked)

The track registry, model_tracks.py and execution_sources.py are D-053 artifacts of open PR #124 (program/dynamic-execution-sources); develop has none of them. Work therefore branches off program/dynamic-execution-sources and the PR targets that branch (stacked), to be retargeted to develop after #124 merges. No clean-worktree bypass.

Complexity boundary

~6 existing files + 2–3 test files + docs; no new modules; no selector/ranking code changes. Budget: default 120 min wall clock.

Non-goals

No GLM-5.3 base changes, no zai track creation, no profile-minima changes, no preference_order changes, no merge (human merge gate).

Acceptance

Full gate green: uv sync --only-dev, make check, uv run basedpyright, git diff --check.
program/dynamic-execution-sources..HEAD contains only this issue's commits.
Calibration tests pin the post-change eligible sets; light-family enforcement tests present.
PR open for human review; not merged.

## Scope (frozen) Change the capability-estimation mechanism so that a designated **light model family** (weak models: GPT Luna, GLM Flash) is represented and floor-expanded at **`max` reasoning effort only**. Owner-approved 2026-09-26 after a selector investigation showed Luna Medium winning `editorial`/`routine_coding` by the minimal-margin rule while contradicting the owner's accepted workflow roles. ### Rules (owner-accepted) 1. **Designation:** light families are explicit per-track in `model-tracks.json` (new optional `effort_restriction: "max_only"` on `ModelTrack`), never inferred. `openai/luna` is light. GLM Flash is already max-only in the catalog (no zai tracks exist; ratified as-is). `sol`, `astra`, `daybreak` and GLM-5.3 base are NOT light (GLM-5.3 base must stay multi-effort: `m31-repo-review-low` is calibrated against `glm-5.3/low`). 2. **Static catalog:** light families carry `max`-effort entries only. `gpt-5.6-luna/medium` is **removed**; `gpt-6-luna/high` is **re-labelled `gpt-6-luna/max`** with the vector unchanged (3/3/2/4/4/3 = the luna track floor, carried at the family's single allowed effort; owner chose re-label over deletion). Catalog v4 → v5. 3. **Dynamic floor expansion:** `ExecutionSources.derived_catalog_entries()` clamps to `max` for `max_only` tracks — a floor entry is generated at `max` only, and only if the runtime reports a `max` effort. 4. **Policy artifact:** `model-policy.json` `reasoning_effort_policy` gains the light-family rule; policy_version 8 → 9. 5. **No selector-engine changes:** ranking, minima evaluation and margin semantics untouched. 6. **Governance:** decision **D-054** in `docs/decisions.md`; `docs/model-calibration.md` updated; calibration tests re-pinned; new enforcement tests (non-max light entry rejected; floor expansion clamps). ### Verified outcome deltas (selector run, 4 capacity scenarios × 9 profiles) - `editorial`: Luna Medium → **Luna Max** in all scenarios (now matches the owner's `EDITORIAL_AUTHOR` role). - `routine_coding`: winner changes only in the OpenAI-lighter branch (Luna Medium → Luna Max); `glm-5.3/low` still wins elsewhere. - `mechanical`: one runner-up changes in one scenario (gpt-6-luna loses the effort tie-break after re-label). - All other profiles select identically. ## Dependency (not bypassed — stacked) The track registry, `model_tracks.py` and `execution_sources.py` are D-053 artifacts of open PR #124 (`program/dynamic-execution-sources`); `develop` has none of them. Work therefore branches off `program/dynamic-execution-sources` and the PR targets that branch (stacked), to be retargeted to `develop` after #124 merges. No clean-worktree bypass. ## Complexity boundary ~6 existing files + 2–3 test files + docs; no new modules; no selector/ranking code changes. Budget: default 120 min wall clock. ## Non-goals No GLM-5.3 base changes, no zai track creation, no profile-minima changes, no preference_order changes, no merge (human merge gate). ## Acceptance - Full gate green: `uv sync --only-dev`, `make check`, `uv run basedpyright`, `git diff --check`. - `program/dynamic-execution-sources..HEAD` contains only this issue's commits. - Calibration tests pin the post-change eligible sets; light-family enforcement tests present. - PR open for human review; **not merged**.