Selector: make reasoning effort first-class and onboard Terra #57

New issue

Closed

opened 2026-09-07 22:21:23 +00:00 by kilo · 1 comment

kilo commented

2026-09-07 22:21:23 +00:00

Member

Goal

Make reasoning effort an explicit validated catalog configuration property and rank the lowest sufficient effort; onboard the three approved GPT-5.6 Medium configurations.

Why

Opaque variants currently route routine coding to Luna Max and deep coding directly to Sol High, omitting calibrated lower-effort configurations and the Terra intermediate tier.

Scope

Catalog v2 dated 2026-09-08; provider-independent reasoning_effort vocabulary none/low/medium/high/xhigh/max with null distinct from string none; consistency validation; ranking after scarcity and capability margin but before preference and identity. Preserve existing four capability vectors and add exactly Luna Medium (3,4,3,5,5,4), Terra Medium (4,5,4,4,5,4), Sol Medium (5,5,4,5,5,4), in reasoning/coding/scientific/writing/tool/translation order. Terra properties: 1,050,000 input, 128,000 output, tool/vision/reasoning true, openai/codex binding. Explicit curated provenance using existing official-family/developer and owner-calibration source conventions. Update focused contracts, selector/catalog/scenario tests and narrowly authoritative documentation, including next free decision. No capability inference or variant parsing.

Acceptance Criteria

Construction/deserialization validates effort values and reasoning-mode consistency, including null versus none; round-trip and catalog v2 tests pass.
Exactly seven calibrated configurations; accepted Luna Max, Sol High and both GLM vectors unchanged.
OpenAI deterministic outcomes: routine_coding Luna Medium; deep_coding Terra Medium with Sol Medium ahead of Sol High alternatives; scientific_review Sol High; orchestration Luna Max; translation Sol High. Pin all profile eligible sets.
Synthetic tests prove scarcity > margin > effort > preference > identity, capability rejection, conflicting lexical variants and shared scope observations.
CLI/REST/MCP/direct parity and all v1 public schemas remain unchanged.
uv sync --only-dev, make check, uv run basedpyright (zero errors/warnings/notes), git diff --check pass.
One bounded live collection reused for status and five profile recommendations; record only sanitized provider states/selections, no exact personal quota values and no model execution.
One task-only branch/PR to develop, no new merge commits, pre/post push develop SHA unchanged; leave PR open for human-frozen external review.

Constraints

Base prerequisite verified: clean develop ea7332cfb5 contains PR #55 closeout, docs/m3-acceptance.md M3 PASS, and M3c merge #53. Preserve historical M3 acceptance. One agent/session, GPT-6 Astra / low per explicit human assignment override; no switching, subagents or internal reviewer loop. Human performs independent external review and merge. Bounded scope: catalog/types/ranking, focused tests and listed authoritative docs; no new architecture/dependency. Execution budget 120 minutes, at most one diagnosed worker recovery, no internal review rounds. Frozen threat model: existing validated local contracts and synthetic fixtures; credentials remain outside agent output; one existing read-only collection with existing D-018 exception only. Complexity breach or unmet base/push prerequisite stops and escalates.

Do not change provider collectors, capacity v3, scarcity formula, scope matching, reservations, replenishment, profile minima, machine-interface v1, model identity, or model-policy serialized contract. No API-price routing, execution, local inference, database/cache, M4, or Astra onboarding; issue #49 unchanged.

Links

Human assignment: Selector correction — make reasoning effort first-class and onboard GPT-5.6 Terra (2026-09-08), including approved effort-specific calibration and accepted external OpenAI family evidence.
Prerequisite PR #55 and M3c PR #53
docs/capability-model.md, docs/selection-policy.md, docs/model-calibration.md, docs/decisions.md
Independent Astra onboarding: #49

## Goal Make reasoning effort an explicit validated catalog configuration property and rank the lowest sufficient effort; onboard the three approved GPT-5.6 Medium configurations. ## Why Opaque variants currently route routine coding to Luna Max and deep coding directly to Sol High, omitting calibrated lower-effort configurations and the Terra intermediate tier. ## Scope Catalog v2 dated 2026-09-08; provider-independent reasoning_effort vocabulary none/low/medium/high/xhigh/max with null distinct from string none; consistency validation; ranking after scarcity and capability margin but before preference and identity. Preserve existing four capability vectors and add exactly Luna Medium (3,4,3,5,5,4), Terra Medium (4,5,4,4,5,4), Sol Medium (5,5,4,5,5,4), in reasoning/coding/scientific/writing/tool/translation order. Terra properties: 1,050,000 input, 128,000 output, tool/vision/reasoning true, openai/codex binding. Explicit curated provenance using existing official-family/developer and owner-calibration source conventions. Update focused contracts, selector/catalog/scenario tests and narrowly authoritative documentation, including next free decision. No capability inference or variant parsing. ## Acceptance Criteria - Construction/deserialization validates effort values and reasoning-mode consistency, including null versus none; round-trip and catalog v2 tests pass. - Exactly seven calibrated configurations; accepted Luna Max, Sol High and both GLM vectors unchanged. - OpenAI deterministic outcomes: routine_coding Luna Medium; deep_coding Terra Medium with Sol Medium ahead of Sol High alternatives; scientific_review Sol High; orchestration Luna Max; translation Sol High. Pin all profile eligible sets. - Synthetic tests prove scarcity > margin > effort > preference > identity, capability rejection, conflicting lexical variants and shared scope observations. - CLI/REST/MCP/direct parity and all v1 public schemas remain unchanged. - uv sync --only-dev, make check, uv run basedpyright (zero errors/warnings/notes), git diff --check pass. - One bounded live collection reused for status and five profile recommendations; record only sanitized provider states/selections, no exact personal quota values and no model execution. - One task-only branch/PR to develop, no new merge commits, pre/post push develop SHA unchanged; leave PR open for human-frozen external review. ## Constraints Base prerequisite verified: clean develop ea7332cfb5f926c5da19fa615b9a8792f518c5a9 contains PR #55 closeout, docs/m3-acceptance.md M3 PASS, and M3c merge #53. Preserve historical M3 acceptance. One agent/session, GPT-6 Astra / low per explicit human assignment override; no switching, subagents or internal reviewer loop. Human performs independent external review and merge. Bounded scope: catalog/types/ranking, focused tests and listed authoritative docs; no new architecture/dependency. Execution budget 120 minutes, at most one diagnosed worker recovery, no internal review rounds. Frozen threat model: existing validated local contracts and synthetic fixtures; credentials remain outside agent output; one existing read-only collection with existing D-018 exception only. Complexity breach or unmet base/push prerequisite stops and escalates. Do not change provider collectors, capacity v3, scarcity formula, scope matching, reservations, replenishment, profile minima, machine-interface v1, model identity, or model-policy serialized contract. No API-price routing, execution, local inference, database/cache, M4, or Astra onboarding; issue #49 unchanged. ## Links - Human assignment: Selector correction — make reasoning effort first-class and onboard GPT-5.6 Terra (2026-09-08), including approved effort-specific calibration and accepted external OpenAI family evidence. - Prerequisite PR #55 and M3c PR #53 - docs/capability-model.md, docs/selection-policy.md, docs/model-calibration.md, docs/decisions.md - Independent Astra onboarding: #49

kilo commented

2026-09-08 18:16:40 +00:00

Author

Member

Live recovery authorization: the first bounded live harness failed because it passed an unsupported stderr keyword to cli.main after collection. No telemetry was persisted and no provider/quota values were printed. Human explicitly selected 'One Corrected Pass': exactly one additional collection is authorized to recover this diagnosed harness error, with no further live retry and no model execution. The corrected harness will first be checked with synthetic snapshots, then reuse its one real collection for CLI status plus the five profile selections.

No subagents, model switching, internal review or merge. Requested assignment is GPT-6 Astra / low under the human override; runtime reasoning effort is not independently observable from this session and is not claimed verified.

Implementation checkpoint: catalog v2 and D-032 implemented on selector/reasoning-effort-terra; 901 deterministic tests pass, basedpyright reports 0 errors/0 warnings/0 notes, git diff --check passes. Five-profile direct/CLI/REST/MCP v1 parity is pinned. The only model-policy edit is descriptive reasoning_effort_policy.selector_support (not_implemented -> calibrated_configurations); schema/version/minima are unchanged. Live recovery authorization: the first bounded live harness failed because it passed an unsupported stderr keyword to cli.main after collection. No telemetry was persisted and no provider/quota values were printed. Human explicitly selected 'One Corrected Pass': exactly one additional collection is authorized to recover this diagnosed harness error, with no further live retry and no model execution. The corrected harness will first be checked with synthetic snapshots, then reuse its one real collection for CLI status plus the five profile selections. No subagents, model switching, internal review or merge. Requested assignment is GPT-6 Astra / low under the human override; runtime reasoning effort is not independently observable from this session and is not claimed verified.

kilo referenced this issue from a pull request that will close it,

2026-09-08 18:21:13 +00:00

Selector: Make reasoning effort first-class and onboard Terra #58