M2: Freeze selector contracts and implementation plan #31

Merged

creatidy merged 2 commits from m2-selector-planning-gate into main

2026-09-05 22:11:50 +00:00

creatidy commented

2026-09-05 21:56:09 +00:00

(Migrated from github.com)

Closes #30

What this is

The M2 planning gate: a documentation + architecture-contract change that freezes the minimum selector input contracts and decision boundaries before any M2 selector implementation. Per the issue, it intentionally implements nothing: no scarcity scoring, no capability ratings, no select CLI, no simulation, no blackout execution, no reset-credit handling code, no Artificial Analysis client, no provider health client, no compound workflow execution.

Contracts frozen

TaskRequirement (docs/capability-model.md) — exactly four parts: task level L0–L5 (not a capability score; no "L4 ⇒ minimum 4" shortcut), explicit per-dimension capability minima (every specified minimum met, never averaged), typed hard constraints, and single-place profile expansion into those three parts. Explicit inputs may only tighten a profile; contradictions fail validation.
Capability scale — frozen ordinal 1..5 with routing rubric; missing/unknown is represented separately (UNKNOWN_CAPABILITY), never 0 = unknown; unknown never silently passes sufficiency. Dimensions stay the current six; no orchestration or factuality dimension without demonstrated routing need.
Hard constraints — frozen nine-member vocabulary (minimum_input_context_tokens, minimum_output_tokens, requires_tool_use, requires_vision, requires_reasoning_mode, required_provider, required_model, required_variant, privacy_constraint). No local/cloud constraints (D-017); no free-form constraint framework.
Model catalog entry — stable provider/model/variant identity separate from display name, quota bucket, provider aliases and model class; hard properties; context/output allowances; per-dimension ratings with provenance (source, source version/date, assessment date, confidence, rationale); explicit reviewable human override; capacity bindings. Initial catalog only: GPT-5.6 Luna, GPT-5.6 Sol, GLM-5.3, GLM-5.3-Flash. No ratings populated (M2c).
Capacity applicability (critical M2 gate, D-020) — a provider-independent capacity scope concept: (provider, scope_id) identifies one normalized scope; a model binds to one or more scopes; all applicable scopes participate; no optimistic single-scope selection. The v2 window_id is diagnostic and selector code must never parse it for applicability. The first M2 implementation prerequisite is a new normalized capacity-contract version adding an explicit semantic CapacityWindow.scope_id (conceptual shape only here; no schema v3 in this PR, no production capacity class changes).
Scarcity aggregation boundary — only applicable scopes participate; all relevant windows within an applicable scope participate; most restrictive governs under balanced; a healthy short window never hides a critical weekly window; an unrelated bucket never penalizes a candidate; unknown applicability is explicit. No formula or thresholds yet (U-007).
Evidence/policy precedence (D-021) — hard user policy (blackout ⇒ policy_blocked, never unavailable, never rewrites telemetry) > direct capacity evidence > provider-native failure evidence > official public status (advisory degraded confidence only) > external benchmarks (catalog curation only).
Replenishment (D-021) — reset credits are replenishment opportunities, not current capacity; minimal safe ReplenishmentState (provider, kind, available_count, details_known, optional earliest_expiry, retrieved_at); no free-text titles/descriptions; no opaque credit IDs required.
Artificial Analysis boundary (D-021) — offline/periodic catalog evidence; not runtime capacity, not automatic truth, never called during select(); required provenance fields for any future snapshot; API key stays server-side; no response hoarding.
ExecutionBudget (D-022) — max_total_model_calls, max_legs, max_review_rounds, max_remediation_rounds, max_retries_per_leg, max_wall_clock_minutes; all finite/bounded, no unlimited sentinel; a compound recommendation without a complete valid budget is invalid; structural maxima for single/cascade/critique; never "review/fix until clean"; call-count and scope accounting explicit even when token cost is unknown.

Also: HydraFusion positioning updated to the GitHub Copilot CLI Research Preview reality with the sharp distinction (it chooses and executes workflows; Scarcity Router decides which scarce capable resource or bounded plan to spend, without executing) and the roadmap now records the staged sequence M2a → M2b → M2c → M2d → M2e with later optional additions (health advisory, cached AA evidence, compound recommendations) explicitly non-blocking. The one stale M1 wording ("read-only subscription collectors") is corrected per D-018; M1 is not otherwise reopened.

Deliberately unresolved

Exact model ratings and profile numeric minima (U-006, M2c).
Scarcity penalty function and label thresholds (U-007, M2c/M2d scenario calibration).
Final v3 scope-identifier field shape beyond the recorded direction (M2a PR).
Whether unknown capability may proceed under which policies (explicit policy work in M2d).
Actual AA integration, official health advisory integration, compound recommendation implementation (optional, after core selector).

Validation

uv sync --only-dev ✅
make check: 431 tests pass; basedpyright 0 errors, 0 warnings, 0 notes ✅
git diff --check origin/main clean; origin/main..HEAD contains exactly one commit; no merge commits ✅
Docs-only diff (9 files); no production selector code, no credentials, no personal quota values, no real bucket IDs; model-policy.json unchanged ✅

Human merge gate: leaving this PR open and unmerged; M2a does not start before it is merged.

Closes #30 ## What this is The **M2 planning gate**: a documentation + architecture-contract change that freezes the minimum selector input contracts and decision boundaries before any M2 selector implementation. Per the issue, it intentionally implements nothing: no scarcity scoring, no capability ratings, no `select` CLI, no simulation, no blackout execution, no reset-credit handling code, no Artificial Analysis client, no provider health client, no compound workflow execution. ## Contracts frozen 1. **TaskRequirement** (`docs/capability-model.md`) — exactly four parts: task level L0–L5 (not a capability score; no "L4 ⇒ minimum 4" shortcut), explicit per-dimension capability minima (every specified minimum met, never averaged), typed hard constraints, and single-place profile expansion into those three parts. Explicit inputs may only tighten a profile; contradictions fail validation. 2. **Capability scale** — frozen ordinal `1..5` with routing rubric; missing/unknown is represented separately (`UNKNOWN_CAPABILITY`), never `0 = unknown`; unknown never silently passes sufficiency. Dimensions stay the current six; no orchestration or factuality dimension without demonstrated routing need. 3. **Hard constraints** — frozen nine-member vocabulary (`minimum_input_context_tokens`, `minimum_output_tokens`, `requires_tool_use`, `requires_vision`, `requires_reasoning_mode`, `required_provider`, `required_model`, `required_variant`, `privacy_constraint`). No local/cloud constraints (D-017); no free-form constraint framework. 4. **Model catalog entry** — stable `provider`/`model`/`variant` identity separate from display name, quota bucket, provider aliases and model class; hard properties; context/output allowances; per-dimension ratings with provenance (source, source version/date, assessment date, confidence, rationale); explicit reviewable human override; capacity bindings. Initial catalog only: GPT-5.6 Luna, GPT-5.6 Sol, GLM-5.3, GLM-5.3-Flash. No ratings populated (M2c). 5. **Capacity applicability (critical M2 gate, D-020)** — a provider-independent `capacity scope` concept: `(provider, scope_id)` identifies one normalized scope; a model binds to one or more scopes; all applicable scopes participate; no optimistic single-scope selection. The v2 `window_id` is diagnostic and selector code must never parse it for applicability. **The first M2 implementation prerequisite is a new normalized capacity-contract version adding an explicit semantic `CapacityWindow.scope_id`** (conceptual shape only here; no schema v3 in this PR, no production capacity class changes). 6. **Scarcity aggregation boundary** — only applicable scopes participate; all relevant windows within an applicable scope participate; most restrictive governs under `balanced`; a healthy short window never hides a critical weekly window; an unrelated bucket never penalizes a candidate; unknown applicability is explicit. No formula or thresholds yet (U-007). 7. **Evidence/policy precedence (D-021)** — hard user policy (blackout ⇒ `policy_blocked`, never `unavailable`, never rewrites telemetry) > direct capacity evidence > provider-native failure evidence > official public status (advisory degraded confidence only) > external benchmarks (catalog curation only). 8. **Replenishment (D-021)** — reset credits are replenishment opportunities, not current capacity; minimal safe `ReplenishmentState` (`provider`, `kind`, `available_count`, `details_known`, optional `earliest_expiry`, `retrieved_at`); no free-text titles/descriptions; no opaque credit IDs required. 9. **Artificial Analysis boundary (D-021)** — offline/periodic catalog evidence; not runtime capacity, not automatic truth, never called during `select()`; required provenance fields for any future snapshot; API key stays server-side; no response hoarding. 10. **ExecutionBudget (D-022)** — `max_total_model_calls`, `max_legs`, `max_review_rounds`, `max_remediation_rounds`, `max_retries_per_leg`, `max_wall_clock_minutes`; all finite/bounded, no unlimited sentinel; a compound recommendation without a complete valid budget is invalid; structural maxima for `single`/`cascade`/`critique`; never "review/fix until clean"; call-count and scope accounting explicit even when token cost is unknown. Also: HydraFusion positioning updated to the GitHub Copilot CLI Research Preview reality with the sharp distinction (it chooses and executes workflows; Scarcity Router decides which scarce capable resource or bounded plan to spend, without executing) and the roadmap now records the staged sequence **M2a → M2b → M2c → M2d → M2e** with later optional additions (health advisory, cached AA evidence, compound recommendations) explicitly non-blocking. The one stale M1 wording ("read-only subscription collectors") is corrected per D-018; M1 is not otherwise reopened. ## Deliberately unresolved - Exact model ratings and profile numeric minima (U-006, M2c). - Scarcity penalty function and label thresholds (U-007, M2c/M2d scenario calibration). - Final v3 scope-identifier field shape beyond the recorded direction (M2a PR). - Whether unknown capability may proceed under which policies (explicit policy work in M2d). - Actual AA integration, official health advisory integration, compound recommendation implementation (optional, after core selector). ## Validation - `uv sync --only-dev` ✅ - `make check`: 431 tests pass; basedpyright **0 errors, 0 warnings, 0 notes** ✅ - `git diff --check origin/main` clean; `origin/main..HEAD` contains exactly one commit; no merge commits ✅ - Docs-only diff (9 files); no production selector code, no credentials, no personal quota values, no real bucket IDs; `model-policy.json` unchanged ✅ Human merge gate: leaving this PR **open and unmerged**; M2a does not start before it is merged.

creatidy commented

2026-09-05 22:09:06 +00:00

(Migrated from github.com)

Single bounded remediation applied on top of reviewed head 9abac09 (new head 2451048).

MERGE_BLOCKER resolved — ExecutionBudget numeric domain (D-022). The contract required a complete budget including max_review_rounds, max_remediation_rounds and max_retries_per_leg while demanding every value be a finite positive integer, which made a hard upper bound of zero unrepresentable. The numeric domain is now frozen per field:

max_total_model_calls    integer >= 1
max_legs                 integer >= 1
max_review_rounds        integer >= 0
max_remediation_rounds   integer >= 0
max_retries_per_leg      integer >= 0
max_wall_clock_minutes   integer >= 1

Every value remains finite with a documented upper bound; still no unlimited sentinel, infinity, null = unlimited or omitted field = unlimited. Permission semantics are explicit: maxima are hard upper bounds, zero means not permitted by this recommendation (never unknown/missing/unlimited/invalid), and a prohibited phase is never encoded as an artificial budget of 1. Illustrative budgets added for single (zero review/remediation/retry), cascade (escalation leg is a leg, not a review round) and critique (positive maxima permitted, zero legal). D-022 carries an amendment note; no other contract changed.

Changed files: docs/selection-policy.md, docs/decisions.md only. Validation: 431 tests pass, basedpyright 0 errors / 0 warnings / 0 notes, no production code changes, model-policy.json unchanged, no merge commits. Ready for narrow final verification.

Single bounded remediation applied on top of reviewed head `9abac09` (new head `2451048`). **MERGE_BLOCKER resolved — ExecutionBudget numeric domain (D-022).** The contract required a complete budget including `max_review_rounds`, `max_remediation_rounds` and `max_retries_per_leg` while demanding every value be a finite *positive* integer, which made a hard upper bound of zero unrepresentable. The numeric domain is now frozen per field: ```text max_total_model_calls integer >= 1 max_legs integer >= 1 max_review_rounds integer >= 0 max_remediation_rounds integer >= 0 max_retries_per_leg integer >= 0 max_wall_clock_minutes integer >= 1 ``` Every value remains finite with a documented upper bound; still no unlimited sentinel, infinity, `null = unlimited` or omitted field = unlimited. Permission semantics are explicit: maxima are hard upper bounds, zero means *not permitted by this recommendation* (never unknown/missing/unlimited/invalid), and a prohibited phase is never encoded as an artificial budget of `1`. Illustrative budgets added for `single` (zero review/remediation/retry), `cascade` (escalation leg is a leg, not a review round) and `critique` (positive maxima permitted, zero legal). D-022 carries an amendment note; no other contract changed. Changed files: `docs/selection-policy.md`, `docs/decisions.md` only. Validation: 431 tests pass, basedpyright 0 errors / 0 warnings / 0 notes, no production code changes, `model-policy.json` unchanged, no merge commits. Ready for narrow final verification.