M2c: Calibrate initial model catalog and task profiles #36

New issue

Closed

opened 2026-09-06 10:35:12 +00:00 by creatidy · 0 comments

creatidy commented

2026-09-06 10:35:12 +00:00

(Migrated from github.com)

Summary

M2c populates the first accepted CONTENT for the frozen M2a/M2b contracts:

exactly four real model entries in a new model-catalog.json (GPT-5.6 Luna Max, GPT-5.6 Sol High, GLM-5.3 Max, GLM-5.3-Flash Max);
curated six-dimensional capability ratings (frozen 1..5 ordinal scale) with provenance for every known rating;
evidenced hard properties (tri-state; unknown stays unknown);
explicit semantic capacity bindings (M2a v3 scope identities: openai/codex, zai/coding_plan);
the eight calibrated task profiles frozen as calibrated_requirement inside the existing descriptive model-policy.json vocabulary;
one pure authoritative profile-expansion mechanism (TaskProfileDefinition + TaskProfileCatalog.resolve) in scarcity_router/selection_types.py;
calibration acceptance scenario tests proving the capability-only eligible sets have the intended shape.

Non-scope

No scarcity penalty, no reservations, no capacity aggregation, no selector/ranking, no runtime profile loading, no AA client, no provider/capacity/status changes, no catalog expansion beyond the four accepted identities.

Governance

Resolves U-006 (initial capability ratings and profile thresholds) via a new D-025 decision record.
U-007 (scarcity parameters) remains open for M2d.
External evidence is one bounded pass (first-party OpenAI ≤3 pages, first-party Z.ai ≤4 pages, Artificial Analysis ≤4 pages) used only to verify and cite provenance for the accepted calibration — not to invent a different one.

Acceptance

model-catalog.json parses through ModelCatalog.from_dict, deterministic round-trip, exactly four entries, no HumanOverride, complete provenance.
model-policy.json: schema_version stays 1, policy_version increments, numeric minima marked calibrated, exactly one calibrated_requirement per profile.
Capability-only scenario expectations pass (mechanical/routine_coding/general_reasoning: all four; deep_coding: Sol+GLM-5.3; scientific_review/translation: Sol; editorial/orchestration: Luna+Sol).
make check exits 0 (unittest + basedpyright).

## Summary M2c populates the first accepted CONTENT for the frozen M2a/M2b contracts: 1. exactly four real model entries in a new `model-catalog.json` (GPT-5.6 Luna Max, GPT-5.6 Sol High, GLM-5.3 Max, GLM-5.3-Flash Max); 2. curated six-dimensional capability ratings (frozen 1..5 ordinal scale) with provenance for every known rating; 3. evidenced hard properties (tri-state; unknown stays unknown); 4. explicit semantic capacity bindings (M2a v3 scope identities: `openai/codex`, `zai/coding_plan`); 5. the eight calibrated task profiles frozen as `calibrated_requirement` inside the existing descriptive `model-policy.json` vocabulary; 6. one pure authoritative profile-expansion mechanism (`TaskProfileDefinition` + `TaskProfileCatalog.resolve`) in `scarcity_router/selection_types.py`; 7. calibration acceptance scenario tests proving the capability-only eligible sets have the intended shape. ## Non-scope No scarcity penalty, no reservations, no capacity aggregation, no selector/ranking, no runtime profile loading, no AA client, no provider/capacity/status changes, no catalog expansion beyond the four accepted identities. ## Governance - Resolves U-006 (initial capability ratings and profile thresholds) via a new D-025 decision record. - U-007 (scarcity parameters) remains open for M2d. - External evidence is one bounded pass (first-party OpenAI ≤3 pages, first-party Z.ai ≤4 pages, Artificial Analysis ≤4 pages) used only to verify and cite provenance for the accepted calibration — not to invent a different one. ## Acceptance - `model-catalog.json` parses through `ModelCatalog.from_dict`, deterministic round-trip, exactly four entries, no HumanOverride, complete provenance. - `model-policy.json`: schema_version stays 1, policy_version increments, numeric minima marked calibrated, exactly one `calibrated_requirement` per profile. - Capability-only scenario expectations pass (mechanical/routine_coding/general_reasoning: all four; deep_coding: Sol+GLM-5.3; scientific_review/translation: Sol; editorial/orchestration: Luna+Sol). - `make check` exits 0 (unittest + basedpyright).