M1: Implement Z.ai quota response parser #7

Merged

creatidy merged 3 commits from m1-zai-capacity-parser into main

2026-09-03 14:37:53 +00:00

creatidy commented

2026-09-03 12:58:44 +00:00

(Migrated from github.com)

Closes #6

Summary

Pure provider-edge parser for the Z.ai Coding Plan quota response, normalizing
a decoded payload plus a caller-supplied retrieval timestamp into the v1
CapacitySnapshot contract (provider=zai, source=zai_usage_endpoint).
Live acquisition (credential discovery, authenticated HTTP) is explicitly
not implemented and remains future work.

Commits on this branch (no merge commits):

198da5e — initial parser, fixture tests, docs status note
9d588c1 — review hardening (reset-unit validation, plan allowlist, structural fail-closed limits, required level key, total window-ID generation)
15657ba — provider package docstring terminology correction (parser scope, not collection)

Parser boundary

scarcity_router/providers/zai.py exposes exactly one public function:

parse_zai_quota_response(payload: object, *, retrieved_at: str) -> CapacitySnapshot

Pure parsing + normalization only. Zero I/O: no clock, no filesystem, no
environment, no network, no credentials; fully deterministic (same inputs →
identical snapshot). Failure paths use narrow deliberate checks — no broad
Exception, no raw KeyError/TypeError leakage.

Envelope and success-schema validation

Envelope validated deliberately: code (int, non-bool), msg (str),
success (bool), data present. Broken envelope → schema_changed.
code == 401 → status=auth_required, windows=[], auth_required
diagnostic; provider error text never enters output.
Well-formed non-success other than 401 (e.g. 500, 200+success=false, 403)
→ status=unknown + telemetry_unknown (unevidenced as anything else).
For a successful response the evidenced schema structurally requires:
data object, the data.level key present, data.limits a list, and
every limits[] entry an object carrying a string type. Any violation →
schema_changed with windows=[] — no partial decoding, healthy siblings
are not preserved. These are schema incompatibilities, not per-window
degradation.

Provider mappings (adapter-owned, per U-004 evidence)

TOKENS_LIMIT (unit=3, number=5) → tokens/five_hour, duration_seconds=18000.
TOKENS_LIMIT (unit=6, number=1) → tokens/weekly, duration_seconds=604800.
Classification only from validated (unit, number), never array position.
TIME_LIMIT → resource="time", distinct non-token, kind="unknown", no
duration_seconds (period semantics unevidenced), window_semantics_unknown.

Unknown windows and unknown types

Unknown (unit, number), missing or non-integer unit/number: window
preserved as tokens/unknown, no invented duration, deterministic safe
window_id, plus window_semantics_unknown. Known siblings survive.
A limits object with a valid but unevidenced string type (including
"") is preserved as unknown/unknown: no raw type text in output, no
provider-derived window_id, and percentage/reset are omitted with
explicit percentage_unknown/reset_unknown/window_semantics_unknown
diagnostics because their semantics are unvalidated for that type.
Missing weekly window: never synthesized, never represented as zero.
A degraded single window never escalates the snapshot away from ok.

Percentage normalization

Evidenced used orientation: valid non-bool integer percentage in 0..100 →
used_percent=percentage, remaining_percent=100-percentage, always emitted
together. percentage=0 stays known zero (0/100); 100 stays known exhaustion
(100/0). Missing, null, bool, string, float, negative, >100 → pair omitted +
window-scoped percentage_unknown. Never defaulted.

Reset normalization

nextResetTime is evidenced as a 13-digit epoch-millisecond integer;
only that representation normalizes: integer (bool-excluded), positive,
within the 13-digit band, representable → canonical UTC
YYYY-MM-DDTHH:MM:SS.sssZ via integer arithmetic (no float precision loss,
UTC only). Epoch-seconds-shaped values (e.g. 10-digit), zero, negatives,
12-/14-digit values, non-integers and unrepresentable magnitudes are
rejected — resets_at omitted + window-scoped reset_unknown — never
misinterpreted through another unit. No wall-clock comparison.

Plan normalization

data.level must be present (structural requirement above); its value
normalizes to plan only via the adapter evidence allowlist, currently
{"pro"}. Arbitrary safe-ID-shaped provider strings (plus,
user-123, sk-example, pro2, uppercase, punctuation) omit plan —
never sanitized, never copied into diagnostics.

Window identity

Deterministic adapter-owned ID from validated non-secret identity fields
only: fixed type token plus bounded components (tokens_limit-3-5,
time_limit-5-1). Generation is total: identity parts render as decimal
digits only when small non-negative integers, checked before any int→str
conversion, so arbitrary magnitudes (e.g. 10**5000) and negatives degrade
deterministically to a non-colliding placeholder instead of raising; the
result always satisfies the v1 safe-ID grammar and 64-character limit. No
array index, raw JSON, account data or provider free-text. Fully unknown
window types omit provider_metadata entirely.

Fixture coverage

All six existing fixtures are exercised and unchanged:
quota-200-known-windows (full normalization + order independence),
quota-200-unknown-window (sibling survives, no guessed period),
quota-200-weekly-missing (no synthesis), quota-200-degraded-values
(known-zero vs missing, omitted pair, null reset, valid degraded snapshot),
quota-schema-changed (fail-closed, no partial decode),
quota-401-auth-failed (auth_required, no error-text leak).

Additional focused tests: shuffled ordering; percentage 100 / negative /

100 / boolean / string; malformed and unrepresentable nextResetTime;
epoch-seconds misread prevention; zero/negative/12-/14-digit rejection;
13-digit lower bound; missing unit/number; non-integer identity fields;
huge/negative/oversized identity integers (no exceptions, valid snapshots);
structural drift (scalar entry, list entry, missing/non-string type,
healthy sibling not partially preserved); unknown valid string type;
unapproved plan strings; physically removed level key; empty limits;
unrecognized failure codes; deterministic repeated parse; v1 round-trip
validation of every fixture result and adversarial-normal result; serialized
output no-leak assertions (envelope keys, provider values, credential-shaped
strings); invalid retrieved_at raises the typed error; parser-module
purity source check.

Security / non-scope

No credential access, no Kilo auth reading, no environment inspection, no
HTTP/redirect/retry/timeout logic, no live requests; fixtures and tests
remain synthetic and redacted. Not implemented (explicitly out of scope):
live acquisition/collection, caching, freshness policy, CLI/REST/MCP, other
providers, capability catalog, selector, routing, generic provider
framework. No new dependencies; U-008 untouched.

Results

Tests: 137 passed, 0 failures (84 pre-existing + 53 added).
basedpyright (recommended, repo-managed uv env): 0 errors, 0 warnings,
0 notes; type suppressions: 0; no gate or config changes.
uv sync --only-dev + make check: exit 0.
git diff --check origin/main: clean; origin/main..HEAD: three scoped
commits, no merge commits.
Files: scarcity_router/providers/{__init__,zai}.py,
tests/test_zai_parser.py, docs/providers.md (status note only: Z.ai
response parser implemented and fixture-tested; live acquisition not
implemented; collector not marked done).
Fixtures: unchanged since origin/main.

Closes #6 ## Summary Pure provider-edge parser for the Z.ai Coding Plan quota response, normalizing a decoded payload plus a caller-supplied retrieval timestamp into the v1 `CapacitySnapshot` contract (`provider=zai`, `source=zai_usage_endpoint`). Live acquisition (credential discovery, authenticated HTTP) is explicitly **not** implemented and remains future work. Commits on this branch (no merge commits): - `198da5e` — initial parser, fixture tests, docs status note - `9d588c1` — review hardening (reset-unit validation, plan allowlist, structural fail-closed limits, required `level` key, total window-ID generation) - `15657ba` — provider package docstring terminology correction (parser scope, not collection) ## Parser boundary `scarcity_router/providers/zai.py` exposes exactly one public function: ```python parse_zai_quota_response(payload: object, *, retrieved_at: str) -> CapacitySnapshot ``` Pure parsing + normalization only. Zero I/O: no clock, no filesystem, no environment, no network, no credentials; fully deterministic (same inputs → identical snapshot). Failure paths use narrow deliberate checks — no broad `Exception`, no raw `KeyError`/`TypeError` leakage. ## Envelope and success-schema validation - Envelope validated deliberately: `code` (int, non-bool), `msg` (str), `success` (bool), `data` present. Broken envelope → `schema_changed`. - `code == 401` → `status=auth_required`, `windows=[]`, `auth_required` diagnostic; provider error text never enters output. - Well-formed non-success other than 401 (e.g. 500, 200+`success=false`, 403) → `status=unknown` + `telemetry_unknown` (unevidenced as anything else). - For a successful response the evidenced schema structurally requires: `data` object, the `data.level` **key present**, `data.limits` a list, and every `limits[]` entry an object carrying a string `type`. Any violation → `schema_changed` with `windows=[]` — no partial decoding, healthy siblings are not preserved. These are schema incompatibilities, not per-window degradation. ## Provider mappings (adapter-owned, per U-004 evidence) - `TOKENS_LIMIT (unit=3, number=5)` → `tokens/five_hour`, `duration_seconds=18000`. - `TOKENS_LIMIT (unit=6, number=1)` → `tokens/weekly`, `duration_seconds=604800`. Classification only from validated `(unit, number)`, never array position. - `TIME_LIMIT` → `resource="time"`, distinct non-token, `kind="unknown"`, no `duration_seconds` (period semantics unevidenced), `window_semantics_unknown`. ## Unknown windows and unknown types - Unknown `(unit, number)`, missing or non-integer `unit`/`number`: window preserved as `tokens/unknown`, no invented duration, deterministic safe `window_id`, plus `window_semantics_unknown`. Known siblings survive. - A limits object with a valid but unevidenced string `type` (including `""`) is preserved as `unknown/unknown`: no raw type text in output, no provider-derived `window_id`, and percentage/reset are omitted with explicit `percentage_unknown`/`reset_unknown`/`window_semantics_unknown` diagnostics because their semantics are unvalidated for that type. - Missing weekly window: never synthesized, never represented as zero. - A degraded single window never escalates the snapshot away from `ok`. ## Percentage normalization Evidenced used orientation: valid non-bool integer `percentage` in 0..100 → `used_percent=percentage`, `remaining_percent=100-percentage`, always emitted together. `percentage=0` stays known zero (0/100); 100 stays known exhaustion (100/0). Missing, null, bool, string, float, negative, >100 → pair omitted + window-scoped `percentage_unknown`. Never defaulted. ## Reset normalization `nextResetTime` is evidenced as a **13-digit epoch-millisecond integer**; only that representation normalizes: integer (bool-excluded), positive, within the 13-digit band, representable → canonical UTC `YYYY-MM-DDTHH:MM:SS.sssZ` via integer arithmetic (no float precision loss, UTC only). Epoch-seconds-shaped values (e.g. 10-digit), zero, negatives, 12-/14-digit values, non-integers and unrepresentable magnitudes are rejected — `resets_at` omitted + window-scoped `reset_unknown` — never misinterpreted through another unit. No wall-clock comparison. ## Plan normalization `data.level` must be present (structural requirement above); its value normalizes to `plan` only via the adapter evidence allowlist, currently `{"pro"}`. Arbitrary safe-ID-shaped provider strings (`plus`, `user-123`, `sk-example`, `pro2`, uppercase, punctuation) omit `plan` — never sanitized, never copied into diagnostics. ## Window identity Deterministic adapter-owned ID from validated non-secret identity fields only: fixed type token plus bounded components (`tokens_limit-3-5`, `time_limit-5-1`). Generation is total: identity parts render as decimal digits only when small non-negative integers, checked before any int→str conversion, so arbitrary magnitudes (e.g. `10**5000`) and negatives degrade deterministically to a non-colliding placeholder instead of raising; the result always satisfies the v1 safe-ID grammar and 64-character limit. No array index, raw JSON, account data or provider free-text. Fully unknown window types omit `provider_metadata` entirely. ## Fixture coverage All six existing fixtures are exercised and unchanged: `quota-200-known-windows` (full normalization + order independence), `quota-200-unknown-window` (sibling survives, no guessed period), `quota-200-weekly-missing` (no synthesis), `quota-200-degraded-values` (known-zero vs missing, omitted pair, null reset, valid degraded snapshot), `quota-schema-changed` (fail-closed, no partial decode), `quota-401-auth-failed` (`auth_required`, no error-text leak). Additional focused tests: shuffled ordering; percentage 100 / negative / >100 / boolean / string; malformed and unrepresentable `nextResetTime`; epoch-seconds misread prevention; zero/negative/12-/14-digit rejection; 13-digit lower bound; missing `unit`/`number`; non-integer identity fields; huge/negative/oversized identity integers (no exceptions, valid snapshots); structural drift (scalar entry, list entry, missing/non-string `type`, healthy sibling not partially preserved); unknown valid string type; unapproved plan strings; physically removed `level` key; empty `limits`; unrecognized failure codes; deterministic repeated parse; v1 round-trip validation of every fixture result and adversarial-normal result; serialized output no-leak assertions (envelope keys, provider values, credential-shaped strings); invalid `retrieved_at` raises the typed error; parser-module purity source check. ## Security / non-scope No credential access, no Kilo auth reading, no environment inspection, no HTTP/redirect/retry/timeout logic, no live requests; fixtures and tests remain synthetic and redacted. Not implemented (explicitly out of scope): live acquisition/collection, caching, freshness policy, CLI/REST/MCP, other providers, capability catalog, selector, routing, generic provider framework. No new dependencies; U-008 untouched. ## Results - Tests: **137 passed**, 0 failures (84 pre-existing + 53 added). - basedpyright (`recommended`, repo-managed uv env): **0 errors, 0 warnings, 0 notes**; type suppressions: **0**; no gate or config changes. - `uv sync --only-dev` + `make check`: **exit 0**. - `git diff --check origin/main`: clean; `origin/main..HEAD`: three scoped commits, no merge commits. - Files: `scarcity_router/providers/{__init__,zai}.py`, `tests/test_zai_parser.py`, `docs/providers.md` (status note only: Z.ai response parser implemented and fixture-tested; live acquisition not implemented; collector not marked done). - Fixtures: unchanged since `origin/main`.