Agent discoverability pass: intent-differentiated MCP tool descriptions + README selection boundaries (#162) #163

Merged

adrian.tkacz merged 3 commits from docs/162-agent-discoverability into develop 2026-09-29 21:09:05 +00:00

Conversation 2 Commits 3 Files changed 3 +96 -11

kilo commented 2026-09-29 07:15:29 +00:00

Member

Copy link

Closes #162 when merged (issue stays open until human review completes; agents never merge). Representation-only: no routing semantics, authority boundary, machine-interface shape or maturity claim changes. Baseline: canonical develop @ 59538e9; this PR is exactly one commit, 9ed9675, touching only scarcity_router/mcp.py, tests/test_mcp.py, README.md.

What changed and why

1. MCP tool descriptions (scarcity_router/mcp.py) — the material defect. All three tools previously shared one generic prefix and differed only by which REST endpoint they mirror; a retrieval system ranking on name + short description could not separate them by intent, and scarcity_status's description said it "reports or recommends" (false — status never recommends) while never mentioning capacity/quota, the only thing it returns. Each description now states: the intent it serves, its actual output, when to choose it, and which neighbouring tool serves the neighbouring intent — grounded in the already-authoritative examples/agent-routing-prompt.md semantics and the implemented envelopes (snapshots; decision with alternatives/exclusions/reasons and explicit no-solution; SimulationResult = baseline + simulated under typed overrides such as evaluated_at and window-percentage replacement).

Preserved exactly (D-028 and tests/test_mcp.py pins): tool names stay simple and unversioned; every description still opens with "Scarcity Router machine-interface contract v1" and still states it never executes model inference; input schemas, property names/types, required, additionalProperties, envelopes and logical error payloads are byte-identical. No new decision number is needed — D-028 froze names, the contract-v1 documentation statement and the wire contract, not this prose.

2. Test pin (tests/test_mcp.py). The existing test_tool_descriptions_and_advertised_fields gains assertions that each description carries its differentiator ("only reports current state" / "only recommends" / "only answers hypotheticals") and the cross-reference to the neighbouring tool, so the intent split cannot silently regress. No new test framework.

3. README (README.md). One compact "When To Use It" section between the opening paragraph and "What It Does": positive selection boundary (multiple subscription plans, rolling quota windows, want a reproducible routing answer), three explicit negative boundaries (generic OpenAI-compatible reverse proxy — the optional execution gateway serves only explicitly configured authorized resources; autonomous coding agent/orchestrator; hosted multi-user quota pooling), and one honest maturity line (pre-release v0.1.0, not on any package index or registry). No restructuring; all existing prose depth preserved.

Discoverability baseline (stable intent set, issue #162)

Intent	Before	After
Direct	CORRECT_POSITIVE (README/repo description)	unchanged
Indirect positive	CORRECT_POSITIVE	unchanged
Tool-selection → scarcity_select	AMBIGUOUS at MCP surface	CORRECT_POSITIVE (select is named "the routing-decision tool"; neighbours point to it)
Status → scarcity_status	AMBIGUOUS (description claimed "recommends", never mentioned capacity)	CORRECT_POSITIVE (only-reports + snapshots + cross-ref)
Simulation → scarcity_simulate	AMBIGUOUS (no hypothetical semantics in metadata)	CORRECT_POSITIVE (what-if + baseline/simulated comparison + "not the dispatch decision")
Negative: generic gateway	CORRECT_NEGATIVE (README two-mode boundary)	unchanged, sharpened by the new block
Negative: autonomous coding agent	CORRECT_NEGATIVE	unchanged, sharpened
Negative: hosted quota pooling	CORRECT_NEGATIVE	unchanged, sharpened

MCP-surface verdicts are static-metadata assessments, not claims about any model's tool-selection behavior.

Intentionally unchanged

Tool identifiers, schemas, envelopes, error payloads, versioning; CLI/REST surfaces (already well-differentiated); examples/agent-routing-prompt.md (remains the deep routing authority); docs/machine-interfaces.md (its "descriptions state contract v1" sentence remains true); no llms.txt/ARD/registry/topics mutation (owner actions; recommendations in the issue); no Forgejo/GitHub metadata changes.

Gates (exact head 9ed9675)

• uv sync --only-dev OK; make check green in one run — 2190 tests OK; uv run basedpyright 0 errors / 0 warnings / 0 notes; git diff --check clean.
• One intermittent failure in the first full-suite run: test_openai_codex_acquisition.MalformedAndBoundedOutput.test_oversized_line_with_secret_never_leaks (a stderr thread-exception race under full-suite load, in a path this diff does not touch). It passed 3/3 in isolation and the final full run was green — same class as the documented #143 load-sensitive family, not folded in.
• Diff manually inspected: only the three files above; origin/develop..HEAD is one commit, no merge commits.

UX impact: none-to-positive — no workflow, configuration or interface change; the only user-visible difference is richer, accurate tool descriptions and a scannable README selection block. No new required configuration; no first-run regression.

Per repository governance: PR left open for human review; workers never merge.

Closes #162 when merged (issue stays open until human review completes; agents never merge). Representation-only: **no routing semantics, authority boundary, machine-interface shape or maturity claim changes.** Baseline: canonical `develop` @ `59538e9`; this PR is exactly one commit, `9ed9675`, touching only `scarcity_router/mcp.py`, `tests/test_mcp.py`, `README.md`. ## What changed and why **1. MCP tool descriptions (`scarcity_router/mcp.py`) — the material defect.** All three tools previously shared one generic prefix and differed only by which REST endpoint they mirror; a retrieval system ranking on name + short description could not separate them by intent, and `scarcity_status`'s description said it "reports **or recommends**" (false — status never recommends) while never mentioning capacity/quota, the only thing it returns. Each description now states: the intent it serves, its actual output, when to choose it, and which neighbouring tool serves the neighbouring intent — grounded in the already-authoritative `examples/agent-routing-prompt.md` semantics and the implemented envelopes (`snapshots`; `decision` with alternatives/exclusions/reasons and explicit no-solution; `SimulationResult` = baseline + simulated under typed overrides such as `evaluated_at` and window-percentage replacement). **Preserved exactly (D-028 and `tests/test_mcp.py` pins):** tool names stay simple and unversioned; every description still opens with "Scarcity Router machine-interface contract v1" and still states it never executes model inference; input schemas, property names/types, `required`, `additionalProperties`, envelopes and logical error payloads are byte-identical. No new decision number is needed — D-028 froze names, the contract-v1 documentation statement and the wire contract, not this prose. **2. Test pin (`tests/test_mcp.py`).** The existing `test_tool_descriptions_and_advertised_fields` gains assertions that each description carries its differentiator ("only reports current state" / "only recommends" / "only answers hypotheticals") and the cross-reference to the neighbouring tool, so the intent split cannot silently regress. No new test framework. **3. README (`README.md`).** One compact "When To Use It" section between the opening paragraph and "What It Does": positive selection boundary (multiple subscription plans, rolling quota windows, want a reproducible routing answer), three explicit negative boundaries (generic OpenAI-compatible reverse proxy — the optional execution gateway serves only explicitly configured authorized resources; autonomous coding agent/orchestrator; hosted multi-user quota pooling), and one honest maturity line (pre-release v0.1.0, not on any package index or registry). No restructuring; all existing prose depth preserved. ## Discoverability baseline (stable intent set, issue #162) | Intent | Before | After | |---|---|---| | Direct | CORRECT_POSITIVE (README/repo description) | unchanged | | Indirect positive | CORRECT_POSITIVE | unchanged | | Tool-selection → `scarcity_select` | AMBIGUOUS at MCP surface | CORRECT_POSITIVE (select is named "the routing-decision tool"; neighbours point to it) | | Status → `scarcity_status` | AMBIGUOUS (description claimed "recommends", never mentioned capacity) | CORRECT_POSITIVE (only-reports + snapshots + cross-ref) | | Simulation → `scarcity_simulate` | AMBIGUOUS (no hypothetical semantics in metadata) | CORRECT_POSITIVE (what-if + baseline/simulated comparison + "not the dispatch decision") | | Negative: generic gateway | CORRECT_NEGATIVE (README two-mode boundary) | unchanged, sharpened by the new block | | Negative: autonomous coding agent | CORRECT_NEGATIVE | unchanged, sharpened | | Negative: hosted quota pooling | CORRECT_NEGATIVE | unchanged, sharpened | MCP-surface verdicts are static-metadata assessments, **not** claims about any model's tool-selection behavior. ## Intentionally unchanged Tool identifiers, schemas, envelopes, error payloads, versioning; CLI/REST surfaces (already well-differentiated); `examples/agent-routing-prompt.md` (remains the deep routing authority); `docs/machine-interfaces.md` (its "descriptions state contract v1" sentence remains true); no `llms.txt`/ARD/registry/topics mutation (owner actions; recommendations in the issue); no Forgejo/GitHub metadata changes. ## Gates (exact head `9ed9675`) - `uv sync --only-dev` OK; **`make check` green in one run — 2190 tests OK; `uv run basedpyright` 0 errors / 0 warnings / 0 notes; `git diff --check` clean.** - One intermittent failure in the first full-suite run: `test_openai_codex_acquisition.MalformedAndBoundedOutput.test_oversized_line_with_secret_never_leaks` (a stderr thread-exception race under full-suite load, in a path this diff does not touch). It passed 3/3 in isolation and the final full run was green — same class as the documented #143 load-sensitive family, not folded in. - Diff manually inspected: only the three files above; `origin/develop..HEAD` is one commit, no merge commits. **UX impact: none-to-positive** — no workflow, configuration or interface change; the only user-visible difference is richer, accurate tool descriptions and a scannable README selection block. No new required configuration; no first-run regression. Per repository governance: PR left open for human review; workers never merge.

kilo added 1 commit 2026-09-29 07:15:29 +00:00

docs: agent discoverability pass — intent-differentiated MCP tool descriptions and README selection boundaries (#162) ... 9ed9675d14

The three MCP tool descriptions shared one generic prefix and differed
only by which REST endpoint they mirror, so tool-selection systems could
not distinguish them by intent; scarcity_status even claimed it
'reports or recommends'. Each description now states its own intent,
output and when the neighbouring tool is preferable, keeping the D-028
statement (contract v1, never executes model inference) and all tool
names, schemas and envelopes unchanged. README gains a compact
use-it-when / do-not-use-it-when block and an honest maturity line.
The existing description test now pins the differentiation.

adrian.tkacz referenced this pull request from a commit 2026-09-29 20:29:15 +00:00

fix: review remediation on #163 — truthful status/select semantics and acceptance wording (#162)

adrian.tkacz added 1 commit 2026-09-29 20:29:15 +00:00

fix: review remediation on #163 — truthful status/select semantics and acceptance wording (#162) ... 332f17221d

Three MAJOR wording defects from independent review, nothing else:

- scarcity_select no longer claims the 'least scarce capable' (global
  minimization) result: capability and hard constraints gate eligibility,
  capacity/scarcity and the configured selector policy rank the eligible
  candidates, and quota-preference policy may outrank better scarcity
  (docs/selection-policy.md ranking order); the tool recommends the
  top-ranked configuration.
- scarcity_status no longer claims 'current state'/'health': it reports
  collected observations — normalized snapshots with observed windows,
  validated reset information where known and collection/source status —
  explicitly not a real-time availability guarantee (docs/capacity-model.md).
- README maturity line distinguishes the green automated repository gate
  from live/external acceptance work tracked in docs/m10-acceptance.md.

Test markers updated to pin the semantic distinctions and to keep the two
removed overclaims out. Tool names, schemas, envelopes, error contracts
and the D-028 contract-v1 statements unchanged; scarcity_simulate
untouched. Full gate green on this head (2190 tests, basedpyright 0/0/0).

kilo commented 2026-09-29 20:29:33 +00:00

Author

Member

Copy link

Review remediation (REQUEST_CHANGES, 3 × MAJOR) — prior reviewed HEAD 9ed9675 → new HEAD 332f172

One remediation commit on the same branch; base still develop (now @ f79f452 — advanced past the PR's merge base 59538e9 after #157-family integration; the PR remains mergeable and the branch was deliberately not merged with develop per governance). Only scarcity_router/mcp.py, tests/test_mcp.py, README.md changed; scarcity_simulate was already accurate and is untouched.

Finding 1 — scarcity_select optimization overclaim. Problem: the description said the tool "returns the least scarce capable model configuration" — a global-minimization/optimality claim stronger than the implemented algorithm (eligibility stages + policy-aware ranking; e.g. an active happy-hour quota-preference can rank a candidate ahead of one with better scarcity, per docs/selection-policy.md's exact ranking order). Remediation: the description now says the tool "gates candidates on capability and hard constraints, then ranks the eligible ones by current capacity/scarcity and the configured selector policy — quota-preference policy may outrank better scarcity — and recommends the top-ranked configuration with ordered alternatives, exclusions and reasons" (no-solution stays explicit and valid). No minimization/optimality/minimum-scarcity wording remains.

Finding 2 — scarcity_status observation/freshness overclaim. Problem: "only reports current state" and "health" implied real-time availability guarantees the implementation does not make (docs/capacity-model.md: capacity "describes an observation"; "v3 does not claim that a snapshot is fresh merely because it has a timestamp"). Remediation: the description now says "only reports observations", lists "observed quota windows, validated reset information where known, collection/source status, explicit failures and unknowns", keeps "makes no recommendation", and adds "Snapshots are observations at collection time, not a real-time availability guarantee."

Finding 3 — README acceptance overclaim. Problem: "green under this repository's full test and acceptance gates" implied external acceptance is complete while live Windows/Codex acceptance gates remain open. Remediation: the maturity line now reads "the automated repository gate (unit tests and type checks) is green, while live and external acceptance work remains tracked separately in docs/m10-acceptance.md". No surrounding maturity text or links removed; no completion of external acceptance claimed.

Test protection (tests/test_mcp.py, same test, semantic pins not prose snapshots): status → asserts "only reports observations", "snapshot", "collection/source status" and asserts absent the removed overclaims "current state"/"health"; select → asserts "selector policy", "eligible", "only recommends" and asserts absent "least scarce capable"; simulate pins unchanged. Existing contract-v1 and "never executes model inference" assertions untouched and still green.

Contract preservation: no MCP tool names, input schemas, required fields, property names, output envelopes, error contracts or version markers changed — the diff touches description strings, test assertions and one README paragraph only. Routing/capacity/policy algorithms and provider behavior untouched. The D-028 statement ("Scarcity Router machine-interface contract v1") and the no-inference statement remain present in all three descriptions.

Validation (exact head 332f172): uv run python -m unittest tests.test_mcp → 18 tests OK; make check green in one run — 2190 tests OK; uv run basedpyright 0 errors / 0 warnings / 0 notes; git diff --check clean. Diff vs 9ed9675 manually inspected: 3 files, +35/−16, no other file touched, no merge commits.

Ready for a fresh independent review; the PR stays open — workers never merge.

## Review remediation (REQUEST_CHANGES, 3 × MAJOR) — prior reviewed HEAD `9ed9675` → new HEAD `332f172` One remediation commit on the same branch; base still `develop` (now @ `f79f452` — advanced past the PR's merge base `59538e9` after #157-family integration; the PR remains mergeable and the branch was deliberately **not** merged with develop per governance). Only `scarcity_router/mcp.py`, `tests/test_mcp.py`, `README.md` changed; `scarcity_simulate` was already accurate and is untouched. **Finding 1 — `scarcity_select` optimization overclaim.** *Problem:* the description said the tool "returns the least scarce capable model configuration" — a global-minimization/optimality claim stronger than the implemented algorithm (eligibility stages + policy-aware ranking; e.g. an active happy-hour quota-preference can rank a candidate ahead of one with better scarcity, per `docs/selection-policy.md`'s exact ranking order). *Remediation:* the description now says the tool "gates candidates on capability and hard constraints, then ranks the eligible ones by current capacity/scarcity and the configured selector policy — quota-preference policy may outrank better scarcity — and recommends the **top-ranked** configuration with ordered alternatives, exclusions and reasons" (no-solution stays explicit and valid). No minimization/optimality/minimum-scarcity wording remains. **Finding 2 — `scarcity_status` observation/freshness overclaim.** *Problem:* "only reports current state" and "health" implied real-time availability guarantees the implementation does not make (`docs/capacity-model.md`: capacity "describes an observation"; "v3 does not claim that a snapshot is fresh merely because it has a timestamp"). *Remediation:* the description now says "only reports **observations**", lists "observed quota windows, validated reset information where known, **collection/source status**, explicit failures and unknowns", keeps "makes no recommendation", and adds "Snapshots are observations at collection time, **not a real-time availability guarantee**." **Finding 3 — README acceptance overclaim.** *Problem:* "green under this repository's full test and acceptance gates" implied external acceptance is complete while live Windows/Codex acceptance gates remain open. *Remediation:* the maturity line now reads "the automated repository gate (unit tests and type checks) is green, while **live and external acceptance work remains tracked separately in `docs/m10-acceptance.md`**". No surrounding maturity text or links removed; no completion of external acceptance claimed. **Test protection (`tests/test_mcp.py`, same test, semantic pins not prose snapshots):** status → asserts "only reports observations", "snapshot", "collection/source status" and asserts **absent** the removed overclaims "current state"/"health"; select → asserts "selector policy", "eligible", "only recommends" and asserts **absent** "least scarce capable"; simulate pins unchanged. Existing contract-v1 and "never executes model inference" assertions untouched and still green. **Contract preservation:** no MCP tool names, input schemas, required fields, property names, output envelopes, error contracts or version markers changed — the diff touches description strings, test assertions and one README paragraph only. Routing/capacity/policy algorithms and provider behavior untouched. The D-028 statement ("Scarcity Router machine-interface contract v1") and the no-inference statement remain present in all three descriptions. **Validation (exact head `332f172`):** `uv run python -m unittest tests.test_mcp` → 18 tests OK; **`make check` green in one run — 2190 tests OK; `uv run basedpyright` 0 errors / 0 warnings / 0 notes; `git diff --check` clean.** Diff vs `9ed9675` manually inspected: 3 files, +35/−16, no other file touched, no merge commits. Ready for a fresh independent review; the PR stays open — workers never merge.

adrian.tkacz approved these changes 2026-09-29 21:08:01 +00:00

adrian.tkacz added 1 commit 2026-09-29 21:08:10 +00:00

Merge branch 'develop' into docs/162-agent-discoverability e8f59701cb

adrian.tkacz merged commit 8d6d59e0a4 into develop 2026-09-29 21:09:05 +00:00

adrian.tkacz deleted branch docs/162-agent-discoverability 2026-09-29 21:09:05 +00:00

adrian.tkacz referenced this pull request from a commit 2026-09-29 21:09:07 +00:00

Merge pull request 'Agent discoverability pass: intent-differentiated MCP tool descriptions + README selection boundaries (#162)' (#163) from docs/162-agent-discoverability into develop

kilo referenced this pull request 2026-09-29 22:43:14 +00:00

Reasoning output semantics: preserve or fail closed (D-062, #158) #160

Sign in to join this conversation.

Reviewers

No reviewers

adrian.tkacz

Labels

Clear labels   

accessibility

Barrier affecting people with disabilities

  

bug

Something isn't working

  

documentation

Improvements or additions to documentation

  

duplicate

This issue or pull request already exists

  

enhancement

New feature or request

  

good first issue

Good for newcomers

  

help wanted

Extra attention is needed

  

invalid

This doesn't seem right

  

P1

Owner-declared highest explicit implementation priority (loop ordering rank)

  

P2

Required product and integration work

  

P3

Deferred or producer-dependent work

  

question

Further information is requested

  

wontfix

This will not be worked on

No labels

accessibility

bug

documentation

duplicate

enhancement

good first issue

help wanted

invalid

P1

P2

P3

question

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Creatidy/scarcity-router!163

No description provided.