Creatidy/creatidy-kernel
5
0
Fork 0
Code Issues 8 Pull requests 1 Projects Releases Packages Wiki Activity Actions 1

D1-06: Public self-contained operator path for Scarcity Router-backed task execution (#44) #45

Merged
adrian.tkacz merged 6 commits from issue-44-selfcontained-operator into develop 2026-10-04 17:21:47 +00:00
Conversation 2 Commits 6 Files changed 12 +2037 -79
kilo commented 2026-10-04 02:07:19 +00:00
Member
Copy link

Linked issue

Creatidy/creatidy-kernel#44 — D1-06: Public self-contained operator path for Scarcity Router-backed task execution.

Summary

Makes the public Kernel package self-contained for the bounded Scarcity Router-backed task operator path, without any creatidy-onprem dependency and without configuring a local Scarcity Router checkout path.

  • Kernel-owned source acquisition (adapters/source_cache.py): normal operation no longer requires CREATIDY_KERNEL_SOURCE_REPOSITORY. The controller acquires its own read-only cache from the TaskSpec canonical repository (${XDG_CACHE_HOME:-~/.cache}/creatidy-kernel/source/<org>/<repo>), re-verifies the exact canonical origin before every reuse, refreshes with a bounded fail-closed fetch, and never searches sibling directories or workspace layouts. --repo remains an advanced override with the existing exact verification. Acquisition is read-only with respect to the remote; the disposable candidate workspace stays a separate controller-owned tree; exact-base freezing is unchanged.
  • Codex discovery (adapters/codex_discovery.py): CREATIDY_KERNEL_CODEX_BIN/VERSION are optional. Deterministic single-candidate resolution (override, else first codex on the supplied PATH), a --version probe in the closed operational environment, version-shape validation shared with the runtime parser, and closed failure categories (not_found, probe_failed, unsupported_version). A unusable resolved installation is never silently substituted; no thread or turn is ever started during discovery/doctor/preflight.
  • User-local defaults: task state defaults to ${XDG_STATE_HOME:-~/.local/state}/creatidy-kernel/task; --data-dir/env overrides unchanged; SQLite storage-safety checks unchanged.
  • Generic operator profile support (adapters/operator_profile.py): bounded literal dotenv parsing, JSON-profile parsing, validation that reports key names only, and an optional generic SOPS adapter using a user-supplied profile plus the user's own SOPS key environment (--profile PATH --sops; explicit profile values override the ambient environment for that invocation). Not a secrets manager; no committed secrets, no Creatidy-specific recipients, hosts, or paths.
  • Public operator UX: creatidy-kernel config template | config validate, doctor, plus task preflight/run/status/export with defaults. doctor renders the same readiness gate with safe diagnostics and fixed hints.
  • Safe diagnostics: preflight blockers carry an optional closed category. Router failures distinguish endpoint unreachable / HTTP rejection / invalid response / no eligible selection / selection incompatible with the controller binding (real loopback-HTTP tests). Source failures distinguish acquisition / fetch / remote mismatch / checkout suitability / dirty override / exact-base problems.

Preserved (verified by the existing suite)

TaskSpec authority; exact-base freezing and durable recovery; controller-owned candidate creation; closed Codex child environment; inference-free preflight with no Forge writes; no automatic merge/deploy; trusted-development acknowledgment; #143/#166 durable identities; the Scarcity Router boundary through the ResourceAllocator port (no policy embedding, no routing framework).

Validation

  • make check: PASS (uv lock, ruff format/check, basedpyright strict 0 errors, lint-imports KEPT, 966 tests passed, secret scan clean, git diff --check).
  • make package-check: PASS (wheel/sdist rebuild, isolated install, CLI replay/export smoke).
  • make audit: not run (no dependency or tooling changes).

Acceptance evidence

  • New tests/test_source_cache.py and tests/test_selfcontained_operator.py prove, with real Git fixtures and native-shaped fakes, that a zero-inference doctor -> task preflight flow works with no source/state/codex configuration across two unrelated filesystem layouts, that a canonical sibling checkout is never adopted or touched, and that acquisition output is independent of checkout placement.
  • No live #166 execution was performed or authorized by this PR; no merge or deployment is included.
## Linked issue Creatidy/creatidy-kernel#44 — D1-06: Public self-contained operator path for Scarcity Router-backed task execution. ## Summary Makes the public Kernel package self-contained for the bounded Scarcity Router-backed task operator path, without any `creatidy-onprem` dependency and without configuring a local Scarcity Router checkout path. - **Kernel-owned source acquisition** (`adapters/source_cache.py`): normal operation no longer requires `CREATIDY_KERNEL_SOURCE_REPOSITORY`. The controller acquires its own read-only cache from the TaskSpec canonical repository (`${XDG_CACHE_HOME:-~/.cache}/creatidy-kernel/source/<org>/<repo>`), re-verifies the exact canonical origin before every reuse, refreshes with a bounded fail-closed fetch, and never searches sibling directories or workspace layouts. `--repo` remains an advanced override with the existing exact verification. Acquisition is read-only with respect to the remote; the disposable candidate workspace stays a separate controller-owned tree; exact-base freezing is unchanged. - **Codex discovery** (`adapters/codex_discovery.py`): `CREATIDY_KERNEL_CODEX_BIN`/`VERSION` are optional. Deterministic single-candidate resolution (override, else first `codex` on the supplied `PATH`), a `--version` probe in the closed operational environment, version-shape validation shared with the runtime parser, and closed failure categories (`not_found`, `probe_failed`, `unsupported_version`). A unusable resolved installation is never silently substituted; no thread or turn is ever started during discovery/doctor/preflight. - **User-local defaults**: task state defaults to `${XDG_STATE_HOME:-~/.local/state}/creatidy-kernel/task`; `--data-dir`/env overrides unchanged; SQLite storage-safety checks unchanged. - **Generic operator profile support** (`adapters/operator_profile.py`): bounded literal dotenv parsing, JSON-profile parsing, validation that reports key names only, and an optional generic SOPS adapter using a user-supplied profile plus the user's own SOPS key environment (`--profile PATH --sops`; explicit profile values override the ambient environment for that invocation). Not a secrets manager; no committed secrets, no Creatidy-specific recipients, hosts, or paths. - **Public operator UX**: `creatidy-kernel config template | config validate`, `doctor`, plus `task preflight/run/status/export` with defaults. `doctor` renders the same readiness gate with safe diagnostics and fixed hints. - **Safe diagnostics**: preflight blockers carry an optional closed `category`. Router failures distinguish endpoint unreachable / HTTP rejection / invalid response / no eligible selection / selection incompatible with the controller binding (real loopback-HTTP tests). Source failures distinguish acquisition / fetch / remote mismatch / checkout suitability / dirty override / exact-base problems. ## Preserved (verified by the existing suite) TaskSpec authority; exact-base freezing and durable recovery; controller-owned candidate creation; closed Codex child environment; inference-free preflight with no Forge writes; no automatic merge/deploy; trusted-development acknowledgment; #143/#166 durable identities; the Scarcity Router boundary through the `ResourceAllocator` port (no policy embedding, no routing framework). ## Validation - `make check`: PASS (uv lock, ruff format/check, basedpyright strict 0 errors, lint-imports KEPT, 966 tests passed, secret scan clean, `git diff --check`). - `make package-check`: PASS (wheel/sdist rebuild, isolated install, CLI replay/export smoke). - `make audit`: not run (no dependency or tooling changes). ## Acceptance evidence - New `tests/test_source_cache.py` and `tests/test_selfcontained_operator.py` prove, with real Git fixtures and native-shaped fakes, that a zero-inference `doctor -> task preflight` flow works with no source/state/codex configuration across two unrelated filesystem layouts, that a canonical sibling checkout is never adopted or touched, and that acquisition output is independent of checkout placement. - No live #166 execution was performed or authorized by this PR; no merge or deployment is included.
ScarcityRouterUnavailable now carries a closed RouterFailureCategory vocabulary
(endpoint_unreachable, http_rejected, invalid_response, no_eligible_selection,
selection_incompatible, request_unsupported) so operator readiness diagnostics
can distinguish real failure classes without echoing response bodies or free text.
acquire_source clones the TaskSpec canonical HTTPS repository into a
user-local XDG cache, re-verifies the exact origin before every reuse, refreshes
it with a bounded fail-closed fetch, and never searches the filesystem for
candidate checkouts. Failures use a closed safe category vocabulary.
resolve_codex deterministically resolves the explicit override binary or the
first codex on the supplied PATH and probes --version in the closed operational
environment. A single candidate is classified into closed safe categories and
never silently substituted; no thread or turn is ever started.
Generic bounded dotenv/JSON profile parsing plus an optional generic SOPS
decryption adapter over a user-supplied profile and the user's own SOPS key
environment. Validation reports key names only; values are never printed,
logged or serialized. No deployment-specific constants.
Preflight now acquires the controller-owned source cache from the TaskSpec
canonical repository when no explicit source is configured, fills missing Codex
fields from deterministic PATH discovery, defaults state to user-local XDG
paths, and reports closed safe categories on Router/source/Codex blockers.
The CLI gains config template/validate, a doctor diagnostic surface, optional
--profile/--sops loading, and default state roots, so the normal zero-inference
doctor -> task preflight path needs no private repository and no local checkout
configuration. TaskSpec authority, exact-base freezing, closed child
environments, inference-free preflight and recovery semantics are unchanged.
docs: document the public self-contained operator surface (D1-06, #44)
All checks were successful
ci / check (pull_request) Successful in 1m7s
Details
c2944f0da5
kilo reviewed 2026-10-04 09:16:07 +00:00
kilo left a comment
Copy link

Independent full-PR review of the frozen head c2944f0da538e60fb67f091bd70e66eea212e141 (base develop @ fa81734ed10bfbdd6e7a75b138649a89a9724511), performed by a fresh read-only reviewer session dispatched on GLM-5.3 per the owner-mandated model override for this delivery. The reviewer session is distinct from the implementation session and posted nothing itself; this submission persists its verdict. platform_identity_limitation: the configured reviewer identity is the same account that authored this PR, so Forgejo refuses a formal self-approval ("approve your own pull is not allowed"); the substantive verdict below is independent of that platform state, and forgejo_review_state for this review is COMMENT with platform_approval_recorded = false.

PR Review: D1-06: Public self-contained operator path for Scarcity Router-backed task execution (#44) (#45)

Repository: Creatidy/creatidy-kernel
Branch: issue-44-selfcontained-operator -> develop
Frozen head: c2944f0da5
Linked issue: Creatidy/creatidy-kernel#44 (D1-06 — Public self-contained operator path for Scarcity Router-backed task execution)

Frozen-input verification (run this review): Forgejo MCP get_pull_request_by_index(45) returned head issue-44-selfcontained-operator @ c2944f0da538e60fb67f091bd70e66eea212e141, base develop @ fa81734ed10bfbdd6e7a75b138649a89a9724511 (merge base identical), state open, not merged — matches the frozen inputs exactly. Local checkout: git rev-parse HEAD = c2944f0da538e60fb67f091bd70e66eea212e141, clean tree on the head branch. Issue #44 fetched via get_issue_by_index(44); 12 changed files confirmed via list_pull_request_files and git diff --stat fa81734e..c2944f0 (2037 insertions / 79 deletions).

Summary

The PR makes the public package's bounded-task operator path self-contained. It adds (1) a controller-owned read-only source cache acquired from the TaskSpec canonical remote (src/creatidy_kernel/adapters/source_cache.py, new), (2) deterministic single-candidate Codex discovery with a --version-only probe (adapters/codex_discovery.py, new), (3) generic dotenv/JSON operator profiles with an optional generic SOPS decrypt adapter (adapters/operator_profile.py, new), (4) safe closed diagnostic categories on preflight blockers plus Router failure categories (adapters/task_execution.py, adapters/scarcity_router.py), and (5) a public CLI surface config template|validate, doctor, task preflight/run/status/export with user-local XDG defaults (adapters/cli.py). No core or ports files change; README documents the new surface. All changes sit in adapters plus tests and README.

Intent / Acceptance Evidence

Issue criterion Evidence at frozen HEAD
1. Source ownership: normal path needs no CREATIDY_KERNEL_SOURCE_REPOSITORY; controller-owned read-only cache from canonical remote; exact origin verification; read-only remote; no sibling/home/workspace search; --repo stays an advanced override; no generic discovery framework task_execution.py:1837-1853 (resolve_source: override → configured field → canonical cache); task_execution.py:2008-2017 (preflight acquires when unset); source_cache.py:59-70 (URL-shape-validated cache key, no filesystem search anywhere in the module), source_cache.py:139-152 (exact origin re-verified), source_cache.py:168-184 (fresh clone with credential.helper=, http.followRedirects=false, --template=), source_cache.py:276-284 (bounded fail-closed fetch --prune origin refresh), source_cache.py:286-288 (foreign/corrupt entry replaced, never adopted); tests tests/test_source_cache.py:53-98 (two layouts, no placement leakage; canonical sibling never adopted or touched), :100-145 (reuse fetches, failed fetch fails closed)
2. Codex discovery: CODEX_BIN/CODEX_VERSION optional when an unambiguous supported Codex is on PATH; deterministic resolution; --version probe; shared inference-free validation; no silent substitution; no thread/turn codex_discovery.py:95-113 (override else first on PATH, single candidate); codex_discovery.py:63-83 (probe only --version, closed failure mapping, output never echoed); codex_discovery.py:25 + task_execution.py:1563 (one shared version pattern); task_execution.py:1936-1955 (environment_with_discovered_codex); tests tests/test_codex_discovery.py:42-64 (deterministic first-hit, override wins), test_selfcontained_operator.py:165-186 (preflight ready with no manual Codex fields)
3. XDG-compatible user-local state defaults, storage safety preserved, overrides allowed task_execution.py:1806-1811 (default_state_directory honoring absolute XDG_STATE_HOME); task_execution.py:1814-1834 (task_paths: CLI → field → default; source may be None); unchanged state gates at task_execution.py:2018-2052; cli.py:309-316 (status/export default); test test_selfcontained_operator.py:445-454
4. Generic runtime configuration/secrets: no new secrets manager, no committed secrets, no Creatidy recipients/.sops.yaml dependency, env config still supported, generic SOPS adapter, user-local profile paths, secrets never printed/serialized operator_profile.py:39-53 (bounded literal dotenv), :56-71 (JSON profile), :74-105 (generic SOPS: user-supplied profile, user's own SOPS_* env only), :108-127 (symlink/size-refusing loader), :130-141 (field-name-only diagnostics), :144-149 (XDG config default); tests test_operator_profile.py:93-131 (SOPS passthrough verified, failures safe), :147-157 (template generic, host/onprem absent)
5. Public operator UX covering config validation, doctor, preflight, run, status, export; installed CLI over Make targets cli.py:230-243 (config, doctor subcommands), cli.py:116-162 (_config), cli.py:164-202 (_doctor), cli.py:309-339 (status/export/preflight/run); no new Make targets; README "Public Operator Surface" (README.md:281-323); tests test_selfcontained_operator.py:345-442
6. Doctor/preflight fail-closed with safe closed diagnostics (router: unreachable/HTTP/invalid/no-eligible/incompatible; source: real acquisition/remote/base causes); no bodies or credentials exposed scarcity_router.py:38-46 (RouterFailureCategory), :385-394 (HTTP/encoding/deadline classified), :413-424 (request/no-eligible/incompatible), :463-472 (OSError → endpoint unreachable; raise outside handler); source_cache.py:34-40 (source categories); task_execution.py:2053-2089 (checkout/remote/dirty/base subcategories on explicit sources); cli.py:43-105 (fixed hints); tests test_selfcontained_operator.py:239-319 (real loopback HTTP for five router categories, secret-absence asserted), :188-225 (discovery categories)
7. Scarcity Router remains external through ResourceAllocator; no policy embedding, no Router-as-Runtime, no product copy, no routing framework Only category metadata was added to the existing adapter (scarcity_router.py diff); no new port, no policy data, no provider framework; make check ran lint-imports: "Adapters depend inward; core and ports never depend on adapters KEPT"
Acceptance: zero-inference doctor -> task preflight without creatidy-onprem and without a configured checkout path test_selfcontained_operator.py:364-415: CLI doctor then task preflight --json both READY with Codex from PATH, XDG state default, source acquired by the controller, inference_performed/forge_writes_performed false; :345-361: doctor with an empty environment fails closed with categories and hints
Acceptance: two unrelated filesystem layouts; acquisition independent of checkout placement test_source_cache.py:53-79 (two unrelated remote/state layouts, varying cache roots, same logical content, no placement strings in .git/config) and test_selfcontained_operator.py:117-142 (preflight parametrized over two unrelated layouts, same acquired cache evidence)
Constraints: no live #166 execution; no merge or deployment; no Creatidy-specific constants/hosts/recipients in the public product Diff adds no execution/Forge-write path and no deployment code; grep of the new/changed files found no hosts, recipients or onprem references (sole negative assertion at test_operator_profile.py:153); the only forgejo.creatidy.com occurrences are the pre-existing frozen TaskSpec canonical URLs already on base fa81734e (task_execution.py:490, :535) — TaskSpec authority data this PR does not touch
Preserve list (TaskSpec authority, exact-base freezing, controller-owned candidate, closed child env, no ambient controller secrets, inference-free preflight, no Forge writes in preflight, no auto merge/deploy, trusted-development ack, #143/#166 identities, recovery semantics) No TaskSpec or run-path semantic changes: _preflight_base/_resolve_base (task_execution.py:1856-1890), run_task (:1196+), prepare_workspace call (:1296-1298) unchanged; child env stays the closed key set (:1463-1469); preflight payload still reports inference_performed/forge_writes_performed/execution_authorized false (:1789-1793); run still requires --approve --trusted-development (cli.py, unchanged lines in diff context); TASKS registry unchanged (:600); 966 existing+new tests pass

Architecture / Information Flow

All new modules live in adapters and import only adapters (source_cache.py:23-24, operator_profile.py:24, codex_discovery.py imports nothing internal); no core/ports file changed, and the import-linter layers contract was verified KEPT by make check in this session. Authority stays where it was: TaskSpec still solely owns repository identity/base (resolve_source at task_execution.py:1837-1853 reads only task.repository_url; infrastructure defaults never touch TaskSpec, per task_paths's docstring :1821-1827). Information flow is one-directional: environment/profile → typed config → components; the cache namespace is controller-owned under ${XDG_CACHE_HOME:-~/.cache}/creatidy-kernel/source (source_cache.py:51-56), and every reuse re-verifies the exact canonical origin before use (:253-259). Diagnostic flow carries closed vocabularies only: PreflightBlocker.category is regex-validated (task_execution.py:1753-1755) and Router categories are a fixed StrEnum (scarcity_router.py:38-46). Git acquisition runs in a freshly constructed environment (GIT_CONFIG_NOSYSTEM, GIT_CONFIG_GLOBAL=/dev/null, GIT_TERMINAL_PROMPT=0, no ambient GIT_* passthrough — source_cache.py:123-136), with a test proving ambient url.*.insteadOf rewriting cannot redirect it (test_source_cache.py:203-219); run_git_bounded is shell-free, byte-bounded and process-group-killed (forgejo_transport.py:181-211).

Blocking Issues

None. Fail-closed behavior holds on every path traced: discovery failure becomes a codex_not_ready blocker with a closed category and never a substitution; acquisition failure becomes a source_not_ready blocker; a failed cache refresh refuses readiness rather than degrading (source_cache.py:276-284); explicit --repo sources keep the full exact-origin/clean/base verification (task_execution.py:2055-2089).

Non-blocking Suggestions

  1. Cache key omits the host — source_cache_key (source_cache.py:59-70) keys only on the URL path, so two forges hosting the same <org>/<repo> share one cache slot. Safety is intact (exact origin re-verified before every use; mismatch replaces the entry), but alternating tasks on different hosts would discard and re-clone each time.
  2. Configuration blocker can drop Codex field names — in the ConfigurationError handler (task_execution.py:1980-1993), when the user supplied both Codex fields (discovery skipped) and another field is also invalid/missing, the Codex field names fall out of the diagnostics. Fail-closed is preserved; only the reported names are lossy.
  3. config validate ignores the top-level --profile — _config (cli.py:116-162) uses only --config-profile/--config-sops, so creatidy-kernel --profile X config validate silently validates the ambient environment instead of X. Either honor it or reject the combination.
  4. Template embeds live machine values — profile_template() (operator_profile.py:152-179) writes the operator's current HOME and PATH into the generated file. Not a secret leak, but pure placeholders would be cleaner.
  5. Cache replacement race — _discard(target) then staging.rename(target) can race between concurrent invocations on the same machine; worst case is a redundant clone or a transient unsuitable_cache refusal.
  6. evidence["source"]["cache_root"] is recomputed from default_cache_root(enriched) rather than taken from the acquisition itself; identical in production.
  7. Override-misconfiguration reads as not_found — an explicitly configured but relative/non-executable CREATIDY_KERNEL_CODEX_BIN is classified not_found (codex_discovery.py:98-105).

Questions for the Author

  1. Should the top-level --profile also apply to config validate (see suggestion 3), or is the two-flag split intentional enough to document in README?
  2. Was omitting the host from the cache key a deliberate single-forge assumption, or an oversight worth fixing before multi-forge TaskSpecs appear?

Security / Privacy Notes

  • Secrets are never printed or serialized: ConfigurationError carries field names only; ProfileError messages are fixed; the Codex probe never echoes binary output; Router failures never echo response bodies and the exception is raised outside the handler; tests assert synthetic secrets absent from payloads.
  • Closed child environments: the Codex probe and runtime child share the closed key set; Git acquisition runs in the constructed environment described above.
  • Remote interaction is strictly read-only: only clone/fetch/status/config --get/remote set-url (local metadata) are issued; --version is the only probe; no thread or turn is started in discovery/doctor/preflight.
  • No ambient controller secrets reach children; no Creatidy-specific constants, hosts, age recipients or .sops.yaml references exist in the new public code (grep-verified); SOPS is a generic adapter over the user's own key environment.

Test / Documentation Gaps

  • The end-to-end READY proof runs task 143 with the registry monkeypatched to 143 only; task 166 shares the identical composition path but is not itself exercised end-to-end. Acceptable under the issue's no-live-166 constraint.
  • The "two unrelated filesystem layouts" both live under the pytest temporary root; the property the issue names — acquisition independent of checkout placement — is what the tests assert (varying remote/state placement, constant acquisition result, no placement leakage).
  • README matches the implemented behavior; no stale env-var references exist elsewhere in docs/ (grep-verified).
  • make audit not run — consistent, since the diff touches no dependency or tooling file.

Checks run in this review (all as local untrusted checks; nothing pushed, no working-tree modification, no installs): focused new/affected tests → 112 passed; make check → PASS (966 tests, ruff, basedpyright, lint-imports KEPT, secret scan clean); make package-check → PASS. Not verifiable from this public checkout: the reference inspection of private Infrastructure/creatidy-onprem — confirmed instead that the public product contains no runtime dependency on or constants from it.

Follow-Up Candidates

  • Host-scoped source cache key if multi-forge TaskSpecs are foreseeable.
  • Cache-replacement concurrency note or advisory lock if concurrent operators become a supported pattern.
  • A dedicated discovery category for a misconfigured explicit Codex override.

Overall Recommendation

APPROVE

The frozen HEAD implements every required-architecture section of issue #44 with verifiable evidence, preserves the issue's entire "Preserve" list, keeps all changes inside adapters with the layers contract verified KEPT, and passes both repository gates in this session (966 tests; package-check). All findings are non-blocking diagnostics/UX/efficiency refinements. No live #166 execution, merge or deployment is included, per the issue's constraints.

Independent full-PR review of the frozen head `c2944f0da538e60fb67f091bd70e66eea212e141` (base `develop` @ `fa81734ed10bfbdd6e7a75b138649a89a9724511`), performed by a fresh read-only reviewer session dispatched on GLM-5.3 per the owner-mandated model override for this delivery. The reviewer session is distinct from the implementation session and posted nothing itself; this submission persists its verdict. `platform_identity_limitation`: the configured reviewer identity is the same account that authored this PR, so Forgejo refuses a formal self-approval ("approve your own pull is not allowed"); the substantive verdict below is independent of that platform state, and `forgejo_review_state` for this review is COMMENT with `platform_approval_recorded = false`. ## PR Review: D1-06: Public self-contained operator path for Scarcity Router-backed task execution (#44) (#45) **Repository:** Creatidy/creatidy-kernel **Branch:** issue-44-selfcontained-operator -> develop **Frozen head:** c2944f0da538e60fb67f091bd70e66eea212e141 **Linked issue:** Creatidy/creatidy-kernel#44 (D1-06 — Public self-contained operator path for Scarcity Router-backed task execution) Frozen-input verification (run this review): Forgejo MCP `get_pull_request_by_index(45)` returned head `issue-44-selfcontained-operator` @ `c2944f0da538e60fb67f091bd70e66eea212e141`, base `develop` @ `fa81734ed10bfbdd6e7a75b138649a89a9724511` (merge base identical), state open, not merged — matches the frozen inputs exactly. Local checkout: `git rev-parse HEAD` = `c2944f0da538e60fb67f091bd70e66eea212e141`, clean tree on the head branch. Issue #44 fetched via `get_issue_by_index(44)`; 12 changed files confirmed via `list_pull_request_files` and `git diff --stat fa81734e..c2944f0` (2037 insertions / 79 deletions). ### Summary The PR makes the public package's bounded-task operator path self-contained. It adds (1) a controller-owned read-only source cache acquired from the TaskSpec canonical remote (`src/creatidy_kernel/adapters/source_cache.py`, new), (2) deterministic single-candidate Codex discovery with a `--version`-only probe (`adapters/codex_discovery.py`, new), (3) generic dotenv/JSON operator profiles with an optional generic SOPS decrypt adapter (`adapters/operator_profile.py`, new), (4) safe closed diagnostic categories on preflight blockers plus Router failure categories (`adapters/task_execution.py`, `adapters/scarcity_router.py`), and (5) a public CLI surface `config template|validate`, `doctor`, `task preflight/run/status/export` with user-local XDG defaults (`adapters/cli.py`). No core or ports files change; README documents the new surface. All changes sit in `adapters` plus tests and README. ### Intent / Acceptance Evidence | Issue criterion | Evidence at frozen HEAD | |---|---| | 1. Source ownership: normal path needs no `CREATIDY_KERNEL_SOURCE_REPOSITORY`; controller-owned read-only cache from canonical remote; exact origin verification; read-only remote; no sibling/home/workspace search; `--repo` stays an advanced override; no generic discovery framework | `task_execution.py:1837-1853` (`resolve_source`: override → configured field → canonical cache); `task_execution.py:2008-2017` (preflight acquires when unset); `source_cache.py:59-70` (URL-shape-validated cache key, no filesystem search anywhere in the module), `source_cache.py:139-152` (exact origin re-verified), `source_cache.py:168-184` (fresh clone with `credential.helper=`, `http.followRedirects=false`, `--template=`), `source_cache.py:276-284` (bounded fail-closed `fetch --prune origin` refresh), `source_cache.py:286-288` (foreign/corrupt entry replaced, never adopted); tests `tests/test_source_cache.py:53-98` (two layouts, no placement leakage; canonical sibling never adopted or touched), `:100-145` (reuse fetches, failed fetch fails closed) | | 2. Codex discovery: `CODEX_BIN`/`CODEX_VERSION` optional when an unambiguous supported Codex is on PATH; deterministic resolution; `--version` probe; shared inference-free validation; no silent substitution; no thread/turn | `codex_discovery.py:95-113` (override else first on PATH, single candidate); `codex_discovery.py:63-83` (probe only `--version`, closed failure mapping, output never echoed); `codex_discovery.py:25` + `task_execution.py:1563` (one shared version pattern); `task_execution.py:1936-1955` (`environment_with_discovered_codex`); tests `tests/test_codex_discovery.py:42-64` (deterministic first-hit, override wins), `test_selfcontained_operator.py:165-186` (preflight ready with no manual Codex fields) | | 3. XDG-compatible user-local state defaults, storage safety preserved, overrides allowed | `task_execution.py:1806-1811` (`default_state_directory` honoring absolute `XDG_STATE_HOME`); `task_execution.py:1814-1834` (`task_paths`: CLI → field → default; source may be None); unchanged state gates at `task_execution.py:2018-2052`; `cli.py:309-316` (status/export default); test `test_selfcontained_operator.py:445-454` | | 4. Generic runtime configuration/secrets: no new secrets manager, no committed secrets, no Creatidy recipients/`.sops.yaml` dependency, env config still supported, generic SOPS adapter, user-local profile paths, secrets never printed/serialized | `operator_profile.py:39-53` (bounded literal dotenv), `:56-71` (JSON profile), `:74-105` (generic SOPS: user-supplied profile, user's own `SOPS_*` env only), `:108-127` (symlink/size-refusing loader), `:130-141` (field-name-only diagnostics), `:144-149` (XDG config default); tests `test_operator_profile.py:93-131` (SOPS passthrough verified, failures safe), `:147-157` (template generic, host/onprem absent) | | 5. Public operator UX covering config validation, doctor, preflight, run, status, export; installed CLI over Make targets | `cli.py:230-243` (`config`, `doctor` subcommands), `cli.py:116-162` (`_config`), `cli.py:164-202` (`_doctor`), `cli.py:309-339` (status/export/preflight/run); no new Make targets; README "Public Operator Surface" (`README.md:281-323`); tests `test_selfcontained_operator.py:345-442` | | 6. Doctor/preflight fail-closed with safe closed diagnostics (router: unreachable/HTTP/invalid/no-eligible/incompatible; source: real acquisition/remote/base causes); no bodies or credentials exposed | `scarcity_router.py:38-46` (`RouterFailureCategory`), `:385-394` (HTTP/encoding/deadline classified), `:413-424` (request/no-eligible/incompatible), `:463-472` (OSError → endpoint unreachable; raise outside handler); `source_cache.py:34-40` (source categories); `task_execution.py:2053-2089` (checkout/remote/dirty/base subcategories on explicit sources); `cli.py:43-105` (fixed hints); tests `test_selfcontained_operator.py:239-319` (real loopback HTTP for five router categories, secret-absence asserted), `:188-225` (discovery categories) | | 7. Scarcity Router remains external through `ResourceAllocator`; no policy embedding, no Router-as-Runtime, no product copy, no routing framework | Only category metadata was added to the existing adapter (scarcity_router.py diff); no new port, no policy data, no provider framework; `make check` ran `lint-imports`: "Adapters depend inward; core and ports never depend on adapters KEPT" | | Acceptance: zero-inference `doctor -> task preflight` without creatidy-onprem and without a configured checkout path | `test_selfcontained_operator.py:364-415`: CLI `doctor` then `task preflight --json` both READY with Codex from PATH, XDG state default, source acquired by the controller, `inference_performed`/`forge_writes_performed` false; `:345-361`: `doctor` with an empty environment fails closed with categories and hints | | Acceptance: two unrelated filesystem layouts; acquisition independent of checkout placement | `test_source_cache.py:53-79` (two unrelated remote/state layouts, varying cache roots, same logical content, no placement strings in `.git/config`) and `test_selfcontained_operator.py:117-142` (preflight parametrized over two unrelated layouts, same acquired cache evidence) | | Constraints: no live #166 execution; no merge or deployment; no Creatidy-specific constants/hosts/recipients in the public product | Diff adds no execution/Forge-write path and no deployment code; grep of the new/changed files found no hosts, recipients or onprem references (sole negative assertion at `test_operator_profile.py:153`); the only `forgejo.creatidy.com` occurrences are the pre-existing frozen TaskSpec canonical URLs already on base `fa81734e` (`task_execution.py:490`, `:535`) — TaskSpec authority data this PR does not touch | | Preserve list (TaskSpec authority, exact-base freezing, controller-owned candidate, closed child env, no ambient controller secrets, inference-free preflight, no Forge writes in preflight, no auto merge/deploy, trusted-development ack, #143/#166 identities, recovery semantics) | No TaskSpec or run-path semantic changes: `_preflight_base`/`_resolve_base` (`task_execution.py:1856-1890`), `run_task` (`:1196+`), `prepare_workspace` call (`:1296-1298`) unchanged; child env stays the closed key set (`:1463-1469`); preflight payload still reports `inference_performed/forge_writes_performed/execution_authorized` false (`:1789-1793`); run still requires `--approve --trusted-development` (cli.py, unchanged lines in diff context); `TASKS` registry unchanged (`:600`); 966 existing+new tests pass | ### Architecture / Information Flow All new modules live in `adapters` and import only adapters (`source_cache.py:23-24`, `operator_profile.py:24`, `codex_discovery.py` imports nothing internal); no core/ports file changed, and the import-linter layers contract was verified KEPT by `make check` in this session. Authority stays where it was: TaskSpec still solely owns repository identity/base (`resolve_source` at `task_execution.py:1837-1853` reads only `task.repository_url`; infrastructure defaults never touch TaskSpec, per `task_paths`'s docstring `:1821-1827`). Information flow is one-directional: environment/profile → typed config → components; the cache namespace is controller-owned under `${XDG_CACHE_HOME:-~/.cache}/creatidy-kernel/source` (`source_cache.py:51-56`), and every reuse re-verifies the exact canonical origin before use (`:253-259`). Diagnostic flow carries closed vocabularies only: `PreflightBlocker.category` is regex-validated (`task_execution.py:1753-1755`) and Router categories are a fixed StrEnum (`scarcity_router.py:38-46`). Git acquisition runs in a freshly constructed environment (`GIT_CONFIG_NOSYSTEM`, `GIT_CONFIG_GLOBAL=/dev/null`, `GIT_TERMINAL_PROMPT=0`, no ambient `GIT_*` passthrough — `source_cache.py:123-136`), with a test proving ambient `url.*.insteadOf` rewriting cannot redirect it (`test_source_cache.py:203-219`); `run_git_bounded` is shell-free, byte-bounded and process-group-killed (`forgejo_transport.py:181-211`). ### Blocking Issues None. Fail-closed behavior holds on every path traced: discovery failure becomes a `codex_not_ready` blocker with a closed category and never a substitution; acquisition failure becomes a `source_not_ready` blocker; a failed cache refresh refuses readiness rather than degrading (`source_cache.py:276-284`); explicit `--repo` sources keep the full exact-origin/clean/base verification (`task_execution.py:2055-2089`). ### Non-blocking Suggestions 1. **Cache key omits the host** — `source_cache_key` (`source_cache.py:59-70`) keys only on the URL path, so two forges hosting the same `<org>/<repo>` share one cache slot. Safety is intact (exact origin re-verified before every use; mismatch replaces the entry), but alternating tasks on different hosts would discard and re-clone each time. 2. **Configuration blocker can drop Codex field names** — in the `ConfigurationError` handler (`task_execution.py:1980-1993`), when the user supplied both Codex fields (discovery skipped) and another field is also invalid/missing, the Codex field names fall out of the diagnostics. Fail-closed is preserved; only the reported names are lossy. 3. **`config validate` ignores the top-level `--profile`** — `_config` (`cli.py:116-162`) uses only `--config-profile`/`--config-sops`, so `creatidy-kernel --profile X config validate` silently validates the ambient environment instead of `X`. Either honor it or reject the combination. 4. **Template embeds live machine values** — `profile_template()` (`operator_profile.py:152-179`) writes the operator's current `HOME` and `PATH` into the generated file. Not a secret leak, but pure placeholders would be cleaner. 5. **Cache replacement race** — `_discard(target)` then `staging.rename(target)` can race between concurrent invocations on the same machine; worst case is a redundant clone or a transient `unsuitable_cache` refusal. 6. **`evidence["source"]["cache_root"]` is recomputed** from `default_cache_root(enriched)` rather than taken from the acquisition itself; identical in production. 7. **Override-misconfiguration reads as `not_found`** — an explicitly configured but relative/non-executable `CREATIDY_KERNEL_CODEX_BIN` is classified `not_found` (`codex_discovery.py:98-105`). ### Questions for the Author 1. Should the top-level `--profile` also apply to `config validate` (see suggestion 3), or is the two-flag split intentional enough to document in README? 2. Was omitting the host from the cache key a deliberate single-forge assumption, or an oversight worth fixing before multi-forge TaskSpecs appear? ### Security / Privacy Notes - Secrets are never printed or serialized: `ConfigurationError` carries field names only; `ProfileError` messages are fixed; the Codex probe never echoes binary output; Router failures never echo response bodies and the exception is raised outside the handler; tests assert synthetic secrets absent from payloads. - Closed child environments: the Codex probe and runtime child share the closed key set; Git acquisition runs in the constructed environment described above. - Remote interaction is strictly read-only: only `clone`/`fetch`/`status`/`config --get`/`remote set-url` (local metadata) are issued; `--version` is the only probe; no thread or turn is started in discovery/doctor/preflight. - No ambient controller secrets reach children; no Creatidy-specific constants, hosts, age recipients or `.sops.yaml` references exist in the new public code (grep-verified); SOPS is a generic adapter over the user's own key environment. ### Test / Documentation Gaps - The end-to-end READY proof runs task 143 with the registry monkeypatched to 143 only; task 166 shares the identical composition path but is not itself exercised end-to-end. Acceptable under the issue's no-live-166 constraint. - The "two unrelated filesystem layouts" both live under the pytest temporary root; the property the issue names — acquisition independent of checkout placement — is what the tests assert (varying remote/state placement, constant acquisition result, no placement leakage). - README matches the implemented behavior; no stale env-var references exist elsewhere in `docs/` (grep-verified). - `make audit` not run — consistent, since the diff touches no dependency or tooling file. Checks run in this review (all as local untrusted checks; nothing pushed, no working-tree modification, no installs): focused new/affected tests → 112 passed; `make check` → PASS (966 tests, ruff, basedpyright, lint-imports KEPT, secret scan clean); `make package-check` → PASS. Not verifiable from this public checkout: the reference inspection of private `Infrastructure/creatidy-onprem` — confirmed instead that the public product contains no runtime dependency on or constants from it. ### Follow-Up Candidates - Host-scoped source cache key if multi-forge TaskSpecs are foreseeable. - Cache-replacement concurrency note or advisory lock if concurrent operators become a supported pattern. - A dedicated discovery category for a misconfigured explicit Codex override. ### Overall Recommendation APPROVE The frozen HEAD implements every required-architecture section of issue #44 with verifiable evidence, preserves the issue's entire "Preserve" list, keeps all changes inside `adapters` with the layers contract verified KEPT, and passes both repository gates in this session (966 tests; package-check). All findings are non-blocking diagnostics/UX/efficiency refinements. No live #166 execution, merge or deployment is included, per the issue's constraints.
adrian.tkacz deleted branch issue-44-selfcontained-operator 2026-10-04 17:21:47 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
adrian.tkacz
Labels
Clear labels   
bug
Something is not working

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
help wanted
Need some help

  
invalid
Something is wrong

  
question
More information is needed

  
wontfix
This won't be fixed

No labels
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Creatidy/creatidy-kernel!45
No description provided.