D1-06 — Public self-contained operator path for Scarcity Router-backed task execution #44

Open
opened 2026-10-04 00:59:54 +00:00 by kilo · 3 comments
Member

Problem

Public creatidy-kernel currently depends operationally on private Infrastructure/creatidy-onprem for the normal bounded-task operator path.

The private repository currently owns generic Kernel product/operator behavior:

  • scripts/kernel_operator.py
  • Kernel profile generation/validation
  • SOPS-facing operator glue
  • kernel-env-*
  • kernel-doctor.dev
  • kernel-task.preflight.dev
  • kernel-task.run.dev
  • task status/export wrappers
  • machine/path discovery
  • readiness result validation

This violates the product boundary. creatidy-kernel is public and must be independently usable without access to any private Creatidy repository.

A concrete symptom is source-path discovery: different valid machines may contain the target repository at paths such as:

  • /home/adrian/workspace/scarcity-router
  • /home/adrian/workspace/creatidy/scarcity-router

These are both legitimate. Normal Kernel operation must not depend on repository-relative path guessing or require the user to configure the local checkout path.

Goal

Make the public Kernel package self-contained for bounded Scarcity Router-backed task execution.

A user with:

  • public creatidy-kernel;
  • access to the target Forge repository;
  • a supported Codex installation/authentication;
  • a reachable Scarcity Router;
  • the required scoped credentials;

must be able to validate and run a registered bounded task without cloning or accessing creatidy-onprem.

Required architecture

1. Source repository ownership

The normal path must NOT require CREATIDY_KERNEL_SOURCE_REPOSITORY.

Kernel already owns canonical repository URL, repository identity and base branch through TaskSpec.

For normal operation Kernel must prepare its own controller-owned read-only source clone/cache/worktree from that canonical repository.

Requirements:

  • no assumption about sibling-directory layout;
  • no search for /home/<user>/workspace/...;
  • no repository-relative path guessing;
  • exact canonical remote verification;
  • exact base SHA still frozen by preflight;
  • source acquisition remains read-only with respect to the remote;
  • disposable candidate workspace remains separate;
  • an explicit --repo may remain as an advanced/local override, but must not be required.

Do not introduce a generic repository discovery framework.

2. Codex discovery

Normal operation must not require manual configuration of:

  • CREATIDY_KERNEL_CODEX_BIN
  • CREATIDY_KERNEL_CODEX_VERSION

when an unambiguous supported Codex is available on PATH.

Kernel should:

  • resolve the executable deterministically;
  • probe codex --version;
  • validate supported version/schema using the existing inference-free mechanisms;
  • permit explicit overrides for advanced cases;
  • never silently choose between ambiguous incompatible installations.

No thread or turn during discovery/doctor/preflight.

3. Local state/evidence defaults

Normal operation must not require manual state/evidence path configuration.

Use platform-appropriate user-local defaults, preferably XDG-compatible on Linux, with the existing storage-safety requirements preserved.

Explicit path overrides remain allowed.

4. Runtime configuration / secrets

Move the GENERIC operator/profile behavior required by Kernel out of creatidy-onprem and into the public Kernel product.

Do not build a new secrets manager.

Requirements:

  • no committed secrets;
  • no Creatidy-specific age recipient;
  • no dependency on creatidy-onprem/.sops.yaml;
  • environment-based configuration must remain supported;
  • if SOPS support is retained/adopted, make it an optional generic profile adapter using a user-supplied profile/key, not repository-specific infrastructure;
  • local profile paths belong under user configuration/state locations, not inside a source checkout;
  • secrets must never be printed or serialized into evidence.

5. Public operator UX

Provide a coherent public CLI/operator surface equivalent to the useful behavior currently hidden in creatidy-onprem.

It must cover at least:

  • configuration/profile validation where applicable;
  • doctor;
  • task preflight;
  • task run;
  • task status;
  • task export.

Prefer installed CLI commands over product-specific Make targets.

The user should not need to know internal environment-variable names for normal operation.

6. Doctor diagnostics

doctor and preflight must preserve fail-closed behavior but provide useful safe diagnostics.

For example, a router_not_ready result should distinguish safe categories such as:

  • endpoint unreachable;
  • HTTP/protocol rejection;
  • invalid response/schema;
  • no eligible selection;
  • selection incompatible with controller runtime binding;

without exposing response bodies, credentials or unsafe free text.

Likewise source readiness should identify a real acquisition/remote/base problem rather than merely returning an opaque path failure.

7. Scarcity Router boundary

Scarcity Router remains an external independent product through the existing ResourceAllocator port.

Do NOT:

  • embed Router policy into Kernel;
  • turn Router into Kernel Runtime;
  • copy the Scarcity Router product into this repository;
  • add a generic provider-routing framework.

The goal is to make Kernel's PUBLIC integration/operator path self-contained, not to merge the two products.

Preserve

  • TaskSpec authority.
  • exact-base freezing.
  • controller-owned candidate creation.
  • closed child environment.
  • no ambient controller secrets.
  • inference-free preflight.
  • no Forge writes during preflight.
  • no automatic merge/deploy.
  • trusted-development acknowledgment.
  • #143 and #166 durable identities/contracts.
  • all existing recovery/reconciliation semantics.

Migration evidence

Inspect the current operator implementation in Infrastructure/creatidy-onprem as REFERENCE behavior, but do not create a runtime dependency on that repository.

Identify which behavior is:

  • product-generic and must move to Kernel;
  • owner/deployment-specific and must NOT move.

Do not copy Creatidy-specific constants, paths, Forge host assumptions or SOPS recipient configuration into the public product.

Acceptance

A clean public Kernel checkout on a machine with varying workspace layout must be capable of a ZERO-INFERENCE:

doctor -> task preflight --task 166

without creatidy-onprem and without configuring a local Scarcity Router repository checkout path.

Tests must demonstrate at least two unrelated filesystem layouts and prove that source acquisition is independent of checkout placement.

No live #166 execution is authorized by this issue.

No merge or deployment.


Why

See ### Problem above: the public product's normal bounded-task operator path must not depend on any private repository.

Scope

See ### Required architecture (sections 1-7). Non-goals are explicit there: no generic repository discovery framework, no new secrets manager, no Router policy embedding, no generic provider-routing framework.

Constraints

See ### Preserve plus: no committed secrets; no Creatidy-specific constants, Forge host assumptions or SOPS recipient configuration in the public product; no live #166 execution; no merge or deployment.

Acceptance Criteria

See ### Acceptance above. Verifiably: a clean public checkout supports zero-inference doctor then task preflight --task 166 without creatidy-onprem and without a configured local Scarcity Router checkout path, and tests prove source acquisition is independent of checkout placement across at least two unrelated filesystem layouts.

### Problem Public `creatidy-kernel` currently depends operationally on private `Infrastructure/creatidy-onprem` for the normal bounded-task operator path. The private repository currently owns generic Kernel product/operator behavior: - `scripts/kernel_operator.py` - Kernel profile generation/validation - SOPS-facing operator glue - `kernel-env-*` - `kernel-doctor.dev` - `kernel-task.preflight.dev` - `kernel-task.run.dev` - task status/export wrappers - machine/path discovery - readiness result validation This violates the product boundary. `creatidy-kernel` is public and must be independently usable without access to any private Creatidy repository. A concrete symptom is source-path discovery: different valid machines may contain the target repository at paths such as: - `/home/adrian/workspace/scarcity-router` - `/home/adrian/workspace/creatidy/scarcity-router` These are both legitimate. Normal Kernel operation must not depend on repository-relative path guessing or require the user to configure the local checkout path. ### Goal Make the public Kernel package self-contained for bounded Scarcity Router-backed task execution. A user with: - public `creatidy-kernel`; - access to the target Forge repository; - a supported Codex installation/authentication; - a reachable Scarcity Router; - the required scoped credentials; must be able to validate and run a registered bounded task without cloning or accessing `creatidy-onprem`. ### Required architecture #### 1. Source repository ownership The normal path must NOT require `CREATIDY_KERNEL_SOURCE_REPOSITORY`. Kernel already owns canonical repository URL, repository identity and base branch through `TaskSpec`. For normal operation Kernel must prepare its own controller-owned read-only source clone/cache/worktree from that canonical repository. Requirements: - no assumption about sibling-directory layout; - no search for `/home/<user>/workspace/...`; - no repository-relative path guessing; - exact canonical remote verification; - exact base SHA still frozen by preflight; - source acquisition remains read-only with respect to the remote; - disposable candidate workspace remains separate; - an explicit `--repo` may remain as an advanced/local override, but must not be required. Do not introduce a generic repository discovery framework. #### 2. Codex discovery Normal operation must not require manual configuration of: - `CREATIDY_KERNEL_CODEX_BIN` - `CREATIDY_KERNEL_CODEX_VERSION` when an unambiguous supported Codex is available on `PATH`. Kernel should: - resolve the executable deterministically; - probe `codex --version`; - validate supported version/schema using the existing inference-free mechanisms; - permit explicit overrides for advanced cases; - never silently choose between ambiguous incompatible installations. No thread or turn during discovery/doctor/preflight. #### 3. Local state/evidence defaults Normal operation must not require manual state/evidence path configuration. Use platform-appropriate user-local defaults, preferably XDG-compatible on Linux, with the existing storage-safety requirements preserved. Explicit path overrides remain allowed. #### 4. Runtime configuration / secrets Move the GENERIC operator/profile behavior required by Kernel out of `creatidy-onprem` and into the public Kernel product. Do not build a new secrets manager. Requirements: - no committed secrets; - no Creatidy-specific age recipient; - no dependency on `creatidy-onprem/.sops.yaml`; - environment-based configuration must remain supported; - if SOPS support is retained/adopted, make it an optional generic profile adapter using a user-supplied profile/key, not repository-specific infrastructure; - local profile paths belong under user configuration/state locations, not inside a source checkout; - secrets must never be printed or serialized into evidence. #### 5. Public operator UX Provide a coherent public CLI/operator surface equivalent to the useful behavior currently hidden in `creatidy-onprem`. It must cover at least: - configuration/profile validation where applicable; - `doctor`; - task `preflight`; - task `run`; - task `status`; - task `export`. Prefer installed CLI commands over product-specific Make targets. The user should not need to know internal environment-variable names for normal operation. #### 6. Doctor diagnostics `doctor` and preflight must preserve fail-closed behavior but provide useful safe diagnostics. For example, a `router_not_ready` result should distinguish safe categories such as: - endpoint unreachable; - HTTP/protocol rejection; - invalid response/schema; - no eligible selection; - selection incompatible with controller runtime binding; without exposing response bodies, credentials or unsafe free text. Likewise source readiness should identify a real acquisition/remote/base problem rather than merely returning an opaque path failure. #### 7. Scarcity Router boundary Scarcity Router remains an external independent product through the existing `ResourceAllocator` port. Do NOT: - embed Router policy into Kernel; - turn Router into Kernel Runtime; - copy the Scarcity Router product into this repository; - add a generic provider-routing framework. The goal is to make Kernel's PUBLIC integration/operator path self-contained, not to merge the two products. ### Preserve - TaskSpec authority. - exact-base freezing. - controller-owned candidate creation. - closed child environment. - no ambient controller secrets. - inference-free preflight. - no Forge writes during preflight. - no automatic merge/deploy. - trusted-development acknowledgment. - #143 and #166 durable identities/contracts. - all existing recovery/reconciliation semantics. ### Migration evidence Inspect the current operator implementation in `Infrastructure/creatidy-onprem` as REFERENCE behavior, but do not create a runtime dependency on that repository. Identify which behavior is: - product-generic and must move to Kernel; - owner/deployment-specific and must NOT move. Do not copy Creatidy-specific constants, paths, Forge host assumptions or SOPS recipient configuration into the public product. ### Acceptance A clean public Kernel checkout on a machine with varying workspace layout must be capable of a ZERO-INFERENCE: `doctor -> task preflight --task 166` without `creatidy-onprem` and without configuring a local Scarcity Router repository checkout path. Tests must demonstrate at least two unrelated filesystem layouts and prove that source acquisition is independent of checkout placement. No live #166 execution is authorized by this issue. No merge or deployment. --- ### Why See `### Problem` above: the public product's normal bounded-task operator path must not depend on any private repository. ### Scope See `### Required architecture` (sections 1-7). Non-goals are explicit there: no generic repository discovery framework, no new secrets manager, no Router policy embedding, no generic provider-routing framework. ### Constraints See `### Preserve` plus: no committed secrets; no Creatidy-specific constants, Forge host assumptions or SOPS recipient configuration in the public product; no live #166 execution; no merge or deployment. ### Acceptance Criteria See `### Acceptance` above. Verifiably: a clean public checkout supports zero-inference `doctor` then `task preflight --task 166` without `creatidy-onprem` and without a configured local Scarcity Router checkout path, and tests prove source acquisition is independent of checkout placement across at least two unrelated filesystem layouts. ### Links - Creatidy/creatidy-kernel#166 — bounded TaskSpec registration (support merged; execution remains separately owner-authorized) - Creatidy/creatidy-kernel#143 — frozen task identity - Creatidy/creatidy-kernel#34 — preflight readiness - Creatidy/creatidy-kernel#37 — bounded task execution promotion - Infrastructure/creatidy-onprem#43 — private operator counterpart (reference only; no runtime dependency)
Author
Member

D1-06 delivery ledger (issue #44 / PR #45)

  • Issue: Creatidy/creatidy-kernel#44
  • PR: Creatidy/creatidy-kernel#45 (issue-44-selfcontained-operator -> develop)
  • Base: develop @ fa81734ed10bfbdd6e7a75b138649a89a9724511
  • Current HEAD: c2944f0da538e60fb67f091bd70e66eea212e141
  • max_remediation_cycles = 10, remediation_used = 0
  • Review dispatch policy (owner override for this delivery): implementation model GLM-5.3-Flash; each fresh independent review uses exactly GLM-5.3. No routing substitution.
  • Validation at this HEAD: make check PASS (966 tests, ruff, basedpyright strict 0 errors, lint-imports KEPT, secret scan clean), make package-check PASS.

Review log

(no completed reviews yet)

Remediation log

(none)

## D1-06 delivery ledger (issue #44 / PR #45) - Issue: Creatidy/creatidy-kernel#44 - PR: Creatidy/creatidy-kernel#45 (`issue-44-selfcontained-operator` -> `develop`) - Base: `develop` @ `fa81734ed10bfbdd6e7a75b138649a89a9724511` - Current HEAD: `c2944f0da538e60fb67f091bd70e66eea212e141` - `max_remediation_cycles = 10`, `remediation_used = 0` - Review dispatch policy (owner override for this delivery): implementation model GLM-5.3-Flash; each fresh independent review uses exactly GLM-5.3. No routing substitution. - Validation at this HEAD: `make check` PASS (966 tests, ruff, basedpyright strict 0 errors, lint-imports KEPT, secret scan clean), `make package-check` PASS. ### Review log (no completed reviews yet) ### Remediation log (none)
Author
Member

D1-06 delivery ledger (issue #44 / PR #45) — update 1

  • Issue: Creatidy/creatidy-kernel#44
  • PR: Creatidy/creatidy-kernel#45 (issue-44-selfcontained-operator -> develop)
  • Base: develop @ fa81734ed10bfbdd6e7a75b138649a89a9724511
  • Current HEAD: c2944f0da538e60fb67f091bd70e66eea212e141 (unchanged since initial push)
  • max_remediation_cycles = 10, remediation_used = 0

Review log

Review cycle 1 (initial review; costs zero remediation cycles)

  • Reviewed HEAD: c2944f0da538e60fb67f091bd70e66eea212e141; base fa81734ed10bfbdd6e7a75b138649a89a9724511 (frozen inputs verified by the reviewer against live Forgejo state before reviewing)
  • Review profile: full-repository PR review per .kilo/command/review-pr.md at architecture/security criticality. Routing policy for this delivery is the explicit owner override: reviewer model exactly GLM-5.3 (no Scarcity Router selection was consulted; no catalog/policy versions apply). Dispatched identity: account:zai-individual-coding-plan/GLM-5.3 (reasoning level max).
  • Dispatch: ZCode dynamic workflow dwfrun-f09e08f3-8981-4a1a-b1d8-4877568ad1f4, subagent independent-reviewer, fresh read-only context distinct from the implementation session.
  • Dispatch verification: DISPATCH_VERIFIED — the harness resolved and ran the subagent on account:zai-individual-coding-plan/GLM-5.3$max; the reviewer session self-identified as GLM-5.3 (account:zai-individual-coding-plan/GLM-5.3). An Agent-Manager-style Resolved models: string does not exist in this harness; the recorded evidence is the dispatch configuration plus reviewer self-identification, both consistent with the owner-mandated model.
  • Independent review reference: PR #45 review id 687 (the reviewer itself posts nothing; its complete review was persisted verbatim on the PR, bound to the frozen HEAD).
  • substantive_verdict: APPROVE (### Overall Recommendation), zero blocking findings, 7 non-blocking suggestions, 2 author questions (recorded below as follow-up candidates).
  • forgejo_review_state: COMMENT; platform_approval_recorded: false. platform_identity_limitation: the only available reviewer identity is the PR author account (kilo); Forgejo rejects formal self-approval ("approve your own pull is not allowed"). No branch-protection rule is observable with this account's permissions, so any formal-approval merge requirement is reported separately as unobservable, not assumed.
  • Reviewer-run validation at the frozen HEAD: focused tests 112 passed; make check PASS (966 tests); make package-check PASS. Forgejo CI at c2944f0 (pull_request event, run #2192): success.
  • Blocking finding count: 0. Remediation status: none. Resulting HEAD: unchanged. remediation_used: 0 / 10.

Remediation log

(none — no valid in-scope blocking findings)

Follow-up candidates recorded from review (not implemented; owner triage)

  • Host-scoped source cache key (multi-forge thrash).
  • Configuration blocker can omit Codex field names when both Codex fields were supplied and another field is also invalid.
  • creatidy-kernel --profile X config validate ignores the top-level --profile (uses config validate --profile).
  • Profile template embeds live HOME/PATH values instead of pure placeholders.
  • Cache replacement race under concurrent operators on one machine (worst case: redundant clone or transient unsuitable_cache).
  • evidence["source"]["cache_root"] recomputed instead of taken from the acquisition result.
  • A dedicated discovery category for a misconfigured explicit Codex override (currently not_found).

Readiness summary

For the exact frozen HEAD c2944f0da538e60fb67f091bd70e66eea212e141: verified independent dispatch on the owner-mandated GLM-5.3, completed fresh full review with substantive APPROVE, zero unresolved blocking findings, make check + make package-check PASS (implementation and reviewer sessions), green Forgejo CI (run #2192), acceptance criteria evidenced on the PR. Formal platform APPROVED is unavailable solely due to the shared author identity (recorded above). The issue remains open and the PR unmerged; merge authority stays with the owner.

## D1-06 delivery ledger (issue #44 / PR #45) — update 1 - Issue: Creatidy/creatidy-kernel#44 - PR: Creatidy/creatidy-kernel#45 (`issue-44-selfcontained-operator` -> `develop`) - Base: `develop` @ `fa81734ed10bfbdd6e7a75b138649a89a9724511` - Current HEAD: `c2944f0da538e60fb67f091bd70e66eea212e141` (unchanged since initial push) - `max_remediation_cycles = 10`, `remediation_used = 0` ### Review log **Review cycle 1** (initial review; costs zero remediation cycles) - Reviewed HEAD: `c2944f0da538e60fb67f091bd70e66eea212e141`; base `fa81734ed10bfbdd6e7a75b138649a89a9724511` (frozen inputs verified by the reviewer against live Forgejo state before reviewing) - Review profile: full-repository PR review per `.kilo/command/review-pr.md` at architecture/security criticality. Routing policy for this delivery is the **explicit owner override**: reviewer model exactly **GLM-5.3** (no Scarcity Router selection was consulted; no catalog/policy versions apply). Dispatched identity: `account:zai-individual-coding-plan/GLM-5.3` (reasoning level `max`). - Dispatch: ZCode dynamic workflow `dwfrun-f09e08f3-8981-4a1a-b1d8-4877568ad1f4`, subagent `independent-reviewer`, fresh read-only context distinct from the implementation session. - Dispatch verification: **DISPATCH_VERIFIED** — the harness resolved and ran the subagent on `account:zai-individual-coding-plan/GLM-5.3$max`; the reviewer session self-identified as GLM-5.3 (account:zai-individual-coding-plan/GLM-5.3). An Agent-Manager-style `Resolved models:` string does not exist in this harness; the recorded evidence is the dispatch configuration plus reviewer self-identification, both consistent with the owner-mandated model. - Independent review reference: PR #45 review id 687 (the reviewer itself posts nothing; its complete review was persisted verbatim on the PR, bound to the frozen HEAD). - `substantive_verdict`: **APPROVE** (`### Overall Recommendation`), zero blocking findings, 7 non-blocking suggestions, 2 author questions (recorded below as follow-up candidates). - `forgejo_review_state`: COMMENT; `platform_approval_recorded`: false. `platform_identity_limitation`: the only available reviewer identity is the PR author account (`kilo`); Forgejo rejects formal self-approval ("approve your own pull is not allowed"). No branch-protection rule is observable with this account's permissions, so any formal-approval merge requirement is reported separately as unobservable, not assumed. - Reviewer-run validation at the frozen HEAD: focused tests 112 passed; `make check` PASS (966 tests); `make package-check` PASS. Forgejo CI at `c2944f0` (pull_request event, run #2192): success. - Blocking finding count: 0. Remediation status: none. Resulting HEAD: unchanged. `remediation_used: 0 / 10`. ### Remediation log (none — no valid in-scope blocking findings) ### Follow-up candidates recorded from review (not implemented; owner triage) - Host-scoped source cache key (multi-forge thrash). - Configuration blocker can omit Codex field names when both Codex fields were supplied and another field is also invalid. - `creatidy-kernel --profile X config validate` ignores the top-level `--profile` (uses `config validate --profile`). - Profile template embeds live `HOME`/`PATH` values instead of pure placeholders. - Cache replacement race under concurrent operators on one machine (worst case: redundant clone or transient `unsuitable_cache`). - `evidence["source"]["cache_root"]` recomputed instead of taken from the acquisition result. - A dedicated discovery category for a misconfigured explicit Codex override (currently `not_found`). ### Readiness summary For the exact frozen HEAD `c2944f0da538e60fb67f091bd70e66eea212e141`: verified independent dispatch on the owner-mandated GLM-5.3, completed fresh full review with substantive `APPROVE`, zero unresolved blocking findings, `make check` + `make package-check` PASS (implementation and reviewer sessions), green Forgejo CI (run #2192), acceptance criteria evidenced on the PR. Formal platform `APPROVED` is unavailable solely due to the shared author identity (recorded above). The issue remains open and the PR unmerged; merge authority stays with the owner.
Author
Member

Documentation-only identity/status correction under #46, without changing the completed D1-06 acceptance scope: #44 is already integrated through PR #45 at eb4f4a2956 despite this issue remaining open. The Links section's Creatidy/creatidy-kernel#143/#166 qualification is incorrect; actual target identities are BioMedical-IT/scarcity-router#143 and BioMedical-IT/scarcity-router#166. Kernel task support is #32/#42. Historical #34/private rollout is not a current public-operator prerequisite. Expanded cache ownership/concurrency/dispatch safety and ciphertext/platform compatibility are distinct new work #61/#59/#62, not retroactively added to this completed delivery. No functional task or live execution is authorized by this correction.

Documentation-only identity/status correction under #46, without changing the completed D1-06 acceptance scope: #44 is already integrated through PR #45 at eb4f4a2956712bfaf39a3271e1523e7f77a91e26 despite this issue remaining open. The Links section's `Creatidy/creatidy-kernel#143/#166` qualification is incorrect; actual target identities are https://forgejo.creatidy.com/BioMedical-IT/scarcity-router/issues/143 and https://forgejo.creatidy.com/BioMedical-IT/scarcity-router/issues/166. Kernel task support is #32/#42. Historical #34/private rollout is not a current public-operator prerequisite. Expanded cache ownership/concurrency/dispatch safety and ciphertext/platform compatibility are distinct new work #61/#59/#62, not retroactively added to this completed delivery. No functional task or live execution is authorized by this correction.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Creatidy/creatidy-kernel#44
No description provided.