Flaky test fixture: fake-process _feed thread can write after pipe close #166

New issue

Open

opened 2026-10-01 00:11:41 +00:00 by kilo · 1 comment

kilo commented

2026-10-01 00:11:41 +00:00

Member

Provenance: remediation evidence from PR #164 (#143). Test-infrastructure only — no production impact is implied.

Affected area

tests/test_openai_codex_acquisition.py
_FakeAppServer (tests/test_openai_codex_acquisition.py:94): fake Codex app-server process on a real pipe, with a daemon feeder thread _feed (tests/test_openai_codex_acquisition.py:149) writing scripted lines through a non-blocking write fd.
_close_write() (tests/test_openai_codex_acquisition.py:166) is the single shutdown seam: it sets the stop event, joins the feeder with a timeout (join(timeout=5.0), line 174), then closes the write fd exactly once under a lock.

Race

The fixture's documented invariant — "the write end is closed only after the feeder thread has joined" — is only enforced up to the join timeout. Under deep-suite/load timing the feeder can be descheduled after passing its _stop check but before (or between) os.write() attempts; if the bounded join lapses, _close_write() proceeds to close the write fd and the resumed feeder calls os.write() on the closed fd. The uncaught OSError (EBADF) surfaces as Exception in thread ... (_feed) on stderr.

Downstream effect: tests asserting empty captured stderr can then fail on that leaked traceback even when the behavior under test is correct.

Observed evidence

The failure was observed on pristine develop, while the #164 branch could pass under equivalent concurrent load.
Module-isolated loops did not reproduce it reliably — consistent with a load-sensitive fixture race rather than a deterministic defect.

Acceptance for the future fix

Deterministic synchronization/ownership of feeder shutdown: the closer must provably not race the feeder's final write.
No arbitrary timeout increase as the fix.
No leaked thread traceback on stderr.
Repeated/load-sensitive regression validation (e.g., full-suite or artificial-concurrency loops — module-isolated runs alone demonstrably do not exercise the race).

Non-goal: do not conflate this with production Codex acquisition behavior unless investigation finds an actual production impact.

Provenance: remediation evidence from PR #164 (#143). Test-infrastructure only — no production impact is implied. ## Affected area - `tests/test_openai_codex_acquisition.py` - `_FakeAppServer` (`tests/test_openai_codex_acquisition.py:94`): fake Codex app-server process on a real pipe, with a daemon feeder thread `_feed` (`tests/test_openai_codex_acquisition.py:149`) writing scripted lines through a non-blocking write fd. - `_close_write()` (`tests/test_openai_codex_acquisition.py:166`) is the single shutdown seam: it sets the stop event, joins the feeder **with a timeout** (`join(timeout=5.0)`, line 174), then closes the write fd exactly once under a lock. ## Race The fixture's documented invariant — "the write end is closed only after the feeder thread has joined" — is only enforced up to the join timeout. Under deep-suite/load timing the feeder can be descheduled after passing its `_stop` check but before (or between) `os.write()` attempts; if the bounded join lapses, `_close_write()` proceeds to close the write fd and the resumed feeder calls `os.write()` on the closed fd. The uncaught `OSError` (EBADF) surfaces as `Exception in thread ... (_feed)` on stderr. Downstream effect: tests asserting empty captured stderr can then fail on that leaked traceback even when the behavior under test is correct. ## Observed evidence - The failure was observed on pristine `develop`, while the #164 branch could pass under equivalent concurrent load. - Module-isolated loops did not reproduce it reliably — consistent with a load-sensitive fixture race rather than a deterministic defect. ## Acceptance for the future fix - Deterministic synchronization/ownership of feeder shutdown: the closer must provably not race the feeder's final write. - No arbitrary timeout increase as the fix. - No leaked thread traceback on stderr. - Repeated/load-sensitive regression validation (e.g., full-suite or artificial-concurrency loops — module-isolated runs alone demonstrably do not exercise the race). Non-goal: do not conflate this with production Codex acquisition behavior unless investigation finds an actual production impact.

kilo referenced this issue

2026-10-03 08:46:28 +00:00

F. First-class worker service lifecycle (systemd) #138

kilo referenced this issue from Creatidy/creatidy-kernel

2026-10-03 10:04:41 +00:00

D1-05: Add bounded Scarcity Router #166 TaskSpec (#42) #43

kilo referenced this issue from Creatidy/creatidy-kernel

2026-10-04 22:40:26 +00:00

D1-06 — Public self-contained operator path for Scarcity Router-backed task execution #44

adrian.tkacz added the

wontfix

label

2026-10-05 10:24:16 +00:00

kilo referenced this issue

2026-10-24 17:16:25 +00:00

SUPERSEDED by #216: retire obsolete reviewer delivery (#205) #206

kilo referenced this issue

2026-10-24 17:16:52 +00:00

Harden independent reviewer result capture, safe diagnostics and observable CLI recovery #205

kilo referenced this issue

2026-10-27 21:23:25 +00:00

Record D-083 backlog disposition: re-home excluded wontfix requirements, correct #183/#186 prerequisites #228

kilo referenced this issue

2026-10-27 21:24:30 +00:00

Representative-client compatibility + operations hardening program (ZCode acceptance follow-up) #132

kilo commented

2026-10-27 21:24:50 +00:00

Author

Member

2026-10-27 owner backlog disposition (D-083, governance issue #228). This issue remains open with its wontfix label unchanged; history preserved. Owner disposition: the join(timeout=5.0)/descriptor-close race is recorded as a test-infrastructure limitation — no demonstrated production dependency exists. No successor issue is registered; one is to be created only if a concrete CI failure or newly demonstrated product impact warrants it. See D-083 in docs/decisions.md.

**2026-10-27 owner backlog disposition (D-083, governance issue #228).** This issue remains open with its `wontfix` label unchanged; history preserved. Owner disposition: the `join(timeout=5.0)`/descriptor-close race is recorded as a **test-infrastructure limitation** — no demonstrated production dependency exists. No successor issue is registered; one is to be created only if a concrete CI failure or newly demonstrated product impact warrants it. See D-083 in `docs/decisions.md`.

kilo referenced this issue

2026-10-30 09:03:48 +00:00

Bind worker credentials to paired origin (#224) #236