Creatidy/scarcity-router

Fork 0

M04: Generic OpenAI-compatible HTTP adapter and Ollama integration #89

New issue

Closed

opened 2026-09-19 13:06:31 +00:00 by kilo · 2 comments

kilo commented

2026-09-19 13:06:31 +00:00

Member

Goal

Support configurable OpenAI-compatible APIs and local inference through ONE generic HTTP adapter with explicit provider-specific differences — not a separate gateway per provider.

Module of the execution-gateway program; architecture and contracts come from A0 (#85).

User value

Users bring their existing OpenAI, DeepSeek, OpenRouter and Z.ai API keys, plus their local Ollama, and everything appears as routable resources with PAYG and entitlement channels kept distinct.

Relationship to current code

New provider execution adapter(s) following the existing provider-adapter discipline (docs/providers.md): provider differences at the adapter edge, core stays provider-independent.
Reuses M01's resource representation and M03's execution coordinator. Ollama is accessed directly over HTTP when network-accessible, or through the worker (M05) when localhost-only — one inference implementation, two transports, never two separate Ollama implementations.

Scope

Generic HTTP adapter with explicit, evidence-based provider differences; presets or documented configurations for at least OpenAI, DeepSeek, OpenRouter and Z.ai.
Keep PAYG and Coding Plan or other entitlement channels distinct where applicable.
Ollama: direct HTTP and worker-bridged (the bridged path sequences with M05); model discovery where reliable; explicit capability configuration; health checks.
Provider URL and credential sources come from administrator configuration (M09), never from arbitrary client request content.

Non-goals

No automatic model downloads; no GPU driver installation; no GPU lifecycle management; no per-provider gateways; no claiming full OpenAI compatibility where features differ.

Contracts / interfaces affected

Execution adapter contract (A0); resource registry entries (M01); compatibility matrix entries (M03).

Dependencies

Depends on: A0 (#85), M03.
The worker-bridged Ollama path additionally sequences with M05 (documented inside this issue as a sub-scope ordering; deliberately no Forgejo dependency so the API-only core path is never blocked by the worker).
Feeds: M10 (E2E).

Acceptance criteria

API-only operation works without a worker.
A localhost-only Ollama instance on another host can be accessed through the worker without opening a public Ollama port (with M05).
Providers are not labeled fully OpenAI-compatible when features differ; the compatibility matrix reflects reality.
No automatic model downloads, no GPU driver installation, no GPU lifecycle management.
Resources without verified capability ratings remain unrated/unknown rather than automatically sufficient.
Full validation gate green; redacted fixtures only.

Security considerations

Outbound provider HTTP: verified TLS (no verify=false), provider credentials bound to configured origins, SSRF protection, no Authorization forwarding across unsafe redirects (reject cross-origin redirects), and explicit bounded exceptions for plain-HTTP localhost only where justified (e.g. Ollama loopback). Credentials are never logged and never sourced from client request content.

Risks

Provider drift on "OpenAI-compatible" APIs — mitigated by evidence-based compatibility matrices and fail-closed feature negotiation.

Evidence still required

Dated compatibility evidence per preset provider and for Ollama's OpenAI-compatibility surface (https://docs.ollama.com/api/openai-compatibility — re-verify at implementation time with date and version).

## Goal Support configurable OpenAI-compatible APIs and local inference through ONE generic HTTP adapter with explicit provider-specific differences — not a separate gateway per provider. Module of the execution-gateway program; architecture and contracts come from A0 (#85). ## User value Users bring their existing OpenAI, DeepSeek, OpenRouter and Z.ai API keys, plus their local Ollama, and everything appears as routable resources with PAYG and entitlement channels kept distinct. ## Relationship to current code - New provider execution adapter(s) following the existing provider-adapter discipline (`docs/providers.md`): provider differences at the adapter edge, core stays provider-independent. - Reuses M01's resource representation and M03's execution coordinator. Ollama is accessed directly over HTTP when network-accessible, or through the worker (M05) when localhost-only — one inference implementation, two transports, never two separate Ollama implementations. ## Scope - Generic HTTP adapter with explicit, evidence-based provider differences; presets or documented configurations for at least OpenAI, DeepSeek, OpenRouter and Z.ai. - Keep PAYG and Coding Plan or other entitlement channels distinct where applicable. - Ollama: direct HTTP and worker-bridged (the bridged path sequences with M05); model discovery where reliable; explicit capability configuration; health checks. - Provider URL and credential sources come from administrator configuration (M09), never from arbitrary client request content. ## Non-goals - No automatic model downloads; no GPU driver installation; no GPU lifecycle management; no per-provider gateways; no claiming full OpenAI compatibility where features differ. ## Contracts / interfaces affected Execution adapter contract (A0); resource registry entries (M01); compatibility matrix entries (M03). ## Dependencies - Depends on: A0 (#85), M03. - The worker-bridged Ollama path additionally sequences with M05 (documented inside this issue as a sub-scope ordering; deliberately no Forgejo dependency so the API-only core path is never blocked by the worker). - Feeds: M10 (E2E). ## Acceptance criteria - [ ] API-only operation works without a worker. - [ ] A localhost-only Ollama instance on another host can be accessed through the worker without opening a public Ollama port (with M05). - [ ] Providers are not labeled fully OpenAI-compatible when features differ; the compatibility matrix reflects reality. - [ ] No automatic model downloads, no GPU driver installation, no GPU lifecycle management. - [ ] Resources without verified capability ratings remain unrated/unknown rather than automatically sufficient. - [ ] Full validation gate green; redacted fixtures only. ## Security considerations Outbound provider HTTP: verified TLS (no `verify=false`), provider credentials bound to configured origins, SSRF protection, no `Authorization` forwarding across unsafe redirects (reject cross-origin redirects), and explicit bounded exceptions for plain-HTTP localhost only where justified (e.g. Ollama loopback). Credentials are never logged and never sourced from client request content. ## Risks - Provider drift on "OpenAI-compatible" APIs — mitigated by evidence-based compatibility matrices and fail-closed feature negotiation. ## Evidence still required - Dated compatibility evidence per preset provider and for Ollama's OpenAI-compatibility surface (https://docs.ollama.com/api/openai-compatibility — re-verify at implementation time with date and version).

kilo referenced this issue

2026-09-19 13:11:01 +00:00

A0: Architecture, decisions, and contracts for the optional execution gateway #85

kilo added the

enhancement

label

2026-09-19 13:11:40 +00:00

kilo added a new dependency

2026-09-19 13:12:00 +00:00

#85 A0: Architecture, decisions, and contracts for the optional execution gateway

kilo added a new dependency

2026-09-19 13:12:00 +00:00

#88 M03: OpenAI-compatible gateway and execution coordinator

kilo added a new dependency

2026-09-19 13:12:18 +00:00

#95 M10: Distribution, installation, update, and end-to-end acceptance

kilo referenced this issue

2026-09-19 19:52:09 +00:00

M01: Resource registry, state, and collectors for executable resources (issue #86) #99

kilo referenced this issue

2026-09-20 00:01:35 +00:00

docs(m07): close ZCode feasibility track — owner Stage-2 NO-GO decision D-047 (issue #92) #103

kilo referenced this issue

2026-09-20 00:02:01 +00:00

M07: ZCode adapter: feasibility proof and experimental integration #92

kilo commented

2026-09-20 00:02:01 +00:00

Author

Member

Scope clarification after the M07 closeout (owner decision D-047, 2026-09-20; M07 Stage 1 complete, Stage 2 cancelled — see #92). No change to this issue's scope; provenance notes for the implementing agent:

Z.ai Coding Plan is a candidate generic OpenAI-compatible HTTP provider for this adapter. The vendor documents OpenAI-compatible endpoints for the Coding Plan (https://api.z.ai/api/coding/paas/v4 — re-verify with date and version at implementation time; terms suitability tracked under U-009). It flows through the ONE generic adapter with the existing preset discipline (docs/providers.md); no Z.ai-specific duplicated gateway architecture is wanted or needed.
ZCode Desktop is NOT an M04 backend. M07 Stage 1 (evidence: docs/zcode-adapter-stage1-evidence.md) found no official supported programmatic surface, and the owner cancelled M07 Stage 2 (D-047). Nothing in this issue wraps or shells out to the ZCode desktop runtime.
The M07 NO-GO does not remove Z.ai HTTP/API support. Subscription-backed Z.ai access stays on this issue's generic HTTP path as a distinct resource/entitlement channel per D-042 (PAYG vs Coding Plan remain distinct resources and pools).

**Scope clarification after the M07 closeout** (owner decision D-047, 2026-09-20; M07 Stage 1 complete, Stage 2 cancelled — see #92). No change to this issue's scope; provenance notes for the implementing agent: - **Z.ai Coding Plan is a candidate generic OpenAI-compatible HTTP provider for this adapter.** The vendor documents OpenAI-compatible endpoints for the Coding Plan (`https://api.z.ai/api/coding/paas/v4` — re-verify with date and version at implementation time; terms suitability tracked under U-009). It flows through the ONE generic adapter with the existing preset discipline (`docs/providers.md`); no Z.ai-specific duplicated gateway architecture is wanted or needed. - **ZCode Desktop is NOT an M04 backend.** M07 Stage 1 (evidence: `docs/zcode-adapter-stage1-evidence.md`) found no official supported programmatic surface, and the owner cancelled M07 Stage 2 (D-047). Nothing in this issue wraps or shells out to the ZCode desktop runtime. - **The M07 NO-GO does not remove Z.ai HTTP/API support.** Subscription-backed Z.ai access stays on this issue's generic HTTP path as a distinct resource/entitlement channel per D-042 (PAYG vs Coding Plan remain distinct resources and pools).

adrian.tkacz referenced this issue from a commit

2026-09-20 18:48:12 +00:00

Merge M04 generic OpenAI-compatible HTTP adapter and Ollama integration (issue #89) into program integration branch

kilo referenced this issue

2026-09-20 18:49:11 +00:00

Program integration: M04 generic HTTP adapter + Ollama (#89), M05 native worker + pairing + transport (#90), M09 configuration + web UX + diagnostics (#94) #105

adrian.tkacz referenced this issue from a commit

2026-09-20 20:32:20 +00:00

Merge pull request 'Program integration: M04 generic HTTP adapter + Ollama (#89), M05 native worker + pairing + transport (#90), M09 configuration + web UX + diagnostics (#94)' (#105) from program/m04-m05-m09-parallel into develop

kilo commented

2026-09-21 17:04:48 +00:00

Author

Member

Closeout (v0.1.0 release-readiness reconciliation, 2026-09-21). M04 is implemented and its acceptance criteria are satisfied on develop @ 9d6192e.

ONE generic OpenAI-compatible HTTP adapter (providers/openai_http_adapter.py + openai_http_core.py, 50+50 tests) with evidence-backed presets (providers/openai_http_evidence.py) and the shared translation core; Ollama is served through the same core direct (server-direct HTTP) and worker-bridged (worker_local_translation.py — one inference implementation, two transports, no second Ollama implementation).
API-only operation works without a worker (tests/test_e2e_acceptance.py scenarios); a localhost-only Ollama behind the worker is covered by the worker-bridged e2e scenarios (M10); the compatibility matrix reflects real, dated evidence — providers are not labeled fully OpenAI-compatible where features differ, generic stays all-UNKNOWN and fails closed; no automatic model downloads/GPU management anywhere.
Since PR #108 the composed server builds its production matrix from configuration through server_composition (M04 default_cells_for per bound preset resource), so un-evidenced presets still fail closed with compatibility_unknown.
Security: verified TLS with origin-bound credentials, cross-origin redirects refused and never followed (test_redirects_are_refused_and_never_followed), plain HTTP only for explicit loopback origins, no credential from client request content — pinned in tests/test_openai_http_adapter.py and the M10 security matrix.
The scope-clarification comment above (Z.ai via the generic HTTP path; no ZCode desktop backend) is honored: presets carry the evidenced Z.ai Coding-Plan endpoint discipline; nothing shells out to ZCode.
Validation gate green: make check 1868 tests OK, basedpyright clean, make package-check green; Forgejo CI green on develop (run #25).

Closing as complete; docs/roadmap.md now records M04 complete.

**Closeout (v0.1.0 release-readiness reconciliation, 2026-09-21).** M04 is implemented and its acceptance criteria are satisfied on `develop` @ `9d6192e`. - ONE generic OpenAI-compatible HTTP adapter (`providers/openai_http_adapter.py` + `openai_http_core.py`, 50+50 tests) with evidence-backed presets (`providers/openai_http_evidence.py`) and the shared translation core; Ollama is served through the same core direct (server-direct HTTP) and worker-bridged (`worker_local_translation.py` — one inference implementation, two transports, no second Ollama implementation). - API-only operation works without a worker (`tests/test_e2e_acceptance.py` scenarios); a localhost-only Ollama behind the worker is covered by the worker-bridged e2e scenarios (M10); the compatibility matrix reflects real, dated evidence — providers are not labeled fully OpenAI-compatible where features differ, `generic` stays all-`UNKNOWN` and fails closed; no automatic model downloads/GPU management anywhere. - Since PR #108 the composed server builds its production matrix from configuration through `server_composition` (M04 `default_cells_for` per bound preset resource), so un-evidenced presets still fail closed with `compatibility_unknown`. - Security: verified TLS with origin-bound credentials, cross-origin redirects refused and never followed (`test_redirects_are_refused_and_never_followed`), plain HTTP only for explicit loopback origins, no credential from client request content — pinned in `tests/test_openai_http_adapter.py` and the M10 security matrix. - The scope-clarification comment above (Z.ai via the generic HTTP path; no ZCode desktop backend) is honored: presets carry the evidenced Z.ai Coding-Plan endpoint discipline; nothing shells out to ZCode. - Validation gate green: `make check` 1868 tests OK, basedpyright clean, `make package-check` green; Forgejo CI green on `develop` (run #25). Closing as complete; `docs/roadmap.md` now records M04 complete.

kilo closed this issue

2026-09-21 17:05:12 +00:00