Architecture and Literature

For: people who want to understand why tether works the way it does, and what each design choice costs or saves.

Most agent frameworks were designed around massive frontier models with 200k-token context windows. When you point those same frameworks at a local 8B or 9B model (ollama/qwen3.5:9b, ollama/gemma4:e4b), they immediately fail—not because the model cannot reason, but because 3,000 to 8,000 tokens of nested JSON tool schemas and raw tool outputs flood the attention window before the agent takes its second step.

Tether's design translates four foundational works in recent AI systems research into a compact, zero-bloat Python micro-harness.

flowchart LR
    subgraph Envelope["Deterministic Envelope (Python + SQLite WAL)"]
        MUX["Action Multiplexer\nact(cmd, stdin)"]
        CTX["ContextAssembler\nHead+Tail L1 + K-Turn Folding"]
        DB[("SQLite WAL + FTS5\n.tether/tether.db")]
        SB["Security Sandbox\nbwrap + Path Jail"]
    end
    subgraph Core["Probabilistic Core"]
        LLM["LLM / SLM\n(Qwen 9B, Gemma 4, Claude, GPT)"]
    end

    CTX -->|"Immutable Prefix + L1 Window"| LLM
    LLM -->|"act(cmd, stdin)"| MUX
    MUX --> SB
    SB --> DB
    DB -->|"Observations + BM25 Chunks"| CTX

---

1. Theoretical Foundation

Probabilistic Core, Deterministic Envelope — Maia et al. (2026)

In "LLM-Native Applications as a Paradigm for AI-Driven Software Architecture", Maia et al. (2026) argue that reliable AI software must strictly separate the probabilistic reasoning kernel (the LLM) from a deterministic execution envelope.

In Tether:

• The model is responsible only for deciding the next command (act(cmd, stdin)).
• Everything else—CLI argument parsing, type coercion, KV-cache prefix preservation, observation folding, loop-stall detection, SQLite state transitions, and OS sandboxing—is enforced deterministically in Python and SQLite.

Why a Single act(cmd, stdin) Multiplexer? — Wang et al. (CodeAct, ICML 2024)

Traditional frameworks register 15–30 separate JSON function schemas (read_file, write_file, search_web, run_bash, ...). Small models frequently hallucinate nested JSON keys or pick the wrong schema.

In "Executable Code Actions Elicit Better LLM Agents" (ICML 2024), Wang et al. demonstrated that consolidating fragmented JSON schemas into a unified executable action space improves agent success rates by up to 20% while drastically shrinking prompt overhead. Tether exposes a single JSON tool—act(cmd, stdin)—that routes every built-in and @tool-decorated Python function through a familiar POSIX CLI grammar.

Why Progressive --help & Bounded Views? — Yang et al. (SWE-agent, NeurIPS 2024)

In "SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering" (NeurIPS 2024), Yang et al. showed that raw shell interfaces and verbose outputs degrade agent performance, whereas purpose-built Agent-Computer Interfaces (ACIs) with compact documentation and guardrailed feedback loops significantly boost accuracy.

Tether implements this as Progressive --help Disclosure:

• The system prompt lists only a \(\le\) 12-token summary line per enabled tool.
• Full parameter signatures (\(\le\) 60 tokens) are revealed Just-In-Time (JIT) only when the model runs <cmd> --help or makes a syntax mistake (returning a self-healing usage hint rather than a Python traceback).

Why Tiered L1/L2 Memory & Observation Folding? — Packer et al. (MemGPT, 2023)

In "MemGPT: Towards LLMs as Operating Systems", Packer et al. (2023) introduced virtual context management by treating the LLM's context window as bounded L1 working RAM backed by tiered L2 disk storage.

Tether implements a deterministic three-tier memory hierarchy backed by .tether/tether.db (SQLite WAL + FTS5):

1. Head + Tail L1 Active Elision: Even within the active \(K=2\) turn window, if a command emits an oversized output (more than a third of max_context_tokens in chars, at least 2,400), ContextAssembler keeps the first 60% of the cap (headers/command summary) + the last 40% (stack trace/exit status) and elides the middle with a pointer to mem recall <obs_id>.
2. Ephemeral \(K\)-Turn Folding: Once the history fills more than half of max_context_tokens (below that nothing is folded by age), an observation older than \(K=2\) steps is folded into a single-line tombstone:

   [obs_1 folded: `web fetch https://...` (3,420 chars) | use `mem recall obs_1` or `mem search <query>`]
   

   Exception (#69): the latest read of each of the last 8 files (edit view <p>, bash cat/head/tail/sed -n <p>; paths normalized against the workspace) is not folded until <p> is edited (edit write|replace, or any other bash command naming it: sed -i, tee, redirects, python -c) or re-read. The head+tail elision still applies (an observation may use up to a third of max_context_tokens, at least 2,400 chars), and the hard context budget remains #81. The mem recall hint is shown only when mem is enabled (else "re-run the command").
3. Sub-Millisecond SQLite FTS5 BM25 RAG: Every raw observation and final deliverable is automatically chunked (~150 tokens per paragraph/section) and indexed into an FTS5 virtual table (chunks_fts) so the agent (or a parent agent) can retrieve exact passages via mem search or agent query.

---

2. High-Efficiency Execution Engine

100% KV-Cache Prefix Immutability

Local inference engines (Ollama, llama.cpp, vLLM) and cloud providers (Anthropic ephemeral prompt caching, OpenAI prefix caching) reuse KV-cache blocks only when the message prefix matches byte-for-byte.

• Tether never mutates the system prompt or tools array mid-session.
• Working memory pins ([PINNED MEMORY]) are attached to the tail message of the assembled prompt rather than injected near the top of the message list, ensuring all historical turn messages remain 100% byte-identical across steps.
• AnthropicProvider automatically marks the static system prompt and act tool schema with cache_control: {"type": "ephemeral"}.

Deterministic Loop-Stall Breaker & Pre-done Verification Gates

Small 8B/9B models occasionally fall into a probability trap where they emit the exact same command (web search "..." or bash ...) twice in a row, or prematurely call done while tests or linters are failing:

• Loop-Stall Breaker: Harness.execute_session tracks the (cmd, stdin) signature of the previous step and intercepts consecutive duplicates before executing the tool, returning a [LOOP DETECTED] nudge that forces the model to advance (direct wording from the second consecutive repeat). A read of an unchanged file whose earlier output is still live (live_read_obs) is answered once with a one-line [obs_N unchanged] pointer instead of re-running. A companion cycle detector (detect_cycle) catches a 2-command cycle repeated 3 times, or a 3- or 4-command cycle repeated twice (within the last 8 actions), skips the repeated command, nudges with [CYCLE DETECTED] naming the commands, and records cycle_stall. Reading one file over varying line ranges 7 times in a row without a change (edit view, sed -n, cat, grep) is also a cycle_stall, which a repeated-command check cannot see. Four stall warnings without a change restart the session from a short description of its state (twice at most, see context and store) and then hand it back to the caller as WAITING_INPUT instead of spending every step; any edit or other non-read command resets the count.
• Pre-done Verification Gates: When configured via Harness(verify=[...]), CLI --verify, or Skill YAML verify: [...], Tether intercepts done and runs each verification command inside the session's SandboxPolicy. It consumes 0 static prompt tokens unless a gate exits non-zero—at which point termination is rejected and a compact [VERIFICATION GATE FAILED] observation is injected so the model can self-heal. A gate that errors rather than fails (sandbox violation, exit 124/126/127) ends the session FAILED immediately as gate_error instead of consuming retries.

Failing Test Source in the Observation — Olausson et al. (ICLR 2024), Zhong et al. (LDB, ACL 2024)

Self-repair is limited by the quality of the feedback, not by the number of tries (Olausson et al.), and showing the runtime values behind a failure improves repair (Zhong et al.). A Python traceback shows the failing assert line (self.assertEqual(total(basket), 10240)) but not the inputs set on the lines above it, so a small model guesses or re-reads the whole test file. When a recognised test command fails, the harness appends the source of up to 3 failing test functions named in the traceback (files inside the workspace only, at most 1,500 chars, skipped when the output already shows it, as pytest does):

[failing test source]
# book_store_test.py:12
    def test_four_groups_of_four(self):
        basket = [1, 1, 2, 2, 3, 3, 4, 5, 1, 1, 2, 2, 3, 3, 4, 5]
        self.assertEqual(total(basket), 10240)

0 static prompt tokens, no model call; the text is part of that observation and folds with it.

Naming Regressions Between Test Runs — Huang et al. (ASE 2026), Wang-Lin et al. (2026)

Iterative LLM repair often breaks code that already worked: across six models, 40% to 73% of multi-turn tasks lose behaviour that was correct earlier, and checking each new version against the earlier tests is the one mitigation that helped every model (Huang et al.). Repeated fixing also falls into cycles where the same change is added and removed (Wang-Lin et al.). A test summary like FAILED (failures=3) hides this: three failing before and after can be two different sets. The harness remembers the failing tests of the last run of the same suite (same number of tests) and, when tests that passed then fail now, appends one line:

[regression] Your last change broke 1 test(s) that passed before: test_o_wins; now passing: test_x_wins. Keep what worked.

0 static prompt tokens, no model call, at most 400 chars; nothing is added when no test broke.

The Repair Loop Around Tests — Ridnik et al. (AlphaCodium, 2024), Zhong et al. (LDB, ACL 2024), Olausson et al. (ICLR 2024), Aider auto-test

On a small local model most of the time goes into typing code: a whole-file rewrite of a 100-line module costs qwen3.5:9b 30 to 100 seconds, while running the tests takes under one. So the harness spends its own (free) time around each test run, and never the model's:

• Tests after every edit. Once the model has run a recognised test command, the harness re-runs it after each command that writes an existing workspace file and appends the result under [auto-test]. The model sees whether a version works before it types the next one.
• Say what the tests expect, first. When the model reads the test file of a file it is about to write (paasio_test.py next to paasio.py), one [BEFORE WRITING] line asks it to state in plain words what the 3 hardest tests call and expect, then write. Reasoning about the tests before the code raised GPT-4 from 19% to 44% on CodeContests in AlphaCodium.
• Show what the code did. When the same tests fail again after a change, or only a few still fail, the harness runs the smallest failing test once more under a tracer and appends the calls into workspace code, with their arguments and results (2 levels deep, at most 30 lines):

[runtime trace of test_f: calls into your code, arguments and results]
test_f(self=<m_checks.T testMethod=test_f>)
  f()
  -> 1
  raised AssertionError: 1 != 3

- Start fresh, or keep the near pass. When the same check fails 3 times with edits between, the files the model changed go back to how they started and the history restarts from a short description that names the failing tests, not the failed code: for weaker models a new attempt beats repairing a wrong one (Olausson et al.). A version that already passes all but at most 3 tests is kept instead: when later edits do not beat it, its files come back in place with one [RESTORED] line and the history stays, so the model does not spend its last minutes re-reading the tests.

0 static prompt tokens and no extra model call for any of these. On the paasio exercise with qwen3.5:9b (600 s cap), these moved the score from a mean of 0.63 over 3 runs to 0.94 over 5 runs, with 1 of 5 passing; the rest ended at the time cap with 22 to 24 of 25 tests passing.

Naming a Rewrite That Changed Nothing — Wang-Lin et al. (2026)

The same repair cycles show up in their smallest form: a model rewrites a whole file with the code it already had, sometimes byte for byte, sometimes with only a comment changed, then runs the tests and gets the same failure. In one 8-exercise run of qwen3.5:9b, 9 of 62 rewrites were like this. Before a command that writes a file that already exists (edit write|replace <p>, > p, >> p, tee p), the harness keeps its bytes; after the command, when every such file holds the same code, it appends one line:

[NUDGE] hangman.py: only comments or spacing changed, the code is the same. The tests will fail the same way; change the logic.

"Same code" means identical bytes, or for a .py file an identical syntax tree (comments and spacing do not count). New files, files over 256 KB and files outside the workspace are not compared. 0 static prompt tokens, no model call, at most 200 chars.

The same check names a rewrite that changed only a few lines: when a whole-file write of a file of 20 or more lines changed at most a fifth of them, one line says how many lines changed and points to edit replace, once per file. Typing a 100-line file again to change two lines costs a 9B model about a minute; a replace costs seconds.

Connection Pooling & Thread-Local SQLite WAL Reuse

• Model providers reuse persistent httpx.AsyncClient connection pools across all turns in a session, eliminating repeated TCP/TLS handshakes.
• SQLiteStore caches thread-local SQLite connections configured with PRAGMA journal_mode=WAL; PRAGMA synchronous=NORMAL; PRAGMA temp_store=MEMORY; PRAGMA mmap_size=134217728;.

---

References

1. Maia, P. H. M., et al. (2026). LLM-Native Applications as a Paradigm for AI-Driven Software Architecture.
2. Wang, X., Chen, Y., Yuan, L., Zhang, Y., Li, Y., Peng, H., & Ji, H. (2024). Executable Code Actions Elicit Better LLM Agents (CodeAct). In ICML 2024. arXiv:2402.01030
3. Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., & Press, O. (2024). SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. In NeurIPS 2024. arXiv:2405.15793
4. Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., & Gonzalez, J. E. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560
5. Olausson, T. X., Inala, J. P., Wang, C., Gao, J., & Solar-Lezama, A. (2024). Is Self-Repair a Silver Bullet for Code Generation? In ICLR 2024. arXiv:2306.09896
6. Zhong, L., Wang, Z., & Shang, J. (2024). Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step (LDB). In Findings of ACL 2024. arXiv:2402.16906
7. Huang, Y., Ma, L., Tahir, A., Zhang, Q., Xiao, L., & Xiao, L. (2026). Regression Accumulation in Multi-Turn LLM Programming Conversations. In ASE 2026. arXiv:2607.01855
8. Wang-Lin, X., Isopoussu, A., & Mahon, L. (2026). If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMs. arXiv:2609.10123