Junta Model Council: recreation guide

Summary

In brief

Junta is a provider-neutral multi-model council: several LLMs from different vendors answer independently, review each other anonymously, and a chairman synthesizes a final answer with dissent preserved.
Three council types on one engine: deliberation (research/Q&A), decision (choose one option), and grounded implementation review (code, evidence- required).
This guide has everything to rebuild it from scratch: architecture, the exact prompts, config format, protocol state machines, adapter design, and a component checklist. Nothing here is specific to any private deployment.

1. 1. What it is and why

A single strong model has correlated blind spots. A council of models from different families corrects them, but only if built to avoid the known failure modes: identity leakage (judges favoring their own family), position bias, verbosity bias, premature consensus, and silent member dropouts. Junta is a thin, provider-neutral core that enforces the mitigations structurally.

Design stance: a clean core, not a fork of any single-app council. Providers are adapters; protocols are config; the session ledger is the product.

2. 2. Architecture (the load-bearing rules)

Identity firewall. Model identities never enter any prompt. Members are referred to only by opaque labels (A, B, C…). A sealed label→model map lives in the ledger, outside every prompt. This kills self-preference and identity leakage by construction.
Anonymized answers + randomized order. Peer responses are presented as "Response A / Response B…" with per-judge randomized ordering; the permutation is recorded for replay. Defeats position bias.
Event-sourced ledger. Every stage transition, model call, permutation, ballot, and cost charge is an immutable event with content-addressed blobs. Replay = re-render; re-run = a new session referencing the old.
Quorum, not silence. A stage declares a quorum; failed members are typed events, never silent drops. Synthesis is told "N of M contributed."
Budget as a hard field. Each turn reserves its worst-case cost before the call; if the reservation exceeds the remaining budget, the request's max_tokens is capped down to a floor rather than refused, so a small budget degrades gracefully instead of deadlocking. Submission preflight rejects a budget that can never satisfy quorum, with zero spend.
Adaptive retries. On timeout, escalate the deadline (×1.5 per attempt up to a transport ceiling); on truncation (finish_reason=length), escalate max_tokens instead of resending an identical request.
Isolated tool execution. In the implementation council, deliberating models receive tool results as evidence records, never tool access. Only a scoped read-only executor touches the repo snapshot.

3. 3. Protocols (state machines, config-driven)

One engine runs all three; the config's stages list picks the protocol.

Deliberation council — generate → peer_review → synthesize. Members answer independently; each anonymously ranks all responses; the chairman synthesizes. Rankings aggregate by Borda count; ties broken by the chairman. Best for research and open questions.

Decision council — propose → pairwise → recommend. Members each propose one option (structured: option, rationale, trade-offs, risks); pairwise blind comparisons produce standings; the chairman recommends one course with minority positions preserved verbatim. Best for "choose one."

Implementation council — snapshot → inspect → cross_review → report. Pin a commit; each member independently inspects scoped files and reports defects that MUST quote the exact offending span (unquotable findings are discarded mechanically); peers cross-review each finding for confirm/reject; the chairman writes a report grouped by severity from verified findings only. Findings dedupe by file + overlapping line range + claim similarity.

4. 4. The prompts (verbatim — this is the core IP)

Generate (system):

You are one anonymous member of a deliberation council. Answer the question directly, factually, and concisely. Do not identify yourself or your model name. (temperature 0.7)

Peer review (system):

You are an anonymous peer reviewer on a deliberation council. Judge the responses strictly on accuracy and insight — never on length or style. Do not speculate about which model wrote a response.

Peer-review (user template):

Question:
{question}

Peer responses:

### Response {label}
{text}
... (all responses, anonymized, per-judge randomized order)

Rank the responses from best to worst. Respond with only a JSON object of the
form {"ranking": ["<label>", ...]} using exactly these labels: {labels}.

Ballot validation: the ranking MUST be an exact permutation of the allowed labels (no omissions) — an omitted label would let a judge zero-out a rival's Borda score.

Synthesis (system):

You are the chairman of an anonymous deliberation council. Synthesize the best final answer from the peer responses and their ranking. Preserve substantive dissent instead of averaging it away. Do not speculate about which model wrote a response.

Synthesis (user template):

Question:
{question}

{contributed} of {total} council members contributed responses.
Peer ranking (best first): {ranking}.

Peer responses:

### Response {label}
{text}
...

Write the council's final answer.

Propose (system):

You are one anonymous member of a decision council. Propose the single option you would choose, with rationale, trade-offs, and risks. Do not identify yourself or your model name. (structured output: {option, rationale, trade_offs[], risks[]}; temperature 0.7)

Pairwise (user template, reuses the peer-review system prompt):

Question:
{question}

### Option 1
{first.text}

### Option 2
{second.text}

Which option is the better answer to the question, judged strictly on
substance (never length or style)? Respond with only a JSON object
{"winner": "1"} or {"winner": "2"}.

Recommend (system):

You are the chairman of an anonymous decision council. Recommend one course of action based on the proposals and their pairwise standings. Preserve substantive minority positions verbatim in the dissent list — never average them away. Do not speculate about which model wrote a proposal.

Recommend (user template):

Question:
{question}

{contributed} of {total} council members contributed proposals.
Pairwise standings (best first): {standings}.

Proposals:

### Response {label}
{text}
...

Respond with only a JSON object {"recommendation": "...", "dissent": ["..."]}.

Inspect (system):

You are one anonymous reviewer on an implementation council. Report concrete defects in the code you are shown. Every finding MUST quote the exact span it rests on with its file and line range — findings whose quotes do not match the code are discarded mechanically. Do not identify yourself or your model name.

Inspect (user template): a review-focus line (optional), then "Report defects in the following snapshot (commit {sha12}). Cite file, 1-based line range, and the EXACT quoted span for every finding," a JSON schema instruction, and the 1-based line-numbered file bodies.

Cross-review (system):

You are an anonymous cross-reviewer on an implementation council. For each numbered finding, judge strictly whether the quoted evidence supports the claim. Confirm only findings a maintainer should act on.

Report (system):

You are the chairman of an implementation council. Write a review report strictly from the verified findings given — do not add new claims. Group by severity; keep each finding's file citation.

5. 5. Config format

name: council-basic
members:
  - id: "openrouter:moonshotai/kimi-k3"
  - id: "openrouter:openai/gpt-5"
  - id: "gemini:gemini-2.5-pro"
    usd_per_m_input: 1.25      # required where the provider API exposes no pricing
    usd_per_m_output: 10.0
chairman: "gemini:gemini-2.5-pro"
stages:
  - name: answers
    kind: generate           # generate | peer_review | synthesize |
  - name: review             #   propose | pairwise | recommend
    kind: peer_review
  - name: synthesis
    kind: synthesize
    policy:                   # optional per-stage overrides
      timeout_s: 600
      max_tokens: 4096        # cap ballots/small stages so reservations don't starve members
budget:
  max_usd: 2.0

Member id = provider:model. Use four families for the diversity mitigation (e.g. an OpenRouter-routed OpenAI model, an Anthropic model, a Google model, and an open-weights model like Kimi). The chairman should be a strong synthesizer and ideally not the same family as the members it judges.

6. 6. Provider adapters

One protocol, per-provider shims:

class ModelAdapter(Protocol):
    id: str                                  # "provider:model"
    async def capabilities() -> Caps         # ctx window, structured_output?, tools?, $/1k
    async def preflight() -> Health          # cheap availability + auth check
    async def complete(req: TypedRequest) -> TypedResponse   # schema-enforced

API adapters: one HTTP core, per-provider auth/shape shims. Use native structured output where offered; JSON-repair fallback otherwise.
Schema dialects: pre-compile the JSON Schema into each provider's accepted dialect BEFORE the first call (providers reject different keyword sets: $defs, missing required, number/array constraints). A runtime repair fallback works but burns a paid attempt per member per stage.
Structured output: enforce at the tool-call layer; on mismatch the model retries. Repair a bare top-level array into the expected container when the element schema matches, before hard-failing.
A fake/offline adapter is essential for deterministic tests (fixed responses, no network, no spend).

7. 7. Components to build (checklist)

adapters/ — base protocol, one HTTP core, per-provider shims, a fake adapter, a registry that resolves provider:model ids, schema-dialect pre-compilation, structured-output parse+repair.
core/ — the engine (stage loop), the identity firewall (label↔model seal + anonymization + per-judge order randomization), typed budget/quorum/retry policies, the stage prompt builders (section 4).
aggregation/ — Borda from rankings; pairwise standings.
protocols/ — deliberation and decision state machines (thin config over the engine).
ledger/ — event store (append-only), content-addressed blobs, typed events.
inspection/ — snapshot (commit-pinned worktree), read-only executor, finding dedup, the implementation-council orchestrator.
eval/ — a harness that scores a council against fixtures with a chairman-solo baseline and member-ablation (does each member earn its seat?).
render/ — replay an event log to text; render a session report.
cli/ — run, preflight (check availability before spending), replay, report, eval, inspect.
Optional: an MCP server exposing run/status/result, and a durable handoff inbox + worker for async submit-then-poll.

8. 8. Setup and run

# Python 3.12+, uv
uv sync
cp .env.example .env          # add only the provider keys you will use
uv run junta preflight -c examples/council-basic.yaml   # check availability, no spend
uv run junta run -c examples/council-basic.yaml -q "your question" --budget 2.0
uv run junta replay <session-id>     # no provider calls
uv run junta report <session-id> --format md
uv run junta inspect --repo <path> --rev <sha> -c <config> --paths a.py,b.py --test-cmd "pytest -q"

.env holds provider keys (OPENROUTER_API_KEY, GEMINI_API_KEY, …) and is never committed. Model ids and pricing live in the YAML config.

9. 9. Evaluation (does the council beat one model?)

Ship the harness with the MVP. Baselines: chairman-solo, best-single-member. Ablations: council-minus-member (jury ablation), protocol swaps. Metrics: quality per benchmark plus the cost multiple vs a single model, plus reproducibility (replay divergence). A council that cannot prove it beats its own chairman running alone is just expensive overhead — the harness is how you find out.

10. Method / provenance

This guide is a clean-room description of the Junta design and prompts, prepared to share. It contains no session data, no API keys, no deployment- specific integrations, and no third-party or personal content — only the reusable architecture and the prompt set needed to rebuild the system.