Back to projects
Active

RightModel

Earn-as-you-route model selection: data taint, availability, capability, and cost cascade with a standing multi-model council and falsifiable outcome ledger.

PythonPydanticMulti-Agent SystemsLLM Routing
GitHub RepositoryRead Architecture & Manifesto

Dynamic model selection and trust management substrate for multi-agent fleets. Routes tasks through an objective safety-and-cost cascade, convenes multi-provider consensus councils, and updates an empirical outcome ledger. The trust and routing pillar of Stewardship Engineering.

The Problem

Sending every single prompt to a frontier model wastes quota and capital. Conversely, routing blindly to cheap models leads to silent failures, hallucinated code, and broken refactors.

Furthermore, static model benchmarks (like MMLU or Chatbot Arena) fail to predict whether a specific model can reliably handle specific tasks—such as updating database schemas or refactoring React hooks—inside your actual codebase.

The Routing Cascade

RightModel evaluates four gates sequentially:

[Candidate Models]
        │
        ▼
1. Taint Check        ──► Drops models unauthorized for the sensitivity tier (Public, Internal, Confidential)
        │
        ▼
2. Availability Check ──► Drops models experiencing provider outages, rate limits, or circuit trips
        │
        ▼
3. Capability Check   ──► Verifies empirical track record for the specific task class
        │
        ▼
4. Cost Minimization  ──► Selects the most cost-effective candidate surviving gates 1-3
        │
        ▼
[Chosen Execution Model]

The Standing Multi-Model Council

For high-stakes decisions—architectural forks, legal risk triage, or critical strategy—RightModel convenes a multi-provider council in parallel (spanning Anthropic, OpenAI, Google Gemini, and open weights).

A synthesis judge compiles consensus, flags divergences, and produces an actionable unified recommendation. Running disparate model families avoids the groupthink trap common in single-model multi-persona debate.

The Outcome Ledger & Anti-Goodhart Guard

Authority is not static. After an action settles, the result is appended to the outcome ledger:

  • Pass/fail rates are tracked per model and per task class.
  • Models earn the authority to run specific classes unattended only after establishing a statistically sound track record.
  • An anti-Goodhart guard analyzes predictions to detect and reject trivially easy bets designed to artificially inflate trust scores.

Open Source & Reading