AI Compass / Knowledge cards

Decision models

when a model should only choose

Checked 16 October 2026 · Michael Engel · Automation · Costs

A decision model receives a text and a question with fixed answer options. It returns one of those answers with a probability. One example is Jev from TypeSafe. A large language model (LLM) such as Claude, by contrast, writes free-form text and can operate tools.

What it is for

Many steps in agent workflows are pure decisions: Is this email spam? Which category fits? How urgent is it? A decision model handles these more cheaply and faster than a large language model. The large model is then left with text and tools only.

Steps

1. Find the places in the workflow where something is only being decided: yes/no, a choice from a list, or a score.
2. Define the question and the permitted answers before the model sees them.
3. Pass along only the text that is needed for the decision. Jev processes up to 32,000 tokens.
4. Set a confidence threshold, that is, the minimum certainty at which the answer counts. If the answer falls below it, the large model or a person decides. In one example, AI LABS uses 80%.
5. Writing text and calling tools stay with the large model. Decision models like Jev cannot do that.
6. Check samples regularly. Even a model that only decides can classify wrongly.

Limits

• Jev is proprietary. It runs through a dedicated decisions interface at OpenRouter, not through the chat endpoint. At OpenRouter that interface is called “alpha”.
• According to AI LABS (28 September 2026), TypeSafe is not accepting new sign-ups on its own platform. Access is available through OpenRouter or Vercel AI Gateway.
• The cost comparisons in the videos were measured by the creators themselves, with no prompts disclosed. List price at OpenRouter: $0.042 per million input tokens, output free of charge.
• Creator Magic sees this as protection against injected instructions (prompt injection), because the model cannot output text. That is the author’s claim and has not been verified. A decision model can also be pushed into a wrong classification by injected text.
• Confidence is not hit rate. Calibration has not been tested for yes/no answers. A fixed threshold assumes that the confidence is roughly right. So check samples (step 6).

Uses shown in the videos

• Rule hook: blocks a change when the decision model sees, with at least 80% certainty, a violation of the project rules (AI LABS, 28 September, 11:24-11:56).
• Review layer: For changes to access rules, the model asks whether every documented rule has a test (4:51-5:24).
• File search: first a keyword search, then the model ranks each batch of 20 hits by relevance instead of starting a search agent (7:35-8:40).
• Model routing: Based on a short description of each model, the decision model chooses which language model gets a task (Jack Roberts, 19 September, 9:40-10:47).
• Inbox: see Pre-sorting your inbox.

Before switching it on for real, count false alarms for a week (my recommendation).

Open alternative: a small local classification model, or a small language model such as Claude Haiku with fixed answer options. That takes more effort or costs more (compare list prices before choosing), but it ties you to no single interface.

Links

• Yes/no guards for Claude Code: ready-made hook tools that use a decision model as the checker.
• Checkpoints and gates: A decision model works as a cheap pre-check before the review.
• Model choice: cost per task: the right model for the right task.
• Web data: settle the access path first: Whether collected third-party text can be pre-filtered this way is open (see Limits).
• Pre-sorting your inbox: the worked-out pattern for email.
• Wake gate for long-running jobs: A decision model can answer the question “Has anything changed?”

Sources

• Jack Roberts: Jev AI Just Dropped, 19 September 2026, chapter 10:59
• Creator Magic: Jev Sorted 250 Emails for 1 Cent, 21 September 2026, chapters 03:41, 04:09-05:25. Description contains affiliate links
• AI LABS: Insane Jev Use Cases You Need To Use Right Now, 28 September 2026, chapters 02:19, 09:05, 11:15. Sponsored by Zapier
• OpenRouter: Jev 1.13 model page, published 18 September 2026, retrieved 2 October 2026
• OpenRouter: What Is Jev?, 21 September 2026
• AI LABS: This Just Fixed Your Claude Code Setup’s Biggest Flaw, 7 October 2026. Sponsored by Decodo

Status

Checked 2 October 2026, extended 9 October, 11 October and 16 October 2026 · Maturity new · Basis for the videos of 19-28 September: title, description and chapter markers first; the timestamps under “Uses shown in the videos” come from my review of the transcripts. Video of 7 October: transcript.

Want to try this with your team? That’s what the AI Lab is for. See the AI Lab