Judge Haiku 5.5 by the cost of an accepted job

Anthropic launched Claude Haiku 5.5 on 7 October 2026, positioning it for high-volume tasks such as classification, summaries and narrower subagent work. It is also the first Haiku model with adjustable effort. That makes it a useful candidate for a controlled cost trial.

Choose one operational question for the trial: can this model perform a particular task to the required standard at a lower total cost? A broad instruction to replace the current model everywhere will make the answer harder to establish.

Select work with a clear acceptance rule

Good first candidates have bounded inputs, a stable output format and checks that can detect a meaningful share of mistakes. Examples include assigning approved categories to product descriptions or extracting a small set of fields from a standard document type.

Write down what counts as acceptance. A category must be valid and correct for the record. An extraction must preserve the source value and distinguish missing information from zero. A summary must retain the required exception. Valid JSON is only one part of those rules.

Keep ambiguous cases in the test set. If the easy examples dominate, a model can look economical while moving the difficult work into a hidden review queue. Include the languages, document lengths and unusual inputs that the live service will encounter.

Make effort part of the configuration

Anthropic's effort guide says Haiku 5.5 defaults to medium. It warns that low can skip searches or checks in longer agent prompts, and recommends using evaluations to justify higher settings. Record the selected effort with the model ID and prompt version.

Compare a small number of useful configurations. Start with the default and one plausible alternative, rather than an exhaustive grid with too few examples in each cell. Keep tool access and the acceptance standard consistent.

Inspect the actual work performed. A shorter response is welcome if the task is complete. A shorter response produced by omitting a required lookup is a failure even when it sounds confident. Count missing checks and premature stopping explicitly for tool-using tasks.

Leave adequate output capacity for the selected configuration. If the request repeatedly ends before producing the required fields, investigate the limit and task design before attributing every failure to model capability.

Count the complete cost

The Claude pricing documentation separates input, output and cache charges, and gives Haiku 5.5 different rates above and below a 100,000-token prompt boundary. Use the applicable dated price schedule and actual usage. A previous model's token count is not a reliable substitute for measuring the candidate.

A useful internal measure is:

cost per accepted job = total trial operating cost / accepted jobs

Define what is included. At minimum, count all model attempts, including retries and escalations. If tools carry charges, include them. Track human correction time separately or convert it using an agreed internal rate. Keep the API-only and fully loaded measures visible so people understand the comparison.

For an illustrative trial, a cheap first pass may create enough escalations that the complete route costs more than using the existing model once. It may still help if it removes a useful amount of routine work, but that benefit needs to survive the arithmetic. Do not report the first-pass token bill as the saving for the whole process.

Bound escalation before enabling it

Define the conditions that send a case elsewhere: invalid structure, missing source evidence, a prohibited task category or an application rule that cannot be satisfied. Do not rely entirely on a model's self-reported confidence.

Set a maximum number of automated attempts. Preserve the original input and each attempt's outcome so a reviewer can understand the route. Repeating the same request until one answer happens to pass a superficial check can conceal an unstable process.

Keep authority consistent across models. Escalating from Haiku to a larger model should not grant additional tools or bypass the existing approval and permission boundaries. Capability routing and access control answer different questions.

Make the release decision on held-out work

After tuning, run an untouched set of representative cases. Compare acceptance rate, serious error categories, escalation volume, completed-job latency and cost. Have a person inspect a sample of apparent passes as well as the failures.

If the result is useful, enable the new route for a limited task class and retain the previous configuration. Monitor whether the review queue grows as real inputs arrive. Expand only after the live workload supports the trial's conclusion, and keep the cheaper model out of categories where its exceptions remove the expected benefit.

References

• Anthropic, Introducing Claude Haiku 5.5, 7 October 2026. Accessed 17 October 2026.
• Anthropic, Effort. Accessed 17 October 2026.
• Anthropic, Claude Platform pricing. Accessed 17 October 2026.