AI & Automation

LLM Cost Control: How to Keep AI Running Costs Predictable in Production

November 11, 2026 · 4 min read · Etzal Group Team

An AI feature has a property that most software does not: every use costs money. Each request to a language model is billed by the amount of text sent and received. A pilot with ten users gives a pleasant invoice. The same feature with five hundred users and long conversations can produce a surprising one. Controlling LLM cost is ordinary engineering once you know where the money goes.

What you are billed for

Providers charge per token, which is roughly a word fragment. Two facts matter:

  • Input and output are priced separately, and output usually costs several times more per token.
  • Everything in the request counts as input: your instructions, the conversation so far, any documents you retrieved, and the description of every tool the model may call.

In a typical business assistant the user's question is a small fraction of the input. The rest is context you chose to send. That is where savings are found.

Measure cost per task

Start by logging, for every request: input tokens, output tokens, model, the feature that made the call, and the user or customer it belongs to. Then calculate the cost of one completed task, such as one answered enquiry or one processed invoice.

Cost per task is the number to manage. A request count tells you little, because one task may need six model calls and another only one. Once you have the figure, you can compare it with the value of the task and see which features pay for themselves.

Lever 1: send less

  • Trim the instructions. System prompts grow by accretion. Remove rules that no longer apply and examples that repeat each other.
  • Retrieve fewer, better passages. Five relevant passages beat twenty loosely related ones in both cost and answer quality.
  • Summarise long conversations. After a number of turns, replace the early history with a short summary.
  • Offer only the tools the task needs. Each tool definition is sent with every request. A reply-drafting step does not need the description of forty tools.

Lever 2: reuse what repeats

  • Prompt caching. Put the stable part of the prompt first: instructions, policies, tool definitions. Put the changing part last. Providers that support caching bill the repeated prefix at a reduced rate.
  • Answer caching. If many users ask the same question and the answer does not depend on who is asking, store it and serve it without calling the model.
  • Precompute. Summaries of documents, product descriptions and classifications can be produced once and stored.

Lever 3: use the right size of model

Not every step needs the most capable model. A common arrangement:

  • A small, fast model sorts incoming messages and handles simple, well-defined replies.
  • A larger model is called only for cases the small one marks as complex, or when the output will go to a customer without review.

Decide the routing with evidence. Run your evaluation set on both models. Where the smaller one matches the larger, move that step to it.

Lever 4: control the output

Output tokens are the expensive ones. Ask for the length you need: a classification label, a short reply, a structured object with named fields. Set a maximum output length for each feature. A model asked for "a brief answer" in the instructions still benefits from a hard limit.

Lever 5: set limits

  • Per-user and per-customer budgets. A daily or monthly allowance, with a clear message when it is reached.
  • Loop limits for agents. An agent that keeps calling tools without finishing should stop after a fixed number of steps.
  • Alerts. A notification when daily spend passes a threshold catches a runaway process on the same day, not at the end of the month.

Enforce these in code. An instruction that says "do not use too many steps" is not a limit.

Keep quality in view

Every saving should be checked against the evaluation set before it ships. Shorter prompts, smaller models and tighter retrieval can each reduce accuracy if applied carelessly. The aim is the lowest cost at which the feature still passes its tests, and the tests are what keep cost-cutting from quietly degrading the product.

A monthly review

Once a month, look at four figures: total spend, cost per task for each feature, the share of requests served from cache, and the share routed to each model. Small drifts are easy to correct when they are noticed early.

Cost design is part of how we build production AI systems in our Enterprise Software work, and an AI Consulting review can estimate running costs before you commit to a design. Contact us if a feature you already run is costing more than expected.

Summary

Measure cost per task. Send less, reuse what repeats, match the model to the difficulty, bound the output, and set hard limits. Check each change against the same quality tests, and review the figures every month.

Frequently asked questions

Why is our AI bill higher than the pilot suggested?

Pilots usually run with short conversations and few users. In production, conversation history, retrieved documents and tool definitions are sent with every request, so the input grows. Volume also rises as more staff adopt the tool. Measure cost per completed task, not per request.

Does using a cheaper model always reduce quality?

Not for every task. Classification, extraction from clear documents and short replies are often handled well by smaller models. Test with your own evaluation set. Send only the hard cases to the larger model and keep the rest on the smaller one.

What is prompt caching?

Some providers charge much less for the part of a prompt that is identical to a recent request. If your instructions and reference material stay the same and only the user's question changes, placing the stable part first lets it be billed at the cached rate.

Author

Etzal Group Team

Etzal Group Team, helping businesses grow with technology. Contact us.

Related service

WhatsApp AI Chatbot

We build, train, and run a WhatsApp AI agent that answers customers 24/7, qualifies leads, and follows up.