Guardrails, evaluation, and cost

A demo that works once is easy. An agent you can leave running on real customers is a different thing, and the gap between them is guardrails and evaluation. This is the part most people skip and most projects fail on.

Know the failure modes

• Hallucination: confidently states something untrue. Mitigate with RAG and "answer only from the provided facts".
• Wrong tool or wrong arguments: calls the refund tool when asked a question. Mitigate with sharp tool descriptions and validation.
• Loops: keeps acting without converging. Mitigate with a step limit.
• Prompt injection: a user (or a retrieved document) tries to override your instructions. No complete fix exists today, so never treat input as instructions and gate every real action.

Guardrails

Guardrails are checks around the model, not pleas inside the prompt. Validate the model's output before acting on it. Keep an allow-list of actions it may take autonomously, and route the rest to a human. For anything touching money or irreversible change, require explicit confirmation. The model proposes, your system disposes.

Evaluation: prove it works

You cannot improve what you do not measure, and "it felt good in testing" is not measurement. Build a small evaluation set: 20 to 50 real questions with the answer or action you expect. Run the agent against it whenever you change the prompt or tools, and track how many it gets right. This catches regressions, the silent kind where a prompt tweak fixes one case and breaks three others.

# A tiny eval set (input -> expected)
"jam buka minggu?"            -> mentions 07:00-22:00
"harga latte?"               -> 32000, no invented number
"refund pesanan kemarin"     -> hands off to human (no refund tool call)
"pesan 50 box, kapan siap?"  -> asks for contact, flags as bulk lead

Score after each change. A drop means your last edit hurt something.

Cost and latency

You pay per token, input and output, and bigger models cost more and respond slower. Practical levers: use a smaller, faster model for easy steps and a stronger one only where it is needed, keep prompts lean, cache answers to repeat questions, and set a token budget per request. Measure the real cost per conversation early, it is usually small, but you want to know the number rather than guess.