Every project of this shape starts the same way: a business is answering the same four questions all day, the answers already exist in their systems, and a chatbot trial has already failed. Understanding why the trial failed is the useful part.

It failed because a language model with no access to your data is a very fluent guesser. Asked whether an item is in stock, it produces a plausible sentence. Asked what shipping costs, it produces a plausible number. Plausible numbers are worse than no answer.

Retrieval is where the work is

Most of the engineering effort in a support agent goes into retrieval, not prompting. The catalog is the hard part. In one build, the same physical dimension appeared four ways across the product data — as a fraction, a decimal, a metric conversion, and a trade abbreviation. A customer asking for a half-inch fitting matched none of them.

So normalisation happens before embedding, not after. Units are standardised, part numbers are indexed with and without separators, and trade abbreviations are expanded from a synonym list built out of real search logs. The vector store then holds content a human would recognise as consistent.

  • Normalise units, dimensions and part-number formats on ingest
  • Chunk on semantic boundaries — a product, a policy, a resolved ticket — not fixed token counts
  • Store structured metadata alongside each chunk so retrieval can filter by category or stock
  • Re-embed on change, automatically, or the index quietly drifts out of date

If retrieval returns the wrong three documents, no amount of prompt engineering rescues the answer. Measure retrieval separately from generation.

Tool calling, tightly constrained

An agent becomes useful when it can act: look up an order, check stock, draft a quotation. It becomes dangerous at exactly the same moment. The rule we apply is that the model chooses which tool to call and with what arguments — and nothing else.

// the model may request this; it may not compute it
{
  "tool": "quote.draft",
  "args": { "sku": "BF-050-BRS", "qty": 12 }
}

// tiered pricing, minimums and totals are resolved in Laravel
// the model receives the finished quote as data, not as a suggestion

Every tool is a validated endpoint in the application. Pricing tiers, minimum quantities, weight bands and shipping rules are computed in code that can be tested. The model never performs arithmetic, because a model that is right about arithmetic 99% of the time is unusable when money is involved.

Authorisation lives in the same layer. An order lookup requires a verified customer context, so an agent cannot be talked into showing one customer another customer’s history.

Designing the escalation path first

The temptation is to measure an agent on resolution rate. That incentive produces a system that answers everything, including the things it should not. We measure the handover instead: when the agent escalates, does the human receive the conversation, the retrieved context, and the reason for escalation?

Escalation triggers are explicit. No supporting content retrieved, a tool call that failed, a request outside the supported set, or any signal of an unhappy customer. All four hand over immediately.

Why self-hosted, and when not

Two reasons come up repeatedly. The first is legal: a team that has already refused to send customer records to a third-party API will approve the same feature if inference happens inside their own network. The second is cost, which only bites once the system is genuinely used — per-token pricing scales with success.

On Ollama, DeepSeek and Qwen both handle this workload well. Model choice matters far less than people expect once retrieval and tools are solid.

The honest counter-argument: if your volume is low, your data is not sensitive, and you need the strongest possible reasoning, a hosted frontier model is the better engineering decision. We have recommended that. What matters is making the choice deliberately rather than by default.

What we would tell you before starting

  • Budget most of the timeline for data normalisation and retrieval evaluation
  • Write the tool schema before writing any prompt
  • Log every retrieval, tool call and answer — you cannot improve what you cannot review
  • Ship to internal users first; they forgive the failure modes customers will not

Done this way, the result is unglamorous and effective: in one production system, median ticket resolution time fell 72%, and no customer data left the client’s infrastructure.