Every call, transcribed, attributed and searchable.
The client recorded every call and read almost none of them. The information customers gave on the phone never reached the CRM, and quality review meant listening to a handful of calls a week.
The problem
Recordings nobody could read.
Thousands of hours of call audio sat in storage. It was auditable in theory and useless in practice, because nothing in it was searchable.
Quality assurance consisted of a supervisor listening to a small sample. Patterns across calls — a recurring objection, a competitor mentioned repeatedly, a promise made too often — were invisible.
Hosted transcription services were ruled out: the recordings contain customer details the client will not send to a third party.
The hard parts
Telephony audio is difficult.
- Call audio is narrowband and noisy, with crosstalk and interruptions throughout
- Attribution matters: knowing who said a thing is often more important than the words
- Sentiment across a whole call is close to meaningless — it has to be per segment
- Volume means transcription has to be queued and batched, not synchronous
- Everything had to run on the client’s own hardware
Architecture
Audio to structured record.
- 01captureAsterisk writes call audio to storage the client controls
- 02queueCompleted calls are queued for processing rather than transcribed inline
- 03transcribeWhisperX produces a timestamped transcript
- 04diarizeSpeech is separated by speaker, so each utterance has an owner
- 05analyseA local model extracts sentiment, emotion and intent per segment
- 06summariseA call-level summary and action list are generated from the segments
- 07deliverStructured output is written to the CRM and made searchable
Stack
- WhisperX
- Whisper
- Ollama
- Python
- Laravel
- Asterisk
- Diarization
- Redis
What shipped
What the system produces.
- Timestamped transcript for every call
- Speaker diarization so each utterance is attributed
- Per-segment sentiment, emotion and intent rather than one call-level score
- Call summary with extracted action items
- Full-text search across every call ever recorded
- Structured output pushed into the CRM
- Entirely self-hosted, no audio leaving the client’s infrastructure
Result
412
Attributed utterances per call, on average
Calls became data.
An average call now yields 412 separately transcribed and attributed utterances — the grain the analysis layer works at, and the reason per-segment sentiment is meaningful where a call-level score is not.
Quality review moved from sampling to searching. A supervisor can find every call where a specific objection or competitor came up.
All processing runs on the client’s own hardware, which is why the project was possible at all.
Related
Read next.
Next step