# AI Voice & Call Intelligence

**Every call, transcribed, attributed and searchable.**

- Category: Voice AI · Call operations
- Granularity: 412 utterances / call avg
- Runtime: Self-hosted
- Status: In production

Canonical: https://technologiesninja.com/project-voice-call-intelligence.html

## The problem

Thousands of hours of call audio sat in storage. It was auditable in theory and useless in practice, because nothing in it was searchable.

Quality assurance consisted of a supervisor listening to a small sample. Patterns across calls — a recurring objection, a competitor mentioned repeatedly, a promise made too often — were invisible.

Hosted transcription services were ruled out: the recordings contain customer details the client will not send to a third party.

## The hard parts

- Call audio is narrowband and noisy, with crosstalk and interruptions throughout
- Attribution matters: knowing who said a thing is often more important than the words
- Sentiment across a whole call is close to meaningless — it has to be per segment
- Volume means transcription has to be queued and batched, not synchronous
- Everything had to run on the client’s own hardware

## Architecture

1. **capture** — Asterisk writes call audio to storage the client controls
2. **queue** — Completed calls are queued for processing rather than transcribed inline
3. **transcribe** — WhisperX produces a timestamped transcript
4. **diarize** — Speech is separated by speaker, so each utterance has an owner
5. **analyse** — A local model extracts sentiment, emotion and intent per segment
6. **summarise** — A call-level summary and action list are generated from the segments
7. **deliver** — Structured output is written to the CRM and made searchable

**Stack:** WhisperX · Whisper · Ollama · Python · Laravel · Asterisk · Diarization · Redis

## What shipped

- Timestamped transcript for every call
- Speaker diarization so each utterance is attributed
- Per-segment sentiment, emotion and intent rather than one call-level score
- Call summary with extracted action items
- Full-text search across every call ever recorded
- Structured output pushed into the CRM
- Entirely self-hosted, no audio leaving the client’s infrastructure

## Result

**412** — Calls became data.

An average call now yields 412 separately transcribed and attributed utterances — the grain the analysis layer works at, and the reason per-segment sentiment is meaningful where a call-level score is not.

Quality review moved from sampling to searching. A supervisor can find every call where a specific objection or competitor came up.

All processing runs on the client’s own hardware, which is why the project was possible at all.

## Read next

- **Asterisk VoIP Platform** (Case study) → /project-asterisk-voip
- **AI Customer Support System** (Case study) → /project-ai-customer-support
- **VoIP & Voice Systems** (Service) → /service-voip-voice
- **Voice AI with Whisper, WhisperX and Ollama** (Article) → /blog-whisper-voice-stack

---
© 2026 TechnologiesNinja · Built and maintained in-house