Answer · Updated 16 July 2026
AI Call Summaries: Turn Every Call into Notes, Decisions & Action Items
An AI call summary is a structured, machine-generated record of a phone conversation — a short narrative summary, the decisions reached, and the action items assigned — produced automatically from the call audio within seconds of hang-up and written onto the customer record in a CRM.
Two models produce it. An automatic speech recognition (ASR) model turns the call audio into a speaker-labelled transcript; a large language model reads that transcript and emits fields rather than prose — summary, decisions, action items, owner, next step, outcome. A connector writes those fields onto the CRM record.
The distinction that decides whether any of this saves time is the last step. A summary that lives in a transcription tool is a document someone still has to copy into the CRM. A summary written onto the customer record is a system of record. Only the second removes the work. Salesforce Research measured the size of that work: reps spend 28% of their week actually selling and 8.8% manually entering customer and sales information. The summary is not the product — the write-back is.
Held to that standard, Veera is only partly in the “yes” column today, and this page says so rather than leaving you to find out in a pilot. Smart Notes write the summary, the decisions, and the action items onto the contact record in Veera after every call — that is live. Pushing that summary into GoHighLevel or HubSpot automatically is built and activating, not usable today.
How AI call summarization works
The pipeline has three stages, and each one can be bought separately — which is why two tools that both advertise “AI call summaries” can behave nothing alike.
- 1.Transcription (ASR). The audio becomes text with speaker labels and word-level timestamps. Diarisation — deciding who spoke which words — is a separate problem from recognising the words, and it is the one that degrades first on a noisy mobile line or when two people talk over each other.
- 2.Structured extraction (LLM). A language model reads the transcript against a schema and returns typed fields, not an essay. The schema is what makes the output useful downstream: a free-text paragraph cannot be filtered, counted, or routed, whereas an
action_items[]array with an owner and a due date can drive a task queue. - 3.CRM write-back. A connector creates an activity on the contact record over the provider API. In HubSpot that is a call engagement created with
POST /crm/v3/objects/calls, carrying the summary in thehs_call_bodyproperty and associated to the contact, company, or deal it belongs to.
Stage 3 is where most products stop short, and it is the only stage the person who bought the tool actually feels. Evaluate vendors on it directly: after the call ends, does the summary appear on the contact record in your CRM without anyone typing? Ask us the same question and the honest answer is a qualified one — we answer it for ourselves below.
What belongs in a call summary
A summary that only narrates the call reproduces the transcript at lower fidelity. The fields below are the ones that survive contact with a pipeline review, because each answers a question somebody asks later.
| Field | The question it answers | Why it has to be structured |
|---|---|---|
| Summary | What happened on this call? | Read before the next call, so it must fit on one screen |
| Decisions | What did we agree to, and did they say yes? | The disputed field months later — needs a transcript anchor |
| Action items | Who does what, by when? | Only fields with an owner become tasks; prose never does |
| Next step | What moves this forward? | Drives the follow-up date on the record |
| Outcome / disposition | Where does this contact sit now? | An enum, not prose — it maps to a pipeline stage |
Decisions and action items are the two fields people actually reopen a record for, and they are the two that generic summarisation collapses into narration. “They discussed pricing” is narration. “Agreed to a 14-day pilot at the standard rate; Priya sends the scope doc by Friday” is a decision and an action item.
Why it matters: the 72% problem
Call summarisation exists because rep time is consumed by the paperwork that follows a conversation rather than by the conversation. Salesforce Research, State of Sales, 5th edition — a double-anonymous survey of 7,775 sales professionals across 38 countries, fielded 24 August to 30 September 2022 — broke down how a rep's average week divides. The 6th edition, fielded 8 March to 18 April 2024, puts the same split at 30% selling to 70% not and rounds the data-entry line to 9%, calling that selling figure “virtually unchanged”; the decimal-level breakdown quoted here is the 5th edition's:
- 28% of the week is selling. 72% is not. The non-selling majority is, in the report's words, “critical, but tedious tasks like deal management and data entry.”
- 8.8% goes to manually entering customer and sales information — nearly a third as much time as reps spend selling, spent retyping what they already know into a system that was supposed to know it.
- Only 37% of sales professionals strongly agree that their organisation fully utilises its CRM — and reps rank data entry automation the second most useful CRM feature, behind only process and workflow automation.
Those numbers explain a failure mode worth naming. Manual notes are not merely slow; they are written by the person with the strongest incentive to write them favourably, at the end of a day, from memory. The CRM then inherits whatever survived that filter. A deal-review meeting spent reconstructing what was said is a symptom of notes taken this way. Automatic summarisation fixes the coverage problem — every call gets a record, including the ones a rep would have skipped — before it fixes anything about quality.
Where the pipeline fails
AI call summaries are accurate enough to trust for recall and not accurate enough to trust blindly for commitments. The reason is structural: errors compound across the two stages. The language model treats the transcript as ground truth, so an ASR error does not surface as a garbled sentence — it surfaces as a confident, fluent summary of something nobody said.
The failure is measured, not hypothetical. Koenecke et al., Careless Whisper: Speech-to-Text Hallucination Harms(ACM Conference on Fairness, Accountability, and Transparency, FAccT ’24, June 2024; arXiv:2402.08021), found that roughly 1% of audio transcriptions contained entirely hallucinated phrases or sentences that did not exist in any form in the underlying audio, and that 38% of those hallucinations carried explicit harms such as perpetuating violence or inventing inaccurate associations. The study also found hallucination rates were higher for speakers with aphasia than for a control group — the error does not fall evenly across who is on the call.
Two practices follow directly. Read the action items before acting on a price, a date, or a contractual commitment. And keep the transcript addressable from the record, so any disputed line can be checked against what was actually said rather than against a paraphrase of it.
The law that applies before an AI summarises anything
A summary is downstream of a recording, and the recording is the regulated act. Four bodies of law bear on it, and they stack rather than substitute.
- Recording consent. Federal law (18 U.S.C. § 2511) permits recording with one party's consent, but roughly a dozen US states require all-party consent for confidential communications — including California (Cal. Penal Code § 632) and Illinois (720 ILCS 5/14-2). The stricter rule of the two parties' locations is the safe assumption.
- The transcript is personal data. Under the GDPR (Regulation (EU) 2016/679) a transcript needs a lawful basis, Article 5(1)(c) data minimisation argues against retaining raw audio you no longer need, and Article 17 gives the contact a right to erasure of it.
- An AI voice on the call is a robocall. FCC Declaratory Ruling 24-17 (CG Docket No. 23-362, adopted 2 February 2024, released 8 February 2024) held that the TCPA's existing restrictions on an “artificial or prerecorded voice” (47 U.S.C. § 227) already reach AI technologies that emulate a human voice — so an AI caller inherits the consent burden of a pre-recorded robocall. Calling hours are a separate rule, and that ruling does not address them: 47 CFR § 64.1200(c)(1) bars a telephone solicitationto a residential subscriber before 8 a.m. or after 9 p.m. local time at the called party's location, and § 64.1200(e) extends paragraph (c) to wireless numbers. That window is scoped to solicitation, so it does not by itself reach every AI call.
- Disclosure is arriving. From 2 August 2026, Article 50(1) of the EU AI Act (Regulation (EU) 2024/1689) requires that people be informed they are interacting with an AI system. The timing sits one paragraph down: Article 50(5) requires that the information be given “in a clear and distinguishable manner at the latest at the time of the first interaction or exposure.” Penalties under Article 99(4) reach €15,000,000 or 3% of total worldwide annual turnover, whichever is higher.
Consent is a call-design question, not a summarisation feature. No summarisation vendor can supply a lawful basis for the call that produced the audio.
How Veera writes summaries, and where they land today
Veera places the call and summarises it. Autonomous outbound calling is live and runs at production scale — 42 languages, 700+ voices, with mid-call instruction and supervisor takeover available while the conversation is running.
After every call, Smart Notes writes the summary, the decisions, and the action items onto the contact record in Veera. That much is live. The CRM connection is live too, and native to GoHighLevel and HubSpot: OAuth-connected, with two-way contact sync, pipeline-stage mapping, and reads of your deals and stages. Veera is not an alternative to your CRM and does not try to replace one. Your pipeline, reporting, and automations stay where your team already works; Veera adds the call and the record it produces. Veera is free to start.
The step this page argues matters most is the one we have not switched on. Pushing that summary into GoHighLevel or HubSpot is built and activating — not usable today. It is verified against provider fixtures, but it ships behind a flag that stays off until live sandbox parity is signed off, so today the summary lives on the Veera record and no automatic write-back reaches your CRM. Two details are worth knowing before that changes: the push is operator-initiated rather than a side effect of the call ending, and it is designed to land as a note or task on the contact rather than as the call engagement described above. We would rather you read that here than discover it in a pilot.
What is honest to say about the rest: SMS, WhatsApp, and email send channels, multi-step sequences, and the unified inbox are built and activating — not usable today. The single live send exception is in-call WhatsApp document delivery, which sends a brochure or quote during the conversation itself.
On the compliance surface above, three enforcement behaviours are implemented and verified rather than promised: quiet hours are enforced per call and timezone-aware, so a call outside the recipient's local window is not placed; opt-outs are suppressed before the next send, with one-click unsubscribe; and erasure is honoured when a contact asks to be forgotten. Those are specific mechanisms, not a certification: no tool can make an outreach programme lawful on its own, and disclosure duties depend on where you dial.
Frequently asked questions
What is an AI call summary?
An AI call summary is a structured, machine-generated record of a phone conversation: a short narrative summary, the decisions reached, and the action items assigned. It is produced automatically from the call audio within seconds of hang-up, without a human taking notes. A summary that stays in a transcript tool is a document; a summary written onto the customer record in a CRM is a system of record. The second is the useful one.
How does AI call summarization actually work?
Two models run in sequence. First, an automatic speech recognition (ASR) model converts the call audio into a speaker-labelled transcript with word-level timestamps. Second, a large language model reads that transcript and emits structured fields rather than free prose: summary, decisions, action items, owner, next step, and an outcome disposition. A connector then writes those fields into the CRM activity object. In HubSpot that is a call engagement created with POST /crm/v3/objects/calls, carrying the summary in the hs_call_body property.
Are AI call summaries accurate?
Accurate enough to trust for recall, not accurate enough to trust blindly for commitments. Errors compound across the two stages: the language model treats the transcript as ground truth, so an ASR error becomes a confident summary of something nobody said. Koenecke et al., "Careless Whisper: Speech-to-Text Hallucination Harms" (ACM FAccT 2024, arXiv:2402.08021) found roughly 1% of audio transcriptions contained entirely hallucinated phrases or sentences absent from the underlying audio, and that 38% of those hallucinations carried explicit harms. The practical rule: read the action items before acting on a price, a date, or a contractual commitment, and keep the transcript addressable so any disputed line can be checked against what was actually said.
Do AI call summaries write into my CRM, or do I have to copy them over?
It depends entirely on the tool, and this is the distinction worth evaluating on. Transcription-first tools produce a summary in their own interface and leave the CRM update to you, which reintroduces the manual data entry the summary was supposed to remove. Write-back tools push the summary into the CRM activity object over the provider API as a side effect of the call ending. Ask any vendor one question: after the call, does the summary appear on the contact record in my CRM without anyone typing? Held to that standard, Veera’s own answer is a qualified one. Live today: Smart Notes write the summary, decisions, and action items onto the contact record in Veera after every call, and Veera syncs into GoHighLevel and HubSpot over OAuth with two-way contact sync, pipeline-stage mapping, and reads of your deals and stages. Not live today: pushing that summary into either CRM is built and activating rather than usable — it is verified against provider fixtures but ships behind a flag that stays off until live sandbox parity is signed off, so the summary currently lives on the Veera record.
Do I need consent to record a call before an AI can summarize it?
Frequently, yes, and the answer depends on where both parties are sitting. Federal law (18 U.S.C. § 2511) permits recording with one party’s consent, but roughly a dozen US states require all-party consent for confidential communications, including California (Cal. Penal Code § 632) and Illinois (720 ILCS 5/14-2). Under the EU’s GDPR (Regulation (EU) 2016/679), a transcript is personal data, so it needs a lawful basis, and Article 5(1)(c) data minimisation argues against retaining raw audio you no longer need. Separately, from 2 August 2026 Article 50 of the EU AI Act (Regulation (EU) 2024/1689) requires that people be told when they are interacting with an AI system. Consent is a call-design question, not a summarization feature.
Does Veera replace my CRM?
No. Veera is not an alternative to a CRM and does not try to be one. It places the call, summarizes it, and syncs into the CRM you already run — GoHighLevel and HubSpot are natively supported over OAuth, with two-way contact sync, pipeline-stage mapping, and reads of your deals and stages. Writing the call summary itself back into either CRM is built and activating rather than usable today; for now that summary lands on the contact record in Veera. Your pipeline, reporting, and automations stay where your team already works. Veera is free to start.
This answer is published by Veera, an AI Business Aide that places the call and writes the summary, decisions, and action items onto the contact record in Veera. Veera syncs into GoHighLevel and HubSpot rather than replacing them; pushing the summary itself into either CRM is built and activating, not usable today. Statistics are cited to their original sources: Salesforce Research, State of Sales, 5th edition (fielded 24 August – 30 September 2022, 7,775 respondents across 38 countries), and Koenecke et al., “Careless Whisper: Speech-to-Text Hallucination Harms” (ACM FAccT ’24, arXiv:2402.08021). See also: What is an AI Business Aide? and State of Outbound Business AI 2026.