How to Extract Action Items from Meeting Transcripts (Without Uploading Your Audio Anywhere)

Published July 11, 2026

Published July 11, 2026 · 10 min read

The most common complaint about AI notetakers in 2026 is not that they miss words — it's that they stop at transcription. You end a call with a 6,000-word verbatim transcript and no answer to the only question that matters: who owes what by when? This guide walks through a complete, fully on-device workflow for turning raw meeting audio into a structured list of action items — owners, deadlines, decisions — without a single byte leaving your Mac or iPhone. It targets Apple Silicon users on iOS 26 / macOS 26 and relies on Apple's SpeechAnalyzer for transcription and the Foundation Models framework for structured extraction.

Why this matters: the average knowledge worker now spends roughly 11.3 hours per week in meetings, plus another 3–4 hours in adjacent overhead like preparation, follow-up, and recap notes. Microsoft's Work Trend Index goes higher, putting it at 15.4 hours per week in meetings against only 12.1 hours of uninterrupted focus work. That inverted ratio is the problem action-item extraction is supposed to solve — and cloud notetakers are solving it by trading privacy for convenience. There is now a better way.

The problem: transcript ≠ notes

A transcript is a firehose. A useful meeting note is a filter. Turning one into the other is a language-model problem, and until roughly late 2025 it required paying OpenAI or Anthropic for tokens. That trade-off dragged sensitive audio through third-party servers whose privacy policies were written for the vendor, not you.

Two things changed in 2026. First, Apple's on-device SpeechAnalyzer framework matured enough that transcription accuracy on M-series chips rivals cloud services with no audio ever leaving the phone. Second, Apple shipped the Foundation Models framework, which exposes a ~3 billion parameter on-device language model to any third-party Swift app. That model, according to Apple's machine learning research team, excels at exactly the tasks this workflow needs: summarization, entity extraction, text understanding, and refinement.

The end-to-end on-device pipeline

Here is the whole pipeline at a glance. Every stage runs on the user's device — no API keys, no cloud, no BAA required:

1. Capture system + microphone audio locally on Mac or iPhone.
2. Transcribe with SpeechAnalyzer (or Whisper via MLX for legacy devices).
3. Chunk the transcript into ~1,500-token segments.
4. Summarize each chunk with Apple's Foundation Model.
5. Extract action items as a typed Swift array using guided generation.
6. Write results into Apple Notes / Reminders / Calendar.

Each stage below has a specific technical primitive attached. Vague "on-device" language loses to competitors who quote APIs; here are the actual APIs.

Stage 1: Capture — no bot, no browser extension

The first architectural decision is whether to send a bot into your Zoom, Teams, or Meet call, or to capture audio directly from the device. Bots are the reason Otter and Fireflies have to upload — a bot is a participant, and its audio stream lives in someone else's cloud from the moment it joins.

Bot-free capture reads the system audio bus locally. Tools like TranscribeNext and muesli capture system audio directly on macOS with no participant indicator on the call and no server round-trip. On iPhone, Basil AI records via the on-device microphone for in-person meetings, phone calls (where legally consented), and hybrid setups.

For a broader treatment of why bot-free capture is the privacy-critical choice — including Google's March 2026 clampdown on third-party notetaker bots in Meet — see our bot-free vs on-device AI notetaker breakdown.

Stage 2: Transcribe with SpeechAnalyzer (or Whisper via MLX)

Apple's SpeechAnalyzer, gated on iOS 26+ and macOS 26+, is now the recommended path for on-device speech-to-text on Apple Silicon. For pre-iOS-26 devices, or for languages Apple's model doesn't cover well, a Whisper variant compiled for MLX runs on Metal-accelerated GPUs. The trnscrb macOS tool is one open example: it detects meetings from Zoom, Google Meet, Microsoft Teams, Slack, and FaceTime and runs Whisper via Apple Silicon Metal acceleration, ensuring that no audio or transcript data is sent to the cloud.

Two implementation notes matter here. First, request on-device recognition explicitly (in Apple's older SFSpeechRecognizer API this was requiresOnDeviceRecognition; in SpeechAnalyzer the equivalent is exposed through the analyzer configuration). Second, save the raw audio locally alongside the transcript so users can tap a line and hear the exact moment it was said, as Speechy documents in its transcript player. Playback recovery of source audio is what makes transcript errors survivable.

Stage 3: Chunk before you summarize

Apple's on-device model is optimized for what one developer guide describes as "mid-level text logic" — classifying short text, extracting structured information, rewriting fragments, and summarization. It is not a frontier reasoner. In practice this means two things: don't paste a 50KB transcript in one shot, and don't ask it multi-hop reasoning questions.

The Foundation Models context window is finite and degrades with length; the working recipe is to chunk long transcripts, summarize each chunk, then summarize the summaries. In typical dialogs the context window holds up for 20–30 message exchanges before hitting contextWindowExceeded. Design accordingly.

A working chunk size

Empirically, 1,200–1,800 tokens per chunk (~8–12 minutes of conversation) gives the model enough context to identify commitments without overflowing. Overlap chunks by ~100 tokens so an action item that straddles a boundary isn't cut in half.

Stage 4: Guided generation — the trick that makes this reliable

The unlock for reliable action-item extraction is Apple's guided generation, introduced with the framework in 2025. Instead of asking the model for JSON and hoping, you annotate a Swift struct with @Generable and the framework constrains the model's sampling so it can only produce output matching your type.

As Apple's ML research team describes it, guided generation is "an intuitive Swift approach to constrained decoding" enabled by "vertical integration with the model, the operating system, and the Swift programming language." In WWDC 2025's introductory session, Apple showed how the @Generable macro enables description of types for model-generated instances, which can be constructed from primitives, arrays, and composed types.

A minimal action-item schema in Swift looks like:

import FoundationModels

@Generable
struct ActionItem {
  @Guide(description: "The person committed to doing the task")
  let owner: String

  @Guide(description: "Concise task description, imperative voice")
  let task: String

  @Guide(description: "Deadline mentioned in the meeting, ISO date or null")
  let deadline: String?

  @Guide(description: "Priority 1 (low) to 5 (critical)")
  let priority: Int
}

@Generable
struct MeetingActions {
  let actionItems: [ActionItem]
  let decisions: [String]
  let openQuestions: [String]
}

Because Apple's decoder is constrained by these types, the model cannot return malformed JSON. As a developer guide from Stora puts it, "the type system is a free prompt." Instead of writing three paragraphs of prompt engineering, you shape output by shaping types.

Stage 5: The macOS 26 fm CLI — shell-scripting transcripts

If you don't want to ship a Swift app just to process transcripts you already have on disk, macOS 26 (WWDC 2026's release) added a command-line entry point to Apple's on-device model. As Apple detailed in its WWDC 2026 session on the Foundation Models framework, the new fm CLI "gives on-device and PCC access for everyday productivity" — you can pipe transcripts into fm respond to summarize, extract, or generate content.

A working batch script, adapted from a local-AI walkthrough of the fm command:

# 1. Define a schema for the output
fm schema object --name ActionItems --string items --array > schema.json

# 2. Extract action items from a transcript, constrained by the schema
fm respond "Extract the action items from this meeting: \
  $(cat meeting.txt)" --schema schema.json | jq -r '.items[]'

Three local engines — SpeechAnalyzer or MLX Whisper for audio-to-text, fm for summarization, and MLX for larger open models — chain into a single script with, as the tutorial puts it, "zero cloud calls." That is a real architectural shift from where AI note-taking sat a year ago.

Cloud notetaker vs on-device pipeline: the trade-off table

The vendor policies underpinning the left three columns are worth reading first-hand: Otter's privacy policy, Fireflies' privacy policy, and Zoom's privacy policy each grant broad rights to store and process recordings server-side.

How Basil AI solves this

Basil AI is the fully on-device implementation of the pipeline above. Audio is captured on iPhone or Mac; transcription runs through Apple's on-device speech APIs; summarization and action-item extraction run through Apple's Foundation Models. Nothing is uploaded to Basil, OpenAI, Anthropic, or Google. The transcript, the audio, and the extracted action items are written into Apple Notes via your own iCloud account — the same iCloud you already use for everything else.

For lawyers, this architecture matters because the tool is not a "third-party vendor" in the sense state bars care about — there is no vendor receiving the audio. For a fuller treatment, see our privilege-waiver buyer guide for AI notetakers. For healthcare workers under HIPAA, the same argument holds: no BAA is required for a processor that never exists. And for consultants running back-to-back client calls where each conversation belongs to a different NDA, on-device processing lets you use one tool across every meeting without cross-contamination — a workflow we walked through in the consultants' back-to-back calls playbook.

Regulatory scenarios: matching the workflow to your rules

The pipeline above is architecturally suited to a range of regulatory contexts. Concretely:

HIPAA (US healthcare)

Protected Health Information (PHI) discussed in a clinical consult must be handled under 45 CFR §164. A cloud notetaker vendor becomes a Business Associate under HHS's covered entity and BA rules, requiring a BAA and vendor risk assessment. An on-device pipeline where no vendor sees the audio is architecturally outside the BA relationship for the model itself.

GDPR (EU)

Article 5(1)(c) of the GDPR requires data minimization — personal data must be "adequate, relevant and limited to what is necessary." See the text at Article 5 of the GDPR. Uploading a 60-minute meeting to extract a 200-word summary fails the minimization test if a local model can produce the same output. The on-device pipeline is the technically appropriate design.

Two-party consent states (US)

California, Florida, Illinois, Massachusetts, Maryland, Michigan, Montana, Nevada, New Hampshire, Pennsylvania, and Washington require consent from all parties to record a private conversation. On-device or cloud, you still need consent — but keeping the transcript local reduces third-party subpoena exposure once the recording is lawful.

Practical implementation checklist

If you're building this workflow yourself, or evaluating a tool that claims to offer it, verify:

• Speech recognition is set to on-device — SpeechAnalyzer configured for local processing, or SFSpeechRecognizer with requiresOnDeviceRecognition = true.
• Summarization runs through import FoundationModels and SystemLanguageModel, not a HTTPS call to api.openai.com.
• Output types are annotated with @Generable and @Guide, not parsed from freeform strings.
• Transcripts are stored in the app's sandbox (or Apple Notes via iCloud), not synced to a vendor bucket.
• Availability is device-gated: as one implementation guide from Blake Crosley on the FoundationModels tool protocol notes, not every iOS 26 device has Apple Intelligence, so SystemLanguageModel.default.availability should be checked at runtime.

What Apple's on-device model does poorly (and how to work around it)

The ~3B on-device model is not GPT-5. Where it falls down, per the same iOS 26 developer guide: math (use standard code, never the model), long multi-step reasoning (break into short prompts, feeding each result as context to the next), and factual questions about recent world events ("the model will hallucinate with absolute confidence").

For meeting-notes specifically, this maps to three concrete workarounds. If a speaker says "we need this by end of Q3," don't ask the model to compute the calendar date — extract the phrase and let a Swift date parser handle it. If the meeting hops across five topics, run one extraction pass per topic segment, not one pass over the whole transcript. And if there's ambiguity about who owns an item, keep the raw sentence in the output so the user can adjudicate.

Where this fits in the broader on-device AI shift

Apple's third-generation foundation models, shipped on June 8, 2026, include a 20-billion-parameter sparse on-device model that activates only 1–4B parameters per prompt via a technique Apple Research calls Instruction-Following Pruning. That model, and the Foundation Models framework's new image-input capability, mean the ceiling on what can run entirely on your phone keeps rising. Structured extraction from a meeting transcript is well below that ceiling — this is now boring, everyday territory for on-device inference.

That's the punchline. "Extract action items from meeting transcripts without uploading audio anywhere" is no longer a privacy compromise or a technical stretch. As of macOS 26 and iOS 26, it's the shortest architectural path — fewer moving parts, no API keys, no per-token bills, and no vendor privacy policy standing between you and your own words.

Getting started with Basil AI

If you'd rather not build the pipeline yourself, Basil AI ships it out of the box: bot-free capture on iPhone and Mac, on-device transcription, on-device summarization and action-item extraction, and direct write-through to Apple Notes. Meetings up to 8 hours are supported for full-day workshops and depositions. Nothing is uploaded, ever.

Try Basil AI — Privacy-First AI Notetaker

8-hour continuous recording. 100% on-device transcription and action-item extraction. Nothing ever uploaded.