Local AI for Therapists: Private SOAP Note Drafting

Published April 23, 2026 · Updated August 23, 2026 · 15 min read

You can draft SOAP, DAP and BIRP notes from your own session audio without any of it leaving your laptop: whisper.cpp transcribes locally, Ollama runs the model locally, and the draft lands in a Markdown file you edit and sign in your own EHR. The stack is free, runs on a 16GB machine, and removes the third-party disclosure problem that every cloud AI scribe creates. What it does not remove is your obligation to get informed consent, review every line, and write the risk sections yourself.

Most therapy software vendors will swear their cloud is secure. The problem is not whether their cloud is secure - the problem is that any time a session transcript leaves your machine, your client's protected health information is one breach, one subpoena, or one leaky integration away from disclosure. The HHS Office for Civil Rights breach portal logged 725 healthcare breaches of 500+ records in 2023, affecting more than 133 million people. If you are a licensed therapist, social worker, or LPC, that is not an abstract risk. That is your license, your insurance, and your clients' trust.

Quick Start: The 4-Step Therapist Stack

1. Install Ollama for the LLM runtime: curl -fsSL https://ollama.com/install.sh | sh
2. Pull an open model that handles clinical vocabulary: ollama pull llama3.1:8b-instruct-q5_K_M
3. Install whisper.cpp for offline transcription: brew install whisper-cpp (Mac) or build from source
4. Wire a 30-line shell script that turns audio into a draft progress note in one command

Total cost: $0 in subscription fees. Disk budget: roughly 5.7GB for the Llama 3.1 8B Q5_K_M weights plus about 1.5GB for the Whisper medium.en model, so call it 8GB with the tooling around them.

---

Table of Contents

1. Why Cloud Therapy AI is a HIPAA Liability
2. System Requirements & Hardware
3. Step-by-Step Local Stack Setup
4. Choosing a Model, and How to Evaluate It
5. SOAP, DAP, and BIRP Prompt Templates
6. Whisper Transcription Pipeline
7. End-to-End Session Note Workflow
8. Clinical Safeguards & Limits
9. Common Pitfalls
10. FAQ

---

Why Cloud Therapy AI is a HIPAA Liability

Most "AI scribes" pitched to therapists send your audio to OpenAI, Anthropic, or a thin SaaS wrapper that uses one of them under the hood. Even with a Business Associate Agreement, you are trusting:

• That the vendor's logging is actually disabled (the OpenAI BAA covers Azure OpenAI, not the consumer ChatGPT API)
• That model providers do not retain your prompts for "abuse monitoring" beyond what is disclosed
• That a future subpoena cannot compel disclosure of session transcripts stored on infrastructure you do not control
• That the chain of subprocessors (CDN, observability, payment) does not silently change

A local model has none of those failure modes. Your audio never leaves the machine. Your transcript is encrypted at rest by FileVault or BitLocker. The "vendor" is your own SSD.

The American Psychological Association's HIPAA Privacy Rule overview spells out the duty: minimum necessary disclosure, with a documented data flow. A self-hosted pipeline gives you a one-line data flow: "Audio recorded on encrypted laptop. Transcribed locally. Notes drafted locally. Original audio deleted within 24 hours."

---

System Requirements & Hardware

Minimum (works, slow)

Recommended (fast, comfortable)

Why those numbers

The memory floor is arithmetic, not opinion. At Q5_K_M a model costs roughly 0.7GB per billion parameters, so Llama 3.1 8B is about 5.7GB of weights; Whisper medium.en adds about 1.5GB while it is loaded. Both fit alongside an operating system in 16GB, which is why 16GB is the floor and 32GB is comfortable if you want to keep both models resident between sessions rather than loading each time.

Speed depends entirely on your machine's memory bandwidth, and we do not publish first-party timings for hardware we do not own. The useful upper bound is memory bandwidth (GB/s) ÷ model size (GB) — on a machine with 100GB/s of bandwidth that is under 18 tokens/sec for a 5.7GB model, before any of the overhead the formula ignores. Time one session on your own machine before you redesign your charting workflow around it.

---

Step-by-Step Local Stack Setup

1. Install Ollama

# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh

# Verify
ollama --version
# ollama version is 0.5.x

# Start the server (runs as background service on Mac)
ollama serve

On Windows, download the installer from ollama.com - it installs as a tray app and exposes the same localhost:11434 endpoint.

2. Pull the Right Model

For a 16GB machine, Llama 3.1 8B Instruct at Q5_K_M is the default worth starting from: it is the largest general instruct model that comfortably fits, and Q5 keeps more precision than the Q4 default for a task where a wrong word changes clinical meaning. Whether it handles your vocabulary and note style is something you have to check on your own transcripts - see the evaluation protocol below.

ollama pull llama3.1:8b-instruct-q5_K_M
# ~5.7GB of weights (0.7GB per billion parameters at Q5_K_M)

If you have 32GB RAM and an Apple Silicon Pro/Max chip, upgrade to:

ollama pull qwen2.5:14b-instruct-q5_K_M
# Better for complex case formulations

3. Install whisper.cpp for Offline Transcription

# macOS
brew install whisper-cpp

# Or build from source for Metal/CUDA acceleration
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
# Mac with Metal
WHISPER_METAL=1 make -j

# NVIDIA Linux
WHISPER_CUBLAS=1 make -j

# Download the medium English-only model (best speed/accuracy for clinical English)
bash ./models/download-ggml-model.sh medium.en

The medium.en model file is about 1.5GB. Transcription speed varies enormously with the build: whisper.cpp compiled with Metal or CUBLAS is dramatically faster than the default CPU build, which is the single biggest speed lever you control here. Compile with acceleration, then time one real session on your own machine.

4. Lock Down the Filesystem

# macOS - confirm FileVault is on
fdesetup status

# Create a dedicated, encrypted-by-volume folder
mkdir -p ~/Documents/PrivatePractice/{audio,transcripts,notes}
chmod 700 ~/Documents/PrivatePractice

On Windows, enable BitLocker on the drive that holds these folders. On Linux, use LUKS. This is non-negotiable - HIPAA's Security Rule explicitly addresses encryption at rest as an addressable specification, and unencrypted laptop loss is one of the most common breach categories on the HHS Wall of Shame.

---

Choosing a Model, and How to Evaluate It Yourself

There is no published benchmark for "quality of a psychotherapy progress note", and anyone who shows you a leaderboard for it has invented the scoring. What exists is a memory budget and a short list of candidates that fit inside it:

(Weights column is the 0.7GB-per-billion-parameters constant for Q5_K_M, not a download you have to take on trust — check it against the file size after you pull.)

Because this is a clinical document, run your own evaluation rather than trusting anyone's ranking, including this page's. A protocol that takes an afternoon:

1. Build a test set from synthetic transcripts, not client audio. Write or role-play ten sessions covering the presentations you actually see. Never use real PHI to benchmark a model.
2. Run the same prompt through each candidate, saving every output.
3. Score three things per note: did it capture affect, content, intervention and plan; did it invent anything that was not in the transcript; does it read like a clinician rather than a chatbot.
4. Count invented content specifically. A fluent note with one fabricated symptom is worse than a clumsy accurate one. This is the number that should decide your choice.
5. Re-run it when you change models or quantization. A Q4 build of the same model is a different model for this purpose.

Start with Llama 3.1 8B at Q5_K_M on a 16GB machine, and step up to a 14B model if you have 32GB and your own scoring says the bigger model earns it.

---

SOAP, DAP, and BIRP Prompt Templates

The model is only as good as the prompt. The templates below are built around one constraint that matters more than wording: they forbid the model from adding anything the transcript does not contain. Save them as text files and pipe them into Ollama.

SOAP Note Prompt

You are a clinical scribe assisting a licensed psychotherapist. Draft a SOAP note from the session transcript below. Use only information present in the transcript - do not invent symptoms, history, or diagnoses. If something is unclear, write [unclear in transcript].

Format:
S (Subjective): client's reported experience, mood, presenting concerns
O (Objective): observable affect, behavior, MSE elements (appearance, speech, thought process)
A (Assessment): clinical impression, treatment progress, risk factors. No new diagnoses.
P (Plan): interventions used, homework assigned, next session focus, referrals

Keep each section under 80 words. Use third person. No client name.

TRANSCRIPT:
{transcript}

DAP Note Prompt

Draft a DAP note from the transcript. Sections:
D (Data): what occurred, client report, observations
A (Assessment): clinical interpretation grounded in the transcript
P (Plan): next steps, homework, follow-up

Tone: professional, neutral. No interpretation beyond what was discussed.

TRANSCRIPT:
{transcript}

BIRP Note Prompt (used in CMHCs and many state Medicaid systems)

Draft a BIRP note. Sections:
B (Behavior): client's presenting behaviors and statements
I (Intervention): interventions used by clinician
R (Response): client's response to intervention
P (Plan): plan for next session

Use measurable, observable language. Avoid jargon.

TRANSCRIPT:
{transcript}

---

Whisper Transcription Pipeline

Here is the actual command chain. Save audio as 16kHz mono WAV (most recorders can export this directly, or use ffmpeg).

# 1. Convert any audio to whisper-friendly format
ffmpeg -i session_2026-04-23.m4a -ar 16000 -ac 1 -c:a pcm_s16le session.wav

# 2. Transcribe locally
whisper-cpp -m ~/whisper.cpp/models/ggml-medium.en.bin \
  -f session.wav \
  -otxt \
  -of session_transcript

# Output: session_transcript.txt (plain text, no timestamps)

At ordinary conversational speech rates - call it 130-180 words per minute across both speakers - a 50-minute session works out to roughly 6,000-9,000 words of transcript. That is the number to keep in mind when you set the model's context window: 9,000 words is on the order of 12,000 tokens, so an 8K context window will silently truncate a full session.

---

End-to-End Session Note Workflow

Save this as ~/bin/note.sh and chmod +x it. Run note.sh session.wav after each session.

#!/usr/bin/env bash
set -euo pipefail

AUDIO="$1"
STAMP=$(date +%Y-%m-%d_%H%M)
WORKDIR=~/Documents/PrivatePractice
TRANSCRIPT="$WORKDIR/transcripts/$STAMP.txt"
NOTE="$WORKDIR/notes/$STAMP-soap.md"
PROMPT="$WORKDIR/prompts/soap.txt"

# 1. Normalize audio
ffmpeg -y -i "$AUDIO" -ar 16000 -ac 1 -c:a pcm_s16le /tmp/_session.wav

# 2. Transcribe
whisper-cpp -m ~/whisper.cpp/models/ggml-medium.en.bin \
  -f /tmp/_session.wav -otxt -of "${TRANSCRIPT%.txt}"

# 3. Build prompt and send to local LLM
{
  cat "$PROMPT"
  echo "TRANSCRIPT:"
  cat "$TRANSCRIPT"
} | ollama run llama3.1:8b-instruct-q5_K_M > "$NOTE"

# 4. Securely shred the temp WAV
rm -P /tmp/_session.wav

echo "Draft note saved to $NOTE"
echo "Review and edit before signing in your EHR."

Output is a draft, not a final note. You always review, correct, and sign in your actual EHR. Time the script on your own hardware the first week; whether it saves you anything depends on how fast your machine is and how much you end up rewriting.

If you want to layer this with a private RAG system over your treatment plans and prior notes, see our private AI knowledge base walkthrough - it pairs naturally with this stack.

---

Clinical Safeguards & Limits

A local LLM is a drafting tool, not a clinician. Some non-negotiables:

• You are still the author. The note is your professional record. Read every line.
• Do not let the model assess risk. SI/HI determination, Tarasoff judgments, and CPS reporting are clinical decisions, not generative ones. Strip risk language from the model output and write that section yourself.
• Do not use the model for diagnosis. It will happily invent V-codes. Use it for narrative, not coding.
• Audio retention policy. Document a written retention policy: e.g., raw audio destroyed within 24 hours of note finalization. Most state boards expect this if you record at all.
• Informed consent. If you record, your consent form must say so, including that recording is processed locally and deleted after note generation. Get an attorney to review your form.
• Backups. If you back up the notes folder, the backup destination must also be encrypted. Time Machine to a FileVault-protected external drive is fine. iCloud is a different conversation - many state boards consider iCloud a third-party disclosure.

For privacy posture more broadly, our local AI privacy guide walks through threat modeling for solo practitioners.

---

Common Pitfalls

1. Using the consumer ChatGPT app "just for the draft." OpenAI's consumer terms allow training on your inputs unless you opt out and you are on an Enterprise plan with a BAA. The free and Plus tiers are not HIPAA-eligible. Period.

2. Letting Whisper auto-detect language. medium.en is dramatically more accurate than medium for English-only sessions. Use the .en variant.

3. Skipping audio normalization. Whisper handles bad audio okay, but a 16kHz mono WAV is 3-4x faster than a 48kHz stereo M4A and equally accurate.

4. Forgetting to delete temp files. The shell script uses rm -P (overwrite then remove) on the temp WAV. Do not skip this on a shared machine.

5. Trusting the model's "Assessment" section blindly. It will sometimes write "client appears at low risk for self-harm" when the transcript said no such thing. Always rewrite that section.

6. Running on a personal laptop your kids also use. Local AI does not magically protect against another user account reading your files. Use a dedicated practice machine, or at minimum a separate, encrypted user account.

---

Frequently Asked Questions

Is local AI HIPAA-compliant by default?

No software is "HIPAA-compliant" - compliance is a property of your overall practice, not a single tool. But running an open model entirely on your encrypted device removes the third-party disclosure issue that most cloud AI scribes create. You still need a written privacy policy, encryption at rest and in transit, audit controls, and a documented retention schedule.

Can I use this with an EHR like SimplePractice or TherapyNotes?

Yes - the output is plain Markdown text. Copy/paste into the progress note field. Some practitioners build a small AppleScript or PowerShell hotkey to paste directly into the EHR's note editor.

How accurate is Whisper for therapy sessions?

Accurate enough to be worth editing, not accurate enough to trust unread - and the variable that dominates is audio quality, not the model. One microphone, two speakers, a quiet room and no HVAC gets you a transcript that needs light correction; a laptop microphone in a noisy office does not. Clinical terms, drug names and acronyms are the usual error category, so proofread those specifically. Use the .en model variant for English-only sessions rather than the multilingual build.

What about telehealth - can I record a Zoom session locally?

Yes, but get explicit consent in writing, document it in the chart, and check your state board's rules on telehealth recording. Zoom's local recording option saves an MP4 to your machine - extract the audio with ffmpeg and run the same pipeline.

Will the model leak transcripts back through any update or telemetry?

Ollama and whisper.cpp are open source and run fully offline once installed. Block them at the firewall if you want belt-and-suspenders. On macOS, Little Snitch is a clean way to deny them outbound network access entirely after the initial model download.

How is this different from Freed, Heidi, or Suki?

Those are SaaS scribes. They send audio or text to a cloud LLM, usually OpenAI or Anthropic. Some offer BAAs, some do not. None of them give you the property that "no PHI ever leaves the device." That is the unique value of a local stack.

What if the model writes something clinically wrong?

It will, occasionally. That is why the workflow ends with you reading and editing. What the model saves is the structural drafting time - turning a transcript into the right sections in the right register. It does not replace clinical judgment. Treat the output like a transcriptionist's first pass, not a finished note.

Will my notes survive a state board audit?

The audit artifact is the signed, edited, clinician-authored note in your EHR - it does not matter that a local LLM produced the first draft. What matters is that you reviewed and signed it, that you have documented informed consent for any recording, and that your retention and destruction policies are written down and followed. Boards care about the final record and the process around it, not the typing tool.

Can a small group practice share one local server?

Yes - run Ollama on a Mac Studio in the office, expose it only on the LAN, require Tailscale for remote access, and have each clinician hit the same endpoint from their workstation. See our Ollama production deployment guide for hardening.

---

Wrapping Up

The economics are simple enough to work out yourself: take the per-clinician monthly price of whichever SaaS scribe you are considering, multiply by twelve, and compare it against an afternoon of setup on hardware you already own. The software here is free and there is no per-seat meter.

But the real reason to do this is not money. It is that your client trusted you with their darkest five years, and that trust deserves a workflow where their words never become someone else's training data, telemetry, or breach notification letter. A local model is the only honest answer to "where does my session go?"

Set it up over a quiet weekend, run it for two weeks in parallel with however you currently chart, and keep it only if the drafts genuinely need less work than starting from a blank note.