Local AI Podcast Production: Transcribe, Edit, and Publish Privately

Published on April 23, 2026 — 23 min read

Podcast post-production has quietly turned into a subscription stack. Descript for transcription and text-based editing, Riverside for AI clips, Auphonic for loudness — each modest on its own, and together a few hundred dollars a year for a workflow whose first step is uploading raw guest audio to somebody else's cloud.

That last part is what bites. Independent shows interview people who say things candidly because they trust the host, and guests at companies with a "no third-party AI processing" policy increasingly ask what happens to the recording. "It goes to two US vendors for AI processing" is an awkward answer, and it is the true one for most hosted podcast tooling.

Every AI step in that stack now has a local equivalent that runs on hardware you already own. Whisper handles transcription. pyannote.audio splits speaker turns. A local model through Ollama writes show notes, chapter markers and episode descriptions, and pulls quote candidates for social. The ongoing cost is electricity.

This guide is the full pipeline: the scripts, the prompts, and the integration glue for the podcast hosts most independent shows are on (Transistor, Buzzsprout, Captivate, and self-hosted via Castopod).

Quick Start: From Raw WAV to Show Notes

# 1. Install the audio AI stack
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5:14b
pip install faster-whisper pyannote.audio sounddevice ffmpeg-python jinja2

# 2. Authenticate pyannote (one-time, free Hugging Face account)
huggingface-cli login

# 3. Drop your episode WAV into a folder and run
python pipeline.py episode-042.wav --speakers 2 --show-name "Friday Discourse"

You'll get back a folder with: a clean transcript, speaker-labeled SRT, chapter markers in plain text and Apple Podcasts JSON, four candidate social pull-quotes, a 200-word episode description, and a list of name/term corrections to review before publishing. How long that takes is dominated by the transcription step — the throughput section below has the arithmetic, plus a two-minute way to measure your own machine before you plan a workflow around it.

The rest of this guide explains every piece, including the prompts that make show notes sound like notes, not AI slop.

---

Table of Contents

1. Why Local Beats Descript for Most Shows
2. The Pipeline: 6 Stages
3. What Drives Throughput
4. Stage 1: Transcribe with faster-whisper
5. Stage 2: Speaker Diarization with pyannote
6. Stage 3: Cleanup and Disfluency Removal
7. Stage 4: Show Notes, Chapters, and Quotes
8. Stage 5: Episode Descriptions for SEO
9. Stage 6: Publishing to Transistor / Buzzsprout / Captivate
10. Comparison: Descript, Riverside, Auphonic, and This Build
11. Pitfalls and How to Avoid Them
12. Quality Checks Before You Publish
13. FAQs

---

Why Local Beats Descript for Most Shows

Three reasons that compound:

Guest privacy. Hosted transcription and AI-clip products process audio on their own infrastructure — that is the product. What varies is retention, region and whether your audio can be used to improve their models, and those terms differ by vendor and by plan and change without much fanfare. Read the current ones for whatever you are on. For shows covering therapy, journalism, or founder candor, "nothing leaves this machine" is a materially easier sentence to say to a guest than a summary of someone's data-processing addendum.

Control over names and jargon. Every transcription system stumbles on uncommon proper nouns, because they are rare in training data by definition. Whisper exposes an initial_prompt parameter that is fed to the model as prior context, biasing decoding toward the spellings you supply — so you can hand it "Aoife Ní Mhurchú, Rinkebysorm AB, Helsinki" before each episode and get those spellings back. Hosted products either do not expose that knob or restrict it to a fixed custom-vocabulary list. It is the single biggest quality difference in practice, and it is a parameter rather than a model upgrade.

Cost that stops scaling. Subscription spend scales with seats and with how many shows you run; hardware does not. The payback calculation is entirely yours to run — annual subscription spend divided into the price of a used 12 GB GPU — but the structural point holds regardless of the numbers: the open-source side of this stack gets better each year without a price increase, and it does not lose features when a vendor repackages its tiers.

For more on the privacy angle, the local AI privacy guide covers the threat model that drives most professional adoption.

---

The Pipeline: 6 Stages

Six discrete stages, each replaceable independently:

RAW WAV
   │
   ▼
[1] Whisper (faster-whisper)  ──► transcript.json (word-level timestamps)
   │
   ▼
[2] pyannote.audio diarization ──► speakers.rttm
   │
   ▼
[3] Cleanup (disfluency, normalize) ──► transcript.clean.json
   │
   ▼
[4] Ollama notes/chapters/quotes ──► notes.md, chapters.txt, quotes.json
   │
   ▼
[5] Description generator ──► description.txt (3 lengths: 150w, 300w, 50w)
   │
   ▼
[6] Hosting platform upload (Transistor API / RSS) ──► published episode

Decoupling matters. If a better diarization model ships next year, swap Stage 2 without touching the rest. If you want to A/B test Llama vs Qwen for show notes, only Stage 4 changes.

---

What Drives Throughput (and How to Measure Yours)

Two very different workloads live in this pipeline, and they are slow for different reasons. Understanding which is which tells you where to spend money.

Transcription scales with audio length

Whisper processes audio in 30-second windows, so transcription time scales linearly with episode length. The useful unit is the real-time factor (RTF): seconds of compute per second of audio. An RTF of 0.05 means a 90-minute episode takes about 4.5 minutes; an RTF of 0.5 means 45.

RTF depends on the model size, the compute type (float16 vs int8), and — by far the largest factor — whether you are on a GPU or a CPU. The gap between GPU float16 and CPU inference is more than an order of magnitude, which is why the same pipeline is a coffee break on one machine and a lunch break on another. faster-whisper's README reports roughly 4× the throughput of the reference openai/whisper implementation at equivalent accuracy, which is the reason this guide uses it rather than the original.

Diarization behaves similarly: pyannote runs a segmentation and an embedding model over the same audio, so it also scales with duration, and it also cares a great deal about GPU vs CPU.

The LLM stages are memory-bandwidth bound

Stage 4 and Stage 5 are different. Local LLM decoding reads every model weight once per generated token, so the ceiling is set by memory bandwidth:

tokens/sec ceiling = memory bandwidth (GB/s) ÷ (params in billions × 0.6)

The 0.6 is roughly the GB per billion parameters at Q4_K_M. For the 14B model this guide uses (about 8.4 GB read per token) and an 8B alternative (about 4.8 GB):

These are arithmetic upper bounds, not measurements. Real decode lands below them; nothing puts you above them.

Now size the work. Stage 4 and Stage 5 together produce show notes, chapters, quote candidates and three descriptions — on the order of 2,000-2,500 tokens of output across four calls. Divide by the ceiling above and the LLM half of this pipeline has a floor well under a minute on a 3090 and a few minutes on an M3 Pro. The other half of that cost is prefill over a 90-minute transcript, which is tens of thousands of tokens; prefill is compute-bound and parallel rather than bandwidth-bound, so a card with weak memory bandwidth but strong compute behaves differently there.

The practical conclusion: transcription is what you buy hardware for, not the LLM stages.

Measure your own RTF in two minutes

Do this before planning a workflow around anyone's numbers:

# Cut a 5-minute sample from a real episode
ffmpeg -i episode.wav -t 300 -c copy sample.wav

# Time a real transcription pass on it
time python transcribe.py sample.wav sample.json

Divide the wall-clock seconds by 300 to get your RTF, then multiply by your usual episode length. That single number tells you whether this pipeline is a background job or a blocking one on your machine — and it is worth re-running whenever you change model size or compute type.

---

Stage 1: Transcribe with faster-whisper

faster-whisper is the same Whisper weights reimplemented on top of CTranslate2; its README reports roughly 4× the throughput of the reference implementation at equivalent accuracy, with lower memory use. For podcasts specifically, large-v3 is the right starting model — the distilled variants trade accuracy for speed, and long-form conversational audio with crosstalk is where that trade shows up first.

# transcribe.py
from faster_whisper import WhisperModel
import json, sys

model = WhisperModel("large-v3", device="cuda", compute_type="float16")

def transcribe(audio_path, glossary=None):
    initial_prompt = None
    if glossary:
        initial_prompt = "Glossary of names and terms in this episode: " + ", ".join(glossary)

    segments, info = model.transcribe(
        audio_path,
        language="en",
        word_timestamps=True,
        vad_filter=True,
        vad_parameters=dict(min_silence_duration_ms=500),
        initial_prompt=initial_prompt,
        beam_size=5,
    )

    out = []
    for seg in segments:
        out.append({
            "start": seg.start,
            "end": seg.end,
            "text": seg.text.strip(),
            "words": [{"start": w.start, "end": w.end, "word": w.word} for w in seg.words] if seg.words else []
        })
    return out, info

if __name__ == "__main__":
    glossary = ["Aoife Ní Mhurchú", "Rinkebysorm AB", "Helsinki", "founder mode"]  # example
    segments, info = transcribe(sys.argv[1], glossary=glossary)
    json.dump({"segments": segments, "language": info.language}, open(sys.argv[2], "w"), ensure_ascii=False, indent=2)

Three knobs that matter for podcast audio:

• beam_size=5 keeps several candidate transcriptions alive instead of committing to the highest-probability token at every step. It costs decode time and generally earns it back on noisy or accented audio. Greedy decoding (beam_size=1) is faster — run both over one episode and keep whichever you prefer reading.
• The VAD filter trims silence before the model sees it. Without it, Whisper is known to hallucinate text during long pauses, because a 30-second window of near-silence still has to be decoded into something.
• initial_prompt is the highest-leverage knob on this page. Whisper conditions on it as prior context, so listing the episode's names and terms biases decoding toward the right spellings. It is the difference between "Eva Nee Murcoo" and "Aoife Ní Mhurchú", and it costs thirty seconds of typing per episode.

For a deeper dive on Whisper variants and tuning, see our Whisper local speech-to-text guide.

---

Stage 2: Speaker Diarization with pyannote

pyannote.audio 3.1 is the open-source standard for speaker diarization. Its model card publishes diarization error rates across a set of benchmark corpora — check the one whose recording conditions look most like your show rather than trusting a single headline number, because the spread between a clean studio corpus and a noisy meeting corpus is large. In practice the biggest quality factor is not the model: it is whether your speakers are on separate microphones with minimal bleed.

# diarize.py
from pyannote.audio import Pipeline
import torch, sys

pipe = Pipeline.from_pretrained(
    "pyannote/speaker-diarization-3.1",
    use_auth_token=True
).to(torch.device("cuda" if torch.cuda.is_available() else "cpu"))

def diarize(audio_path, num_speakers=None):
    kwargs = {"num_speakers": num_speakers} if num_speakers else {}
    diarization = pipe(audio_path, **kwargs)
    turns = []
    for turn, _, speaker in diarization.itertracks(yield_label=True):
        turns.append({"start": turn.start, "end": turn.end, "speaker": speaker})
    return turns

if __name__ == "__main__":
    turns = diarize(sys.argv[1], num_speakers=int(sys.argv[2]))
    import json
    json.dump(turns, open(sys.argv[3], "w"), indent=2)

Then merge transcript and diarization into a speaker-labeled output:

# merge.py
def merge_transcript_diarization(segments, turns):
    out = []
    for seg in segments:
        midpoint = (seg["start"] + seg["end"]) / 2
        speaker = next((t["speaker"] for t in turns if t["start"] <= midpoint <= t["end"]), "UNKNOWN")
        out.append({**seg, "speaker": speaker})
    return out

For interview podcasts, always pass num_speakers=2. Telling pyannote the true count removes an entire source of error — otherwise it has to estimate the number of speakers and assign turns, and an over-estimate shows up as one person being split into two labels halfway through the episode. For panel shows, pass the real count if you know it; diarization gets harder as speakers are added and harder still with crosstalk.

---

Stage 3: Cleanup and Disfluency Removal

Whisper's raw output is verbatim. For show notes and SEO descriptions, you want lightly cleaned prose — fillers removed, repetitions collapsed, sentence boundaries preserved. Don't actually edit the audio; this cleaned version exists only for the LLM stages.

# cleanup.py
import re

DISFLUENCIES = re.compile(r"\b(um|uh|er|ah|like|you know|i mean|sort of|kind of)\b", re.IGNORECASE)
REPEATED_WORDS = re.compile(r"\b(\w+)( \1\b)+", re.IGNORECASE)

def clean(text):
    text = DISFLUENCIES.sub("", text)
    text = REPEATED_WORDS.sub(r"\1", text)
    text = re.sub(r"\s+", " ", text).strip()
    return text

def cleaned_corpus(merged_segments):
    return "\n\n".join(
        f"[{s['speaker']}] {clean(s['text'])}"
        for s in merged_segments if clean(s['text'])
    )

This is dumb on purpose. Don't run an LLM over the transcript to "polish" it — that's where hallucinations creep in. Save the LLM for downstream tasks where invention is acceptable (notes, descriptions, social).

---

Stage 4: Show Notes, Chapters, and Quotes

Now the production magic. One Ollama call per task, each with a tightly-scoped prompt.

# notes.py
import ollama, json

MODEL = "qwen2.5:14b"

NOTES_PROMPT = """You are writing show notes for an interview podcast. Output Markdown with:
- A 1-paragraph episode summary (60-80 words)
- 5-7 bullet points covering the main topics in order
- A "Mentioned in this episode" section listing books, papers, tools, people referenced
- Honest, direct voice. No marketing speak. No 'dive deep' or 'fireside chat'.

Transcript:
{transcript}"""

CHAPTERS_PROMPT = """Generate 8-14 chapter markers for this podcast episode. Output JSON:
{
  "chapters": [
    {"title": "<5-9 word title, no quotes>", "start_seconds": <int>}
  ]
}
Use natural topic transitions. First chapter must be at 0 seconds. Titles should be specific (e.g., "How Aoife landed her first 100 customers" not "Customer acquisition").

Transcript with timestamps:
{timed_transcript}"""

QUOTES_PROMPT = """Pick 4 quote candidates for social media from this transcript. Each must be:
- 12-30 words
- A complete thought that makes sense without context
- Said by a guest (not the host) when possible
- Specific, not generic ("the hardest part of fundraising was the rejection email at 11pm" beats "fundraising is hard")

Output JSON: {"quotes": [{"text": "...", "speaker": "SPEAKER_01", "approx_time": "00:32:14"}]}

Transcript:
{transcript}"""

def call(prompt, json_mode=False):
    r = ollama.chat(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}],
        format="json" if json_mode else "",
        options={"temperature": 0.5, "num_ctx": 32768}
    )
    return r["message"]["content"]

def make_notes(transcript_text):
    return call(NOTES_PROMPT.format(transcript=transcript_text[:60000]))

def make_chapters(timed_transcript):
    raw = call(CHAPTERS_PROMPT.format(timed_transcript=timed_transcript[:60000]), json_mode=True)
    return json.loads(raw)

def make_quotes(transcript_text):
    raw = call(QUOTES_PROMPT.format(transcript=transcript_text[:60000]), json_mode=True)
    return json.loads(raw)

Three things worth knowing about this prompt set:

• Prompt tightness matters more than model size here. A 14B instruct model is the sweet spot: large enough to hold a 90-minute conversation's through-line together, small enough to run four calls back to back. Before assuming bigger is better, run one episode through a 70B-class model and read both outputs — the usual failure mode of large models on summary tasks is verbosity, not error, and verbose show notes are worse show notes.
• Cap the context you send. Long-context models degrade on material buried in the middle of a very long input — the "lost in the middle" effect documented in the retrieval literature. If your notes consistently ignore the first half of the episode, that is the cause; chunk the transcript by section and summarise hierarchically instead of raising the cap.
• Always ask for JSON on structured outputs. format="json" constrains decoding rather than relying on the model's good manners. Markdown is fine for prose-only outputs like the show notes.

For details on Ollama's JSON mode and tool calling, see the Ollama Python API guide.

---

Stage 5: Episode Descriptions for SEO

Every podcast app truncates episode descriptions somewhere, at different lengths, with different "show more" behaviour — and the cutoffs move between app versions. Rather than chase an exact word count, generate three lengths once and pick per platform. Your RSS <description> field has no hard limit, but the opening lines carry the weight everywhere they are shown.

DESCRIPTION_PROMPT = """Write three episode descriptions for "{show}" episode {num}: {title}.

Length 1 (50 words): for Twitter/social.
Length 2 (150 words): for Apple Podcasts above-the-fold.
Length 3 (300 words): for the show website with SEO keywords woven in naturally.

Tone: factual, no buzzwords, no "in this episode we explore" or "join us as we dive into".
Audience: {audience}
Keywords to incorporate naturally where relevant: {keywords}

Episode summary: {summary}
Top topics: {topics}

Output JSON: {{"50w": "...", "150w": "...", "300w": "..."}}"""

The keyword integration matters because in-app podcast search indexes episode metadata, not just show titles — an episode is discoverable on its own terms. That makes the 300-word version a real surface rather than filler, provided the keywords read like sentences a person wrote.

---

Stage 6: Publishing to Transistor / Buzzsprout / Captivate

Three patterns:

Transistor has a clean REST API. Upload episode, set title/description/chapters, schedule publish. Roughly 30 lines of Python.

Buzzsprout has an API as well, but file uploads go through their proprietary endpoint. Slightly more involved.

Captivate doesn't expose a public API for uploads at the time of writing. Workaround: drop the audio + chapters JSON into a watch folder and use the GUI for the final upload trigger.

Self-hosted via Castopod lets you push directly to your own RSS feed. The pipeline can write the RSS XML if you want full control.

# publish_transistor.py
import requests, os

API = "https://api.transistor.fm/v1"
TOKEN = os.environ["TRANSISTOR_API_KEY"]
SHOW_ID = os.environ["TRANSISTOR_SHOW_ID"]

def publish(audio_path, title, description, chapters, season, number):
    # 1. Authorize upload
    upload = requests.get(f"{API}/episodes/authorize_upload",
        headers={"x-api-key": TOKEN},
        params={"filename": os.path.basename(audio_path)}
    ).json()
    requests.put(upload["data"]["attributes"]["upload_url"], data=open(audio_path, "rb"))

    # 2. Create episode
    body = {
        "episode": {
            "show_id": SHOW_ID,
            "audio_url": upload["data"]["attributes"]["audio_url"],
            "title": title,
            "summary": description[:255],
            "description": description,
            "season": season,
            "number": number,
        }
    }
    ep = requests.post(f"{API}/episodes", headers={"x-api-key": TOKEN}, json=body).json()
    return ep["data"]["id"]

For chapters, both Apple Podcasts and Spotify support PSC (Podlove Simple Chapters) embedded in the audio. ffmpeg writes them:

ffmpeg -i episode.wav -i chapters.txt -map_metadata 1 -codec copy episode-with-chapters.mp3

The chapters.txt format is documented in the Apple Podcasts Connect chapter spec. Stage 4's chapter generator outputs in that format directly.

---

Comparison: Descript, Riverside, Auphonic, and This Build

A capability comparison, not a benchmark. Transcription accuracy is not a single number you can copy from a table — it depends on your microphones, your accents and your jargon — so measure it on your own audio (see the FAQ below for how) rather than trusting anyone's figure, including ours. Pricing moves constantly; check each vendor's current page.

A reasonable hybrid: keep Auphonic for levelling and loudness normalisation, which is genuinely worth paying for and is not an LLM problem, and replace the AI-derived outputs with this pipeline. What you save depends entirely on what you pay now — the structural change is that the largest part of the bill stops scaling with how much you publish.

---

Pitfalls and How to Avoid Them

Pitfall 1: Trying to use Whisper for the master audio edit. Whisper transcribes; it doesn't edit. The transcripts are for show-note generation, not for cutting audio. Use a real DAW (Reaper, Audacity, Logic) for actual editing.

Pitfall 2: Skipping the glossary. Every episode has a handful of proper nouns, technical terms and company names. Spend 30 seconds writing them down before transcription and pass them as initial_prompt — Whisper conditions on them and gets the spellings right far more often. Skipping this is the most common reason a local transcript reads worse than a hosted one.

Pitfall 3: Letting the LLM rewrite the transcript. That's where hallucinations live. Use the LLM for derivative outputs (notes, chapters, descriptions) and treat the Whisper transcript as ground truth.

Pitfall 4: Not caching intermediate outputs. A 90-minute Whisper pass is slow. Save transcript.json to disk and re-run downstream stages from it without re-transcribing.

Pitfall 5: Generic chapter titles. "Introduction" and "Closing thoughts" are useless for SEO and listener navigation. Tighten the prompt to require specific titles ("How Aoife landed her first 100 customers" not "Customer acquisition").

Pitfall 6: Forgetting to compare to a human-written episode. Run your first three episodes through both this pipeline and your existing process. If the output sounds AI-generated, tighten prompts before going all-in.

---

Quality Checks Before You Publish

Everything this pipeline produces is a draft. A short, fixed review pass catches the failure modes that would otherwise reach listeners:

• Spot-check ten transcript segments against the audio, weighted toward the noisy parts and the crosstalk — not ten random ones. Errors cluster where the audio is hard.
• Verify chapter timestamps land on real topic shifts, not mid-sentence. The model infers boundaries from text and cannot hear the pause that actually marked the transition.
• Read each quote candidate with the surrounding context removed. This is where an LLM most often flatters a speaker into saying something cleaner and stronger than they said. If it is going on social with their name attached, it has to be what they actually said.
• Confirm the description does not claim anything the episode does not contain. Descriptions are generated from a summary of a summary, which is where invented specifics appear.
• Re-read every proper noun against your glossary. Getting a guest's name wrong in the show notes is the one error they will definitely notice.

Instrument it rather than trusting a table

If you want per-stage timings, measure your own — they are worth far more than anyone's published numbers, since they reflect your hardware and your episode lengths:

import time, contextlib

@contextlib.contextmanager
def stage(name):
    t0 = time.perf_counter()
    yield
    print(f"{name}: {time.perf_counter() - t0:.1f}s")

with stage("whisper"):
    segments, info = transcribe("episode.wav", glossary=glossary)
with stage("pyannote"):
    turns = diarize("episode.wav", num_speakers=2)
with stage("notes"):
    notes = make_notes(transcript_text)

Run that on one real episode and you will know exactly which stage to optimise — and it is almost always transcription.

---

Frequently Asked Questions

The full FAQ schema is in the page metadata. Practical highlights:

• Multi-speaker shows work, but diarization gets harder with every speaker added and harder still with crosstalk. Pass the true speaker count and record to separate tracks where you can.
• Non-English shows are supported. Whisper's model card lists the 99 languages it covers along with per-language error rates — look yours up there rather than assuming.
• The pipeline can run unattended via cron: drop a WAV in a watch folder and collect the outputs when it finishes.
• For audio levelling and loudness compliance, Auphonic remains the right tool. This pipeline handles the AI-derived outputs only.

---

Wrapping Up

The goal of a pipeline like this is to disappear. A recording goes into a watch folder; a clean transcript, draft show notes, chapter markers, pull-quote candidates and three description lengths come out the other side. The machine does the mechanical work. The editorial judgment — what the episode is actually about, which quote is fair to the guest, what the title promises — stays with you, which is the whole design and the reason the review checklist above is not optional.

Whether the hardware pays for itself depends on what you publish and what you currently pay, so run that arithmetic with your own numbers rather than anyone else's. The parts that do not depend on your numbers are the ones worth having regardless: guest audio never leaves the building, the glossary knob is yours, and nothing in the workflow disappears when a vendor reshuffles its pricing tiers.