Voice AIStatus: Case StudyYear: 2026

Voice Cloning Pipeline

Voice cloning from a short browser recording, running locally on Apple silicon.

Voice Cloning PipelineVoice AI

Python
FastAPI
Qwen3-TTS

A look inside

3 screens from the real build. Select any to see it full size.

Voice Cloner web app in a dark theme. Left: a Reference Audio panel with a 9-second recorded preview and an orb visualizer, and a Target Text panel with a sentence containing a date, a time, a five-digit code, an email address and a phone number, plus a transcript of the reference clip. Right: an output panel reading READY and Inference: 57.94s, an audio player at 0:02 of 0:17, and a Download WAV link. — One full cloning run on an Apple M4 laptop (MPS) with Qwen3-TTS 0.6B Base. The FastAPI backend builds a voice prompt from the reference clip, using the transcript when one is given, and returns a WAV that the page plays and offers for download. The reference voice here is a synthetic macOS voice, not a real person.

Voice Cloner while recording: the reference panel shows a live cyan waveform with a red REC 0:02 timer and a Stop button, a glowing orb visualizer below it, a greyed-out Generate Speech button, and an empty output panel reading Output will appear here. — Reference audio is recorded in the browser with MediaRecorder. A canvas draws the live microphone waveform, and the orb reacts to input level. Generate Speech stays disabled until a recording exists.

FastAPI Swagger page for POST /api/clone with ref.wav attached, the text and transcript fields filled in, the generated curl command, and a 200 response whose headers include content-type audio/wav and an x-inference-time of about 53 seconds. — The backend is one FastAPI endpoint that takes the reference clip, the text to speak and an optional transcript. Without a transcript it uses the faster speaker-embedding-only mode. It returns the WAV with the inference time in a response header.

Overview

A local voice-cloning app: record a short clip in the browser, type what it should say, and get speech in that voice from Qwen3-TTS running on a Mac.

The problem

Trying voice cloning usually means a cloud API and uploading someone's voice. This keeps the whole loop on one laptop.

What I built

Built the app: in-browser recording, the FastAPI cloning endpoint, and local inference setup on Apple silicon.

What it does

In-browser recording with a live waveform
Optional reference transcript
WAV playback and download

System and architecture

A React frontend records reference audio, and a FastAPI endpoint builds a voice prompt for Qwen3-TTS 0.6B on Apple's MPS backend and returns the generated speech.

The browser records reference audio with MediaRecorder and draws the live microphone waveform. A FastAPI endpoint takes the clip, the text and an optional transcript, builds a voice prompt for Qwen3-TTS 0.6B on Apple's MPS backend, and returns a WAV with the inference time in a response header. Without a transcript it uses the faster speaker-embedding-only mode.

System map3 layers

Interface
API & services
AI & models

One FastAPI endpoint for cloning
Transcript-guided or speaker-embedding-only modes
Local inference on Apple silicon

Technology

Themes

Challenges and trade-offs

The transcript has to match the reference clip, or the model speaks the leftover words.
Structured text such as email addresses and phone numbers still needs normalizing before synthesis.

Outcome

At a glance

Qwen3-TTS 0.6B · FastAPI · Apple MPS

Runs entirely on a laptop, with no cloud TTS calls.
Reports inference time with every generated clip.

Learnings

Cloning quality depends on a clean reference clip and an accurate transcript.
Local models make experimenting cheap, but setup details such as audio codecs decide whether it runs at all.

Next steps

Normalize emails, phone numbers and dates before synthesis.
Add torchcodec to requirements so fresh installs work.