Luke Oliff.
Luke Oliff · Writing

Dynamic video narration using the Speechify Voice Cloning API

·Voice AI·8 min read·Luke Oliff

Dynamic video narration using the Speechify Voice Cloning API

It’s 2026 and every TikTok video sounds the same. With Speechify, a regular text-to-speech request with us can be transformed by switching out one of our voices with your own, as easily as changing the voice_id to the one you just cloned.

This walks the native (no-SDK) version of the clone, use, delete lifecycle. The full recipe is in the cookbook at recipes/audio/typescript/native/voice-cloning, with a runnable end-to-end version in speechify-ai-demos/voice-cloning-narration. You’ll need a Speechify API key to run any of it. Sign up here if you don’t have one yet.

What “voice cloning” actually does

You POST an audio sample of someone speaking. The API returns a voice_id. Any future synthesis call with that voice_id produces audio in the cloned voice. The clone persists on your account until you delete it.

That’s it. No fine-tuning step, no waiting queue, no separate “is the model ready” poll. The clone is usable on the next request.

The sample should be 10 to 30 seconds of clean speech. Single speaker, no music, minimal background noise. Phone-quality audio works but produces phone-quality output, so this is the one place where the input matters more than the prompt.

Voice cloning

A clone is just a reusable voice_id

Upload a consented sample, get a voice id back, and pass that id into the regular speech endpoint.

  1. Step 1 Upload sample 10 to 30 second WAV, one speaker, plus consent JSON in multipart form data. POST /v1/voices
  2. returns id
  3. Step 2 Store voice_id Persisted on your account and reusable across future synthesis calls. { "id": "voice_abc123…" }
  4. use as voice_id
  5. Step 3 Synthesize as normal Same endpoint and body shape as a catalogue voice. POST /v1/audio/speech

One-off jobs delete the voice after use with DELETE /v1/voices/{id}. Long-running narrator identities keep the id and reuse it.

The bit that shocked me

I cloned my own voice as a smoke test for this post, expecting the usual uncanny-valley approximation. What came back was close enough that I’m posting both clips here so someone else can tell me if I’m imagining it.

Both samples below are me, in the sense that neither one is anyone else. One is a real microphone recording, the other is Speechify synthesizing text using a voice cloned from that same recording. I’m not going to tell you which is which, that would ruin the joke.

Sample A

Sample B

Cloning a voice with the REST API

The clone endpoint is POST /v1/voices, multipart form-data. The fields are name, gender, consent, and sample. One gotcha the cookbook recipe gets right that everyone gets wrong the first time: when you pass a FormData to fetch, do NOT set Content-Type: multipart/form-data manually. fetch sets the boundary itself from the FormData instance. If you set the header yourself, your boundary value is missing or wrong and the upload fails with a non-obvious error. I’ve watched this trip up three different engineers on three different teams, including past-me.

import "dotenv/config";
import fs from "node:fs";
import path from "node:path";

const token = process.env.SPEECHIFY_API_KEY;
if (!token) throw new Error("Set SPEECHIFY_API_KEY");

const BASE = "https://api.speechify.ai";
const samplePath = path.resolve(import.meta.dirname, "fixtures/spacewalk.wav");

const form = new FormData();
form.append("name", "narrator-v1");
form.append("gender", "male");
form.append("consent", JSON.stringify({
  fullName: "Jane Doe",
  email: "jane@example.com",
}));

const sampleBytes = fs.readFileSync(samplePath);
form.append("sample", new Blob([sampleBytes], { type: "audio/wav" }), "spacewalk.wav");

const createRes = await fetch(`${BASE}/v1/voices`, {
  method: "POST",
  headers: { Authorization: `Bearer ${token}` }, // no Content-Type here
  body: form,
});

if (!createRes.ok) {
  if (createRes.status === 402) {
    console.error("Voice cloning isn't included in your current Speechify plan.");
    console.error("Upgrade at https://speechify.ai/pricing");
    process.exit(1);
  }
  throw new Error(`POST /v1/voices → ${createRes.status} ${await createRes.text()}`);
}

const voice = await createRes.json() as { id: string; display_name: string; type: string };
console.log(`Created voice ${voice.id}`);

That 402 path is real, I hit it on my own dev account the first time I ran this. The response envelope is the standard Speechify shape:

{
  "error": {
    "code": "voice_cloning_not_included",
    "message": "current billing plan does not have access to voice cloning"
  },
  "request_id": "993cae7e..."
}

voice_cloning_not_included is the error code to branch on if you’re shipping this to customers whose plan may or may not include cloning. Don’t catch the generic 402.

consent is required. The API rejects the request without it. The shape is a JSON string attesting you have the speaker’s permission to clone their voice, including a name and email of the consenting person.

form.append("consent", JSON.stringify({
  fullName: "Jane Doe",
  email: "jane@example.com",
}));

Don’t fake this. Don’t paste in lorem ipsum for a quick test and forget to swap it out. Voice cloning is the API in the Speechify catalogue with the most obvious abuse vector, and the consent record is what differentiates legitimate use from “I scraped a podcast.” For your demos, use the bundled spacewalk.wav sample in the cookbook (NASA ISS audio, public domain). For your product, use a real consent flow with a real person.

Synthesizing with the cloned voice

Once you have a voice_id, the synthesis call is identical to the default-voice version. Same endpoint, same body shape, the cloned id slots in where george or any other catalogue voice would go:

const speechRes = await fetch(`${BASE}/v1/audio/speech`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${token}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    input: "Hello from a voice cloned with the Speechify API.",
    voice_id: voice.id, // the id you just got back from /v1/voices
    audio_format: "mp3",
    model: "simba-english",
  }),
});

const speech = await speechRes.json() as { audio_data: string };
fs.writeFileSync("narration.mp3", Buffer.from(speech.audio_data, "base64"));

If you read the Node.js TTS post, this is the same request you’ve already made, just with a different value for one field. The cloned voice acts identical to a catalogue voice from this point on, which means streaming, speech marks (see the captions post if you want to wire those up), and SSML all work exactly as they do for any other voice.

Cleaning up

Cloned voices stick around on your account until you delete them. For one-off video narration jobs, the lifecycle is usually clone, synthesize, delete:

try {
  // ... synthesis ...
} finally {
  const delRes = await fetch(
    `${BASE}/v1/voices/${encodeURIComponent(voice.id)}`,
    { method: "DELETE", headers: { Authorization: `Bearer ${token}` } },
  );
  if (!delRes.ok) {
    console.error(`Cleanup failed: ${delRes.status} ${await delRes.text()}`);
  }
}

The finally matters. If your synthesis throws, you still want the voice gone, otherwise you’ve got an orphan on your account every time a job fails halfway. voices.list() is eventually consistent so a just-deleted voice may show up for a second or two on the next list call, but the delete itself is immediate. A subsequent delete returns 404.

For long-running narrator identities (a podcast host, a recurring character), keep the voice and reuse the voice_id. Don’t re-clone every time, it costs an upload and adds latency.

Stitching audio into a video

Once you’ve got narration.mp3, the rest is ffmpeg. If your video is silent footage plus the narration:

ffmpeg -i video.mp4 -i narration.mp3 \
  -c:v copy -c:a aac -shortest \
  output.mp4

-c:v copy keeps your video stream untouched (no re-encode, no quality loss). -c:a aac re-encodes the narration to AAC, which is the audio codec MP4 wants. -shortest trims the output to whichever stream ends first, which is how you avoid trailing silence if the narration is shorter than the visuals.

For a static-image video (talking head replacement, podcast cover art, audiobook splash):

ffmpeg -loop 1 -i cover.png -i narration.mp3 \
  -c:v libx264 -tune stillimage -c:a aac \
  -pix_fmt yuv420p -shortest \
  output.mp4

That’s enough for YouTube Shorts, podcast clips, audiobook samples, and anywhere else you’d otherwise be paying for a stock TTS voice a million other pipelines are already using.

FAQ

Is voice cloning included on every Speechify plan?

No. Voice cloning is plan-gated. The API returns 402 voice_cloning_not_included if your current plan does not include it. Pricing and which tiers include cloning live on the Speechify pricing page. Branch on the voice_cloning_not_included error code (not the generic 402 status) so your customer-facing error message is precise.

How long should the audio sample be?

Between 10 and 30 seconds of clean speech, single speaker, minimal background noise. Less than 10 seconds and the clone is hit-or-miss on prosody. More than 30 seconds adds upload time without improving the output meaningfully. Phone-recorded audio works but produces phone-quality clones, so for a polished narrator voice the input has to be polished too.

Can I use a cloned voice with streaming and speech marks?

Yes. Once /v1/voices gives you back a voice_id, it acts identically to any catalogue voice. You can stream it through /v1/audio/stream for low time-to-first-byte playback, and the non-streaming /v1/audio/speech endpoint returns word-level speech marks alongside the audio for caption generation. Both work without any additional clone-specific configuration. The full parameter surface for both endpoints lives in the Speechify docs .

Yes. It must be a JSON string with fullName and email fields identifying the person whose voice is being cloned. The API rejects requests without it. Don’t fake the values, this is the audit record that distinguishes legitimate use from scraping. For your own dev testing, use the bundled spacewalk.wav (NASA public-domain audio) so consent is unambiguous.

← NewerThe API now exposes more safety limits at the edge
All posts