Fish Audio Drama 3 Guide: Voice References and Dialogue Workflows

Fish Audio Drama 3 gives a dialogue team a concrete workflow to evaluate: provide reference audio, label the characters, generate a take and review whether the performance matches the script. The difficult part is usually maintaining the intended voice and timing across revisions, especially when a scene contains several speakers.

This guide focuses on reference preparation, dialogue structure and listening checks. It describes documented integration options and a proposed evaluation plan; it does not claim that the sample scenes have been generated or commercially cleared.

What Drama 3 documents

Fish Audio describes Drama 3 as a text-to-speech model with multi-speaker generation, voice cloning and natural-language control. The model card specifies a 35B MoE text backbone with 3B active parameters, speaker tags from <|speaker:0|> through <|speaker:15|>, and automatic chunking of long text. It provides an SGLang-Omni deployment path and reference-audio request examples. Drama 3 model card.

Keep the weights and hosted service separate in your plan. Downloadable artifacts do not establish a hosted account's rate limits, prices or commercial permissions. Vercel also lists Drama 3 on AI Gateway under fish-audio/drama-3; that identifier differs from the Hugging Face repository spelling. Confirm which surface your application calls. AI Gateway availability.

The release links a custom Fish Audio license. Read the license attached to the specific artifacts you use rather than assuming an Apache-style grant. Voice-source permissions and the service agreement are separate checks. Linked license file.

Prepare references before writing the request

Make a small reference inventory with a clip identifier, transcript, intended character and permission record. Use audio you are entitled to provide. For a prototype, your own recorded lines or expressly permitted material are easier to manage than an unidentified clip copied from a video.

Listen for background music, room echo and other voices. Those details can complicate the interpretation of a reference. Keep the transcript faithful to the clip instead of using it to describe the voice. Store a separate direction such as “calm explanation” if your workflow needs one.

Start with one character and one short passage. Include a name, a number and a sentence with punctuation that changes the reading. This gives a reviewer specific things to listen for: pronunciation, omitted words and phrasing. Do not begin with an entire episode and try to diagnose every problem at once.

For multilingual work, choose review material in the language your listeners will hear. A clean English take does not validate a Mandarin brand name, and intelligible words do not automatically imply natural regional pronunciation. Ask a fluent reviewer to assess those distinctions.

Build dialogue with stable character labels

Keep a character map outside the prose. Assign one speaker tag per character and retain that assignment across revisions. A script editor should be able to change a line without silently swapping the voices attached to two roles.

Separate spoken text from production notes in your authoring system. Only pass the supported controls to the generation request. A parenthetical note intended for the director can become an unwanted spoken line if it enters the wrong field.

The official examples show point tags such as [sighs] and [laughs]. Use a small number initially and compare the take with the same passage without them. Their placement is part of the performance instruction; moving a tag to the wrong sentence can change the scene's meaning.

For each revision, keep the script version, reference identifiers and request settings. Label the output with a take number. A listening session becomes much easier when the reviewer can say “take four dropped the final phrase” and you can reconstruct that request.

Choose local or hosted integration deliberately

The local recipe uses SGLang-Omni. Follow its current cookbook for environment and streaming details instead of inventing a generic text-model server command. Reserve time for loading the full checkpoint and audio components; active parameters alone are not a memory estimate. Drama 3 cookbook.

A hosted route can simplify infrastructure, but your application still needs request limits, error handling and an output delivery path. First verify the exact model in the service's current catalog. Then check whether the service exposes the reference and dialogue controls your scene requires. Availability of a model name does not guarantee parity with every local feature.

Streaming also deserves an application test. Measure when playable audio arrives, whether successive chunks join cleanly, and what happens if a listener stops playback. A server emitting bytes is only one stage of the user experience.

Review performances with a repeatable listening sheet

Use a review sheet that separates transcription accuracy from performance quality:

Listen once without reading the script, then compare against the text. The first pass catches confusing delivery; the second catches subtle omissions. Retain a rejected take with a short reason rather than deleting every failure. It becomes a regression case when you change the pipeline.

Worked scene: revising a two-character support dialogue

Imagine a training clip in which a support agent explains a return process and a customer asks one follow-up question. Write a short synthetic script with a fictional order number and no real customer information. Assign the two characters stable speaker tags and keep a versioned mapping from each tag to its permitted reference material.

Create the initial scene with ordinary spoken text. Review it for words, names and turn boundaries before adding acting directions. If the order number is wrong, adding a more expressive delivery will not make the training clip correct. Fix the content error first and preserve the failed take as a pronunciation test.

In a second revision, change only the customer's question. Compare the agent's unchanged lines with the earlier take. You are looking for continuity of voice and intended tone, not necessarily identical samples. If the unaffected lines change in a way that disrupts the scene, consider regenerating smaller segments and reviewing the assembled result.

In a third revision, move a pause to let the customer interrupt naturally. Record whether the generated timing follows the direction and whether the dialogue remains understandable without reading the script. A line can sound emotionally plausible while arriving too late to function as an interruption.

Retain a revision table with script changes, reference changes and output notes. This prevents a director from attributing an improvement to a new prompt when someone also replaced the reference clip. The table can be simple, but each take should resolve to one reproducible input package.

For final review, let one listener evaluate the clip without the script and another compare every word against it. The two perspectives answer different questions: whether the scene communicates clearly and whether it faithfully says what was approved. Do not average away a wrong deadline or number because the performance sounds good.

Design a reference library that survives a longer project

A production reference library needs stable identifiers and an ownership trail. Store the approved clip, its transcript, the associated character and the terms under which it can be used. Keep references with different recording conditions distinct rather than silently replacing one file behind the same name.

Normalize your preparation procedure, but retain the originals. If you trim silence or adjust levels for a working copy, document that transformation. An audio reviewer should be able to determine whether an artifact comes from the source recording, preprocessing or generation. Aggressive cleanup can also remove cues you wanted the model to follow.

Separate character identity from scene direction in the metadata. A reference can define a consistent voice while the current scene calls for a different emotional state. If every new direction requires a new unidentified reference, continuity and permission tracking become difficult across a long episode.

Choose a naming convention that links scripts and assets without encoding private information. A fictional character identifier and take number are usually enough. Do not use a person's sensitive details as a convenient filename when the production team only needs to distinguish approved voices.

Periodically audit which references the project actually uses. Remove unused working copies according to the project's retention rules, and keep the approved originals where their access is controlled. A reusable voice library is a managed production asset, not merely a folder of convenient WAV files.

Test streamed playback rather than server output alone

Define observable events along the delivery path: request accepted, first audio received, first audio played and playback completed. Measure those times in the intended client. A fast first network chunk is not the same as a listener hearing a stable opening word if the player waits for a larger buffer.

Test short and long passages separately. A short sentence may finish before a user notices buffering, while a long scene can reveal gaps between chunks. Listen at the boundaries and retain a waveform or interval note for each discontinuity. This gives the engineering team a location to investigate rather than a general complaint that streaming feels rough.

Add cancellation to the client test. Start a long take, stop playback, then immediately start a different passage. Confirm that the old audio cannot continue into the new scene and that the request state is cleared. This is especially relevant in preview tools where editors repeatedly regenerate a line.

For network failures, decide whether the preview resumes, restarts or reports a partial take. Label partial output clearly so it does not enter the approved asset folder by accident. Do not join an incomplete generated segment with a newly started segment and assume the voice and timing will remain consistent.

Track concurrency at the scene level. If two editors preview different characters simultaneously, keep their responses and audio buffers separate. A successful request log does not prove that the application played the right response in the right panel. Include the character and take identifier in development diagnostics.

Use a pronunciation and continuity regression set

Create a small set of lines containing the vocabulary your production repeats: product names, acronyms, dates, currencies and mixed-language phrases. Record the intended reading in a review note. Where pronunciation has several acceptable forms, state them before listening to outputs rather than changing the rubric to fit a take.

Include sentences with ambiguous punctuation and a corrected version with clearer phrasing. This helps identify whether an authoring change can solve the problem more reliably than a model setting. A script intended for speech often benefits from spelling out a confusing abbreviation or dividing a long sentence, but those edits still need editorial approval.

For continuity, compare the same character across calm, urgent and explanatory lines. Ask the reviewer to distinguish an intentional style change from a different perceived voice. Keep that judgment descriptive. A universal identity percentage would require a defined measurement method and actual data.

Use a separate category for nonverbal cues. A laugh placed on the wrong phrase can change the speaker's attitude. A sigh can become distracting when repeated. Review these cues in the complete scene rather than scoring isolated sounds as successful features.

Choose a release rule that treats meaningful content errors as blockers. A training clip with a wrong procedure cannot pass because its audio is clean. Likewise, a correct script may need another take if clipping or abrupt joins make it hard to understand. Record which requirement failed so revision work stays focused.

Choose the next investment from the failure pattern

If most failures are wrong words, improve the script and pronunciation cases first. If the words are correct but characters drift, investigate reference preparation and stable mapping. If the generated take is good but playback breaks, work on delivery and buffering. These branches lead to different fixes.

Only expand the scene length or number of characters after the current workflow has a clear review process. A larger scene multiplies the places where references, labels and timing can go wrong. Maintain a short reliable scene as a smoke test whenever you change a model revision, serving runtime or hosted integration.

Document the decision with an approved sample package and rejected cases, not just a promotional reel. The result should tell the next editor what settings were used, what needs listening review and which kinds of lines remain difficult. That is enough to make a careful pilot repeatable without claiming universal production readiness.

Conclusion

Start Drama 3 with one permitted reference and a short scene. Expand to longer dialogue only after character mapping, pronunciation and audio delivery survive revisions. A traceable take history will help your team more than a collection of impressive clips with no reproducible inputs.

FAQ

Can Drama 3 handle multiple characters?

The card documents speaker tags for up to 16 characters. Validate the voice assignments and transitions in the integration you choose.

Are downloadable weights unrestricted for commercial use?

Do not assume that. Review the linked custom license and any hosted-service terms for your intended use, alongside rights to the reference voices.

Is a streamed response ready for publication?

It still needs listening review. Check script accuracy, character consistency and joins between chunks before approving a final audio asset.