A tool to produce actor-accurate dubs on a local PC that keeps the original music and effects intact while replacing dialogue in another language or accent. An AI coding agent directs the process line-by-line: it assigns speakers using voice- and frame-matching, corrects transcripts, times translations to each actor’s delivery, and prompts a voice engine so each line is acted and timed like the original. Rendered lines are transcribed back for QA and retried up to three times; between lines the original soundtrack is preserved. Outputs are a single video with the dub as the default track, the original audio as a secondary track, and bilingual subtitles. Supported pairs include Japanese/Spanish/Korean/Chinese → English and English → Japanese/Spanish, and agents can add more pairs.
The pipeline and tooling are concrete and reproducible: it uses VoiceStudio plus OmniVoice/IndexTTS/Chatterbox for cloning, Whisper large-v3 (with secondary recognizers) for transcription, and Mel-Band Roformer for music/FX separation. Work is split into ~3-minute parts, mixed without re-encoding, and assembled into a final MKV. Requirements: Windows with an NVIDIA GPU (≥6 GB; 16 GB recommended), large disk overhead (~40 GB for models, ~85 GB during encode), and an AI agent runtime (Claude Code, Codex, etc.). Runtime for a full movie is roughly seven GPU-hours on an RTX 4090. Everything runs locally (agent transcripts may be sent where the agent runs); documentation, demos, and Python scripts let you run or customize the pipeline without the agent. Responsible-use and rights/consent guidance are included.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.