Under review at ICLR 2027

Continuo: A Large-Scale Dataset for Controllable Long-Form Speech Generation

Anonymous Authors

Paper under double-blind review

Code Testset Audio Samples
Continuo-Dialogue example · 4 speakers · 30 min, one continuous recording 0:00 / 30:00
Figure 1. A long-form dialogue drawn as a score (illustrative). Each staff is one speaker and the waveform marks who holds the floor. Italic markings are natural-language style instructions, and bracketed tags are nonverbal vocalizations (NV). Hover to scrub.

Abstract

Generating long-form speech requires models to keep content, speaker identity, turn-taking, and expressive style consistent over time. Yet most open training corpora contain examples lasting only tens of seconds and open long-form benchmarks cover at most two speakers per recording. We introduce Continuo, a 229K-hour multilingual dataset for controllable long-form speech generation. From in-the-wild data, Continuo constructs short utterances, single-speaker long-form speech, and multi-speaker dialogues, preserving transcripts, speaker identities, pauses, and acoustic transitions. Examples longer than five minutes account for 50.0% of single-speaker long-form hours and 66.8% of dialogue hours. To support expressive control, Continuo provides natural language instructions for 4.77M examples and nonverbal-vocalization tags for 933K examples. We also introduce Continuo-testset, 23 hours of bilingual recordings of up to 30 minutes and four speakers, each speaker paired with reference audio. We train autoregressive (AR) and non-autoregressive (NAR) models on Continuo. Our 0.6B-backbone NAR model achieves the lowest Chinese CER and English WER on TTSD-Eval (5.27%/9.93%) among compared systems, while our AR model is on par with Qwen3-TTS-12Hz-VD on InstructTTSEval (zh/en: 76.2/78.3 vs. 77.1/77.9). These results validate Continuo as a comprehensive data foundation for long-form speech generation and minute-scale expressive control.

Continuo Dataset

Continuo-Pipe turns in-the-wild recordings into three views that share the same source audio. Long-form views keep the continuous recording, including native pauses and transitions between speakers, instead of concatenating trimmed clips. Annotated subsets add natural-language style instructions and NV tags placed among the spoken words.

Continuo-Pipe overview: data construction (speech enhancement, diarization, transcription, merge), additional annotation (instruction and NV annotation), and the three resulting example types.
Figure 2. Overview of Continuo-Pipe. Recordings are enhanced, diarized, transcribed, and merged; segments are annotated with style instructions and NV tags, then reassembled with speaker and timing information into long-form speech and dialogue examples.
Speaker 1 Speaker 2 Extracted example

Figure 3. Three views cut from one illustrative 12-minute source recording, with the size of each view in Continuo. After deduplication, the corpus holds 229,388 hours across 41 languages.

Continuo-testset. 118 human-verified Chinese and English recordings (23.3 h): 42 single-speaker long-form cases and 76 dialogues with 2–4 speakers, up to 30 min each. Every case provides a speaker-attributed script and reference audio for each speaker, and the whole script must be synthesized as a single recording. The testset is available on Hugging Face.

Training Data Preview

Examples from each view of Continuo, drawn from the training set with test-set recordings excluded. Long-form and dialogue examples are continuous recordings that keep native pauses and turn-taking; dialogue transcripts are speaker-attributed.

Browse the full dataset on 🤗 Hugging Face

Instruction Data Preview

Examples from the instruction-annotated subsets of Continuo, each paired with its natural-language style instruction in Chinese or English. Instructions take one of four forms: an acoustic-parameter specification (APS), a descriptive-style directive (DSD), a role-play direction (RP), or a free-form voice and scene description. Long-form instructions can describe how attributes change over the recording; dialogue instructions are given per speaker.

NV Data Preview

Examples from the NV-annotated subsets of Continuo. Multilingual-NVASR transcribes Chinese and English speech together with nonverbal-vocalization (NV) events, such as [Laughter], [Cough], and [Sigh], and this interleaved sequence serves as the training transcript. For long-form and dialogue examples, the NV column lists the speech segments that contain NV events.

Results

We train two models with a 0.6B backbone on Continuo: Continuo-AR (autoregressive with flow matching) and Continuo-NAR (masked generative). Both are competitive with much larger systems on long-form dialogue and learn instruction and NV control, which no open-source baseline supports for long-form speech.

Bold and underlining mark the best and second-best generated-speech scores per column. Parameter counts cover the sequence backbone only.

Audio Samples

Speech generated by Continuo-AR and Continuo-NAR on Continuo-testset. The long-form cases compare whole-sequence inference, which generates the complete script in one pass, with chunked inference, which generates short chunks of about 25–30 s independently and concatenates them. The instruction and NV cases come from the expressive-control evaluation sets built from Continuo-testset cases shorter than 5 min.

BibTeX

@inproceedings{continuo2027,
  title     = {Continuo: A Large-Scale Dataset for Controllable Long-Form Speech Generation},
  author    = {Anonymous},
  booktitle = {Submitted to the International Conference on Learning Representations},
  year      = {2027}
}