# WaveGen: Truly End-to-End Waveform Generation via Internal Spectral Trajectory Learning

**Authors:** Sang-Hoon Lee, Ha-Yeong Choi  
**Lab:** Ajou university · **Published:** 9 Nov 2026 · **Upvotes:** 17 · **Level:** Advanced  
**Links:** [arXiv](https://arxiv.org/abs/2611.12527) · [PDF](https://arxiv.org/pdf/2611.12527) · [Code](https://github.com/sh-lee-prml/WaveGen) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2611.12527.html)

## Curator's notes

**TL;DR:** WaveGen directly generates speech waveforms from text using a diffusion model that learns internal spectral trajectories.

- **What's new:** It removes all intermediate modules like spectrogram vocoders, duration predictors, and self‑supervised embeddings, training a single end‑to‑end model.
- **Key result:** WaveGen achieves a word error rate of 1.47 and a similarity score of 0.75 on the Seed‑en benchmark.
- **Why it matters:** Builders can simplify TTS pipelines to a single model, reducing engineering overhead and latency.

## Key numbers

- **0.6B**: …the scalability of end-to-end TTS models by scaling the model size from 0.1B to 0.6B parameters.

## Abstract

This work proposes WaveGen, a fully end-to-end diffusion-based text-to-waveform framework based on Spectral Forcing, which jointly learns an internal spectral trajectory and the target waveform trajectory within a single model, enabling direct end-to-end text-to-waveform generation. Unlike recent TTS pipelines, WaveGen does not rely on separately trained or externally decoded intermediate representations, such as neural audio codecs, Mel-spectrogram vocoders, or VAE-based latent models. In addition, WaveGen performs internal text-speech alignment within a single model, eliminating the need for external alignment modules such as duration predictors. To preserve a truly end-to-end formulation, our framework further avoids dependence on self-supervised representations, such as BERT-based text embeddings and Wav2Vec 2.0-based semantic representations, as well as semantic distillation methods for training optimization. To achieve this, we carefully design Spectral Forcing, a model architecture with disentangled diffusion heads, and a training objective based on a multi-scale spectral $v$-loss. We further validate the scalability of end-to-end TTS models by scaling the model size from 0.1B to 0.6B parameters. Using a single architecture, open-source data, and one-stage training, without external modules such as duration predictor or REPA, WaveGen demonstrates fully end-to-end waveform generation achieving state-of-the-art performance on the Emilia training dataset. In particular, WaveGen achieves a WER of 1.47 and a SIM of 0.75 on the Seed-en benchmark. Source code and checkpoints are available at https://github.com/sh-lee-prml/WaveGen

## What you could build

1. Create a low‑latency voice assistant using WaveGen for on‑device speech synthesis.
2. Develop a multilingual TTS service by fine‑tuning WaveGen on new language data.
3. Build an audio book generator that directly converts text chapters to high‑quality waveforms.

## Quiz yourself

What two performance metrics does WaveGen report on the Seed‑en benchmark?

**Answer:** It reports a word error rate (WER) of 1.47 and a similarity (SIM) score of 0.75.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.