High-Performance Local TTS: Implementing On-Device Inference with Piper and ONNX

In the current AI landscape, Text-to-Speech (TTS) is often offloaded to cloud APIs like OpenAI or Google Cloud. However, for applications requiring zero latency, strict data privacy, or offline functionality, cloud dependencies are a bottleneck.

This guide focuses on implementing Piper, a fast, local neural TTS system that leverages ONNX Runtime for high-performance inference on consumer hardware.

Meta Description

Learn how to implement production-ready, local text-to-speech inference in Python using Piper and ONNX Runtime for offline-first applications.

Prerequisites

• Python 3.10+
• ONNX Runtime: For model execution.
• piper-phonemize: For converting text to phoneme sequences.
• NumPy: For audio buffer manipulation.

pip install onnxruntime piper-phonemize numpy

The Problem Context

Most neural TTS models (like Tacotron 2 or VITS) are computationally expensive, often requiring dedicated GPUs to achieve real-time factors (RTF) below 1.0. Piper solves this by using a heavily optimized VITS architecture exported to ONNX, making it capable of running on Raspberry Pi 4 or low-end mobile CPUs while maintaining high vocal quality.

Architecture & Mental Model

The inference pipeline consists of three distinct stages: text normalization, phonemization, and neural synthesis.

graph TD
    A[Raw Text Input] --> B[Text Normalizer]
    B --> C[Phonemizer - piper-phonemize]
    C --> D[Phoneme IDs]
    D --> E[ONNX Runtime - VITS Model]
    E --> F[Raw PCM Audio Buffer]
    F --> G[Audio Playback / Export]

1. Phonemization: Converts text (e.g., "Hello") into phoneme sequences (e.g., /həˈloʊ/) and then into numerical IDs mapped to the model's vocabulary.
2. Synthesis: The ONNX model takes these IDs and generates raw PCM audio data.
3. Post-processing: Converting the raw buffer into a playable format (WAV) or streaming it directly to an audio output.

Step-by-Step Implementation

1. Initializing the Inference Engine

We encapsulate the logic into a LocalTTS class to manage the ONNX session and phonemizer state efficiently.

tts_engine.py

import os
import json
import numpy as np
import onnxruntime as ort
from piper_phonemize import phonemize_espeak

class LocalTTS:
    def __init__(self, model_path: str, config_path: str):
        """
        Initializes the TTS engine with an ONNX model and its config.
        :param model_path: Path to the .onnx file
        :param config_path: Path to the .json config file
        """
        if not os.path.exists(model_path):
            raise FileNotFoundError(f"Model not found at {model_path}")
        
        # Load configuration
        with open(config_path, "r") as f:
            self.config = json.load(f)
            
        self.sample_rate = self.config["audio"]["sample_rate"]
        
        # Initialize ONNX Runtime Session
        # We use CPU as default; for GPU, use ['CUDAExecutionProvider']
        self.session = ort.InferenceSession(
            model_path, 
            sess_options=ort.SessionOptions(),
            providers=['CPUExecutionProvider']
        )

    def synthesize(self, text: str) -> np.ndarray:
        """
        Converts text to raw PCM audio.
        """
        # 1. Phonemization
        phonemes = phonemize_espeak(text, self.config["espeak"]["voice"])
        
        # 2. Map phonemes to IDs (Simplified logic)
        phoneme_ids = self._text_to_ids(phonemes)
        
        # 3. Prepare ONNX inputs
        # Models usually expect phoneme_ids, lengths, and scales
        input_ids = np.array(phoneme_ids, dtype=np.int64)[None, :]
        input_lengths = np.array([input_ids.shape[1]], dtype=np.int64)
        scales = np.array([0.667, 1.0, 0.8], dtype=np.float32) # noise, length, phoneme noise

        # 4. Run Inference
        inputs = {
            "input": input_ids,
            "input_lengths": input_lengths,
            "scales": scales
        }
        
        try:
            audio = self.session.run(None, inputs)[0]
            # Flatten and return as float32 PCM
            return audio.flatten()
        except Exception as e:
            print(f"Inference error: {e}")
            return np.array([], dtype=np.float32)

    def _text_to_ids(self, phonemes):
        # Implementation of ID mapping based on config['phoneme_id_map']
        # Piper models include this map in their JSON config
        id_map = self.config["phoneme_id_map"]
        ids = [id_map.get(p, id_map.get(" ")) for p in phonemes[0]]
        return ids

2. Handling Streamed Output

For production apps, waiting for the full audio buffer can feel slow. We can break the text into sentences and synthesize them in parallel or sequentially to reduce "Time to First Audio" (TTFA).

main.py

import soundfile as sf
from tts_engine import LocalTTS

def main():
    # Paths to local assets
    MODEL_PATH = "en_US-lessac-medium.onnx"
    CONFIG_PATH = "en_US-lessac-medium.onnx.json"
    
    try:
        engine = LocalTTS(MODEL_PATH, CONFIG_PATH)
        text = "At Zavont Labs, we build offline-first applications that respect user privacy."
        
        print("Synthesizing...")
        audio_buffer = engine.synthesize(text)
        
        # Save to disk
        output_file = "output.wav"
        sf.write(output_file, audio_buffer, engine.sample_rate)
        print(f"Audio saved to {output_file}")
        
    except Exception as e:
        print(f"Critical Failure: {e}")

if __name__ == "__main__":
    main()

Performance & Edge Cases

Memory Management

ONNX models for TTS can range from 20MB to 200MB. In resource-constrained environments (like edge devices), ensure you:

1. Reuse the Session: Do not re-instantiate InferenceSession for every request.
2. Quantization: Use int8 quantized models to reduce memory footprint by 4x with minimal quality loss.
3. Thread Limiting: Limit ONNX threads (intra_op_num_threads) to avoid starving the main application thread on low-core CPUs.

Audio Quality vs. Speed

Piper offers three tiers: low, medium, and high.

• Low: ~22kHz, extremely fast, suitable for IoT.
• High: ~44kHz, higher fidelity, requires more CPU cycles.

The Zavont Advantage

At Zavont Labs, we focus on offline-first architecture. By moving TTS from the cloud to the device, we eliminate network latency and the unpredictability of API pricing. This implementation pattern is a cornerstone of how we build resilient, private, and high-performance media tools for our clients.

Conclusion

Local TTS is no longer a compromise. With Piper and ONNX, you can achieve professional-grade voice synthesis entirely on-device. Try implementing a sentence-level queue to handle long-form text and see the performance gains for yourself.

Ready to build something private and fast? Check out our other guides or get in touch.