Click to see the available input types for synthesize request.
The `input` parameter can be a string or, alternatively, an array
of strings or inputs items.
Type | Description | Example
---|---|---
`text` | Speak the text in `value`. This is equivalent to giving a pure string input. | {
type: "text",
value: "This is an example."
}
`speech` | Speak the text in `value` with corresponding subtitles in `subtitles` (optional). This type allows the spoken words to be different that the subtitles. | {
type: "text",
value: "One two three",
subtitles: "123"
}
`phonetic` | Speak the model specific phonetic alphabets in `value` with corresponding `subtitles` (optional). | {
type: "phonetic",
value: "mˈɜɹʧəndˌIz",
subtitles: "merchandise"
}
`characters` | Speak the `value` character-by-character with corresponding `subtitles` (optional). Supports also numbers that are read digit-by-digit. | {
type: "characters",
value: "ABC-123-8",
subtitles: "ABC-123-8"
}
`number` | Speak the number in `value` with corresponding `subtitles` (optional). The number should presented as a string. | {
type: "number",
value: "123.5",
subtitles: "123.5"
}
`date` | Speak the date in `value` with corresponding `subtitles` (optional). The date is presented as milliseconds from epoch. | {
type: "date",
value: Date.now(),
subtitles: "02/05/2025"
}
`time` | Speak the time in `value` with corresponding `subtitles` (optional). The time is presented as milliseconds from epoch. | {
type: "time",
value: Date.now(),
subtitles: "6:45 PM"
}
`break` | The length of the break in milliseconds in `value` with corresponding `subtitles` (optional). | {
type: "break",
value: 2000,
subtitles: "..."
}
*TODO: Add support for `audio` type.*
An example using an array of input items:
```javascript
{
type: "synthesize",
id: 14, // Unique request identifier.
data: {
input: [
"There were ",
{ type: "speech", value: "over two hundred ", subtitles: ">200 " },
"items of",
{ type: "phonetic", value: "mˈɜɹʧəndˌIz ", subtitles: "merchandise " },
"on sale."
]
}
}
```
---
# NodeJS WebSocket/RESTful Server: `headtts-node.mjs`
Install using NPM (requires Node.js v20+):
```bash
npm install @met4citizen/headtts
```
Start the server:
```bash
npm start
```
HeadTTS
**HeadTTS** is a free JavaScript text-to-speech (TTS) solution that
provides timestamps and Oculus visemes for lip-sync, in addition
to audio output (WAV/PCM). It uses
[Kokoro](https://huggingface.co/onnx-community/Kokoro-82M-v1.0-ONNX-timestamped)
neural model and voices, and inference can run entirely in
the browser (via WebGPU or WASM), or alternatively
on a Node.js WebSocket/RESTful server (CPU-based).
- **Pros**: Free. Doesn't require a server in in-browser mode.
WebGPU support. Uses neural voices with a StyleTTS 2 model.
Great for lip-sync use cases and fully compatible with the
[TalkingHead](https://github.com/met4citizen/TalkingHead).
MIT licensed, doesn't use eSpeak NG or any other GPL-licensed
module.
- **Cons**: WebGPU is only supported by default in Chrome and Edge
desktop browsers. Takes time to load (the first time) and uses
a lot of memory. WebGPU support in onnxruntime-node is still experimental
and not released, so server-side inference is relatively slow. English
is currently the only supported language.
**If you're using a Chrome or Edge desktop browser, check out the
[In-browser Demo](https://met4citizen.github.io/HeadTTS/)!**
The project uses [websockets/ws](https://github.com/websockets/ws) (MIT License),
[hugginface/transformers.js](https://github.com/huggingface/transformers.js/)
(Apache 2.0 License) and
[onnx-community/Kokoro-82M-v1.0-ONNX-timestamped](https://huggingface.co/onnx-community/Kokoro-82M-v1.0-ONNX-timestamped)
(Apache 2.0 License) as runtime dependencies. For information on
language modules and dictionaries, see Appendix B. Using
[jest](https://jestjs.io) for testing.
You can find the list of supported English voices and voice samples
[here](https://huggingface.co/onnx-community/Kokoro-82M-v1.0-ONNX-timestamped#voicessamples).
---
# In-browser Module: `headtts.mjs`
The HeadTTS JavaScript module enables in-browser text-to-speech
using Module Web Workers and WebGPU/WASM inference. Alternatively, it can
connect to and use the HeadTTS Node.js WebSocket/RESTful server.
Create a new `HeadTTS` class instance:
```javascript
import { HeadTTS } from "./modules/headtts.mjs";
const headtts = new HeadTTS({
endpoints: ["ws://127.0.0.1:8882", "webgpu"], // Endpoints in order of priority
languages: ['en-us'], // Language modules to pre-load (in-browser)
voices: ["af_bella", "am_fenrir"] // Voices to pre-load (in-browser)
});
```