Xuan-Son Nguyen
5c0b49a23b
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* mtmd: add audio out stream api * add /tts endpoint * add docs * wire up mtmd_helper_model_can_chat * upload speaker_ref via form-data * expose seed to api * fix header for pcm * wip * fix penalty_last_n * fix server_output_limits * fix pocket-tts stream * wip * add server_schema for tts * add server routing * server: coerce seed and stream from multipart form fields Form fields arrive as strings, so -F seed=42 and -F stream=true were rejected by the schema. seed joins the integer fields, parsed as a long long so a full uint32 seed fits, and stream is read as a boolean when it comes as a string. --------- Co-authored-by: Pascal <admin@serveurperso.com>
2.1 KiB
2.1 KiB
llama.cpp TTS
This is a tool to demonstrate audio generation capability in llama.cpp via libmtmd. It was added via PR #26254
Note: this tool used to serve as a demo for OuteTTS, but it was converted to a more model-agnostic tool.
Common usage
Simple usage:
llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF -p "Hello world" --output out.wav
Note: it's highly recommended to always provide a speaker reference voice (via --tts-speaker-file); otherwise, the model's performance may be degraded.
Common params:
- Sampling params such as
--top-k,--top-p,--temp, etc. -n <number_of_frames>limits the output length, e.g.-n 500. Note that how many milliseconds each frame represents varies by model- Core inference params such as
-ngl,-b,-ub, etc.
Qwen3-TTS
Available params:
--tts-langcan bezh,en,de,it,pt,es,ja,ko,fr,ru(default:en)--tts-speaker-fileshould point to a speaker reference audio file (wav, mp3)
Example usage:
llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF \
-p "Hello world" \
--tts-lang english \
--tts-speaker-file speaker.mp3 \
--output out.wav
Pocket TTS
Available params:
--tts-speaker-fileshould point to a speaker reference audio file (wav, mp3). It is required, the model produces almost no audio without it- Note:
langis not used, the language is a property of the weights
Example usage:
llama-tts -m pocket-tts.gguf \
-mm mmproj-pocket-tts.gguf \
-p "Hello world" \
--tts-speaker-file speaker.mp3 \
--output out.wav
Note for GGUF conversion:
The upstream repository holds one complete model per language under languages/, next to a set of shared files at the root. Convert one of the languages/<name> directories, not the root directory:
python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --outfile pocket-tts.gguf
python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --mmproj --outfile mmproj-pocket-tts.gguf