eaRS is a Rust-based streaming speech-to-text stack built on Kyutai's models. The tool is now delivered as a single CLI with three responsibilities:
- Server management:
ears server start|stoplaunches and controls the inference backend. - Client capture: Running
earswithout subcommands streams microphone audio to the server and prints live transcripts. - Dictation:
ears dictation start|stopenables system-wide dictation with hotkey control, dictionary replacements, and transcript history. - Companion UI:
ears-ui(seeapps/ears-ui) is an optional native desktop app (GPUI) with a floating dictation island, tray icon and settings for dictionaries, context profiles and overlay preferences. Optional word/subtitle/pointer previews retain the current pill and defaults.
@byteowlz/ears-browser provides standalone
English/German Moonshine CPU/WASM recognition and an explicit eaRS WebSocket
adapter. Callers own capture and UI; browser-local mode needs no speech server.
See the runtime compatibility and browser proof.
eaRS uses just as a command runner. Install it first:
# macOS
brew install just
# Arch Linux
sudo pacman -S just
# Ubuntu/Debian
sudo apt install just
# Or via cargo
cargo install justThen install eaRS with automatic hardware detection:
# Install with all features and auto-detected acceleration
just install-all
# Interactive installation - detects your hardware and offers feature selection
just install-ears
# Or use presets for specific configurations:
just install-ears-metal # macOS with Metal acceleration (Apple Silicon)
just install-ears-cuda # Linux/Windows with NVIDIA GPU (CUDA)
just install-ears-default # CPU only, no acceleration
# Install with custom feature combinations
just install-ears-features "apple,parakeet-rs,hooks"The just install-ears command will:
- Check and install system dependencies (sentencepiece on Linux)
- Detect your hardware (GPU type, Wayland session, etc.)
- Set appropriate environment variables (CUDA paths, Wayland compat, etc.)
- Let you choose which features to enable
- Build and install the binaries
Available features:
| Feature | Description |
|---|---|
nvidia |
CUDA acceleration for NVIDIA GPUs |
apple |
Metal/CoreML acceleration for Apple Silicon |
amd |
ROCm acceleration for AMD GPUs |
directml |
DirectML acceleration for Windows |
parakeet-rs |
Enable Nemotron cache-aware streaming through parakeet-rs |
moonshine |
Optional native CPU Moonshine Tiny Streaming, including local fine-tuned ORT exports |
transcribe-cpp |
Enable ggml GGUF streaming models via transcribe.cpp (built from source, needs cmake; apple/nvidia add Metal/CUDA compute) |
hooks |
Enable shell command hooks for dictation state changes |
If you prefer not to use just:
cargo install --locked --path . # CPU only
cargo install --locked --path . --features apple # Apple Silicon
cargo install --locked --path . --features nvidia # NVIDIA GPU
cargo install --locked --path . --features parakeet-rs # Nemotron streaming
cargo install --locked --path . --features apple,parakeet-rs # Kyutai Metal + Nemotron
cargo install --locked --path . --features apple,transcribe-cpp # Kyutai Metal + transcribe.cpp (GGUF, Metal)On Linux, eaRS uses the system sentencepiece library to avoid protobuf conflicts with ONNX Runtime.
Ubuntu/Debian:
sudo apt-get update
sudo apt-get install -y libsentencepiece-dev sentencepieceFedora/RHEL:
sudo dnf install -y sentencepiece-develArch Linux:
sudo pacman -S sentencepieceNote: The
just install-*commands automatically check for and install sentencepiece.
On macOS, sentencepiece is compiled from source and statically linked during the build process. eaRS also depends on Opus through kaudio; install the system Opus package so audiopus_sys does not fall back to its bundled CMake build:
brew install pkg-config opus cmakeIf you see an audiopus_sys error mentioning Compatibility with CMake < 3.5 has been removed from CMake, Opus was not found by pkg-config. Install the packages above and rebuild. The just check-deps and just install-* recipes install these Homebrew dependencies automatically.
On Windows, sentencepiece is compiled from source and statically linked during the build process. No manual installation required.
eaRS dictation uses the Linux uinput kernel interface for reliable keyboard input. This approach works on both Wayland and X11, unlike X11-only tools like xdotool.
Wayland's security model prevents applications from injecting keystrokes into other windows using traditional X11 methods. eaRS solves this by creating a virtual keyboard device at the kernel level via /dev/uinput, which:
- Works on all Wayland compositors (Sway, Hyprland, KWin, GNOME, etc.)
- Works on X11 as well (unified codebase)
- Is compositor-agnostic (no protocol negotiation needed)
- Provides reliable, low-latency input
Run the automated setup script:
./scripts/setup-dictation-linux.shThe script will:
- Load the uinput kernel module
- Configure uinput to load on boot
- Create a udev rule for proper
/dev/uinputpermissions - Add your user to the
inputgroup - Verify the configuration
You must log out and log back in after setup for group changes to take effect.
If you prefer to configure manually:
# 1. Load the uinput kernel module
sudo modprobe uinput
# 2. Make it load on boot
echo "uinput" | sudo tee /etc/modules-load.d/uinput.conf
# 3. Create udev rule for proper permissions
echo 'KERNEL=="uinput", MODE="0660", GROUP="input", OPTIONS+="static_node=uinput"' | \
sudo tee /etc/udev/rules.d/99-uinput.conf
sudo udevadm control --reload-rules
sudo udevadm trigger
# 4. Reload uinput to apply permissions
sudo rmmod uinput && sudo modprobe uinput
# 5. Add your user to the input group
sudo usermod -a -G input $USER
# 6. Log out and log back in (required!)
# 7. Verify setup
groups | grep input
ls -l /dev/uinputExpected output:
crw-rw---- 1 root input 10, 223 Dec 12 10:00 /dev/uinput
| Issue | Solution |
|---|---|
| "Failed to open /dev/uinput" | Check permissions: ls -l /dev/uinput (should show crw-rw---- 1 root input) |
| Wrong group on /dev/uinput | Create/update udev rule (step 3 above), then reload uinput |
| uinput module not loaded | Run sudo modprobe uinput and verify with lsmod | grep uinput |
| Not in input group | Run groups | grep input - if missing, add yourself and log out/in |
| Dictation startup fails | Check logs at $XDG_STATE_HOME/ears/dictation.log (default: ~/.local/state/ears/dictation.log) |
| Server unreachable on start | ears dictation start now fails fast if the endpoint is not stably reachable; verify configured dictation.servers host/port (or --server URL) |
| Still not working after setup | Reboot to ensure all changes take effect |
[Speech] -> [ears server] -> [WebSocket] -> [ears-dictation]
|
v
[uinput device]
|
v
[/dev/uinput]
|
v
[Linux kernel]
|
v
[Focused application]
The src/virtual_keyboard.rs module provides a cross-platform abstraction:
- Linux: Uses uinput directly (works on Wayland and X11)
- macOS/Windows: Falls back to enigo library
For development builds:
cargo build --release # CPU only
cargo build --release --features apple # Apple Silicon (Metal)
cargo build --release --features nvidia # NVIDIA GPU (CUDA)
cargo build --release --features parakeet-rs # Nemotron streaming
cargo build --release --features "apple,parakeet-rs,hooks" # Typical macOS buildAll binaries are emitted into ./target/release/.
| Command | Description |
|---|---|
just |
Show all available commands |
just install |
Prepare environment (detect hardware, fetch deps) |
just install-ears |
Interactive installation with feature selection |
just install-all |
Install with all features + auto-detected acceleration |
just install-ears-features "f1,f2" |
Install with specific features |
just check-deps |
Check/install system dependencies only |
just build |
Debug build |
just build-release |
Release build |
just check |
Fast compile check (no build) |
just test |
Run tests (uses cargo-nextest) |
just fmt |
Format code |
just clippy / just lint |
Run linter |
just fix |
Auto-fix lint warnings |
just clean |
Clean build artifacts |
just update |
Update dependencies |
just docs |
Generate and open documentation |
# 1. Start the transcription server (runs in the background)
./target/release/ears server start
# 2. Stream your microphone to the server and print live text
./target/release/ears
# 3. (Optional) Enable system-wide dictation
./target/release/ears dictation start
# 4. (Optional) Add speech-specific replacements
./target/release/ears dictionary add \
--replacement "Kyutai" \
--phrase "cute eye" \
--phrase "q tie"Press Ctrl+C in the client to stop streaming. When you are done with the backend:
If the local server is not running, ears will start it automatically before connecting.
./target/release/ears server stopears server start \
[--bind 0.0.0.0:8765] \
[--engine kyutai|parakeet-rs|transcribe-cpp] \
[--hf-repo kyutai/stt-1b-en_fr-candle] \
[--parakeet-rs-model /path/to/nemotron-model] \
[--parakeet-rs-lang auto|en|de|es|ja] \
[--transcribe-cpp-model /path/to/model.gguf] \
[--transcribe-cpp-lang auto|en|de|...] \
[--with-kyutai] [--cpu] [--timestamps] [--vad]
# Restart accepts the same arguments as start.
ears server restart --engine parakeet-rs --parakeet-rs-lang de--bind: Override the default bind address (0.0.0.0:<port-from-config>).--engine: Choose the default engine (kyutai, orparakeet-rs/transcribe-cppwhen compiled with their features). The server refuses to start if the default engine cannot be loaded instead of silently serving another one. The legacy prkt-basedparakeetengine andparakeet-*acceleration features have been removed; for Nemotron useparakeet-rswithappleornvidiaas needed.--with-kyutai: Also load Kyutai when another engine is the default, so clients can switch to it withsetengine. Without it aparakeet-rsserver neither downloads nor loads Kyutai (about 2.6 GB RSS instead of ~5 GB).--hf-repo: Choose a different Kyutai Speech repository.--parakeet-rs-model: Override[parakeet_rs].model_dir. parakeet-rs loads ONNX Runtime >= 1.24 at runtime: setORT_DYLIB_PATH, or the server looks next to theearsbinary, its siblinglib/,/opt/homebrew/lib,/usr/local/liband the usual Linux library directories (brew install onnxruntime/pacman -S onnxruntime).--parakeet-rs-lang: Override[parakeet_rs].language; a dictation client--langcommand overrides it per session.--transcribe-cpp-model: Streaming-capable transcribe.cpp GGUF model (overrides[transcribe_cpp].model_path). Accepts either an existing.ggufpath or a known model slug, which is auto-downloaded from thehandy-computerHugging Face org into the shared HF cache on first use. Append@QUANT(e.g.nemotron-3.5-asr-streaming-0.6b@Q5_K_M) to pick a non-default quant. One session at a time: the model's compute lease is held per active stream.
Browse and pre-download models (requires the transcribe-cpp feature):
ears models list # show known streaming models
ears models pull # fzf-pick a model and download it
ears models pull nemotron # seed the fuzzy picker with a query
ears models pull nemotron-3.5-asr-streaming-0.6b # exact slug: download directly
ears models refresh # update the local catalog to the latest listmodels pull prints the cached .gguf path to stdout (so --transcribe-cpp-model "$(ears models pull ...)" works) and falls back to a numbered prompt when fzf is not installed.
The catalog is data, not code: src/server/catalog.json is baked into the binary (so models list works offline) and generated from the handy-computer Hugging Face org by scripts/gen_catalog.py. A weekly GitHub Action regenerates it and opens a PR, so newly published streaming models get picked up automatically. ears models refresh fetches the maintained catalog into the platform config dir (~/.config/ears/models.json on Linux, ~/Library/Application Support/ears/models.json on macOS), which then overrides the baked copy — so you get new models without reinstalling. Set EARS_CATALOG_URL to point at a different source. Any handy-computer/*-gguf slug also works directly via --transcribe-cpp-model, listed or not.
--transcribe-cpp-lang: Override[transcribe_cpp].languagefor multilingual GGUF models.--cpu: Force Kyutai CPU execution (otherwise CUDA/Metal is used when available).--timestamps: Include word timestamps in the server stream.--vad: Enable Kyutai voice-activity detection.
The server writes a PID file to $XDG_STATE_HOME/ears/server.pid (or ~/.local/state/ears/server.pid). ears server stop stops the local server and associated local dictation. ears server restart keeps dictation alive so it reconnects to the new server automatically.
just install-all includes Moonshine on Apple Silicon and Linux x86_64/arm64.
Other targets skip it because native deployment is not yet qualified. You can
also select the feature explicitly:
just install-ears-features "apple,parakeet-rs,transcribe-cpp,hooks,moonshine" # Mac
# CPU-only trial: just install-ears-features "moonshine"
# Public German Tiny base: downloads once, then reuses the HF cache root.
ears server run --engine moonshine
# Your private fine-tune: no download, upload or silent base-model fallback.
ears server run --engine moonshine --moonshine-model /path/to/native-export
# In another terminal (file transcription or ordinary microphone dictation):
ears --server ws://127.0.0.1:8765 -f sample.wav
ears dictation start --server ws://127.0.0.1:8765 --engine moonshine--moonshine-model overrides [moonshine].model_dir in config.toml. With neither set, the official German Tiny quantized_26_08_24 runtime assets download from download.moonshine.ai into $HF_HOME/moonshine/ (default ~/.cache/huggingface/moonshine/), with pinned SHA-256 checks. These are native ORT assets, not the PyTorch checkpoint on HF; they are not presented as an HF Hub snapshot. HF_HUB_OFFLINE=1 requires a complete verified cache.
A local fine-tune must be a Tiny Streaming native export bundle containing adapter.ort, cross_kv.ort, decoder_kv.ort, encoder.ort, frontend.model.ort, frontend.weights.ort, streaming_config.json, and tokenizer.bin. A Transformers checkpoint or LoRA adapter directory cannot be loaded directly: export/merge it first. Existing private ORT bundles are supported without copying them into the public cache. Keep private weights outside git; all model-file symlink targets must be available on the machine running the server.
The native model processes audio incrementally and emits revisable Interim previews. For safe keyboard insertion, only finalized lines become Word events: expect phrase-level insertion, not Nemotron-style word-by-word typing. One client session at a time; CPU execution; language is fixed by the model, not a session --lang prompt. This export has line-level timing, not exact per-word alignment. This is an opt-in trial, not a claim that fine-tune quality or no-speech regressions are resolved.
Repeated partial agreement still permits later word revisions; it is not a safe commit signal.
The vendored moonshine-rs runtime includes the final-audio repair; pinned native build dependencies download once when not cached. Apple Silicon uses static libraries. On Linux, just install-all and just install-ears-features copy the SDK shared libraries beside the installed binaries; a manual cargo install also needs bash scripts/install-moonshine-runtime.sh (optional arguments: binary directory, Cargo target directory). Windows and Intel Mac native deployment are not yet qualified.
./target/release/ears [--device 1] [--server ws://host:port/] [--timestamps] [--list-devices]
--list-devices: Print available input devices and exit.--device: Select a specific capture device by index.--server: Point the client at a remote server (ws://127.0.0.1:<config-port>/by default).ears-dictation --codec opus: Use the bandwidth-saving Opus transport for a remote dictation server. It overrides[dictation].codec; PCM remains the accuracy-first default.--timestamps: Print the final transcript with per-word timing instead of live text.--codec: Audio transport codec,pcm(default) oropus. Opus cuts upstream bandwidth roughly 30x (~768 kbps raw f32 vs ~25 kbps) with no practical accuracy loss — useful when the server is remote or the link is slow. The client sends a{"type":"setcodec","codec":"opus"}command before streaming ogg-opus frames; third-party clients can do the same.
The client streams raw 24 kHz mono PCM to the server (or ogg-opus with --codec opus) and displays each live word as it appears. When the backend signals completion, the final transcript (and optional timestamps) is printed.
eaRS can apply a fast dictionary pass to dictation output before typing. This is useful for project names, product names, acronyms, and words that speech-to-text engines consistently mishear.
Dictionary entries map multiple observed phrases to one canonical replacement:
version = 1
[[entries]]
replace = "Kyutai"
phrases = ["cute eye", "q tie", "kyu tai"]
[[entries]]
replace = "byteowlz"
phrases = ["byte owls", "bite owls", "by alts"]The default global dictionary lives at:
$XDG_CONFIG_HOME/ears/dictionaries/global.toml
# or ~/.config/ears/dictionaries/global.toml
Manage it with:
ears dictionary list
ears dictionary add --replacement "oqto" --phrase "octo" --phrase "O Q T O"
ears dictionary remove --phrase "octo"
ears dictionary remove --replacement "oqto"
ears dictionary test "cute eye inside pie agent"dictionary add is flag-based on purpose: --replacement is the canonical text, and each --phrase is an observed phrase to replace. Repeated adds for the same replacement merge into the existing entry. If a phrase already belongs to another entry, it is moved so each phrase has one owner. Use dictionary remove --phrase "..." to remove observed phrases, or dictionary remove --replacement "..." to remove an entire canonical entry.
ears-dictation hot-reloads dictionary files when they change. Multi-word phrases may span separate streaming events: ordinary text is typed immediately, while a possible dictionary-phrase prefix of at least two characters is held for up to 1.5 seconds awaiting the next word. Single letters remain immediate. After installing a new eaRS binary you still need to restart ears-dictation once; after that, dictionary edits are picked up automatically.
The ingress boundary VAD flushes the final uncommitted word after 300 ms of silence by default. Adjust the latency/phrase-pause trade-off with ears server run --boundary-vad-hangover-ms <MS> (or the equivalent start/restart command).
Dictation transcript history is enabled by default. Live word events and final events are buffered in memory and periodically appended to JSONL files:
$XDG_STATE_HOME/ears/transcripts/YYYY-MM-DD.jsonl
# or ~/.local/state/ears/transcripts/YYYY-MM-DD.jsonl
Unchanged live words omit the replaced field by default:
{"ts":1782720576,"type":"word","raw":"and","changed":false}
{"ts":1782720576,"type":"word","raw":"cute eye","replaced":"Kyutai","changed":true}Buffers flush every few seconds, after enough events, on disconnect, and on shutdown. This avoids writing to disk for every live word while keeping recent dictation persisted.
[dictation.auto_pause] idle_ms = 3000 requests a pause after three seconds of
capture-VAD silence and rearms wake locally, independently of ASR support or
connection. Pending transcription finishes separately; legacy/unconfirmed tails
are reported truthfully, not treated as timer success. Recording retention is
independent, and a held push-to-talk key is never cut. Default: off.
Tune speech_threshold (default 0.5) in config.toml or Settings → Dictation →
Auto-pause; higher rejects more noise but may miss quiet speech.
See configuration, protocol and qualification limits.
With [dictation.audio_history] enabled = true, dictation also keeps the audio of every committed utterance, so your own dictation becomes training and evaluation data (your voice, microphones and vocabulary). It is off by default and never enabled implicitly. While it is on, the "Dictation enabled" notification says (audio history on) and ears dictation status reports it.
$XDG_STATE_HOME/ears/audio-history/index.jsonl # one record per utterance
$XDG_STATE_HOME/ears/audio-history/audio/YYYY-MM-DD/<id>.wav # mono 16-bit, 24 kHz
Each record holds the utterance ID, UTC time, session ID, engine, backend, language, input device, duration, SHA-256, raw and dictionary-replaced transcript, the inserted text and the target app's bundle id (never a window title), with label_status = "unreviewed" and training_eligible = false.
- Local only: directories are
0700, files0600, and a path inside a git repository is refused. - Nothing is kept while an excluded app (password managers by default) is frontmost or while macOS Secure Event Input is active.
max_total_mbandmax_age_daysdelete the oldest utterances first.- Only enable it for your own speech; do not retain meetings or other people. Keeping audio is not consent to voice cloning, speaker enrollment, upload or redistribution.
ears dictation history path # directory and whether retention is on
ears dictation history list [--json]
ears dictation history audio-path ID # verified local WAV path for playback, e.g. afplay on macOS
ears dictation history export --out DIR # copies audio + manifest.jsonl with session-disjoint split hints
ears dictation history purge --id ID | --session ID | --before YYYY-MM-DD | --allFor capture-owned acoustic boundaries instead of ASR-arrival slicing, see client-side VAD. This optional mode saves untranscribed clips and keeps original recognition as explicitly unassigned suggestions; it never silently creates training pairs. Microphone conversion preserves continuous resampling state and original amplitude.
Utterance audio is cut from the first recognised word (minus pre_roll_ms) to the server's end-of-speech boundary, by arrival time. It is suitable as training data but is not authoritative speaker-gate evidence.
Runtime configuration lives at:
$XDG_CONFIG_HOME/ears/config.toml
# or ~/.config/ears/config.toml
Key sections:
[storage]: Override model cache directories and reference audio location.[parakeet_rs]: Set the default Nemotron model directory and multilingual language prompt. CLI server flags override these values; client--langoverrides the language per session.[transcribe_cpp]: Set the default transcribe.cpp GGUF model path and language hint (requires thetranscribe-cppfeature).[dictation].codec: Set"pcm"(default, accuracy-first) or"opus"(bandwidth-saving for remote servers).ears-dictation --codecoverrides the config value.[server]: Default WebSocket port used byears server startand the capture client.[dictation]: Enable live typing and configure in-app hotkeys.- Linux specifics (tray, Hyprland window rules, microphone,
uinputtyping): docs/linux.md. [hotkeys]: Set hotkey bindings and mode (toggle,push_to_talk, orhybrid).[hotkeys].push_to_talk: Add a dedicated hold-to-talk key alongside any mode, including modifier-only ones (right_option,right_cmd,fn) andcmd+…chords. Hold, speak, release to insert; a release under 300 ms, or another key during a modifier-only hold, discards the take instead of inserting it (the Hex app's rule).[dictation]:start_paused = true|falsecontrols whether dictation starts paused or live.[dictation.notifications]: Toggle desktop popups and customise start/pause/stop messages shown for dictation state changes.[dictation.hooks](requirescargo build --features hooks): Run shell commands on start, pause, or stop transitions (e.g., change colours in status bars).[replacement]: Enable dictionary replacement, configure dictionary paths, and choose case-sensitive matching.[transcripts]: Enable transcript JSONL history and configure buffering.
Example engine, replacement, and transcript settings:
[parakeet_rs]
model_dir = "~/Models/parakeet-rs/nemotron-3.5-asr-streaming-0.6b-onnx"
language = "auto"
[replacement]
enabled = true
dictionary_paths = ["~/.config/ears/dictionaries/global.toml"]
case_sensitive = false
[transcripts]
enabled = true
path = "default"
store_raw = true
store_replaced = true
store_unchanged_replaced = false
flush_interval_ms = 2000
flush_max_events = 50If the file does not exist, it is created on first run together with the reference audio bundle.
The server emits JSON events:
{"type":"word","word":"hello","start_time":1.23,"end_time":null}– live word updates.{"type":"final","text":"…","words":[…]}– final transcript with timestamp list.
Clients may send {"type":"stop"} to end the current session (the capture client does this automatically when interrupted).
ears server startreports "already running" – Useears server stopto terminate the existing instance. If the PID no longer exists,stopwill clean up the stale PID file.- Client prints "failed to connect" – Ensure the server is running and reachable at the URL passed via
--server(check the configured port). - High latency – Run the server on the same machine as the client or enable GPU acceleration (
--features cudaor--features metal).
