Skip to content
PatMartinPublic

About

OPAL or Open Pal is a suite of projects providing interchangeable robot parts such as speech, speech recognition, vision and inference.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

21 Commits

Folders and files

Repository files navigation

OPAL

OPAL

Build local-first AI tools that can hear, speak, see, reason, and act. Start with one service. Add another when you need it. A Raspberry Pi can handle voice commands while a separate computer runs the language model.

Setup · Listen · Say · Look · Event · Think · Combine services

Repository What it does
opal-listen Recognize speech, match wakewords and intents, extract slots, and run local reactions.
opal-say Generate speech with Piper or Kokoro. Use voices, profiles, and personas to pick how it sounds.
opal-look Manage vision sources and pipelines, analyze images, and work with observations and media.
opal-event Publish events, match rules, execute registered actions, and inspect what ran.
opal-think Send tasks to Ollama or OpenAI, keep conversation sessions, and call configured Event actions.
opal-api Read the OpenAPI 3.1 contracts and shared kernel for all five services.

Setup

Use Python 3.12 and uv. Clone the service you want to try and open its checkout in two terminals. Run its setup and service start in one terminal, then run the client examples in the other. Run every command on its own. Where a command uses an ID such as TASK_ID, paste the value returned by the preceding command.

Each checkout owns its own opal entry point. Running uv run inside the corresponding checkout keeps the environments separate. Use the service-specific help to see what your checkout exposes.

The examples use local development endpoints. Listen and Look both default to port 8000, so run them one at a time or move Look to another port and pass the same endpoint to its client.

Listen: test a voice command

In the Listen checkout, install the package and start the service. Download and extract the small US English Vosk model first. Replace the example path with the extracted model directory.

uv sync
OPAL_LISTENER_VOSK_MODEL_PATH=/absolute/path/to/vosk-model-small-en-us-0.15 uv run opal-listen-svc

Test the built-in kitchen-light intent as text, then replay a fixture WAV through speech recognition. The text test is useful when you're editing phrases and want to isolate intent matching from microphone quality.

uv run opal listen service ping
uv run opal listen service doctor
uv run opal listen wakeword list
uv run opal listen intent list
uv run opal listen intent get light_turn_on
uv run opal listen slot values room_name
uv run opal listen test text "computer, turn on the kitchen light" --expect-intent light_turn_on
uv run opal listen test audio resources/cmds/cmd-kitchen-lights-on.wav
uv run opal listen test regression

On Debian, Ubuntu, or Raspberry Pi OS, install libportaudio2 for live microphone use. With a microphone attached to the service host, check the devices and listen for 20 seconds. Use the logs and performance summary to investigate a miss.

uv run opal listen service devices
uv run opal listen test listen-debug --duration 20
uv run opal listen log list --no-match --limit 10
uv run opal listen service perf summary

Trigger a local reaction

Run these commands in the same configuration environment as the Listen service. Reaction commands edit its local reactions.yaml. The test below checks the match without executing the action. To try live dispatch, enable reactions and speak the command into the microphone. This example uses /bin/echo on Linux.

uv run opal listen reaction add kitchen-demo --cond "intent_name eq light_turn_on" --run "/bin/echo Kitchen light requested"
uv run opal listen reaction test --text "computer, turn on the kitchen light"
uv run opal listen setting set reactions_enabled true
uv run opal listen test listen --duration 20
uv run opal listen reaction log
uv run opal listen reaction remove kitchen-demo

Say: make a WAV file

In the Say checkout, install Piper support. Download the matching en_US-amy-low.onnx and en_US-amy-low.onnx.json files to ~/piper/voices on the service host. See Say model setup for links and Kokoro setup.

uv sync --extra piper
uv run --extra piper opal-say-svc

The catalog ID is en-us-amy-low. Validate it before generating audio. Open the WAV files in your audio player to compare the two speeds.

uv run --extra piper opal say info ready
uv run --extra piper opal say engine list
uv run --extra piper opal say engine validate piper
uv run --extra piper opal say voice list --engine piper
uv run --extra piper opal say voice validate en-us-amy-low
uv run --extra piper opal say generate "Welcome to the workshop." --voice en-us-amy-low --format wav --output welcome.wav
uv run --extra piper opal say generate "Welcome to the workshop." --voice en-us-amy-low --speed 0.85 --format wav --output welcome-slow.wav
uv run --extra piper opal say profile list
uv run --extra piper opal say persona list

Look: test a vision pipeline

In the Look checkout, use Python 3.12. Use an editable checkout so the client can import its CLI package from src. Add PyYAML to the run environment for YAML configuration support. Port 8002 keeps this example separate from Listen.

OPAL_LOOK_PORT=8002 uv run --python 3.12 --with PyYAML opal serve

Put a JPG named sample.jpg in the Look checkout. Use the built-in synthetic model to exercise the pipeline and inspect persisted observations. Synthetic labels are test data. For RTSP capture and the limits of trained-model inference and live viewing, see Look's capability notes.

uv run --with PyYAML opal look --url http://127.0.0.1:8002 info health ready
uv run --with PyYAML opal look --url http://127.0.0.1:8002 engines list
uv run --with PyYAML opal look --url http://127.0.0.1:8002 models list
uv run --with PyYAML opal look --url http://127.0.0.1:8002 pipelines add --body '{"id":"lab","name":"Image lab","model_id":"synthetic-vision","task":"detection","retention":{"persist_observations":true,"ttl_seconds":3600}}'
uv run --with PyYAML opal look --url http://127.0.0.1:8002 analyze image sample.jpg --request '{"pipeline_id":"lab"}'
uv run --with PyYAML opal look --url http://127.0.0.1:8002 observations list
uv run --with PyYAML opal look --url http://127.0.0.1:8002 media upload sample.jpg --ttl 3600
uv run --with PyYAML opal look --url http://127.0.0.1:8002 media list

Event: turn a message into an action

In the Event checkout, create a new file named demo-event.yaml with the following contents. The server uses its default local endpoint on port 8080. The demo registers a Linux executable and a rule in this configuration.

persistence:
  database: ./demo-event.sqlite3

Start the service with that file.

uv sync
uv run opal-event-svc --config demo-event.yaml

Check health, register the action and rule, then dry-run the rule. Publishing the event triggers the action. The execution history can take a moment to show the result.

uv run opal event info health --ready
uv run opal event actions create --json '{"id":"demo_greet","type":"shell","enabled":true,"mode":"exec","executable":"/bin/echo","args":["Hello, {{data.name}}!"],"stdin":"none","timeout_seconds":10}'
uv run opal event rules create --json '{"id":"demo_greeting_rule","enabled":true,"topic":"demo.greeting","conditions":[{"path":"data.name","operator":"exists","value":true}],"actions":["demo_greet"]}'
uv run opal event rules test demo_greeting_rule --event-json '{"topic":"demo.greeting","data":{"name":"Ada"}}'
uv run opal event publish demo.greeting --source workshop --data-json '{"name":"Ada"}'
uv run opal event list --topic demo.greeting
uv run opal event executions list --rule-id demo_greeting_rule
uv run opal event info stats
uv run opal event backup export --output event-backup.json
uv run opal event backup validate --file event-backup.json

Think: ask a model and track its work

Start Ollama with a downloaded model before trying the reasoning commands. Think uses local Ollama by default at http://127.0.0.1:11434. See Think setup to point it at another host.

uv sync
uv run opal-think-svc

Make a one-off request, then try a task you can inspect independently. Replace TASK_ID with the ID printed by task create.

uv run opal think health --ready
uv run opal think model list
uv run opal think ask "Explain what an event rule does in one sentence."
uv run opal think task create "Give me three names for a workshop voice assistant." --detach --timeout 60
uv run opal think task watch TASK_ID
uv run opal think task show TASK_ID
uv run opal think task list

Use a session for follow-up questions. Paste the ID returned by session add into both requests.

uv run opal think session add --title "Workshop assistant"
uv run opal think ask "My assistant is called Ada. Give her a one-line greeting." --session-id SESSION_ID
uv run opal think ask "Make that greeting shorter." --session-id SESSION_ID
uv run opal think session show SESSION_ID

Combine services

  • Listen → Event. Configure Listen's Event endpoint, then enable publication with uv run opal listen setting set event_publication_enabled true in the Listen checkout. Speak a command with its mic pipeline running. In the Event checkout, run uv run opal event list --topic opal.listen.intent.match before wiring a rule to that topic.
  • Think → Say. Run uv run opal think ask "Write a short welcome for my workshop." in the Think checkout. Paste the response as the text argument to opal say generate in the Say checkout. This makes it easy to adjust the prompt and voice independently.
  • Think → Event. Think can call enabled, registered Event actions when the integration is configured. Put the actual operation and its input constraints in the Event action. See Think's execution semantics.
  • Persona → voice. The shared v1.6.0 persona contract defines a named identity with an optional voice binding outside Say. In Say, list configured personas with uv run --extra piper opal say persona list and pass a configured persona ID with uv run --extra piper opal say generate "I'm ready." --persona PERSONA_ID --output ready.wav. Each service manages its own persona records.

API and more examples

The five reference contracts and the shared kernel currently use v1.6.0. The kernel defines common errors, correlation headers, service identity, personas, and configuration concurrency conventions. Use the service's own --help and its pinned OpenAPI file when checking CLI syntax or HTTP behavior.

Service orchestrator (opal CLI)

This repository also provides opal, a lightweight local orchestrator that runs the OPAL services as independent background processes. Supported on Linux and Windows.

Install

Install it into its own virtual environment — every service checkout ships its own opal entry point, so keep them separate:

uv venv
uv pip install -e ".[dev]"
opal service --help

Then work from a directory containing an opal.yaml (see opal.yaml.example).

Configuration

Precedence: $OPAL_CONFIG if set, otherwise ./opal.yaml in the current directory. Runtime state and logs live under .opal/ next to the config file ($OPAL_STATE_DIR overrides). opal.yaml is gitignored — it contains machine-local paths.

services:
  look:
    command: ["C:/gh/opal/opal-look/.venv/Scripts/python.exe", "-m", "opal_look.main"]
    cwd: "C:/gh/opal/opal-look"          # optional; defaults to the config dir
    host: "127.0.0.1"                     # optional
    port: 8002                            # optional; defaults: look 8002, listen 8000, say 8001, think 8080
    ready_path: "/api/v1/health/ready"    # optional; else a TCP-accept probe is used
    env: {SOME_VAR: "value"}              # optional extra env vars
  • command is always an argv array (never a shell string). First element is the executable — point it at the service virtualenv's Python to keep environments separate. Verified launches: -m opal_look.main, -m opal_listener, -m opal_say, -m opal_think.
  • Host/port injection mechanism (the only one): the orchestrator sets each service's own env vars in the child — OPAL_LOOK_HOST/PORT, OPAL_LISTENER_HOST/PORT, OPAL_SAY_HOST/PORT, OPAL_THINK_HOST/PORT. Do not put --host/--port flags in command.
  • Saved vs active ports: update --port rewrites opal.yaml and applies on the next start/restart; start --port overrides for that launch only and never edits the file. list/status/show display both columns.
  • Writes are atomic (temp file + replace) under a lock (<state>/opal.lock), so concurrent opal invocations cannot corrupt config/state or double-launch.

Commands

Command Effect
opal service list Saved settings + runtime state for all four services
opal service show SVC Full config, runtime record, readiness detail
opal service update SVC --port P Change the saved port (next start/restart)
opal service start SVC [--port P] Background-start one service (bounded startup + readiness wait)
opal service start --all Start every configured service
opal service stop SVC / stop --all Verified, escalating shutdown
opal service restart SVC Stop + start using the saved configuration
opal service status [SVC] Process state + readiness (saved vs active)
opal service logs SVC [--follow] Last 200 lines; --follow streams until Ctrl+C

SERVICE is one of look, listen, say, think. start/stop require exactly one of SERVICE or --all. Exit codes: 0 success, 1 any failed/unavailable operation, 2 usage errors. Bulk operations always attempt every service, print a result per service, and exit nonzero if any failed (partial success is explicit).

Process model

  • Services are spawned detached (start_new_session on POSIX; DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP | CREATE_BREAKAWAY_FROM_JOB on Windows) and survive CLI exit. The Windows breakaway flag escapes job objects that would otherwise kill the service when the launching terminal closes; spawn falls back silently if the job forbids breakaway. stdout/stderr append to .opal/logs/<name>.log — no pipes that could block.
  • Ownership is verified by pid plus process creation time (/proc starttime / GetProcessTimes), never pid alone — OSes reuse pids. Records that fail verification are unknown and are never signaled; pid reuse is reported and the record is dropped.
  • On Windows a venv python.exe is a launcher shim that re-execs the real interpreter, so the spawned pid is not the serving pid. listen/say/think honor OPAL_LISTENER_PID_FILE / OPAL_SAY_PID_FILE / OPAL_THINK_PID_FILE (set to .opal/pids/<name>.pid); look always writes logs/opal_look.pid under its working directory. The orchestrator adopts the pidfile's real server pid — checked at start, and re-adopted lazily by status/stop since pidfiles appear only once the app is serving — so status and shutdown track the process actually bound to the port.
  • status distinguishes running from ready: ready means the configured HTTP ready_path returned 200, or — with no ready_path — that the port accepts TCP (labeled tcp accept, not application readiness). A live pid alone never counts as ready; unverifiable results report unknown.
  • Stop escalation: POSIX sends SIGTERM to the service's process group, waits up to 10s, then SIGKILLs. Windows has no graceful console-signal path for detached processes, so taskkill /T /F terminates the tree immediately. Both cover child processes. restart never starts a replacement after a failed stop.

Troubleshooting

  • config file not found — run from the directory containing opal.yaml, or set OPAL_CONFIG.
  • ... is already in use — the advisory port check found a listener; pick another port (update/--port). The check is advisory: a real bind error still appears in the service log.
  • cannot be verified — a recorded pid can't be identified; the orchestrator refuses to signal it. Inspect opal service status; if the process isn't yours, remove the record from .opal/state.json.
  • readiness unknown — the process runs but didn't verify within the bounded wait; check opal service logs SVC and status again shortly.
  • Corrupt state.json is quarantined to state.json.corrupt automatically.

MVP limitations

Local host only; no crash recovery, boot-time startup, dependency ordering, or supervisor daemon; no remote/REST interface. The Python PYTHONUNBUFFERED=1 env var is set for children unless already defined. Developed and tested on Windows; the Linux paths (process groups, /proc identity, fcntl locking) are implemented but were not exercised in this environment.

About

OPAL or Open Pal is a suite of projects providing interchangeable robot parts such as speech, speech recognition, vision and inference.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages