feat: known-good CPU audio + 1080p VAAPI + KMS display wall

Working snapshot before the OpenVINO MetricGAN GPU spike:
thread-capped SpeechBrain DelaySum+MetricGAN on CPU, hop 200 ms,
video hold for A/V sync, Arc VAAPI H.264, headless kmssink wall.
32 tests passing. Do not treat this as a GPU audio path.
This commit is contained in:
root
2026-08-28 07:14:21 +00:00
parent 11776adf52
commit 736eccb932
17 changed files with 1421 additions and 46 deletions
+20 -16
View File
@@ -5,7 +5,8 @@ LiveKit room.
Each camera becomes one LiveKit participant (`cam-01` .. `cam-20`) that
publishes:
- a camera video track (1280x720 @ 30 fps by default, MJPG from V4L2)
- a camera video track (640x360 @ 30 fps MJPG, H.264 VAAPI — sized
for low-latency wall decode, not max quality)
- a microphone audio track (16 kHz mono, cleaned with speechbrain)
Identity is the USB serial, pinned by udev (`/dev/camNN` + `slots.json`).
@@ -72,28 +73,31 @@ re-run `--write` unless you intend to remap slots.
## Headless displays (LiveKit -> GStreamer kmssink)
This host has no X11/Wayland. `run_displays.py` joins the LiveKit room as
`display-wall` (subscribe-only) and pipes each camera's video into a
GStreamer `kmssink` on a DRM connector.
`display-wall` (subscribe-only) and drives three DRM roles with GStreamer
`kmssink`:
Default mapping: `cam-01`, `cam-02`, `cam-03` onto the first 3 DRM heads
(connected first, Arc `xe` HDMI preferred). Today HDMI-A-7 is the live 4K
panel; plug two more HDMI cables into the Arc card (`HDMI-A-5`/`HDMI-A-6`)
and they light up on the same process.
1. **grid** — 5x4 mosaic of cam-01..cam-20
2. **speaker** — designated speaker camera (`DISPLAY_SPEAKER_CAMERA`, default
`cam-01`; set to `active` to follow LiveKit active speaker)
3. **screenshare** — stub placeholder until the production LiveKit server
is online (no subscribe, no speaker login yet)
Default connector order: connected Arc HDMI first (today HDMI-A-7 is live;
HDMI-A-5 / HDMI-A-6 light up when cabled).
```bash
.venv/bin/python run_displays.py --list
.venv/bin/python run_displays.py --test-pattern # smpte/ball/snow, no LiveKit
.venv/bin/python run_displays.py # subscribe cam-01..03
.venv/bin/python run_displays.py --connectors HDMI-A-7,HDMI-A-6,HDMI-A-5
.venv/bin/python run_displays.py --test-pattern
.venv/bin/python run_displays.py
.venv/bin/python run_displays.py --speaker-camera cam-07
sudo cp deploy/livekit-displays.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now livekit-displays
sudo systemctl restart livekit-displays
journalctl -u livekit-displays -f
```
`DISPLAY_IDENTITIES` / `DISPLAY_CONNECTORS` / `DISPLAY_COUNT` in `.env`.
kmssink takes the DRM master for those connectors (the framebuffer console
on HDMI-A-7 will be replaced by camera video).
`.env`: `DISPLAY_ROLES`, `DISPLAY_CONNECTORS`, `DISPLAY_SPEAKER_CAMERA`,
`DISPLAY_GRID_*`. kmssink takes the DRM master (framebuffer console on
HDMI-A-7 is replaced by the grid).
## Audio cleanup