docs: latency budget, thread cap, and Arc B580 CUDA mismatch

Also extend bench_enhance.py with --stereo --hop for isolated RTF.
This commit is contained in:
root
2026-08-28 06:21:02 +00:00
parent 36c4bbc956
commit f83f0f0683
2 changed files with 68 additions and 22 deletions
+14 -6
View File
@@ -74,13 +74,20 @@ re-run `--write` unless you intend to remap slots.
C920 mics are 16 kHz **stereo**. Each hop (default 200 ms) is kept
stereo through a SpeechBrain preprocessor, then published as mono:
1. `AudioNormalizer(mix="keep")` at 16 kHz
2. Delay-and-Sum beamform (GCC-PHAT TDOA) — the live path from the
SpeechBrain [multi-microphone beamforming tutorial](https://speechbrain.readthedocs.io/en/latest/tutorials/preprocessing/multi-microphone-beamforming.html)
(`speechbrain.lobes.beamform_multimic.DelaySum_Beamformer`)
3. `speechbrain/metricgan-plus-voicebank` neural enhancement on a 1 s
1. Delay-and-Sum beamform (GCC-PHAT TDOA, frozen for
`BEAMFORM_TDOA_EVERY=5` hops) — live path from the SpeechBrain
[multi-microphone beamforming tutorial](https://speechbrain.readthedocs.io/en/latest/tutorials/preprocessing/multi-microphone-beamforming.html)
2. `speechbrain/metricgan-plus-voicebank` neural enhancement on a 1 s
rolling window (`ENHANCE_CHUNK_S=1.0`, `ENHANCE_HOP_S=0.2`)
The 1 s window is **past context**, not +1 s delay: only the latest hop
is published. Algorithmic delay ≈ hop + compute.
Each worker caps Torch/BLAS to `TORCH_NUM_THREADS=1` (default) so 14–20
processes do not oversubscribe the i7-14700. This host has an Intel Arc
B580; the venv Torch is NVIDIA CUDA and **cannot** use it. Do not set
`run_opts={"device":"cuda"}`.
One worker process per camera loads the modules once and reuses them.
`enhance_batch` must be called with `lengths=torch.tensor([1.0])`
(relative, not sample count). `ENHANCE_MODE=off` (or a failed model
@@ -89,7 +96,8 @@ instead of Delay-and-Sum. DSP high-pass + soft-clip is the last-resort
fallback so streams still work.
Until the 1 s window is full, hops use the DSP fallback so LiveKit is
not silent on startup.
not silent on startup. Workers log `process_ms` / `capture_ms` every 50
hops.
## Appliance boot / power button