docs: latency budget, thread cap, and Arc B580 CUDA mismatch
Also extend bench_enhance.py with --stereo --hop for isolated RTF.
This commit is contained in:
@@ -74,13 +74,20 @@ re-run `--write` unless you intend to remap slots.
|
||||
C920 mics are 16 kHz **stereo**. Each hop (default 200 ms) is kept
|
||||
stereo through a SpeechBrain preprocessor, then published as mono:
|
||||
|
||||
1. `AudioNormalizer(mix="keep")` at 16 kHz
|
||||
2. Delay-and-Sum beamform (GCC-PHAT TDOA) — the live path from the
|
||||
SpeechBrain [multi-microphone beamforming tutorial](https://speechbrain.readthedocs.io/en/latest/tutorials/preprocessing/multi-microphone-beamforming.html)
|
||||
(`speechbrain.lobes.beamform_multimic.DelaySum_Beamformer`)
|
||||
3. `speechbrain/metricgan-plus-voicebank` neural enhancement on a 1 s
|
||||
1. Delay-and-Sum beamform (GCC-PHAT TDOA, frozen for
|
||||
`BEAMFORM_TDOA_EVERY=5` hops) — live path from the SpeechBrain
|
||||
[multi-microphone beamforming tutorial](https://speechbrain.readthedocs.io/en/latest/tutorials/preprocessing/multi-microphone-beamforming.html)
|
||||
2. `speechbrain/metricgan-plus-voicebank` neural enhancement on a 1 s
|
||||
rolling window (`ENHANCE_CHUNK_S=1.0`, `ENHANCE_HOP_S=0.2`)
|
||||
|
||||
The 1 s window is **past context**, not +1 s delay: only the latest hop
|
||||
is published. Algorithmic delay ≈ hop + compute.
|
||||
|
||||
Each worker caps Torch/BLAS to `TORCH_NUM_THREADS=1` (default) so 14–20
|
||||
processes do not oversubscribe the i7-14700. This host has an Intel Arc
|
||||
B580; the venv Torch is NVIDIA CUDA and **cannot** use it. Do not set
|
||||
`run_opts={"device":"cuda"}`.
|
||||
|
||||
One worker process per camera loads the modules once and reuses them.
|
||||
`enhance_batch` must be called with `lengths=torch.tensor([1.0])`
|
||||
(relative, not sample count). `ENHANCE_MODE=off` (or a failed model
|
||||
@@ -89,7 +96,8 @@ instead of Delay-and-Sum. DSP high-pass + soft-clip is the last-resort
|
||||
fallback so streams still work.
|
||||
|
||||
Until the 1 s window is full, hops use the DSP fallback so LiveKit is
|
||||
not silent on startup.
|
||||
not silent on startup. Workers log `process_ms` / `capture_ms` every 50
|
||||
hops.
|
||||
|
||||
## Appliance boot / power button
|
||||
|
||||
|
||||
Reference in New Issue
Block a user