feat: OpenVINO CPU MetricGAN mask; 100 ms hop

Per-worker OpenVINO CPU runs the MetricGAN BLSTM mask; SpeechBrain
keeps STFT/ISTFT. ENHANCE_INFER=auto with torch fallback. Hop is
0.1 s after isolated last_ms ~6 ms (model+ov+beamform).
This commit is contained in:
root
2026-08-28 07:25:30 +00:00
parent 42443420be
commit ef9573c294
9 changed files with 211 additions and 26 deletions
+9 -6
View File
@@ -101,22 +101,25 @@ HDMI-A-7 is replaced by the grid).
## Audio cleanup
C920 mics are 16 kHz **stereo**. Each hop (default 200 ms) is kept
C920 mics are 16 kHz **stereo**. Each hop (default 100 ms) is kept
stereo through a SpeechBrain preprocessor, then published as mono:
1. Delay-and-Sum beamform (GCC-PHAT TDOA, frozen for
`BEAMFORM_TDOA_EVERY=5` hops) — live path from the SpeechBrain
[multi-microphone beamforming tutorial](https://speechbrain.readthedocs.io/en/latest/tutorials/preprocessing/multi-microphone-beamforming.html)
2. `speechbrain/metricgan-plus-voicebank` neural enhancement on a 1 s
rolling window (`ENHANCE_CHUNK_S=1.0`, `ENHANCE_HOP_S=0.2`)
rolling window (`ENHANCE_CHUNK_S=1.0`, `ENHANCE_HOP_S=0.1`).
The MetricGAN **mask** runs on OpenVINO CPU (`ENHANCE_INFER=auto`);
STFT/ISTFT stay in Torch. `ENHANCE_INFER=torch` forces SpeechBrain
`enhance_batch`. Do not use the Arc B580 for this net.
The 1 s window is **past context**, not +1 s delay: only the latest hop
is published. Algorithmic delay ≈ hop + compute.
Each worker caps Torch/BLAS to `TORCH_NUM_THREADS=1` (default) so 14–20
processes do not oversubscribe the i7-14700. This host has an Intel Arc
B580; the venv Torch is NVIDIA CUDA and **cannot** use it. Do not set
`run_opts={"device":"cuda"}`.
Each worker caps Torch/BLAS/OpenVINO to `TORCH_NUM_THREADS=1` (default)
so 14–20 processes do not oversubscribe the i7-14700. This host has an
Intel Arc B580; the venv Torch is NVIDIA CUDA and **cannot** use it. Do
not set `run_opts={"device":"cuda"}`.
One worker process per camera loads the modules once and reuses them.
`enhance_batch` must be called with `lengths=torch.tensor([1.0])`