feat: OpenVINO CPU MetricGAN mask; 100 ms hop
Per-worker OpenVINO CPU runs the MetricGAN BLSTM mask; SpeechBrain keeps STFT/ISTFT. ENHANCE_INFER=auto with torch fallback. Hop is 0.1 s after isolated last_ms ~6 ms (model+ov+beamform).
This commit is contained in:
@@ -101,22 +101,25 @@ HDMI-A-7 is replaced by the grid).
|
||||
|
||||
## Audio cleanup
|
||||
|
||||
C920 mics are 16 kHz **stereo**. Each hop (default 200 ms) is kept
|
||||
C920 mics are 16 kHz **stereo**. Each hop (default 100 ms) is kept
|
||||
stereo through a SpeechBrain preprocessor, then published as mono:
|
||||
|
||||
1. Delay-and-Sum beamform (GCC-PHAT TDOA, frozen for
|
||||
`BEAMFORM_TDOA_EVERY=5` hops) — live path from the SpeechBrain
|
||||
[multi-microphone beamforming tutorial](https://speechbrain.readthedocs.io/en/latest/tutorials/preprocessing/multi-microphone-beamforming.html)
|
||||
2. `speechbrain/metricgan-plus-voicebank` neural enhancement on a 1 s
|
||||
rolling window (`ENHANCE_CHUNK_S=1.0`, `ENHANCE_HOP_S=0.2`)
|
||||
rolling window (`ENHANCE_CHUNK_S=1.0`, `ENHANCE_HOP_S=0.1`).
|
||||
The MetricGAN **mask** runs on OpenVINO CPU (`ENHANCE_INFER=auto`);
|
||||
STFT/ISTFT stay in Torch. `ENHANCE_INFER=torch` forces SpeechBrain
|
||||
`enhance_batch`. Do not use the Arc B580 for this net.
|
||||
|
||||
The 1 s window is **past context**, not +1 s delay: only the latest hop
|
||||
is published. Algorithmic delay ≈ hop + compute.
|
||||
|
||||
Each worker caps Torch/BLAS to `TORCH_NUM_THREADS=1` (default) so 14–20
|
||||
processes do not oversubscribe the i7-14700. This host has an Intel Arc
|
||||
B580; the venv Torch is NVIDIA CUDA and **cannot** use it. Do not set
|
||||
`run_opts={"device":"cuda"}`.
|
||||
Each worker caps Torch/BLAS/OpenVINO to `TORCH_NUM_THREADS=1` (default)
|
||||
so 14–20 processes do not oversubscribe the i7-14700. This host has an
|
||||
Intel Arc B580; the venv Torch is NVIDIA CUDA and **cannot** use it. Do
|
||||
not set `run_opts={"device":"cuda"}`.
|
||||
|
||||
One worker process per camera loads the modules once and reuses them.
|
||||
`enhance_batch` must be called with `lengths=torch.tensor([1.0])`
|
||||
|
||||
Reference in New Issue
Block a user