root
42443420be
Arc B580 lost the kill test: batch-14 mask p50 103 ms vs Torch CPU 70 ms enhance / OpenVINO CPU 5 ms. Production stays on known-good CPU workers (tag known-good-cpu-2026-08-28). Hop stays 200 ms.
2.1 KiB
2.1 KiB
Spike: OpenVINO MetricGAN on Arc B580
Date: 2026-08-28
Known-good tag: known-good-cpu-2026-08-28 (736eccb)
Fleet was stopped (cameras + displays). LiveKit server stayed up.
What ran
- Export
enhance_model(EnhancementGenerator BLSTM) to ONNX. Input for 1 s @ 16 kHz:(batch, 63, 257)log-magnitude. STFT / ISTFT stay on Torch CPU (not in the ONNX). - OpenVINO 2026.3.1 in
.venv-ov(not the CUDA project venv). - Devices: CPU i7-14700, GPU.0 UHD 770, GPU.1 Arc B580.
Numbers (batch=14, 1 s windows)
| path | p50 batch | p95 batch | p99 batch | p50 / cam |
|---|---|---|---|---|
Torch CPU enhance_batch (1 thread) |
69.7 ms | 69.9 ms | 69.9 ms | 4.98 ms |
| Torch CPU mask only | 63.6 ms | 63.7 ms | 63.7 ms | 4.54 ms |
| OpenVINO CPU mask | 5.13 ms | 5.49 ms | 5.65 ms | 0.37 ms |
| OpenVINO GPU.0 iGPU mask | 75.0 ms | 75.4 ms | 75.4 ms | 5.36 ms |
| OpenVINO GPU.1 Arc B580 mask | 103.0 ms | 103.1 ms | 103.4 ms | 7.35 ms |
Kill criterion (plan)
OpenVINO GPU ms/camera must be ≥2× thread-capped CPU batch, and p99 including copies < hop/2.
- Arc vs Torch CPU enhance: slower (7.35 vs 4.98 ms/cam). Fail.
- Arc vs OpenVINO CPU mask: 20× slower. Fail.
- Arc p99 103 ms is already half of a 200 ms hop for the mask only, before STFT/ISTFT. Fail for hop cut.
Do not put MetricGAN on the B580. Do not merge a GPU enhance daemon.
What would actually cut hop
OpenVINO CPU mask is ~12× Torch. A single batched CPU enhance
process (STFT + OV-CPU mask + ISTFT) is the only path that looks fast
enough for ENHANCE_HOP_S=0.1. That is a new architecture (workers
DelaySum locally, one enhance server), not a CUDA/device=xpu switch.
This spike does not implement that server. Production stays on the known-good per-worker Torch path (hop 200 ms).
Reproduce
systemctl stop livekit-cameras livekit-displays
.venv/bin/python scripts/spike_export_metricgan_onnx.py
.venv/bin/python scripts/spike_bench_cpu_metricgan.py --batch 14
.venv-ov/bin/python scripts/spike_bench_openvino.py --batch 14
systemctl start livekit-cameras livekit-displays