root
42443420be
Arc B580 lost the kill test: batch-14 mask p50 103 ms vs Torch CPU 70 ms enhance / OpenVINO CPU 5 ms. Production stays on known-good CPU workers (tag known-good-cpu-2026-08-28). Hop stays 200 ms.
56 lines
2.1 KiB
Markdown
56 lines
2.1 KiB
Markdown
# Spike: OpenVINO MetricGAN on Arc B580
|
||
|
||
Date: 2026-08-28
|
||
Known-good tag: `known-good-cpu-2026-08-28` (736eccb)
|
||
Fleet was **stopped** (cameras + displays). LiveKit server stayed up.
|
||
|
||
## What ran
|
||
|
||
- Export `enhance_model` (EnhancementGenerator BLSTM) to ONNX.
|
||
Input for 1 s @ 16 kHz: `(batch, 63, 257)` log-magnitude.
|
||
STFT / ISTFT stay on Torch CPU (not in the ONNX).
|
||
- OpenVINO 2026.3.1 in `.venv-ov` (not the CUDA project venv).
|
||
- Devices: CPU i7-14700, GPU.0 UHD 770, GPU.1 Arc B580.
|
||
|
||
## Numbers (batch=14, 1 s windows)
|
||
|
||
| path | p50 batch | p95 batch | p99 batch | p50 / cam |
|
||
|---|---:|---:|---:|---:|
|
||
| Torch CPU `enhance_batch` (1 thread) | 69.7 ms | 69.9 ms | 69.9 ms | 4.98 ms |
|
||
| Torch CPU mask only | 63.6 ms | 63.7 ms | 63.7 ms | 4.54 ms |
|
||
| OpenVINO CPU mask | 5.13 ms | 5.49 ms | 5.65 ms | 0.37 ms |
|
||
| OpenVINO GPU.0 iGPU mask | 75.0 ms | 75.4 ms | 75.4 ms | 5.36 ms |
|
||
| OpenVINO GPU.1 Arc B580 mask | 103.0 ms | 103.1 ms | 103.4 ms | 7.35 ms |
|
||
|
||
## Kill criterion (plan)
|
||
|
||
OpenVINO GPU ms/camera must be ≥2× thread-capped CPU batch, and p99
|
||
including copies < hop/2.
|
||
|
||
- Arc vs Torch CPU enhance: **slower** (7.35 vs 4.98 ms/cam). Fail.
|
||
- Arc vs OpenVINO CPU mask: **20× slower**. Fail.
|
||
- Arc p99 103 ms is already half of a 200 ms hop for the **mask only**,
|
||
before STFT/ISTFT. Fail for hop cut.
|
||
|
||
**Do not put MetricGAN on the B580. Do not merge a GPU enhance daemon.**
|
||
|
||
## What would actually cut hop
|
||
|
||
OpenVINO **CPU** mask is ~12× Torch. A single batched CPU enhance
|
||
process (STFT + OV-CPU mask + ISTFT) is the only path that looks fast
|
||
enough for `ENHANCE_HOP_S=0.1`. That is a new architecture (workers
|
||
DelaySum locally, one enhance server), not a CUDA/`device=xpu` switch.
|
||
|
||
This spike does **not** implement that server. Production stays on the
|
||
known-good per-worker Torch path (hop 200 ms).
|
||
|
||
## Reproduce
|
||
|
||
```bash
|
||
systemctl stop livekit-cameras livekit-displays
|
||
.venv/bin/python scripts/spike_export_metricgan_onnx.py
|
||
.venv/bin/python scripts/spike_bench_cpu_metricgan.py --batch 14
|
||
.venv-ov/bin/python scripts/spike_bench_openvino.py --batch 14
|
||
systemctl start livekit-cameras livekit-displays
|
||
```
|