Fix/audio stream continuity (#16095)

* fix(audio): add streaming resampler * fix(audio): preserve stream resampling state * fix(audio): keep playback callback nonblocking * fix(audio): decouple capture conversion from dasp * fix(audio): support stateful samplerate backend * refactor(audio): isolate stream callback state * refactor(audio): group capture output options * fix(audio): clear stale playback state after startup failure Reset non-Linux playback state when stream startup fails to prevent new-format audio from using the previous stream or resampler. Add regression tests for failed format changes and successful playback. Signed-off-by: fufesou <linlong1266@gmail.com> * fix(audio): honor capture resampler selection and reuse buffers Use the selected resampling backend for fixed-frame capture. Convert samples directly into the input queue and reuse the PCM frame buffer. Add tests for anti-aliasing, thread transfer, and partial-frame draining. Signed-off-by: fufesou <linlong1266@gmail.com> * refact: reduce diffs Signed-off-by: fufesou <linlong1266@gmail.com> * test(audio): check resampler output count and passband energy Signed-off-by: fufesou <linlong1266@gmail.com> * fix(audio): reset incompatible Linux playback state on startup failure Preserve compatible output streams when replacement startup fails. Clear state when no compatible stream exists and cover both paths in tests. Signed-off-by: fufesou <linlong1266@gmail.com> * perf(audio): reuse PCM buffers in the capture pipeline - Reuse capture framing, resampling, and channel conversion buffers - Deliver borrowed packets and write Sinc output into reusable storage - Add allocation and output-equivalence regression tests Signed-off-by: fufesou <linlong1266@gmail.com> * fix(audio): smooth buffer discard discontinuities Signal receiver PCM discards and fade from the current playback output when the callback reaches the new timeline. Signed-off-by: fufesou <linlong1266@gmail.com> * fix(audio): add missing Cargo.toml Signed-off-by: fufesou <linlong1266@gmail.com> * perf(audio): move capture encoding off the CPAL callback Move Opus encoding and service delivery to a dedicated worker. Use a preallocated bounded PCM queue with explicit loss reporting. Add tests for callback allocations and queue saturation. Signed-off-by: fufesou <linlong1266@gmail.com> * fix(audio): smooth capture gaps and report losses during backlog Signed-off-by: fufesou <linlong1266@gmail.com> * feat(audio): report capture queue high-water mark Track peak queued PCM packets and log the approximate queued audio duration alongside capture loss statistics. Signed-off-by: fufesou <linlong1266@gmail.com> * refact(audio): reduce diffs Signed-off-by: fufesou <linlong1266@gmail.com> * fix(audio): avoid blocking capture on encoder queue contention Use preallocated queues with try_lock in the capture callback. Count and drop the current packet on contention, preserving drop-oldest behavior on overflow. Add regressions for paused workers, buffer reuse, and sequence wrap. Signed-off-by: fufesou <linlong1266@gmail.com> * fix: add the missing files Signed-off-by: fufesou <linlong1266@gmail.com> * fix(audio): isolate zero-gate state per encoder Signed-off-by: fufesou <linlong1266@gmail.com> * refact: reduce diffs Signed-off-by: fufesou <linlong1266@gmail.com> * refact(audio): simple refactor Signed-off-by: fufesou <linlong1266@gmail.com> * fix(audio): avoid waiting on playback callback locks Use one PCM try_lock attempt and preserve queued samples during contention. Replace readiness locking with per-stream atomic status and report callback errors from the receiving thread. Cover callback progress, retained audio, recovery, and poisoned-buffer handling. * fix(audio): restart capture after processing errors Stop further processing until the service recreates the stream. Document the guard as defensive recovery for an unconfirmed failure. Group capture and resampler submodules under their parent directories. Signed-off-by: fufesou <linlong1266@gmail.com> * audio: report capture queue contention drops separately - Add contention_dropped to loss reports while preserving total drop counts - Document packet rejection on contention even when buffers are available - Extend existing contention and saturation test assertions Signed-off-by: fufesou <linlong1266@gmail.com> * refact unit tests Signed-off-by: fufesou <linlong1266@gmail.com> --------- Signed-off-by: fufesou <linlong1266@gmail.com>
2026-09-18 18:31:02 +03:00 · 2026-09-10 16:00:58 +08:00
parent 978e2e28b9
commit c4221469d8
20 changed files with 2957 additions and 221 deletions
--- a/src/audio_resampler/allocation_tests.rs
+++ b/src/audio_resampler/allocation_tests.rs
@@ -0,0 +1,142 @@
+use super::{AudioResamplerConfig, FixedFrameAudioResampler};
+use std::alloc::{GlobalAlloc, Layout, System};
+use std::cell::Cell;
+
+struct CountingAllocator;
+
+thread_local! {
+    static ALLOCATIONS: Cell<Option<usize>> = const { Cell::new(None) };
+}
+
+fn record_allocation() {
+    let _ = ALLOCATIONS.try_with(|count| {
+        if let Some(value) = count.get() {
+            count.set(Some(value + 1));
+        }
+    });
+}
+
+unsafe impl GlobalAlloc for CountingAllocator {
+    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
+        record_allocation();
+        unsafe { System.alloc(layout) }
+    }
+
+    unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
+        record_allocation();
+        unsafe { System.alloc_zeroed(layout) }
+    }
+
+    unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, size: usize) -> *mut u8 {
+        record_allocation();
+        unsafe { System.realloc(ptr, layout, size) }
+    }
+
+    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
+        unsafe { System.dealloc(ptr, layout) }
+    }
+}
+
+#[global_allocator]
+static ALLOCATOR: CountingAllocator = CountingAllocator;
+
+pub(crate) fn assert_no_allocations(process: impl FnOnce()) {
+    struct ResetCounter;
+    impl Drop for ResetCounter {
+        fn drop(&mut self) {
+            ALLOCATIONS.with(|count| count.set(None));
+        }
+    }
+
+    ALLOCATIONS.with(|count| assert!(count.replace(Some(0)).is_none()));
+    let reset = ResetCounter;
+    process();
+    let allocations = ALLOCATIONS.with(|count| count.get().unwrap());
+    drop(reset);
+    assert_eq!(
+        allocations, 0,
+        "PCM processing allocated on the capture thread"
+    );
+}
+
+#[test]
+fn capture_resampling_reuses_buffers() {
+    const PACKETS_PER_SECOND: usize = 100;
+    const PACKET_COUNT: usize = 100;
+    const MAX_STARTUP_DELAY_PACKETS: usize = 1;
+    const SIGNAL_LEVEL: f32 = 0.25;
+    const RATE_PAIRS: [(u32, u32); 6] = [
+        (32_000, 24_000),
+        (44_100, 24_000),
+        (44_100, 48_000),
+        (48_000, 24_000),
+        (96_000, 48_000),
+        (192_000, 48_000),
+    ];
+
+    for (input_rate, output_rate) in RATE_PAIRS {
+        for channels in [1, 2, 4, 6, 8] {
+            let config = AudioResamplerConfig {
+                input_rate,
+                output_rate,
+                channels,
+            };
+            let input =
+                vec![SIGNAL_LEVEL; input_rate as usize / PACKETS_PER_SECOND * channels as usize];
+            let frames = output_rate as usize / PACKETS_PER_SECOND;
+            let mut resampler = FixedFrameAudioResampler::new(config, frames).unwrap();
+            let mut packets = 0;
+            let mut energy = 0.0;
+            assert_no_allocations(|| {
+                for _ in 0..PACKET_COUNT {
+                    resampler
+                        .process_with(&input, |packet| {
+                            assert_eq!(packet.len(), frames * channels as usize);
+                            energy += packet.iter().map(|sample| sample * sample).sum::<f32>();
+                            packets += 1;
+                        })
+                        .unwrap();
+                }
+            });
+            assert!((PACKET_COUNT - MAX_STARTUP_DELAY_PACKETS..=PACKET_COUNT).contains(&packets));
+            assert!(energy > SIGNAL_LEVEL);
+        }
+    }
+}
+
+#[cfg(all(feature = "use_samplerate", not(feature = "use_dasp")))]
+#[test]
+fn sinc_output_matches_the_existing_backend() {
+    use super::AudioResampler;
+
+    const INPUT_FRAMES: usize = 2_048;
+    const CHUNK_FRAMES: usize = 73;
+    const SIGNAL_STEP: f32 = 0.07;
+    for (input_rate, output_rate) in [(44_100, 24_000), (44_100, 48_000), (96_000, 48_000)] {
+        for channels in [1, 2, 4, 6, 8] {
+            let config = AudioResamplerConfig {
+                input_rate,
+                output_rate,
+                channels,
+            };
+            let input: Vec<_> = (0..INPUT_FRAMES * channels as usize)
+                .map(|sample| (sample as f32 * SIGNAL_STEP).sin())
+                .collect();
+            let mut actual = AudioResampler::new(config).unwrap();
+            let expected = samplerate::Samplerate::new(
+                samplerate::ConverterType::SincBestQuality,
+                input_rate,
+                output_rate,
+                channels as usize,
+            )
+            .unwrap();
+            for chunk in input.chunks(CHUNK_FRAMES * channels as usize) {
+                assert_eq!(
+                    actual.process(chunk).unwrap(),
+                    expected.process(chunk).unwrap()
+                );
+                assert_eq!(actual.process(&[]).unwrap(), expected.process(&[]).unwrap());
+            }
+        }
+    }
+}