Files
rustdesk/src/audio_resampler/sinc.rs
fufesou c4221469d8 Fix/audio stream continuity (#16095)
* fix(audio): add streaming resampler

* fix(audio): preserve stream resampling state

* fix(audio): keep playback callback nonblocking

* fix(audio): decouple capture conversion from dasp

* fix(audio): support stateful samplerate backend

* refactor(audio): isolate stream callback state

* refactor(audio): group capture output options

* fix(audio): clear stale playback state after startup failure

Reset non-Linux playback state when stream startup fails to prevent
new-format audio from using the previous stream or resampler.

Add regression tests for failed format changes and successful playback.

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix(audio): honor capture resampler selection and reuse buffers

Use the selected resampling backend for fixed-frame capture.
Convert samples directly into the input queue and
reuse the PCM frame buffer.

Add tests for anti-aliasing, thread transfer, and
partial-frame draining.

Signed-off-by: fufesou <linlong1266@gmail.com>

* refact: reduce diffs

Signed-off-by: fufesou <linlong1266@gmail.com>

* test(audio): check resampler output count and passband energy

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix(audio): reset incompatible Linux playback state on
  startup failure

Preserve compatible output streams when replacement
  startup fails.
Clear state when no compatible stream exists and cover
  both paths in tests.

Signed-off-by: fufesou <linlong1266@gmail.com>

* perf(audio): reuse PCM buffers in the capture pipeline

- Reuse capture framing, resampling, and channel conversion buffers
- Deliver borrowed packets and write Sinc output into reusable storage
- Add allocation and output-equivalence regression tests

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix(audio): smooth buffer discard discontinuities

Signal receiver PCM discards and fade from the current playback output when the callback reaches the new timeline.

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix(audio): add missing Cargo.toml

Signed-off-by: fufesou <linlong1266@gmail.com>

* perf(audio): move capture encoding off the CPAL callback

Move Opus encoding and service delivery to a dedicated worker.
Use a preallocated bounded PCM queue with explicit loss reporting.
Add tests for callback allocations and queue saturation.

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix(audio): smooth capture gaps and report losses during backlog

Signed-off-by: fufesou <linlong1266@gmail.com>

* feat(audio): report capture queue high-water mark

Track peak queued PCM packets and log the approximate
queued audio duration alongside capture loss statistics.

Signed-off-by: fufesou <linlong1266@gmail.com>

* refact(audio): reduce diffs

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix(audio): avoid blocking capture on encoder queue contention

Use preallocated queues with try_lock in the capture callback.
Count and drop the current packet on contention, preserving
drop-oldest behavior on overflow.

Add regressions for paused workers, buffer reuse, and sequence wrap.

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix: add the missing files

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix(audio): isolate zero-gate state per encoder

Signed-off-by: fufesou <linlong1266@gmail.com>

* refact: reduce diffs

Signed-off-by: fufesou <linlong1266@gmail.com>

* refact(audio): simple refactor

Signed-off-by: fufesou <linlong1266@gmail.com>

* fix(audio): avoid waiting on playback callback locks

Use one PCM try_lock attempt and preserve queued samples during contention. Replace readiness locking with per-stream atomic status and report callback errors from the receiving thread.

Cover callback progress, retained audio, recovery, and poisoned-buffer handling.

* fix(audio): restart capture after processing errors

Stop further processing until the service recreates the stream.
Document the guard as defensive recovery for an unconfirmed failure.
Group capture and resampler submodules under their parent directories.

Signed-off-by: fufesou <linlong1266@gmail.com>

* audio: report capture queue contention drops separately

- Add contention_dropped to loss reports while preserving total drop counts
- Document packet rejection on contention even when buffers are available
- Extend existing contention and saturation test assertions

Signed-off-by: fufesou <linlong1266@gmail.com>

* refact unit tests

Signed-off-by: fufesou <linlong1266@gmail.com>

---------

Signed-off-by: fufesou <linlong1266@gmail.com>
2026-09-10 16:00:58 +08:00

113 lines
4.1 KiB
Rust

use super::{AudioResamplerConfig, AudioResamplerError};
use libsamplerate_sys as sys;
use std::ptr::NonNull;
const OUTPUT_MARGIN_FRAMES: usize = 1;
pub(super) struct SincAudioResampler {
state: NonNull<sys::SRC_STATE>,
config: AudioResamplerConfig,
}
impl SincAudioResampler {
pub(super) fn new(config: AudioResamplerConfig) -> Result<Self, AudioResamplerError> {
super::validate_config(config)?;
let ratio = f64::from(config.output_rate) / f64::from(config.input_rate);
if unsafe { sys::src_is_valid_ratio(ratio) } == 0 {
return Err(backend_error(
config,
samplerate::ErrorCode::BadSrcRatio as _,
));
}
let mut error = 0;
// SAFETY: src_new allocates independent state; this owner releases it in Drop.
let state = unsafe {
sys::src_new(
sys::SRC_SINC_BEST_QUALITY as _,
config.channels.into(),
&mut error,
)
};
let state = NonNull::new(state).ok_or_else(|| backend_error(config, error))?;
Ok(Self { state, config })
}
pub(super) fn process_into(
&mut self,
input: &[f32],
output: &mut Vec<f32>,
) -> Result<(), AudioResamplerError> {
super::validate_input(input, self.config.channels as usize)?;
let mut consumed = 0;
loop {
let (used, generated) = self.process_block(&input[consumed..], output)?;
consumed += used;
if consumed == input.len() {
return Ok(());
}
if used == 0 && generated == 0 {
return Err(AudioResamplerError::Backend(
"libsamplerate made no progress while input remained".to_owned(),
));
}
}
}
fn process_block(
&mut self,
input: &[f32],
output: &mut Vec<f32>,
) -> Result<(usize, usize), AudioResamplerError> {
let channels = self.config.channels as usize;
let input_frames = input.len() / channels;
let output_frames = input_frames
.checked_mul(self.config.output_rate as usize)
.map(|frames| frames / self.config.input_rate as usize)
.and_then(|frames| frames.checked_add(OUTPUT_MARGIN_FRAMES))
.ok_or(AudioResamplerError::CapacityOverflow)?;
let start = output.len();
let end = output_frames
.checked_mul(channels)
.and_then(|samples| start.checked_add(samples))
.ok_or(AudioResamplerError::CapacityOverflow)?;
let mut data = sys::SRC_DATA {
data_in: input.as_ptr(),
input_frames: input_frames
.try_into()
.map_err(|_| AudioResamplerError::CapacityOverflow)?,
output_frames: output_frames
.try_into()
.map_err(|_| AudioResamplerError::CapacityOverflow)?,
src_ratio: f64::from(self.config.output_rate) / f64::from(self.config.input_rate),
..Default::default()
};
output.resize(end, 0.0);
data.data_out = output[start..].as_mut_ptr();
// SAFETY: state is exclusively owned; disjoint slices cover the declared frame counts.
let error = unsafe { sys::src_process(self.state.as_ptr(), &mut data) };
let generated = data.output_frames_gen as usize * channels;
output.truncate(start + generated);
if error != 0 {
return Err(backend_error(self.config, error));
}
Ok((data.input_frames_used as usize * channels, generated))
}
}
impl Drop for SincAudioResampler {
fn drop(&mut self) {
// SAFETY: this owner holds the only handle returned by src_new.
unsafe { sys::src_delete(self.state.as_ptr()) };
}
}
fn backend_error(config: AudioResamplerConfig, code: i32) -> AudioResamplerError {
AudioResamplerError::Backend(format!(
"input_rate={}, output_rate={}, channels={}: {:?}",
config.input_rate,
config.output_rate,
config.channels,
samplerate::Error::from_int(code)
))
}