Resampling audio without breaking it

Resampling is the step that looks like a one-line call and quietly decides whether the top of your speech band is intact. Downsampling without a proper anti-aliasing filter folds energy from above the new Nyquist frequency back into the band where speech lives, and nothing downstream can tell that folded energy apart from a fricative.

Upsampling is the easy direction and the less useful one. It cannot restore a band that was never recorded; it only changes the container. The rule that follows is simple: record high, convert down once, and never convert a second time.

Choose a resampler with a real stopband

Three options are good enough for corpus work, with the parameters that matter.

• soxr through librosa: librosa.resample(y, orig_sr=sr, target_sr=16000, res_type="soxr_hq"). The soxr_vhq preset trades more compute for a slightly steeper filter.
• scipy.signal.resample_poly with the integer ratio: for 48 kHz to 16 kHz that is up=1, down=3. It is fast and adequate, but its default Kaiser window has a shallower transition band than soxr, which makes it the wrong pick when the source has strong energy right at the new Nyquist.
• torchaudio.functional.resample, which takes tensors and can run batched on a GPU. Its sinc_interp_kaiser method with the default lowpass_filter_width of 64 is a high-quality choice, and it fits naturally when resampling happens inside a data loader.

Order: mono, trim, resample, normalize

Resample after downmixing to mono, so the filter runs once per file instead of once per channel. Resample after trimming silence, because the filter costs time proportional to the duration and because the anti-aliasing filter rings briefly at file boundaries. That ringing is better placed in silence you are about to remove than in speech you are about to keep.

Resample before loudness normalization, so the gain is computed on the signal that will actually be stored. And resample exactly once. A chain like 48 to 44.1 to 16 applies two filters and takes two chances at folding energy, and 44.1 in the middle of that chain serves no purpose.

The batch trap

A loop that assumes every input file is at 44.1 kHz will silently mangle the ones that are not, and the failure is invisible in a directory listing, because the output files are all labeled with the target rate. The wrong ones are simply wrong. Read the actual rate from each file and compute the ratio from it, or assert the expected rate and fail the batch loudly when a file disagrees.

Assert the output as well. After a conversion run, walk the output directory and check that every file reports the target rate and channel count, with a duration that matches the input within a small tolerance. Count the failures instead of stopping at the first, since the count is what tells you whether a whole folder was mislabeled or a single upload is corrupt.

Verification that catches aliasing

Two tests are worth building into the conversion script, because both fail loudly when the filter is missing or misconfigured.

• The alias test: synthesize a tone above the new Nyquist, for example 15 kHz at a 48 kHz input, downsample it to 16 kHz, and measure the energy near 1 kHz, where an unfiltered tone lands. A correct resampler puts that energy more than 60 dB below the original tone.
• The round-trip test: resample a sweep from 48 kHz to 16 kHz and back, then compare the 0 to 7 kHz band with the original. The passband should be flat within a fraction of a decibel, and there should be essentially no energy above 7.5 kHz, because the first downsample removed it.

Duration arithmetic, and why to compare seconds

Non-integer ratios are where off-by-one errors live. Converting 44.1 kHz to 16 kHz is a ratio of 441 to 160, and the expected output length is the input length multiplied by 160 over 441, rounded. Two libraries can disagree by a single sample on that rounding, which is harmless, and a script that asserts exact length equality will fail on a file that is perfectly fine.

So compare durations in seconds with a tolerance of one sample, not lengths. Keep the tolerance explicit in the code, so the next person does not tighten it and spend an afternoon chasing a rounding difference.