librosa and torchaudio: which one for which job in a dataset pipeline
soundfile to read, torchaudio to train, librosa to verify. Where each library earns its place, and the defaults that cause drift.
Both libraries decode, resample, and compute spectrograms, and that overlap is exactly why pipelines drift: one script uses librosa, another uses torchaudio, the two disagree on defaults, and the features that reach training are not the features that were verified.
The productive answer is not to pick one. It is to assign each library to the part of the pipeline where it is strongest, and then to use the second implementation as an independent check on the first.
Reading and decoding: use soundfile directly
soundfile.read wraps libsndfile, handles WAV, FLAC, and OGG, and returns exactly what is in the file. librosa.load wraps the same library but defaults to a mono mixdown at 22050 Hz, which means a single unqualified call silently resamples a 48 kHz corpus. That default is the most common accidental resample in dataset code.
torchaudio.load returns a tensor shaped channels by time at the native rate, which is what a loader wants, and for compressed formats it hands off to ffmpeg when that is available. For plain reading inside a preparation script, soundfile is the least surprising of the three. The rule that matters either way is to never let a default decide the sample rate.
Resampling: torchaudio in the loader, soxr in the prep script
Inside a training loop, torchaudio.functional.resample runs on tensors, can process a batch in one call, and lives naturally in a data loader worker. For the offline preparation pass, librosa with res_type set to soxr_hq is the better tool, and it is the one to use for anything headed into the archive.
The trap is mixing them across one corpus. The two implementations use different filters, so their output differs slightly in the transition band near Nyquist. Half a dataset resampled one way and half the other is a difference the model will see and nobody will be able to explain. Pick one for the corpus, do it once, and write down which.
Features: torchaudio to train, librosa to verify
torchaudio computes mel spectrograms on GPU tensors inside the training step, which is what you want when features are generated on the fly. librosa has a separate implementation with different defaults, and that difference is what makes it valuable. Run the same file through both with every parameter matched explicitly, align the frames, and compare.
If the parameters are genuinely matched, the maximum absolute difference sits at the level of floating point noise. A constant offset means the mel filterbank differs, usually in the mel scale or the normalization. A one-frame shift means the padding before the FFT differs. Either way, you have found a configuration mismatch on your own machine instead of during a training run.
The defaults that will bite you
None of these raise an error. All of them change the numbers.
- librosa.load defaults to sr=22050 and mono=True. Pass sr=None and mono=False explicitly and handle the downmix yourself.
- torchaudio defaults its mel spectrogram to the HTK scale with no filterbank normalization, while librosa defaults to the Slaney scale with Slaney normalization. Matching n_mels and fmax is not enough to make the two agree.
- Shapes disagree: torchaudio.load returns channels by time, while soundfile.read returns time by channels. A transposed array raises nothing and trains on noise, so assert the shape right after loading.
- Frame counts: both libraries pad before the FFT by default, but the pad mode differs across libraries and versions, which moves the first and last frames. Trim the edge frames before comparing, or align the matrices first.
- Resampled length: output can differ by one sample between implementations. Compare durations in seconds with a one-sample tolerance rather than lengths.
A split that holds up
Preparation and quality assurance scripts in soundfile, numpy, and librosa, because they are readable and easy to inspect by hand. The training path in torchaudio, because it is fast and batchable. The feature configuration in one shared module that both sides import, so the verification script cannot drift away from what training actually uses.
For bulk conversion of a large directory, a shell loop over sox with xargs -P is often the fastest route, since sox combines resampling, gain, and format change in one pass per file. Reserve Python for the steps that need the array in memory.