MFCC or mel spectrogram in practice: picking a front end and pinning its numbers
Log-mel is the default front end now. The real work is pinning its parameters and proving two libraries produce the same features.
The reference pages cover what these two features are. The decision that matters in a project is narrower: what should the front end of your model produce, and which numbers have to be written down so that two machines produce the same thing.
For anything with a neural network in it, the answer today is log-mel. The interesting work is in the parameters, the normalization, and the cross-library check.
Where MFCC is still the right call
MFCC is a decorrelating compression of log-mel: a discrete cosine transform followed by keeping the low coefficients. That compression is both the reason to use it and the reason not to. Three cases where it still wins.
- Keyword spotting and other models that run on a microcontroller, where 13 to 20 coefficients instead of 80 filterbank bins is a real memory and compute difference.
- Reproducing a published recipe, such as an older hybrid recognizer whose acoustic model was trained on MFCC and whose numbers you need to match.
- Bandwidth-limited telephony work where every baseline you are comparing against uses MFCC.
The numbers that have to be pinned
A log-mel configuration is not one parameter but a set, and leaving any of them to a library default means your features depend on the library version.
- The rate the features are computed at, usually 16000, and the audio must already be at that rate rather than resampled inside the feature call.
- Window and hop: 25 ms window with a 10 ms hop, which at 16 kHz means win_length 400 and hop_length 160.
- FFT size: 400 if it follows the window exactly, 512 if you need a power of two for a fast kernel.
- Mel count: 80 bins is the current standard, 40 appears in older recipes, 128 in some large models.
- Frequency range: fmin of 0 or 20 Hz and fmax of 8000 for 16 kHz audio. Setting fmax below Nyquist is a real choice, not a detail.
- Mel scale and filterbank normalization: the two libraries disagree by default. librosa uses the Slaney scale with Slaney area normalization; torchaudio defaults to the HTK scale with no normalization.
The cross-library check
Because the defaults differ, computing the same file with both libraries is a cheap and genuinely useful test. Match every parameter explicitly, including mel_scale and norm, then compare frame by frame. If the configuration is truly matched, the maximum absolute difference sits at the level of floating point noise. A constant offset of a fraction of a decibel means the filterbank differs; a shift of one frame means the padding differs.
Run the check over about a hundred files from the corpus rather than one, and run it again after any library upgrade. It costs a few minutes and it catches the class of bug where training features and inference features quietly stop matching.
Normalization is a separate decision
Most recognizers normalize features per utterance, subtracting the mean and dividing by the standard deviation within the utterance. This makes the model insensitive to recording gain, which is usually what you want, and it destroys absolute loudness information, which you want to keep if the task depends on it. A whispered instruction and a shouted one normalize to the same array.
If you are fine-tuning a model that was trained with a fixed affine mapping rather than per-utterance normalization, keep its mapping. Recomputing features with your own normalization is a common way to make a fine-tune fail for reasons that look like a learning rate problem.
What to ship alongside a dataset
Ship audio plus the script that computes the features, not precomputed features. A feature array locks the buyer into your sample rate, mel scale, log floor, and normalization. Audio does not. If features are requested anyway, ship the configuration as a short document with the exact values above, the library versions, and a hash of the script.
The test of a complete configuration is whether a second person can reproduce your feature matrix on a different machine. If any value is described as the default, the answer is no.