MFCC vs Mel Spectrogram
Short answer. A mel spectrogram is a spectrogram with mel-spaced frequency bands; a log-mel spectrogram is the same thing in decibels. MFCCs are a log-mel spectrogram with a discrete cosine transform applied and all but the first 13 to 20 coefficients thrown away. The transform decorrelates the bands and the truncation compresses the data, but the truncation is lossy: keeping 13 of 80 coefficients changes the representation by about 3 dB per band on average.
How to do it
They are the same front end at different stages
The pipeline runs in one direction: waveform, short-time Fourier transform, mel filterbank, logarithm, then optionally a DCT. Stop after the logarithm and you have a log-mel spectrogram. Continue through the DCT and truncate, and you have MFCCs.
The two are not competitors. MFCC is a lossy compression of the log-mel spectrogram, and the only question is whether the discarded information was useful.
What the DCT buys you
The mel bands overlap, so adjacent band energies are highly correlated. The DCT turns that correlated sequence into coefficients that are close to independent, which is what made MFCCs work well with the diagonal-covariance Gaussian models that were standard for two decades.
That was a real advantage under those models. Under a neural network it is close to irrelevant, because the network can learn the correlation itself given enough data.
What truncation throws away, measured
The first coefficient is the sum of the log-mel bands, which is overall loudness. The next few carry the broad spectral shape. The high coefficients carry fine detail — the precise position of a formant, the harmonic structure inside a band.
The code below measures the loss directly: reconstruct the log-mel spectrogram from the first k coefficients and report the mean absolute error. Thirteen coefficients, the classic default, lose about 3 dB per band. Forty lose about 1.8 dB. That error is exactly the fine spectral detail a modern recognizer can use.
What to use now
Log-mel with 80 bands is the current default for ASR and for most audio neural networks. It keeps the fine detail, the pipeline is simpler, and it gives a convolutional front end something to work with across both time and frequency.
MFCCs remain a reasonable choice when you need a compact fixed-length embedding per utterance rather than a full time-frequency representation — speaker verification and audio classification often use them that way. If you are computing an embedding, the truncation is the point rather than a cost.
Code
Build an 80-band log-mel spectrogram, take its MFCCs, then reconstruct from the first k coefficients to measure how much the truncation discarded.
import numpy as np
import librosa
from scipy.fftpack import dct, idct
SR = 16000
t = np.arange(SR) / SR
f0 = 130.0
sig = sum(np.sin(2 * np.pi * f0 * k * t) / k for k in range(1, 25))
sig *= np.exp(-((t - 0.5) ** 2) / (2 * 0.18 ** 2))
sig += 0.001 * np.random.default_rng(0).standard_normal(len(t))
n_fft, hop, n_mels = 400, 160, 80
mel = librosa.feature.melspectrogram(y=sig, sr=SR, n_fft=n_fft,
hop_length=hop, n_mels=n_mels)
logmel = librosa.power_to_db(mel, ref=np.max)
mfcc = librosa.feature.mfcc(S=logmel, n_mfcc=13)
print(f"mel spectrogram {mel.shape} ({n_mels} mel bands x {mel.shape[1]} frames)")
print(f"log-mel {logmel.shape} {logmel.min():.1f} to {logmel.max():.1f} dB")
print(f"MFCC {mfcc.shape} ({mfcc.shape[0]} coefficients x {mfcc.shape[1]} frames)")
def reconstruct(k):
c = dct(logmel, type=2, axis=0, norm='ortho')[:k]
back = idct(np.pad(c, ((0, n_mels - k), (0, 0))), type=2, axis=0, norm='ortho')
return float(np.abs(back - logmel).mean())
print("\nwhat truncating the DCT throws away:")
for k in (13, 20, 40, 60, 80):
print(f" keep {k:>2} of 80 coefficients mean absolute error {reconstruct(k):>6.2f} dB")- pip install numpy librosa scipy
- The MFCC matrix has the same shape as the log-mel it came from until you truncate; the truncation is what makes it small, not the DCT.
- Keeping 13 of 80 coefficients leaves a mean reconstruction error of about 3.00 dB per band. At 40 it is 1.77 dB, and only at the full 80 does it reach zero.
Where this goes wrong
Truncating to 13 out of 128 mels and expecting no loss
The classic 13-coefficient default was chosen for 20 to 26 mel bands, where 13 keeps most of the information. Applied to a 128-band mel spectrogram, 13 coefficients keep far less of it. The loss is measurable: on the 80-band example on this page, 13 coefficients leave a mean reconstruction error of 3.00 dB per band and 40 leave 1.77 dB. Scale the coefficient count with the band count, or use log-mel.
Forgetting that c0 is loudness
MFCC coefficient zero is proportional to the total log energy of the frame. A model trained on raw MFCCs learns recording level as a feature, which is why cepstral mean normalization exists and why dropping c0 is common in speaker tasks.
Comparing MFCC dimensions across libraries
Libraries differ on whether the mel filterbank is area-normalized, what the log floor is, and whether c0 counts toward n_mfcc. Two MFCC matrices of the same shape from two libraries are not the same features.
Feeding MFCCs to a convolutional model
The DCT mixes all mel bands into every coefficient, so the coefficient axis has no local structure the way a frequency axis does. A convolution kernel sliding down the coefficient axis is mixing information from unrelated frequency regions, which is the opposite of what a spectrogram front end is for. That is one reason log-mel replaced MFCC for neural ASR.
When to buy the data instead of building the pipeline
Build the front end — it is a library call and you should tune the band count on your own data. Buy when the acoustic conditions are the variable: a corpus with consistent recording conditions across speakers, because a front end tuned on studio audio behaves differently on mobile and far-field data. If your evaluation runs on one condition and your deployment runs in another, no feature choice will save you. Match the band count to the model and record it with the dataset.
More technical reference
-
Audio Augmentation
audio augmentation
-
Mean Opinion Score (MOS)
mean opinion score
-
Far-Field Speech Recognition
far field speech recognition
-
PII Redaction in Speech Transcripts
pii redaction
-
Multilingual Speech Recognition
multilingual speech recognition
-
Signal-to-Noise Ratio (SNR)
signal to noise ratio
Need data for MFCC vs Mel Spectrogram?
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline — not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.