Aligning hour-long recordings: cut points, drift, and cross-chunk boundaries
Whole-file alignment fails slowly. Chunking bounds the damage, but only if the cuts come from the audio and you check the residual at every chunk.
A transcript with one missing sentence at minute twelve breaks whole-file alignment in a specific way. Everything after the omission is offset by the length of the missing sentence, and by minute ninety the offset is measured in minutes rather than milliseconds. The alignment does not fail loudly. It returns a complete, plausible, wrong result.
Chunking bounds that damage to a single chunk, which is the main reason to do it. The second reason is memory, and the third is that a per-chunk report is the only practical way to see drift.
Choose cut points from the audio, not from the clock
Cutting every ten minutes puts cuts through words, and a word cut in half gets mangled on both sides. Cut inside silences instead. Compute RMS over 20 ms frames, mark frames below the twentieth percentile as quiet, find runs of quiet frames lasting at least 0.4 seconds, and take the midpoint of the run nearest your target chunk length, searching within about 30 seconds of it.
With torchaudio.load and numpy that is roughly fifteen lines: reshape the mono waveform into frames, take np.sqrt((frames ** 2).mean(1)) for RMS, convert to decibels, and index the runs.
One guard is worth adding. If the chosen cut point is not at least 12 dB below the median speech level of the file, there was no real silence near the target. Take a longer chunk rather than cutting inside a word, and record that the cut was unreliable. A file of continuous speech will produce several of those, and knowing which cuts were forced tells you how much to trust the result.
Pad the chunks and drop the padding afterwards
Give every chunk two seconds of context on each side before aligning it. Without padding, the first and last words of a chunk get compressed or dropped because the aligner has no evidence on one side of them.
The padding then has to be removed, or you get duplicated words at every boundary. Drop any word lying entirely inside the padding region, and clip the ones that straddle it. Accumulate chunk offsets in samples and convert to seconds once at the end rather than adding float seconds sixty times; that error is small next to real drift, but it is free to avoid.
Detect drift with the residual
The diagnostic is a single number per chunk: the start time of the first aligned word, minus the chunk offset, minus the time of that word in the padded audio. If the transcript is complete and the alignment is right, the residual sits near zero and does not trend.
A slope across chunks is drift, and it means the transcript and the audio have diverged gradually, which usually points at an edit made to one and not the other. A step change of seconds between two consecutive chunks is a transcript problem at a known location: go to the transcript, not the aligner. Either way, plot the residual against chunk index before reading any individual word timing, because the shape of that line tells you whether the timings are usable at all.
Cross-chunk boundary rules
With padded chunks, a word that straddles a cut appears in two chunks with two placements. Keep the copy from the chunk containing the word's midpoint and drop the other, or the output gains duplicate words at every cut.
The two placements also disagree, and that disagreement is information. A boundary that differs by more than about 100 ms between the two copies means at least one of them is wrong, and the word belongs on a review list. On clean audio the two copies usually agree within a frame or two.
The same rule applies to the transcript side: a word cut across a chunk boundary is one word, not two, and it should never be split into two records to make the timings line up.
What to record per file
Keep a small table for every file: chunk count, how many cuts were forced, the maximum absolute residual, how many words were dropped as padding, how many words came back with no timestamp, and the overall aligned coverage.
Without that table, a long alignment is either trusted or not trusted as a whole. With it, you can find the two chunks out of ninety that are wrong and re-run those. That is the entire benefit of chunking, and it only exists if the report is written.