Build or buy: the real cost of collecting training data in-house

The build-versus-buy question for training data usually gets answered with a per-unit cost comparison, and that comparison is almost always wrong in the same direction: it undercounts the in-house option.

The reason is that in-house collection does not look like a data cost. It looks like engineers spending two days a week on recruiting, a legal review that keeps coming back, and a pipeline that sits idle between batches. None of that appears in a cost-per-hour figure, and all of it is real.

The in-house costs that stay off the spreadsheet

These costs are not hypothetical. They are what the build option consists of, once the equipment is bought.

• Recruitment is a continuous job, not a one-time setup. Speakers drop out, screening rejects people late, and scheduling consumes someone every week.
• Legal and consent work multiplies by jurisdiction. Every new country is a new consent form, and often a new review.
• The pipeline is idle between batches, and recording and annotation capacity that is not continuously fed still costs what it costs.
• Your engineers become logistics managers. The people who could improve the model are instead chasing no-shows and fixing file formats.
• Quality control needs to be someone else. If the person who produced the data also accepts it, acceptance means nothing, and splitting those roles takes two people, not one.

The buy-side costs that stay off the quote

Buying is not free of hidden work either, and the omissions run in the other direction.

• Specification time. A brief that prevents rework takes days of senior attention, and it is easy to spend that time badly by writing it once and hoping.
• Review time. You still inspect deliveries. Buying the data removes production, not responsibility.
• Integration. Files arrive in conventions that are not yours, and mapping them into a training pipeline is work with a schedule.
• The renegotiation cost of a producer who cannot deliver, discovered late. Replacing one mid-project costs the elapsed time twice: once wasted, once re-spent.

Three questions that decide it

Before comparing numbers, settle what the data is for, because the answers constrain which option is even available.

• Is the data a durable asset or a one-time input? If the capability depends on owning the corpus, in-house collection is the only route to that position.
• Is collection continuous or occasional? Continuous collection amortizes the pipeline setup and gets cheaper over time. One-off needs rarely justify building one.
• Is the population reachable? If the speakers you need are outside your recruiting range, the build option is not actually available, whatever the spreadsheet says.

The hybrid that most teams land on

The common resolution is to buy the first batch and build the second, or to buy the benchmark set and collect the long tail.

Buying first is the cheaper way to learn the specification. The first delivery teaches what the requirements actually mean, and those lessons transfer directly into an in-house pipeline. The reverse order, building first and buying to fill gaps, is more expensive to learn from, because the pipeline has to be maintained while it happens.

The other hybrid is permanent: buy the general-purpose data, collect the differentiated slice. Anything that depends on your product conditions — your device, your users, your deployment noise — is hard to buy, because it is hard for anyone else to produce.

Where the crossover point actually is

The crossover moves with utilization. In-house collection gets cheaper per unit as the pipeline fills and more expensive per unit as it idles. Buying stays roughly flat and converts a fixed cost into a variable one.

That means the question is not which option is cheaper in principle, but whether a pipeline can be kept busy enough, for long enough, to reach the crossover, and whether that utilization is guaranteed. Teams routinely build a pipeline for what turns out to be a one-off project, and pay the fixed cost anyway.