SLAs for data delivery, and the clauses that make them unusable

An SLA is a measurement agreement

A delivery SLA is not a promise about quality. It is an agreement about how quality will be measured, when the measurement happens, and what follows from the result. Most SLAs that fail in practice fail at the first part: the threshold is stated, the measurement is not, and the argument at delivery becomes one about arithmetic rather than performance.

Four things have to be defined before any threshold means anything: the metric and its reference, the sample it is computed on, the party who computes it, and the clock. Everything else in the document is cadence and consequences.

Cadence, expressed as events

Delivery rhythm is where projects slip first, and it is also the easiest part to define precisely. One clarification saves arguments later: the final delivery date is a milestone rather than a cadence, since the SLA governs the rhythm inside the project while the milestone governs when it ends.

• Batch size and interval as numbers rather than intentions: what a batch contains, how many units, and how often one is due.
• The trigger for each clock. A cadence that starts counting at delivery is different from one that starts when the previous batch is accepted, and the difference is exactly the review time you will spend.
• The minimum viable batch. A batch below the agreed size is either accepted as a partial delivery or rejected, and saying which prevents a producer from clearing a deadline with a small batch.
• What a missed cadence triggers: written notice, a recovery plan with dates, and the point at which repeated misses become a contractual event rather than a scheduling annoyance.

Quality thresholds someone else can compute

A threshold is usable when a second party, given the same files, arrives at the same number. That requires five things to be written down:

• A format check a script performs, returning pass or fail. No judgment is involved, which is exactly why it is worth having even though it sounds trivial.
• A measured accuracy metric with a stated formula and a stated normalizer, so both sides compute the same value from the same files.
• A separate threshold for the hard stratum. A single average across easy and hard material conceals the failure mode you are actually buying against.
• A defect-rate ceiling applied per batch rather than per project. A per-project ceiling lets a bad batch pass because the previous one happened to be good.
• A named reference set. The number is meaningless without the material it was computed against, and that reference set has to exist before production starts rather than being assembled after a dispute.

The repair window, and repair versus replacement

Repair terms decide whether a rejection has consequences. Three definitions are needed: how long the producer has to fix a rejected unit; what fixing means, since re-annotation, re-collection, and substitution with different material are three different costs; and whether the replacement sits inside an allowance or is priced as new work.

The most consequential detail is the clock. If the cadence clock pauses during repair, a producer can miss every date and still be compliant as long as the material is eventually fixed. If the clock keeps running, the repair is a real cost to the producer. Both arrangements are defensible, and the one that exists by accident is the one that causes the argument.

State also whether repaired material is sampled on return. A repaired unit that is never re-inspected is an accepted unit with an unverified fix, and that is how errors re-enter a dataset after they were caught.

Remedies, expressed as percentages and events

Remedies belong in the SLA as a ladder rather than a single penalty, because the appropriate response differs by severity. One principle belongs in writing alongside them: a credit is not a substitute for acceptance, since a batch that fails acceptance stays failed and a credit does not make unusable material usable. If a credit is the only consequence, the producer has bought the right to deliver badly.

• Defect rate slightly over threshold: repair inside the window, no remedy beyond the repair cost, and a note in the batch record.
• Batch fails acceptance: the batch is not payable until it is re-accepted, and the next batch does not start until the review closes.
• Missed cadence: written notice and a recovery plan with dates, escalating to a contractual event on the second consecutive miss.
• Repeated failure on the same criterion: the right to pause new work, re-run a calibration round, or terminate with completed milestones settled.
• Credits stated as a percentage of the affected batch rather than as an amount, so they scale with the project and do not need renegotiating when volumes change.

What makes an SLA unenforceable

These are the defects that turn an SLA into a document nobody can apply. Review the SLA against the first two batches as a matter of maintenance, since that is where the definitions meet reality and the corrections are cheap while the project is still young.

• Thresholds with no reference set, so the number cannot be reproduced by a second party.
• Quality defined by an adjective such as industry standard, with no metric attached to it.
• Measurement performed by the party being measured, with no independent check on the result.
• Exclusions wide enough that every failure has a category. Name the exclusions in advance and require the producer to evidence that an exclusion applies.
• A remedy ladder that stops at a credit, which converts every schedule risk in the project into your risk.