Data contracts between marketing and engineering

On a Monday morning the recommendation block in the weekly email is empty for about a fifth of recipients. Nobody deployed anything on the marketing side. The campaign ran as scheduled, the template is unchanged, and the platform reports no errors.

Three hours later someone works out that a release the previous Thursday renamed a field in the product-view event, from product_id to item_id, as part of a tidy-up. The engineer who made the change had no reason to think anyone outside the app was reading it. The pipeline accepted the new shape without complaint, because it was built to be forgiving. The transformation that fills the recommendation table quietly produced nothing.

This is the most common category of CRM incident we are called into, and it is almost never caused by carelessness. It is caused by an absent agreement. Nobody had written down which events the marketing programme depends on, what shape they must arrive in, and who to tell before changing them.

What a data contract is, in practice

A data contract is a versioned, machine-readable definition of a dataset that a producing team commits to and a consuming team builds against. Strip away the vocabulary and it contains six things:

• The schema: field names, types, whether each is required, and allowed values for anything enumerated.
• Semantics: what each field means in plain English, including the edge cases. Does order total include tax, shipping and discount, and in which currency.
• The producer, named as a team rather than a person.
• Freshness and volume expectations: how late the data may be, and the range of daily volume outside which something is wrong.
• The change policy: what counts as a breaking change, how much notice is required, and how versions coexist during a migration.
• The consumers, so a producer can see who they would affect.

It is not a document in a wiki. A contract that lives where nobody works is a record of an intention. The contract needs to sit in the repository, in a format a test can read, so that violating it fails a build rather than an email campaign.

Start with the events that have already broken something

Do not try to put the whole event stream under contract. The programme that tries this stalls within a month, because the inventory work is large and produces nothing visible until it is finished.

Start from incidents. Pull the last two or three quarters of anything that went wrong with personalisation, segmentation or reporting, and list the events involved. That list is short, specific, and already has organisational agreement that it matters.

Then add the events that feed anything automated and customer-facing. In most programmes that means the small set underneath the decision layer: the signals that drive next best action and the features behind any scoring model. These are the highest-risk data paths in the business, because failure is silent. A broken dashboard is noticed within a day. A model quietly running on a field that has been null for three weeks is not.

Ten to fifteen events is a good first scope. Covering those properly is worth more than covering everything loosely.

Put the contract where engineers already work

The contract should be a file in the producing service's repository, reviewed like code, with the schema expressed in whatever format your stack already uses.

That placement is a deliberate choice about ownership. If the contract lives in the marketing team's space, then changing it is marketing's job and engineers will not read it. If it lives beside the code that emits the event, then a developer changing that code sees the contract in the same diff, and the review conversation happens at the right moment.

Marketing's contribution is the semantic layer, not the schema syntax. The plain-English meaning of each field, the business rules, the list of things that have gone wrong before. Write that section yourself and keep it specific. "Revenue excludes VAT, excludes shipping, and is recorded at the point of dispatch rather than order placement" prevents more incidents than any type declaration.

Version the contract properly. A breaking change means a new version with a stated overlap period during which both shapes are produced, not a rename on a Thursday afternoon.

Enforce at three points, because one is not enough

A contract that is not tested is a description of how things used to work.

At the producer. A test in the producing service's build asserts that emitted events match the contract. This is the cheapest place to catch a problem, because it happens before the change reaches anything else.

In the pipeline. Validate on arrival. Events that violate the contract go to a quarantine table rather than into the main stream, and the count of quarantined events is alerted on. Be careful with the default here: many ELT setups are deliberately permissive, loading whatever arrives so that nothing is lost, which is a sensible default for storage and a poor one for decisions.

At the consumer. Tests on the transformation layer assert the properties the programme actually relies on. In a dbt project this is straightforward: not-null, uniqueness, accepted values, and relationships between tables. Add freshness checks, because stale data passes every schema test ever written.

Alert on all three to a channel the CRM team reads. The failure mode we see most often is an alert that fires correctly into a channel only the data team watches. There it competes with every other alert and gets triaged as low priority, because nobody in that channel knows a campaign depends on it.

Marketing's side of the bargain

Contracts fail when they are treated as a demand rather than an agreement. If marketing asks for guarantees and offers nothing, engineering will treat the process as overhead and comply minimally.

What the marketing side has to bring:

• A named owner for each consuming dataset, who answers questions and approves changes.
• Deprecation. If a field was requested for a campaign two years ago and nothing reads it now, say so and let it be removed. Unused fields under contract are a tax on every future change.
• Fewer one-off requests. A new field added for a single campaign, outside the schema, is how the drift starts.
• Realistic freshness requirements. Asking for real-time on a dataset that a weekly email reads makes every other requirement look negotiable.
• Attendance at the review when a breaking change is proposed, including the ones that arrive at an inconvenient time.

This is also the moment to be honest about the single customer view. Contracts govern the inputs; they do not resolve identity. If two systems disagree about who a person is, well-specified events will simply deliver that disagreement more reliably. Identity resolution is a separate piece of work and should not be smuggled into this one.

What changes once the contracts exist

The immediate effect is fewer silent failures, which is hard to celebrate because it shows up as an absence.

The more interesting effect is on what becomes buildable. Models are only as stable as their features, and the practical constraint on the kind of scoring described in propensity models you can run is rarely the algorithm. It is that the inputs change shape without warning, so a model that validated well in March is making decisions on different data by June.

Contracts also change how event tracking requests get handled. Once there is a defined process for adding an event, marketing stops asking for tags to be added ad hoc and starts specifying properly, which is faster for both sides after the first month.

The part that stays hard is that a contract makes ownership disputes visible without settling them. When an event has three producers and no clear owner, writing the contract is the moment everyone discovers it. That discovery is valuable, and it is also a conversation that a schema file cannot have for you. Expect to spend the first few weeks on ownership rather than syntax, and budget for it openly.