Are Data Contracts the Next Big Data Mistake?

Cameron Price
Feb 19
5 min read

Updated: Jul 21

The data industry is full of conversation around data contracts, especially in the context of data mesh and the creation of data products. Advocates claim that they are essential for improving data quality, governance, and reliability. But is this actually the case?

If you've been in tech long enough, you might recognize a familiar pattern. Data contracts sound a lot like interface contracts from the past, rigid agreements that aimed to standardize interactions between software components. These were supposed to improve consistency but often ended up stifling progress and adding complexity.

This is not a technical comparison between CORBA or EAI and modern data contracts. Rather, it's a reflection on how organizations adopt well meaning solutions in rigid, over engineered ways, often repeating patterns that failed in the past due to cultural and structural friction.

So, will data contracts revolutionize data management, or are we repeating old mistakes?

What Are Data Contracts?

A data contract is an agreement between data producers and consumers that defines (but is not limited to):

Schema (what data is provided, its structure, types, constraints)
SLAs (availability, freshness, quality guarantees)
Ownership and responsibilities (who maintains it, who can change it, how changes are communicated, who is responsible)

The goal is to prevent unexpected data changes that break downstream systems, improve trust in data, and bring software engineering principles to data, while enabling decentralized ownership.

Sounds promising in theory. But in reality? Not so much.

Historical Pattern: Interface Contracts and Their Pitfalls

Interface contracts were created to standardize interactions between software components, but their inflexibility often led to significant issues.

• Enterprise Application Integration (EAI)

A 2003 Gartner report found that 70 percent of EAI projects failed, primarily due to management and organizational challenges. The rigidity of interface contracts created delays, added overhead, and hindered adaptability.

Gartner (2003). “70 Percent of EAI Projects Fail.” Link

• Common Object Request Broker Architecture (CORBA)

CORBA was intended to enable seamless communication between systems. However, it became known for its complexity and inflexibility. Rigid interface definitions and poor interoperability across vendors led to failed implementations and limited adoption.

Emmerich, W. (2000). “Software Engineering and Middleware: A Roadmap.” ACM. https://doi.org/10.1145/336512.336536

The key lesson? Rigid, predefined contracts often become obstacles rather than enablers of progress.

Why Data Contracts Might Fail Like Interface Contracts Did

1. Rigid Agreements Stifle Innovation

Data contracts assume that data producers and consumers can align upfront on schemas and expectations. This is rarely practical.

Business requirements frequently change and often require agility and velocity.
New use cases emerge frequently, especially in the areas of experimentation and innovation
Data evolves organically. Data sources are constantly changing. Organizations are realizing the value of unstructured data to power context within AI, and there are many more sources, both external and internal to the organization.

With strict contracts, every modification requires negotiation and coordination, slowing down data product creation and utilization. Instead of empowering teams, contracts create bureaucratic bottlenecks.

2. Data Consumers Will Find Workarounds

Strict data contracts assume that data consumers can only access structured, controlled data pipelines. But that’s not how real-world data works.

If a data contract blocks a critical use case, users will find other ways to get the data (e.g., direct database queries, screen scraping, or shadow IT pipelines). The concept of shadow IT or shadow BI has been a longstanding topic.
If a contract delays a change, teams will be encouraged bypass official data sources entirely. We see this consistently within organizations where the single source of truth in an elusive dream.

Instead of improving data quality, contracts could drive more fragmentation and less governance, which is the exact opposite of their intent.

3. Contracts Don’t Automatically Improve Data Quality

Proponents argue that data contracts improve data quality. But contracts alone don’t solve the real issue: data quality is an ongoing process, not a one-time agreement.

What actually works?

Schema evolution & versioning. Instead of blocking change, allow structured, backward-compatible evolution.
Data observability. Monitor freshness, completeness, and anomalies in real time.
Proactive governance. Foster collaboration between producers and consumers, not rigid enforcement. Enable the data steward by removing the data governance burden, and powering adoption of ownership.

Data quality isn’t about strict rules; it’s about trust, monitoring, and adaptability.

Modern Implications: Data Contracts in Data Mesh Architectures

Data contracts are often discussed within the data mesh framework, which promotes decentralized data ownership. However, rigid contracts can create challenges:

Cultural Resistance: Transitioning to a data mesh requires a significant cultural shift. Organizations often struggle with decentralizing data ownership, leading to resistance and potential project failures (Profit Optics).
Technical Complexities: Implementing data contracts demands a deep understanding of both the data and its applications. Without proper expertise, organizations may face integration issues, leading to project delays or failures (Amdocs).

BITOL ("Build It, Test Once, Leverage") takes a more pragmatic approach to data contracts by emphasizing test-driven data governance and collaborative enforcement. Unlike traditional rigid contracts, BITOL allows for continuous validation and adaptation, which helps avoid the bottlenecks associated with static contracts.

However, BITOL still faces challenges similar to traditional data contracts if implemented without flexibility. If organizations use BITOL to create excessively rigid testing frameworks, they may encounter the same resistance, workarounds, and stagnation that plagued interface contracts in the past.

"The key to data success isn't rigidity; it's flexibility combined with strong governance." – Zhamak Dehghani, Founder Next Data.

Where Data Contracts Work Well

Not all data contracts are bad. In some cases, they can be helpful, but only when applied selectively:

Regulated Environments. In finance, healthcare, and compliance heavy industries, contracts help enforce rules on data privacy, accuracy, and security.
SLAs for Mission Critical Data. If a business function relies on real-time, high accuracy data, setting clear expectations on availability and quality makes sense.
Standardizing Internal APIs. If your organization has a well-defined data product model, lightweight contracts will help ensure consistency, but only if they’re not too rigid.

"Good governance is about knowing when to be flexible and when to enforce standards." – IBM Data Governance Whitepaper

A More Practical Alternative: Schema Evolution & Data Observability

Instead of enforcing rigid data contracts, modern architectures should focus on:

Schema Evolution & Versioning

Allow controlled, backward compatible changes instead of breaking consumers.
Use soft deprecation instead of hard schema freezes.

Data Observability & Monitoring

Track schema drift, data anomalies, and freshness in real time.
Notify consumers proactively when changes occur.

Collaboration & Metadata Driven Governance

Use data mesh tools to enable consumers to create, configure, and collaborate with data products.
Let data producers and consumers collaborate asynchronously, building a community, rather than relying on bureaucratic approvals. Think of this as having local control with light centralized guard rails.

"AI-driven data management can reduce errors and improve governance without sacrificing agility." – McKinsey Data Report

Final Verdict: Are Data Contracts the Next Big Mistake?

Data contracts are not inherently a mistake. But rigid, inflexible contracts applied without context or flexibility can repeat the same failures we’ve seen before.

Strict, rigid data contracts are likely to fail, just like interface contracts did in software. They will slow down progress, force unnecessary bureaucracy, and lead to workarounds.
Flexible, evolving contracts that focus on collaboration, observability, and adaptability are valuable in specific cases.

In the right context, contracts serve a purpose, but they are far from a universal solution.

"True data agility comes from balancing structure with adaptability, not enforcing rigidity." – Deloitte Data Insights Report

Instead of blindly adopting data contracts as the future of data mesh and data products, organizations should learn from the past and focus on modern alternatives that enable both control and agility.

Join the data conversation,

Cameron Price.