Are Data Contracts the Next Big Data Mistake?

The data industry is buzzing about data contracts, especially in the context of data mesh and the creation of data products. Advocates claim that they are essential for improving data quality, governance, and reliability. But is this true?

If you've been in tech long enough, you might recognize a familiar pattern. Data contracts sound a lot like interface contracts from the past—rigid agreements that aimed to standardize interactions between software components. These were supposed to improve consistency but often ended up stifling progress and increasing complexity.

So, will data contracts revolutionize data management, or are we repeating old mistakes?

What Are Data Contracts?

A data contract is an agreement between data producers and consumers that defines (but not limited to):

• Schema (what data is provided, its structure, types, constraints).

• SLAs (availability, freshness, quality guarantees).

• Ownership and responsibilities (who maintains it, who can change it, how changes are communicated, who is responsible).

  
  

The goal is to prevent unexpected data changes that break downstream systems, improve trust in data, and bring software engineering principles to data, whilst enabling decentralized ownership.

Sounds great in theory. But in reality? Not so much.

Historical Context: Interface Contracts

Interface contracts were designed to standardize interactions between software components, but their inflexibility often resulted in significant issues:

• Enterprise Application Integration (EAI): A 2003 report highlighted that 70% of EAI projects failed, primarily due to management issues rather than technical shortcomings. The rigidity of interface contracts contributed to these failures by creating bureaucratic bottlenecks and hindering adaptability (Wikipedia).

• Common Object Request Broker Architecture (CORBA): Intended to enable seamless communication between systems, CORBA faced criticism for its complexity and inflexibility. The rigid nature of its interface contracts led to poor implementations and limited adoption (Wikipedia).

  
  

The key lesson? Rigid, predefined contracts often become obstacles rather than enablers of progress.

Why Data Contracts Might Fail Like Interface Contracts Did

1.     Rigid Agreements Stifle Innovation

Data contracts assume that data producers and consumers can align upfront on schemas and expectations. This is rarely practical.

• Business requirements frequently change and often require agility and velocity.

• New use cases emerge frequently, especially in the areas of experimentation and innovation

• Data evolves organically. Data sources are constantly changing. Organizations are realizing the value of unstructured data to power context within AI, and there are many more sources, both external and internal to the organization.

  
  

With strict contracts, every modification requires negotiation and coordination, slowing down data product creation and utilization. Instead of empowering teams, contracts create bureaucratic bottlenecks.

2.   Data Consumers Will Find Workarounds

Strict data contracts assume that data consumers can only access structured, controlled data pipelines. But that’s not how real-world data works.

• If a data contract blocks a critical use case, users will find other ways to get the data (e.g., direct database queries, screen scraping, or shadow IT pipelines). The concept of shadow IT or shadow BI has been a longstanding topic.

• If a contract delays a change, teams will be encouraged bypass official data sources entirely. We see this consistently within organizations where the single source of truth in an elusive dream.

  
  

Instead of improving data quality, contracts could drive more fragmentation and less governance—the exact opposite of their intent.

3.   Contracts Aren’t a Silver Bullet for Data Quality

Proponents argue that data contracts improve data quality. But contracts alone don’t solve the real issue: data quality is an ongoing process, not a one-time agreement.

What actually works?

• Schema evolution & versioning → Instead of blocking change, allow structured, backward-compatible evolution.

• Data observability → Monitor freshness, completeness, and anomalies in real time.

• Proactive governance → Foster collaboration between producers and consumers, not rigid enforcement. Enable the data steward by removing the data governance burden, and powering adoption of ownership.

  
  

Data quality isn’t about strict rules; it’s about trust, monitoring, and adaptability.

Modern Implications: Data Contracts in Data Mesh Architectures

Data contracts are often discussed within the data mesh framework, which promotes decentralized data ownership. However, rigid contracts can create challenges:

• Cultural Resistance: Transitioning to a data mesh requires a significant cultural shift. Organizations often struggle with decentralizing data ownership, leading to resistance and potential project failures (Profit Optics).

• Technical Complexities: Implementing data contracts demands a deep understanding of both the data and its applications. Without proper expertise, organizations may face integration issues, leading to project delays or failures (Amdocs).

• 
  

BITOL ("Build It, Test Once, Leverage") takes a more pragmatic approach to data contracts by emphasizing test-driven data governance and collaborative enforcement. Unlike traditional rigid contracts, BITOL allows for continuous validation and adaptation, which helps avoid the bottlenecks associated with static contracts.

However, BITOL still faces challenges similar to traditional data contracts if implemented without flexibility. If organizations use BITOL to create excessively rigid testing frameworks, they may encounter the same resistance, workarounds, and stagnation that plagued interface contracts in the past.

"The key to data success isn't rigidity; it's flexibility combined with strong governance." – Zhamak Dehghani, Founder Next Data.

Where Data Contracts Are Useful

Not all data contracts are bad. In some cases, they can be helpful, but only when applied selectively:

✅ Regulated Environments → In finance, healthcare, and compliance-heavy industries, contracts help enforce rules on data privacy, accuracy, and security.

✅ SLAs for Mission-Critical Data → If a business function relies on real-time, high-accuracy data, setting clear expectations on availability and quality makes sense.

✅ Standardizing Internal APIs → If your organization has a well-defined data product model, lightweight contracts will help ensure consistency—but only if they’re not too rigid.

"Good governance is about knowing when to be flexible and when to enforce standards." – IBM Data Governance Whitepaper

A More Practical Alternative: Schema Evolution & Data Observability

Instead of enforcing rigid data contracts, modern architectures should focus on:

Schema Evolution & Versioning

• Allow controlled, backward-compatible changes instead of breaking consumers.

• Use soft deprecation instead of hard schema freezes.

Data Observability & Monitoring

• Track schema drift, data anomalies, and freshness in real time.

• Notify consumers proactively when changes occur.

  
  

Collaboration & Metadata-Driven Governance

• Use data mesh tools to enable consumers to create, configure, and collaborate with data products.

• Let data producers and consumers collaborate asynchronously, building a community, rather than relying on bureaucratic approvals. Think of this as having local control with light centralized guard rails.

• 
  

"AI-driven data management can reduce errors and improve governance without sacrificing agility." – McKinsey Data Report

Final Verdict: Are Data Contracts the Next Big Mistake?

🚫 Strict, rigid data contracts are likely to fail—just like interface contracts did in software. They will slow down progress, force unnecessary bureaucracy, and lead to workarounds.

✅ Flexible, evolving contracts that focus on collaboration, observability, and adaptability could be valuable in specific cases.

In the right context, contracts may serve a purpose—but they are far from a universal solution.

"True data agility comes from balancing structure with adaptability, not enforcing rigidity." – Deloitte Data Insights Report

Instead of blindly adopting data contracts as the future of data mesh and data products, organizations should learn from the past and focus on modern alternatives that enable both control and agility.

Join the Data Conversation,

Cameron Price.

Reference List:

1. Gartner. (2003). Gartner Says 70 Percent of Enterprise Application Integration Projects Fail. Retrieved from https://www.gartner.com/en/newsroom/press-releases/2003-01-14-gartner-says-70-percent-of-enterprise-application-integration-projects-fail

2. Forrester. (2024). Amid The AI Hype, Agile Still Remains Relevant In 2025. Retrieved from https://www.forrester.com/blogs/amidst-the-ai-hype-agile-still-remains-relevant-in-2025/

3. Fowler, M. (n.d.). Design Stamina Hypothesis. Retrieved from https://martinfowler.com/bliki/DesignStaminaHypothesis.html

4. Snowden, D. (n.d.). The Cynefin Framework. Retrieved from https://www.cognitive-edge.com/the-cynefin-framework/

5. Kim, G. (2013). The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.

6. Moses, B. (n.d.). Data Quality Is a Process, Not a Project. Retrieved from https://www.montecarlodata.com/blog/data-quality-is-a-process-not-a-project/

7. McKinsey & Company. (2024). The Data-Driven Enterprise of 2025. Retrieved from https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-data-driven-enterprise-of-2025

8. Deloitte. (2023). Tech Trends 2023: Data Governance. Retrieved from https://www2.deloitte.com/us/en/insights/focus/tech-trends/2023/data-governance.html

9. IBM. (n.d.). Data Governance Whitepaper. Retrieved from https://www.ibm.com/downloads/cas/EPYWX1EM

10. Dehghani, Z. (2022). Data Mesh: Delivering Data-Driven Value at Scale. O'Reilly Media.