Enterprise AI needs a data strategy, not just a model strategy

Many enterprise AI conversations start with the same question: Which model should we use? With models becoming more and more commoditized, the big differentiator is whether your data is ready for any model or any agent to use.

The stakes are higher now that AI agents are expected to autonomously discover, access, and reason over enterprise data. A model that can’t find the right data is limited. An agent that can’t find the right data—or accesses data it shouldn’t—is a liability. 

According to a 2026 Linux Foundation report, model capability isn't the main hurdle for enterprise AI anymore. Rather, companies struggle with data access, governance, and organizational readiness, among other factors. The study emphasizes that an agent is only as effective as its available data and business context. This reinforces data readiness and runtime control as the biggest unsolved hurdles to scaling AI.

What follows is a practical framework built around 3 dimensions of a data strategy: foundation, trust, and ecosystem.

3 dimensions of a data strategy

These dimensions aren’t a checklist you adopt all at once—they represent a maturity model. The foundation dimension comes 1st: You can’t govern or integrate data you haven’t discovered. But trust and ecosystem aren’t sequential stages. They develop in parallel, reinforcing each other as your data strategy matures.

Foundation: Connect, discover, understand, trace

You can’t use data you can’t find, and you can’t trust data you can’t trace. This dimension is the most unglamorous and the most critical. It covers 4 capabilities: automated discovery, metadata cataloging, standard application programming interfaces (APIs), and end-to-end lineage. 

Crucially, this foundation must encompass both structured analytics data and the massive volumes of unstructured data—like PDFs, internal wikis, and communications—that power generative AI and retrieval-augmented generation (RAG) workflows.

• Discovery and cataloging: The challenge is familiar: data that’s scattered across dozens of sources, schemas that are undocumented or out of date, metadata that’s missing or locked inside individual teams' heads. What works is automated schema discovery through connectors that introspect sources and centrally catalog metadata. Searchable catalog APIs let any team or system find what data exists, understand its shape, and learn how to access it. For unstructured data, this means effective metadata tagging and integration with vector databases that allow AI models to surface relevant context.
• Lineage and traceability: Lineage isn’t something you bolt on after the fact. It requires instrumentation at every component boundary—from pipelines to transformations to model training to agent use. Tracking runtime lineage prevents upstream data changes from cascading into silent downstream failures. The approach that works is a decoupled event emission model: Every component emits its own lineage events, and a shared server correlates them. This scales better than centralized crawling and gives you provenance without coupling your pipeline components.

In practice, these foundation capabilities map to a layered architecture that flows from data sources through to AI applications:

Figure 1: End-to-end flow of data foundation capabilities.

1. Ingestion and connectivity brings in data from enterprise sources through batch, streaming, and change data capture patterns.
2. Compute engines process and transform data at scale, from document parsing to feature engineering.
3. Data abstraction and cataloging decouples raw data from AI applications, serving both predictive workloads (feature stores) and knowledge retrieval (vector databases, RAG).
4. Orchestration, lineage, and governance provide end-to-end traceability and access control across the entire pipeline.
5. Unified AI experience surfaces these capabilities to developers, agents, and applications through standard APIs/Model Context Protocol (MCP) servers.

Each capability builds on the ones before it. The key design choice is open standards that connect these capabilities rather than proprietary interfaces, so each component can be adopted or replaced independently.

Trust: Governed, controlled, agent-ready

Once data is discoverable and traceable, the next question is whether you can govern who and what accesses it. For regulated industries, this isn’t optional. For organizations deploying AI agents, it’s urgent.

The governance gap is especially stark for AI agents. Organizations are deploying agents faster than they’re governing them. Existing access control can answer whether a specific agent can use a specific tool, but not what data that agent can access for that specific task based on organizational policy.

Closing this gap requires treating governance as a translation layer—taking business-level policy documents and converting them into machine-enforceable access rules that feed into existing enforcement points. However, traditional deterministic security logic like role-based access control (RBAC) is insufficient for non-deterministic large language models (LLMs). An agent might have standard access to a dataset, but that doesn’t prevent prompt injection attacks or stop an LLM from inadvertently summarizing sensitive data in a chat interface. A true AI trust strategy requires system-level access controls paired with semantic output controls, such as LLM guardrails and prompt filtering (for example, TrustyAI).

Generating test scenarios per role, running positive and negative access checks, and producing conformity scores that prove an agent can’t exceed its assigned boundaries are essential. Token scoping for machine identity, audit trails for agent data access, and evaluation frameworks that test agent behavior against governance policies are prerequisites, not follow-ups.

Ecosystem: Extensible by design

Enterprises don’t start from zero. They already have data infrastructure like ingestion pipelines, streaming platforms, storage across cloud and on-premise, Software-as-a-Service (SaaS) analytics platforms, and tuned data warehouses. An AI data platform that demands rip-and-replace adoption stalls at procurement. The question is not whether your platform can do everything, it’s whether your platform can work with what you already have.

Enterprises need a platform designed for integration: open APIs, standards-based interfaces, and a clear separation between what the platform provides natively and what it delegates to the ecosystem.

• Standard APIs as the integration layer: Implement 1 API surface, and every major query engine connects using its built-in client. The same principle applies across the stack for lineage interoperability and cross-platform readability.
• Cross-platform interoperability: Major engines should be able to read data stored in open formats without platform-specific connectors. An open table managed by 1 platform can be queried from another without data movement. The API contract is stable, and the implementation behind it can change without breaking consumers.
• Federate, don’t replace: The right integration model provides governance, catalog, and orchestration capabilities while letting partners and existing infrastructure handle specialized data movement and storage. However, it’s vital to acknowledge the physical limitations of federation for heavy AI workloads. While federation is excellent for discovery and lightweight querying, machine learning training and heavy vector-embedding workloads often require localized, high-speed data access to avoid crippling latency and egress costs.

Extensibility without governance is a liability. The ecosystem and trust dimensions reinforce each other: Standard APIs make governance enforceable across platforms, and governance makes it safe to open the platform to external tools and engines.

What we’re seeing in open source

The open source data infrastructure ecosystem is maturing rapidly, with clear patterns emerging in some categories and healthy fragmentation in others. For leaders evaluating their stack, several trends stand out:

• Storage formats are settling, but catalogs aren’t: Open standards for underlying data storage formats have largely been settled (for example, Apache Iceberg table format). Competition has shifted entirely to the open source catalog layer. Organizations are increasingly looking for catalog solutions that offer backend flexibility without locking them into a single vendor's API (for example, Iceberg REST Catalog Spec).
• The engine stack is shifting: There’s a strong industry movement toward memory-safe, high-performance languages and columnar data formats. Furthermore, embedded analytics engines—which run directly where the data lives rather than requiring a dedicated, heavy compute cluster—are among the fastest-growing open source architectural patterns.
• Orchestration is fragmented by design: Orchestration is the 1 category where no single unified answer is emerging. With multiple archetypes (workflow-centric, data-aware, event-driven, AI-native), this fragmentation reflects the diverse and complex realities of modern enterprise data pipelines.
• AI data infrastructure is becoming a standalone category: Capabilities like document processing for unstructured data, LLM observability, and AI-native extraction and transformation are emerging as distinct investment areas in open source communities. They’re no longer just bolted-on extensions of existing data platforms—they require dedicated strategies.
• Vector search is a real architectural decision: Adding open source vector search capabilities to existing relational databases (for example, adding PGVector to existing PostgreSQL) is often simpler and entirely sufficient for many early generative AI use cases. While dedicated vector databases matter at massive, billion-scale workloads, assuming you need a standalone vector database from Day 1 is a common strategic overinvestment.

When we aligned more than 100 data-related open source projects against the 5 capabilities described earlier (ingestion, compute, catalog, orchestration/governance, unified experience), a clear picture emerged. 

Figure 2: Sample mapping of open source projects in the data space to data foundation capabilities.

Every capability has at least 1 mature open source option, and most have several. The ingestion and catalog layers have the most active competition, with multiple viable projects at different maturity levels. Orchestration and governance have the widest gaps between what’s available and what enterprises need. The foundational layer (Kubernetes, columnar data formats, open lineage standards, data contracts) is the most settled, which is exactly what you want from a foundation.

Deciding whether to build or buy data management software is a real challenge. With an active acquisition wave in the data tooling market, choosing where to invest is genuinely difficult. Back solutions with broad community adoption and foundation governance, not just the newest entrant.

Where to start

Begin with 5 concrete steps:

1. Audit your catalog coverage across all data types: Can every team in your organization find the structured and unstructured data it needs without asking someone? If not, start there. Automated discovery and a searchable catalog solve more problems than any model upgrade.
2. Instrument lineage from Day 1: Don’t treat lineage as a compliance checkbox you’ll get to later. Adopt open lineage standards now, and require every new pipeline to emit lineage events.
3. Adopt open standards, not products: Back API contracts rather than betting on a single vendor's implementation. This is how you maintain flexibility and avoid vendor lock-in.
4. Start your agent strategy with comprehensive governance: Before building agentic data access, define what an agent is allowed to do. Use both traditional access controls and semantic safeguards like prompt filtering.
5. Think hybrid from the beginning, but plan for data gravity: If your data spans clouds and on premise, your data strategy needs to work across all of them. Use a consistent operational model, but be strategic about data placement to minimize latency for computationally heavy AI tasks.

Data is the strategy

Enterprise AI doesn’t stall because of models, and agents will not succeed without data they can discover, trace, and access within governed boundaries.

The enterprises that adopt these principles now will be the ones running production AI at scale, not just proofs of concept.

Explore Red Hat AI capabilities

If your organization is looking to build a security-focused, scalable, and hybrid-ready data foundation for your AI initiatives, Red Hat can help. Explore how Red Hat AI integrates with your existing data infrastructure to bring data discovery, effective governance, and AI workloads together, empowering you to scale enterprise AI without ripping and replacing your current investments.