The race to achieve Return on Investment (ROI) from enterprise AI is becoming increasingly important as enterprises are deploying AI in production. Organizations are moving past simple chatbots and isolated proofs-of-concept into the era of Agentic AI. This transition requires massive computing environments, AI Factories, to support large foundation models and agentic workloads. Yet AI Factories with their massive compute alone are not enough.
An AI factory cannot run without its fuel (data). To build a high-performing Enterprise AI Factory, you must first build a highly available and scalable Enterprise Data Factory — The Dell AI Data Platform.
The AI bottleneck is no longer compute — it’s data
Many organizations face the same hurdle: their most valuable corporate insights are trapped within fragmented departmental silos across structured and unstructured datasets of multiple modalities. If you feed unstructured, unverified or disconnected data into a GPU cluster, your AI applications will fail, at best. At worst, your AI factory will consume orders of magnitude additional CPU and GPU processing power in trying to find and analyze data in various distributed data sources, while trying to handle an incoming request. Garbage in, garbage out was true in the past and becomes even more relevant with agentic applications; the disconnected, unprocessed and raw data cannot be processed by AI factories. They require AI-ready and governed context that can be semantically searched, and this context serves as the primary data for agents.
In enterprise AI deployments, the Enterprise Data Factory is as important as the AI Factory itself. The Dell AI Data Platform plays this role. It sits between enterprise data sources and the Dell AI Factory with NVIDIA, transforming raw information into the high-quality context that models and agents need to reason, decide and act.
This is context engineering. The Dell AI Data Platform does more than clean and move data. It curates metadata, semantic relationships, embeddings and searchable indexes from fragmented enterprise information, then persists that AI-ready context on high-performance Dell storage. Through orchestrated pipelines, it keeps the context current as source data changes. That context is then served to agents and models running in the Dell AI Factory, depending on the data sensitivity, governance requirements and the specific task agents are trying to accomplish.
Anatomy of the data factory: Inside the Dell AI Data Platform
A scalable and highly available enterprise data factory delivers fully managed, always-on pipelines to transform raw enterprise data into AI-ready context that agents and LLMs can use. The core underlying system relies on three layers to ingest raw information, refine it and continuously serve it to models safely:
-
- Data Orchestration & Processing Engines
Rather than moving data passively, the Dell Data Orchestration Engine (DOE) converts raw data into AI-ready context for agents running in Dell AI Factory. It provides automated data pipelines that actively pull information from fragmented enterprise sources (such as databases, Confluence and SharePoint) and unstructured data lakes (documents, images, videos on NAS and Object Storage).
Acting as an enterprise-wide control layer, the DOE routes this freshly pulled data through three specialized subsystems:
-
-
-
- Data Search Engine: Built using Elastic and NVIDIA cuVS, it enables fast, hybrid keyword and vector searches so RAG pipelines can scan billions of files instantly. Integration with NVIDIA cuVS enables 12x faster ingestion and indexing of data.
-
- Data Analytics Engine: Governs and processes structured table formats by connecting to existing enterprise data sources, while accelerating queries and processing by 6x with NVIDIA cuDF.
-
- Data Processing Engine (NVIDIA GPU, cuDF and Apache Arrow Accelerated): Cleans, chunks, transforms and filters raw data into governed context that LLMs can interpret safely. It also transforms structured tabular data into high-quality data products for consumption by agents.
-
-
To eliminate the massive computational bottlenecks for traditional CPU-bound ETL (Extract, Transform, Load) pipelines, the Data Processing Engine leverages NVIDIA cuDF libraries running directly on NVIDIA GPUs. It enables heavy string manipulation, tokenization and regex filtering tasks across thousands of GPU cores.
Apache Arrow data format is utilized across Spark and Dell ObjectScale storage to optimize data movement and to accelerate processing by 6x across CPUs and GPUs. These architectural choices deliver key benefits:
-
-
-
- Drastic pipeline acceleration: Data engineering workloads that previously took hours or days on traditional CPU clusters are completed in minutes.
-
- Zero code rewrites: Because NVIDIA cuDF mirrors standard data science APIs, existing enterprise data pipelines accelerate seamlessly without requiring application changes.
-
- Optimized resource efficiency: Offloading data processing to a hybrid CPU and GPU cluster, and using optimized data formats slashes infrastructure footprint and keeps the entire pipeline tightly coupled to the underlying high-performance storage.
-
-
-
- High-Performance Scale-out Storage
Dedicated, high-performance physical storage is the foundation for the Data Factory:
-
-
-
- Dell PowerScale: A mature scale-out NAS system built to manage persistent file environments and stateful AI application containers.
-
- Dell ObjectScale: A cloud-native, S3-compatible object layer that accelerates data delivery with S3 over RDMA and Apache Arrow, dramatically reducing latency.
-
- LightningFS: A blazing-fast parallel file system designed to handle the hot runtime context and KV cache demands of GPU clusters.
-
-
AI factory compute powering the application cluster for data orchestration engine
The data orchestration engine performs all data processing operations in a Kubernetes-based application cluster.
Inside this application cluster, open-source models, custom AI pipelines and agentic workflows can run safely using the company’s private data. The services running in the application cluster can connect to locally instantiated AI models or externally hosted models for optimized data processing.
High-performance storage feeds clean, high-quality data to the data orchestration cluster. The data travels through secure gateways straight into the AI Factory GPU Cluster. This enables a fast, secure path from raw corporate information to real business value.
Connecting the data factory to the AI factory
An AI Factory Compute and a Data Factory are the two halves of a fully running enterprise AI Factory delivering automated business outcomes with agents. The accelerated compute, networking, model services and application environment provided by Dell AI Factory Compute Cluster powers training, inference and agentic workflows. The Dell AI Data Platform provides the governed data foundation that makes those workloads useful by preparing the right context and delivering it at the right time.
Together, they create a connected path from enterprise data to AI outcomes. The Dell AI Data Platform ingests information from existing sources, enriches it with metadata and semantic understanding, persists it on Dell storage and keeps it refreshed through fully managed, always-on, policy-based, orchestrated pipelines. The Dell AI Factory then uses this trusted context with locally deployed or externally hosted large models to power RAG, agentic workflows, analytics and inference. Results and new data signals can flow back into the Data Factory, creating a continuously improving foundation for enterprise AI.
That is the role of an Enterprise Data Factory: not simply to move data to GPUs, but to engineer the context that allows AI systems to reason over enterprise knowledge safely, efficiently and at production scale. With the Dell AI Data Platform connected to the Dell AI Factory, organizations are deploying the data and compute foundations together instead of treating data readiness as an afterthought.
Ready to build your AI Data Factory?
If you’re looking to turn enterprise data into a real advantage for agentic AI, this is the next step. Dell AI Data Platform is built to help organizations create AI-ready datasets, automate data pipelines and support modern workloads across the full AI lifecycle. Start by registering for the event: Join “From Ambition to AI at Scale”
Explore how Dell is helping customers build an open, modular data foundation for AI:
Dell reported this
Source: www.dell.com
Source link
