The Power of Artificial Intelligence at CPU Prices

Generative AI has moved quickly from experimentation to enterprise deployment — and as organizations build out their AI strategies, one question keeps surfacing: Where does each workload actually run best?

For compute-intensive training and the largest-scale model inference, high-performance graphics processing units remain essential. But as AI models reach production and workloads become better defined, many organizations are discovering that modern CPU infrastructure can handle a broader range of AI tasks than they initially expected.

Deciding where each AI workload should run isn’t just a technology question, it’s an architectural one. CDW helps organizations assess AI workloads across training, inference and production to align each stage with the right mix of CPU and GPU infrastructure, avoiding overprovisioning while preserving performance.

The capability of large language models is defined by the number of variables they can adjust during training to learn patterns and relationships in data. These variables are called parameters, and LLMs can contain tens to hundreds of billions of parameters, depending on their design and intended use.

For many enterprise deployments, models in the range of 7 billion to 15 billion parameters are often sufficient, especially for targeted use cases such as retrieval-augmented generation (RAG), summarization, document processing and domain-specific assistants. This is the sweet spot for Intel Xeon 6 processors, powered by Intel Advanced Matrix Extensions.

For many organizations, the real challenge isn’t model performance, it’s choosing the right compute architecture for production AI. CDW works with customers to map model size, data sensitivity, latency and cost requirements to the most effective accelerated compute design, whether that’s Xeon-powered inference, hybrid CPU/GPU environments or edge deployments.

As AI strategies mature, organizations are putting more scrutiny on where workloads run and why. CDW helps organizations evaluate AI workloads end to end — balancing performance, cost, data gravity and operational complexity — to ensure accelerated compute is deployed where it delivers the most impact.

The result is a growing interest in matching the right compute infrastructure to each stage of the AI lifecycle — using GPU infrastructure for what it does best, and leveraging capable CPU infrastructure, such as Xeon 6, for production workloads where it delivers strong results.

Intelligent monitoring: Computer vision and intelligent monitoring illustrate how a well-tuned, rightsized model can perform every bit as well as one that is overprovisioned. Consider a hospital system that wants to use AI to track staff movement and presence in patient rooms to support compliance, operational efficiency and patient safety. A general-purpose model might be 95% of the way there, capable of detecting human figures and tracking movement, identifying roles by uniform color. A healthcare system might use GPU infrastructure to fine-tune the model for its environment (training it to recognize that nurses wear burgundy scrubs, for example) but then hand off the model to a Xeon 6 processor. This way, organizations can rightsize their production environments while reserving their GPUs for intensive tasks, including model training.

Image processing: Beyond video monitoring, Xeon 6 supports a range of image and document processing use cases. For example, AI models might extract structured data from unstructured visual inputs, classify images at scale or process scanned documents or forms. For organizations dealing with high-volume document workflows (such as insurance claims, logistics receipts or medical imaging), inference-stage image processing on Xeon 6 infrastructure can deliver the performance and throughput these workloads require.

Technical documentation: One of the most compelling use cases for rightsized AI is RAG, a technique that allows an LLM to pull answers directly from a curated set of documents, rather than relying solely on training data. In the military, for example, each aircraft might have a 10,000-page technical manual. When a RAG model is trained on this information, users can ask questions about specific components in natural language and receive instant answers, rather than wading through thousands of pages. Similarly, teams in fields such as healthcare, finance, engineering and law can use RAG-powered tools to quickly access information hiding within dense documentation. By supporting these models with Xeon 6, organizations can both streamline operations and keep their sensitive data in-house.

Business unit support: Xeon 6 processors are powerful enough to run RAG-trained models that support core business functions at the departmental level. In HR, for example, a model trained on employee handbooks, benefits documentation and policy guides can become a self-service resource that handles routine questions without burdening HR staff. A sales model trained on product specifications and pricing documentation can help account managers answer customer questions faster, without escalating to product teams. And in engineering, a RAG deployment can provide an interface that can be queried for vast libraries of specs, design documents and historical project data. Intel has actually deployed exactly this kind of model internally. When a customer asks a detailed question about whether a specific Intel Ethernet card carries a particular compliance certification, the answer is just an AI query away.

As organizations move from AI proof-of-concept to production, success depends on placing the right workloads on the right infrastructure. CDW helps organizations design, deploy and optimize accelerated compute environments, aligning CPUs, GPUs and architectures to real AI workloads so teams can deploy AI confidently, efficiently and at scale. Intel Xeon 6 gives teams a powerful, flexible foundation for running production AI workloads at enterprise scale, complementing the broader AI infrastructure stack and enabling organizations to deploy capable AI more broadly across the business.