AMD EPYC - The Ideal Platform for AI

Although GPUs do much of the heavy lifting in AI workloads, CPUs are still critically important, especially for data preprocessing, data loading, system orchestration, running lightweight inference tasks and managing data movement, input/output operations, and control flow for the entire system. No matter the size or scale of your AI deployments, AMD EPYC 9005 server CPUs give you a high-performance, energy-efficient foundation for enterprise AI alongside general-purpose workloads.

AMD EPYC 9005-series CPUs are ideal to support GPU-accelerated AI workloads due to their high core counts and high memory bandwidth, plus up to 37% more AI instructions per clock (IPC) over previous generations. They also reduce GPU bottlenecks by handling data preparation and post-processing efficiently, resulting in performance gains in many key AI applications.

The EPYC 9005-series CPUs based on the Zen 5 architecture deliver up to 35% faster AI inference AVX-512 vector and matrix operations, compared to previous Zen 4 CPUs, now enabling efficient handling of local LLMs and generative AI workloads.

Performance & Efficiency

AMD EPYC 9005-series CPUs can increase GPU-accelerated system performance by up to 20% compared to competing CPUs, particularly in AI training and inference. The CPUs act as an optimised controller for data, ensuring GPUs remain fully utilised and not waiting for instructions.

• High-frequency processing at up to 5GHz provides up to 28% higher frequency than competitors which significantly speeds up data preparation, kernel launches, and post-processing tasks.
• Certain EPYC 9005 configurations feature up to 384 cores across two sockets (192 cores per CPU), enabling massive parallelism combined with balanced throughput.
• Increased PCIe bandwidth with up to 160 PCIe Gen 5 lanes in dual-socket configurations (or 128 in single socket), prevent bottlenecks by allowing massive data throughput between the CPU and GPUs, essential for AI training.
• 12-channel DDR5-6400 ECC memory efficiency supports massive memory bandwidth of up to 6TB per socket, ensuring data is fed to GPUs faster, reducing read / write cycles.
• Large cache sizes of up to 512MB reduce the latency of retrieving data from slower system RAM, ensuring the GPU spends more time calculating and less time waiting for data.
• AMD works closely with AI software vendors such as NVIDIA CUDA / TensorRT-LLM to ensure optimal performance, leading to up to 50% faster GPU hosting tasks.

You can learn more about how to fine tune AMD EPYC CPUs for AI workloads by reading the AMD Artificial Intelligence / Machine Learning (AI/ML) Tuning Guide.

Security

The exponential growth of data sharing fuelled by AI, collaborative applications and cloud computing, combined with increased global emphasis on privacy regulations and severe penalties for breaches underscores data’s unparalleled value amid rising security risks. To address this, AMD EPYC 9005 CPUs feature the Infinity Guard suite of system-level security features designed for a data-driven world. Built into AMD EPYC processors, its state-of-the-art capabilities are designed to secure sensitive data-in-use - often critical for AI workloads, hence an attractive target for hackers.

The technology includes AMD Secure Boot, that defends against firmware threats, extending the silicon root of trust to help protect the BIOS, enhancing security against malware and attacks in virtualised environments. Also featured is Transparent Secure Memory Encryption (TSME) which helps safeguard against internal security threats with encrypted data protection.

Secure Encrypted Virtualisation (SEV) encrypts and protects the memory of each VM memory space; whilst AMD Shadow Stack offers hardware-enforced stack protection, mitigating malware attacks and maintaining return addresses for integrity checks. Finally Trusted I/O (TIO) extends the Trusted Execution Environment (TEE) to include external trusted devices, such as GPUs and other accelerators. You can learn more about these security features by reading our AMD Server CPU Buyers Guide.

3XS Servers with AMD EPYC

Our in-house 3XS Systems division has been handcrafting PCs, workstations and servers for more than 30 years. We also pioneered the AI dev box back in 2016, so we have a huge amount of experience in building highly reliable AI systems that deliver the most performance for your budget. Here are the key reasons to buy an AMD EPYC-powered GPU server from Scan.