A guide to SeaWulf's AMD nodes

SeaWulf recently introduced new AMD EPYC 96 core nodes. This article will explain how to access these nodes and provide some recommendations for compiler and math library recommendations.

Audience: Faculty, Postdocs, Researchers, Staff and Students

This KB Article References: High Performance Computing
This Information is Intended for: Faculty, Postdocs, Researchers, Staff, Students
Last Updated: February 13, 2024
Average Rating: Not Rated
Your feedback is important to us, help us by logging in to rate this article and provide feedback.

As with the other types of nodes available on SeaWulf, there are multiple queues (Slurm partitions) available for the 96 core AMD nodes.  Depending on the desired number of nodes and required runtime, users can choose from the following queues to access the 96 core nodes:

  • short-96core
  • long-96core
  • extended-96core
  • medium-96core

Here is an example job script for running a simple "Hello World" job on one of the AMD nodes:

#!/usr/bin/env bash

#SBATCH --job-name=AMD_job
#SBATCH --output=AMD_job.log
#SBATCH --nodes=1
#SBATCH --time=05:00
#SBATCH -p short-96core

# load AMD AOCC compiler
module load aocc/4.0.0

# compile the code
clang /gpfs/projects/samples/helloworld/hello.c -o amd_hello

# execute the code
./amd_hello

In this example, we loaded the aocc/4.0.0 module to get access to the latest version of the AMD Optimizing C/C++ and Fortran Compilers, which are well suited for the AMD CPU architecture.  This compiler suite is based on LLVM, so the compiler commands to use are:
 

Language Command
C clang
C++ clang++
Fortran flang

 

In addition, we offer the aocl/4.0.0 module to provide access to AMD's Optimizing CPU Libraries, a suite of numerical libraries that are optimized for AMD's architecture.  The math libraries present in this software suite include implementations of FFTW, SCALAPACK, BLIS (BLAS-like linear algrebra routines), among many others.

While the AOCC and AOCL compilers and libraries are expected to work well on the AMD nodes, SeaWulf users are not limited to only use AMD compilers. GCC and Intel compilers are available on the AMD nodes, just like they are on all other nodes.

Here are a few flags for different compilers that may help with performance on the AMD nodes:
 

  AOCC GCC Intel
Optimization -O3 -O3 -O3
Architecture -march=znver3 -march=znver3 -march=core-avx2
OpenMP -fopenmp -fopenmp -qopenmp

 

Please see the AMD documentation for a more detailed list of recommended compiler flags.

 

Submit a ticket

Additional Information


There are no additional resources available for this article.

Provide Feedback


Your feedback is important to us, help us by logging in to rate this article and provide feedback.

Sign in with NetID

Getting Help


The Division of Information Technology provides support on all of our services. If you require assistance please submit a support ticket through the IT Service Management system.

Submit A Quick Ticket

Supported By


IACS Support System