A guide to SeaWulf's AMD nodes
SeaWulf recently introduced new AMD EPYC 96 core nodes. This article will explain how to access these nodes and provide some recommendations for compiler and math library recommendations.
As with the other types of nodes available on SeaWulf, there are multiple queues (Slurm partitions) available for the 96 core AMD nodes. Depending on the desired number of nodes and required runtime, users can choose from the following queues to access the 96 core nodes:
- short-96core
- long-96core
- extended-96core
- medium-96core
Here is an example job script for running a simple "Hello World" job on one of the AMD nodes:
#!/usr/bin/env bash #SBATCH --job-name=AMD_job #SBATCH --output=AMD_job.log #SBATCH --nodes=1 #SBATCH --time=05:00 #SBATCH -p short-96core # load AMD AOCC compiler module load aocc/4.0.0 # compile the code clang /gpfs/projects/samples/helloworld/hello.c -o amd_hello # execute the code ./amd_hello
In this example, we loaded the aocc/4.0.0 module to get access to the latest version of the AMD Optimizing C/C++ and Fortran Compilers, which are well suited for the AMD CPU architecture. This compiler suite is based on LLVM, so the compiler commands to use are:
Language | Command |
---|---|
C | clang |
C++ | clang++ |
Fortran | flang |
In addition, we offer the aocl/4.0.0 module to provide access to AMD's Optimizing CPU Libraries, a suite of numerical libraries that are optimized for AMD's architecture. The math libraries present in this software suite include implementations of FFTW, SCALAPACK, BLIS (BLAS-like linear algrebra routines), among many others.
While the AOCC and AOCL compilers and libraries are expected to work well on the AMD nodes, SeaWulf users are not limited to only use AMD compilers. GCC and Intel compilers are available on the AMD nodes, just like they are on all other nodes.
Here are a few flags for different compilers that may help with performance on the AMD nodes:
AOCC | GCC | Intel | |
---|---|---|---|
Optimization | -O3 | -O3 | -O3 |
Architecture | -march=znver3 | -march=znver3 | -march=core-avx2 |
OpenMP | -fopenmp | -fopenmp | -qopenmp |
Please see the AMD documentation for a more detailed list of recommended compiler flags.
For More Information Contact
Still Need Help? The best way to report your issue or make a request is by submitting a ticket.
Request Access or Report an Issue