This KB Article References:
High Performance Computing
This Information is Intended for: Faculty, Guests, Researchers, Staff, Students
Last Updated: October 14, 2022
Average Rating: Not Rated
This Information is Intended for: Faculty, Guests, Researchers, Staff, Students
Last Updated: October 14, 2022
Average Rating: Not Rated
Your feedback is important to us, help us by logging in to rate this article and provide feedback.
What does each queue provide?
The following table details resource limits on jobs submitted to each queue via Slurm:
Queues accessed from login1 and login2:
|
Queue |
Default run time |
Max run time |
Max # of nodes |
Min # of nodes |
Max # of simultaneous jobs per user |
| debug-28core | 1 hour | 1 hour | 8 | n/a | n/a |
| extended-24core | 8 hours | 7 days | 2 | n/a | 4 |
| extended-28core | 8 hours | 7 days | 2 | n/a | 6 |
| gpu | 1 hour | 8 hours | 2 | n/a | 2 |
| gpu-long | 8 hours | 48 hours | 1 | n/a | 2 |
| gpu-large | 1 hour | 8 hours | 4 | n/a | 1 |
| p100 | 1 hour | 24 hours | 1 | n/a | 1 |
| v100 | 1 hour | 24 hours | 1 | n/a |
1 |
| large-24core | 4 hours | 8 hours | 60 | 24 | 1 |
| large-28core | 4 hours | 8 hours | 80 | 24 | 1 |
| long-24core | 8 hours | 48 hours | 8 | n/a | 6 |
| long-28core | 8 hours | 48 hours | 8 | n/a | 6 |
| medium-24core | 4 hours | 12 hours | 24 | 8 | 2 |
| medium-28core | 4 hours | 12 hours | 24 | 8 | 2 |
| short-24core | 1 hour | 4 hours | 12 | n/a | 8 |
| short-28core | 1 hour | 4 hours | 12 | n/a | 8 |
Queues accessed from milan1 and milan2:
| Queue | Default run time | Max run time | Max # of nodes | Min # of nodes | Max # of simultaneous jobs per user |
| extended-40core | 8 hours | 7 days | 2 | n/a | 3 |
| extended-96core | 8 hours | 7 days | 2 | n/a | 3 |
| large-40core | 4 hours | 8 hours | 50 | 16 | 1 |
| large-96core | 4 hours | 8 hours | 38 | 16 | 1 |
| long-40core | 8 hours | 48 hours | 6 | n/a | 3 |
| long-96core | 8 hours | 48 hours | 6 | n/a | 3 |
| medium-40core | 4 hours | 12 hours | 16 | 6 | 1 |
| medium-96core | 4 hours | 12 hours | 16 | 6 | 1 |
| short-40core | 1 hour | 4 hours | 8 | n/a | 4 |
| short-96core | 1 hour | 4 hours | 8 | n/a | 4 |
| a100 | 1 hour | 8 hours | 2 | n/a | 2 |
| a100-long | 8 hours | 48 hours | 1 | n/a | 2 |
| a100-large | 1 hour | 8 hours | 4 | n/a | 1 |
In addition to the limits in the tables above, users cannot use more than 32 nodes at one time unless running jobs in one of the large queues.
How are the queues different?
The queues mostly differ in the maximum runtime and number of nodes that can be allocated to jobs run on them. Some sets of queues offer different hardware. More specifically:
- The debug-28core, short-28core, long-28core, extended-28core, medium-28core, and large-28core queues share a set of identical nodes that have a max of 28 Haswell cores.
- The short-24core, long-24core, extended-24core, medium-24core, and large-24core queues share a set of identical nodes that have 24 Haswell cores.
- The short-40core, long-40core, extended-40core, medium-40core, and large-40core queues share a set of identical nodes that have 40 Skylake cores.
- The short-96core, long-96core, extended-96core, medium-96core, and large-96core queues share a set of identical nodes that 96 AMD EPYC Milan cores.
- The gpu and gpu-long queues share a third set of identical nodes that are similar to those used by the short, long, etc. queues but with 4x K80 24GB GPUs each.
- The p100 and v100 queues each allocate jobs to a single node that has two Tesla P100 16GB or 2x V100 32GB GPUs, respectively.
- The a100, a100-long, and a100-large queues have 4x A100 80GB GPUs and 64 cores if Intel Xeon Ice Lake CPUs.
Which queue should I use?
- Use the gpu, gpu-long, p100, or v100 queues only if you need a GPU for your job. Check your software's documentation if you're unsure.
- Use the debug-28core queue for brief interactive jobs. This can be useful for testing or debugging your code.
- Use the extended queues for jobs that you intend to run for longer periods of time, particularly if:
- Adding more nodes (with or without GPUs) will not speed up your job.
- You're monitoring an outside data source (e.g. social media feeds, web-scraping). Remember to watch your disk usage.
- Use the long queue if you're not sure. Once you understand what your job's needs are, pick another one if it's more suitable.
- Use other queues depending on your needs and the level of MPI support offered by your choice of software.