This KB Article References:
High Performance Computing
This Information is Intended for: Faculty, Guests, Researchers, Staff, Students
Last Updated: November 25, 2019
This Information is Intended for: Faculty, Guests, Researchers, Staff, Students
Last Updated: November 25, 2019
What does each queue provide?
The following table details resource limits on jobs submitted to each queue via Slurm:
|
Queue |
Default run time |
Max run time |
Max # of nodes |
Min # of nodes |
Max # of simultaneous jobs per user |
| debug-28core | 1 hour | 1 hour | 8 | n/a | n/a |
| extended-24core | 8 hours | 7 days | 2 | n/a | 4 |
| extended-28core | 8 hours | 7 days | 2 | n/a | 6 |
| extended-40core | 8 hours | 7 days | 2 | n/a | 3 |
| gpu | 1 hour | 8 hours | 2 | n/a | 2 |
| gpu-long | 8 hours | 48 hours | 1 | n/a | 2 |
| gpu-large | 1 hour | 8 hours | 4 | n/a | 1 |
| large-24core | 4 hours | 8 hours | 60 | 24 | 1 |
| large-28core | 4 hours | 8 hours | 80 | 24 | 1 |
| large-40core | 4 hours | 8 hours | 50 | 16 | 1 |
| long-24core | 8 hours | 48 hours | 8 | n/a | 6 |
| long-28core | 8 hours | 48 hours | 8 | n/a | 6 |
| long-40core | 8 hours | 48 hours | 6 | n/a | 3 |
| medium-24core | 4 hours | 12 hours | 24 | 8 | 2 |
| medium-28core | 4 hours | 12 hours | 24 | 8 | 2 |
| medium-40core | 4 hours | 12 hours | 16 | 6 | 1 |
| p100 | 1 hour | 24 hours | 1 | n/a | 1 |
| short-24core | 1 hour | 4 hours | 12 | n/a | 8 |
| short-28core | 1 hour | 4 hours | 12 | n/a | 8 |
| short-40core | 1 hour | 4 hours | 8 | n/a | 4 |
In addition to the limits in the table above, users cannot use more than 32 nodes at one time on the Slurm queues unless running jobs on the large-24core or large-40core queues.
How are the queues different?
The queues mostly differ in the maximum runtime and number of nodes that can be allocated to jobs run on them. Some sets of queues offer different hardware. More specifically:
- The debug, short, long, extended, medium, and large queues share a set of identical nodes that have a max of 28 Haswell cores.
- The short-24core, long-24core, medium-24core, and large-24core queues share a second set of identical nodes that have 24 Haswell cores.
- The short-40core, long-40core, medium-40core, and large-40core queues share a second set of identical nodes that have 40 Skylake cores and use the SLURM scheduler.
- The gpu and gpu-long queues share a third set of identical nodes that are similar to those used by the short, long, etc. queues but with multiple GPUs each and use the SLURM scheduler.
Which queue should I use?
- Use the gpu or gpu-long queue ony if you need a GPU for your job. Check your software's documentation if you're unsure.
- Use the debug queue for brief interactive jobs. This can be useful for testing or debugging your code.
- Use the extended queues for jobs that you intend to run for longer periods of time, particularly if:
- Adding more nodes (with or without GPUs) will not speed up your job.
- You're monitoring an outside data source (e.g. social media feeds, web-scraping). Remember to watch your disk usage.
- Use the long queue if you're not sure. Once you understand what your job's needs are, pick another one if it's more suitable.
- Use other queues depending on your needs and the level of MPI support offered by your choice of software.