This KB Article References:
High Performance Computing
This Information is Intended for: Faculty, Guests, Researchers, Staff, Students
Last Updated: July 01, 2021
Average Rating: Not Rated
This Information is Intended for: Faculty, Guests, Researchers, Staff, Students
Last Updated: July 01, 2021
Average Rating: Not Rated
Your feedback is important to us, help us by logging in to rate this article and provide feedback.
What does each queue provide?
The following table details resource limits on jobs submitted to each queue via Slurm:
|
Queue |
Default run time |
Max run time |
Max # of nodes |
Min # of nodes |
Max # of simultaneous jobs per user |
| debug-28core | 1 hour | 1 hour | 8 | n/a | n/a |
| extended-24core | 8 hours | 7 days | 2 | n/a | 4 |
| extended-28core | 8 hours | 7 days | 2 | n/a | 6 |
| extended-40core | 8 hours | 7 days | 2 | n/a | 3 |
| gpu | 1 hour | 8 hours | 2 | n/a | 2 |
| gpu-long | 8 hours | 48 hours | 1 | n/a | 2 |
| gpu-large | 1 hour | 8 hours | 4 | n/a | 1 |
| large-24core | 4 hours | 8 hours | 60 | 24 | 1 |
| large-28core | 4 hours | 8 hours | 80 | 24 | 1 |
| large-40core | 4 hours | 8 hours | 50 | 16 | 1 |
| long-24core | 8 hours | 48 hours | 8 | n/a | 6 |
| long-28core | 8 hours | 48 hours | 8 | n/a | 6 |
| long-40core | 8 hours | 48 hours | 6 | n/a | 3 |
| medium-24core | 4 hours | 12 hours | 24 | 8 | 2 |
| medium-28core | 4 hours | 12 hours | 24 | 8 | 2 |
| medium-40core | 4 hours | 12 hours | 16 | 6 | 1 |
| p100 | 1 hour | 24 hours | 1 | n/a | 1 |
| short-24core | 1 hour | 4 hours | 12 | n/a | 8 |
| short-28core | 1 hour | 4 hours | 12 | n/a | 8 |
| short-40core | 1 hour | 4 hours | 8 | n/a | 4 |
| v100 | 1 hour | 24 hours | 1 | n/a | 1 |
In addition to the limits in the table above, users cannot use more than 32 nodes at one time on the Slurm queues unless running jobs on the large-24core or large-40core queues.
How are the queues different?
The queues mostly differ in the maximum runtime and number of nodes that can be allocated to jobs run on them. Some sets of queues offer different hardware. More specifically:
- The debug, short, long, extended, medium, and large queues share a set of identical nodes that have a max of 28 Haswell cores.
- The short-24core, long-24core, medium-24core, and large-24core queues share a second set of identical nodes that have 24 Haswell cores.
- The short-40core, long-40core, medium-40core, and large-40core queues share a second set of identical nodes that have 40 Skylake cores.
- The gpu and gpu-long queues share a third set of identical nodes that are similar to those used by the short, long, etc. queues but with multiple GPUs each.
- The p100 and v100 queues each allocate jobs to a single node that has two Tesla P100 or V100 GPUs, respectively.
Which queue should I use?
- Use the gpu, gpu-long, p100, or v100 queues ony if you need a GPU for your job. Check your software's documentation if you're unsure.
- Use the debug queue for brief interactive jobs. This can be useful for testing or debugging your code.
- Use the extended queues for jobs that you intend to run for longer periods of time, particularly if:
- Adding more nodes (with or without GPUs) will not speed up your job.
- You're monitoring an outside data source (e.g. social media feeds, web-scraping). Remember to watch your disk usage.
- Use the long queue if you're not sure. Once you understand what your job's needs are, pick another one if it's more suitable.
- Use other queues depending on your needs and the level of MPI support offered by your choice of software.