How to run a Jupyter notebook on SeaWulf

Jupyter is a popular web-based environment for writing code in Python, as well as other languages.  This article will provide a tutorial on how to launch a Jupyter notebook remotely on SeaWulf and connect to to it in your local browser.  Parts of this tutorial are adapted from the following page:  Jupyter on the Cluster.  

Audience: Faculty, Postdocs, Researchers, Staff and Students

This KB Article References: High Performance Computing
This Information is Intended for: Faculty, Postdocs, Researchers, Staff, Students
Last Updated: June 20, 2023
Average Rating: Not Rated
Your feedback is important to us, help us by logging in to rate this article and provide feedback.

Jupyter is installed on SeaWulf and can be accessed via the following commands:

module load jupyter/7.4.4

To start a new Jupyter notebook on one of the SeaWulf compute nodes, please do the following:

1. Submit a new Slurm job.  For example, using the following template script:

#!/bin/bash

#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH -p short-28core
#SBATCH --time 00:30:00
#SBATCH --job-name jupyter-notebook
#SBATCH --output jupyter-notebook-%J.log

# get tunneling info
XDG_RUNTIME_DIR=""
node=$(hostname -s)
user=$(whoami)
port=9001

# print tunneling instructions to the jupyter-log
echo -e "
Command to create ssh tunnel:
ssh -N -f -L ${port}:${node}:${port} ${user}@login.seawulf.stonybrook.edu

Use a Browser on your local machine to go to:
localhost:${port}"

# load modules or conda environments here
module load jupyter/7.4.4

# Run Jupyter
jupyter notebook --no-browser --port=${port} --ip=${node}

The above script will launch a 30 minute job in the short-28core queue, determine the hostname of the compute node (as well as other useful information), and then launch Jupyter Notebook on the specified port.  Note that the choice of ports is arbitrary.  In this case, we have chosen port 9001 since it's unlikely that this port would be used for any system processes.  

2. Once the job has started running, you will see your log file ("output jupyter-notebook-<job-id>.log") which contains instructions for setting up an ssh tunnel from your computer to the SeaWulf compute node running Jupyter. 

In this case, let's imagine that the job was running on node cn011.  You would then open a new terminal window on your local machine (because you want the tunnel to be opened from the same location as your internet browser) and type the following:

ssh -N -f -L 9001:cn011:9001 <your-netid>@login.seawulf.stonybrook.edu

The above indicates that the ssh command will run in the background and that the port on your local machine (9001) will be forwarded to the port (again 9001 in this case) on SeaWulf compute node cn011.  Please note that if you already have a connection to port 9001 open on your computer, you will need to specify an alternative port number.

3. Open a browser of your choice, and navigate to  http://localhost:9001/

4. If prompted for a password or token, look again in your slurm log file.  There should be a token that you can copy and paste into the browser.  From here, you are ready to start a new notebook or resume an old one!

 

Please note: the instructions above will also work for running R in a Jupyter notebook, except that instead of loading the "jupyter" module, you should do the following instead:

module load r-jupyter/4.0.2

 

SUBMIT A TICKET

 

Additional Information


There are no additional resources available for this article.

Provide Feedback


Your feedback is important to us, help us by logging in to rate this article and provide feedback.

Sign in with NetID

Getting Help


The Division of Information Technology provides support on all of our services. If you require assistance please submit a support ticket through the IT Service Management system.

Submit A Quick Ticket

Supported By


IACS Support System