Jupyter notebook server on Odyssey

This document is to help you run Jupyter server remotely on compute nodes of the cluster and connect to your computational notebook using a local browser of your workstation/laptop. 

Start a job:

After login to the login node on Odyssey, you will need to start a job on the cluster. While you are setting up an environment, we recommend using an interactive node to configure your conda environment. 

$ ssh user@login.rc.fas.harvard.edu

$ srun --pty -p test -t 60 --mem 4000  /bin/bash

Conda environments:

To set up the necessary python packages.

holy7c05314>$ module load Anaconda3/5.0.1-fasrc01

If you already have your own Jupyter environment, activate your environment source activate environment_name and skip to the next step 

holy7c05314>$ conda create -n jupyter_3.6 python=3.6 jupyter  ## you can add any other package to install you might need.

holy7c05314>$ source activate jupyter_3.6

If you want python2, you can create a conda environment with version 2 like conda create -n jupyter_2_7 python=2.7 jupyter. You can create as many conda environments needed and load the one you want to work within your notebook server. 

If you forget to activate the environment Jupyter will not see packages you installed. You have less control over the versions of packages which are part of Anaconda so we recommend you to have a conda environment for you or your lab to share. 

Returning Users:

You can request an interactive node based on the walltime and memory of the job or create your sbatch script to start your Jupyter server.   

Whichever way you are working with SLURM to get access to a compute node, you need to load the modules and source the conda environment created during installation. In case of an interactive node, 

  holy7c05314>$ module load Anaconda3/5.0.1-fasrc01

 holy7c05314>$ source activate jupyter_3.6

Use conda env list to list available environments. 

To help you connect to the Jupyter server, run the following command to get the hostname.

Decide which port to use ( 12345 is just an example. any number around 10000 would work)

$ export myport=12345

$ echo "ssh -NL $myport:$(hostname):$myport $USER@login.rc.fas.harvard.edu"

ssh -NL 12345:holy7c05314.rc.fas.harvard.edu:12345 user@login.rc.fas.harvard.edu

Copy the output of the command for your user and hostname. 

Starting the notebook server:

Finally, you have all the components and you can start Jupyter and connect to it. if the port is 

(jupyter_3.6) holy7c05314>$jupyter-notebook --no-browser --port=$myport --ip=''
[I 16:04:57.029 NotebookApp] Serving notebooks from local directory: /n/home_rc/user
[I 16:04:57.029 NotebookApp] 0 active kernels
[I 16:04:57.030 NotebookApp] The Jupyter Notebook is running at:

New Terminal: 

In a new terminal on your machine, connect to Odyssey using an output of your echo command.

$ ssh -NL 12345:holy7c05314.rc.fas.harvard.edu:12345 user@login.rc.fas.harvard.edu
Verification code:

Note: If this works you will not see anything further output. This means you are able to connect to port 12345 on the interactive node. 

Access Notebook:

Then in your workstation/laptop browser. Make sure to copy the token from the Jupyter notebook server and update the token below.


Adding an R Kernel to Jupyter:

After you activate the jupyter conda environment. Please follow the steps below to setup R Kernel

(jupyter) [rjsingh@holy7c19315 ~]$ module load R

(jupyter) [rjsingh@holy7c19315 ~]$ R


>devtools::install_github('IRkernel/IRkernel') >IRkernel::installspec()

This will take few mins to install required libraries for R and add the R kernel to Jupyter. For more details please check here. Got to the Returning User section to start Jupyter.  


CC BY-NC-SA 4.0 This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Permissions beyond the scope of this license may be available at Attribution.