Julia with Jupyter notebooks on Odyssey

This page is intended to help you run julia kernels in a jupyter notebooks on the Odyssey Cluster.

Important Note: While the currently recommended tool for graphical interactive workflows - including running jupyter notebooks - is our NX remote desktops, please be aware that we are in the process of deploying a new technology that will substitute NX, so stay tuned for updates to this page. 

Connect to the Remote Desktop login

To connect with NX please follow the instructions here 

Setup the necessary Python and Julia packages:

Once you are on holynx01 (or rcnx01) and not already did the setup, follow the steps to download/setup required packages. We would recommend doing the setup in an interactive session. After setup, you will be simply able to load the conda environment to work with your notebooks.

Note: Setup takes more than 30+ mins and we are in process of improving it. 

Open the interactive session on a compute node:

Use srun to open an interactive session.  

holynx01>$ srun --pty -p shared  --mem 4000 -t 300 --tunnel 8911:8888 /bin/bash

Once you are in the interactive session run source centos7-modules.sh, that will enable the CentOS7 specific modules and rectify your run environment. 

The option --tunnel sets up a tunnel between the holynx01 and the compute node, mapping port 8888 (Jupyter's default port). Please note that the port number (8911) is totally arbitrary. You can use any number which is not reserved for system services ( typically in the range 8000-10000). If the port happens to be already in use you will see an error from SLURM, in that case simply pick another port. 

Setup the necessary environment:

Once on the node, you should load the module for Julia and for Python (Anaconda) and create your own conda environment.
Please refer to module-query julia or the portal for information on the different versions of Julia and select the version you would like.

holy7c05314>$ module load Anaconda3/5.0.1-fasrc01
holy7c05314>$ module load julia/0.6.3-fasrc01 

If you already have your own Jupyter environment, activate your environmentsource activate environment_name and skip to the next step to install the Julia kernel 

holy7c05314>$ conda create -n jupyterjulia python=3.6 jupyter ## (and you can add any other package you think you might need)
... omitted output ....
holy7c05314>$ source activate jupyterjulia
(jupyterjulia) holy7c05314>$

Note: If you forget to activate the environment Julia will not find Jupyter and the next step will install it for you. You may have less control over the versions of packages. 

Setup the kernel for Julia:

Now that you have an environment with Jupyter and Julia loaded in your environment, you need to install IJulia package, which binds the Julia kernel for Jupyter.  

(jupyterjulia) holy7c05314>$ julia
julia> Pkg.add("IJulia") 
some output 
INFO: Building IJulia
INFO: Found Jupyter version 4.4.0: jupyter
INFO: Installing Julia kernelspec julia-0.6
[InstallKernelSpec] Installed kernelspec julia-0.6 in /n/yourhomefolder/.local/share/jupyter/kernels/julia-0.6
INFO: Package database updated
julia> exit() 

Note: This can take few mins.

Returning Users:

Load the modules and source the conda environment created during installation. 

 holy7c05314>$ source activate jupyterjulia

Use conda env list to list available environments. 

Use the notebook:

Finally, you have all the components and you can start Jupyter and connect to it.

(jupyterjulia) holy7c05314>$ jupyter notebook 
..some output... including
 Copy/paste this URL into your browser when you connect for the first time,
to login with a token:   

Now open firefox on the Remote Desktop (holynx01) and type http://localhost:8911/?token=LongSecretString  that Jupyter gave while starting and also, please note the port changed in the address. 

To use Julia kernel, select that and your Julia notebook will open in a new tab.


CC BY-NC-SA 4.0 This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Permissions beyond the scope of this license may be available at Attribution.