#

Running Jobs

MPI for Python (mpi4py) on Odyssey

Introduction This web-page is intended to help you running MPI Python applications on the Odyssey cluster using mpi4py. To use mpi4py you need to load an appropriate Python software module. We have the Anaconda Python distribution from Continuum Analytics. In addition to mpi4py, it includes hundreds of the most popular packages for large-scale data processing and scientific computing. You can…

Transferring files to DATA.RC

Users of the data.rc.fas.harvard.edu server have three options for doing so. Please note that if you choose option 2, the settings for FTP/S are not the same as for regular SFTP settings you might use to transfer files to other servers. The connection methods in order of preference: Via Web browser - This is the default means of accessing data.rc…

Common Odyssey Pitfalls

These are some of the common problems that people have when using Odyssey. We hope that these will not be a problem for you as well. Problem Symptom/Reason Throwing multiple cores at Python and R code Without special programming, code written for Python and R is single-threaded. That means give more cores in SLURM to your code will do nothing…

Convenient SLURM Commands

This page will give you a list of the commonly used commands for SLURM. Although there are a few advanced ones in here, as you start making significant use of the cluster, you'll find that these advanced ones are essential! A good comparison of SLURM, LSF, PBS/Torque, and SGE commands can be found here. Also useful: What is FairShare and…

Policy: regal scratch

REGAL SCRATCH RC maintains a large, shared temporary scratch filesystem for general use for high input/output jobs at /n/regal. This workspace is made available with the provisions that each group use no more than 50TB, and that files older than 90 days will be deleted through a periodic purge process (generally during our maintenance period, but also any time the…

SFTP file transfer using Filezilla – Filtering

There may be times when you wish to filter the file listing in the local or remote pane. If you need to do this often, you may want to set up a filter. Unlike the search feature (binoculars icon), filters modify what is shown in the Remote Site: or Local Site: pane. If you simply need to see files grouped together…

SFTP file transfer using Filezilla (Mac/Windows/Linux)

Filezilla is a free and open source SFTP client which is built on modern standards. It is available cross-platform (Mac, Windows and Linux) and is actively maintained. As such Research Computing is recommending its use over previous clients, especially as it does not have some of the quirks of clients like Cyberduck or SecureFX. This document will outline setting up a…

Submitting Large Numbers of Jobs to Odyssey

Introduction Often times one will need to submit a large number of jobs to the cluster for various reasons -- to get high throughput for a big analysis, to vary parameters with one analysis, etc. This document aims to help you become more efficient and to help you take advantage of shell and SLURM resources. This will improve your work…