b. Filesystems and Authorization

Where is ftp?

Modern secure transfer protocols like SFTP and SCP secure data during transit and should be used when moving files from one place to another. However you may still need to use plain, un-secured FTP to download data sets or other files from remote locations while logged into the cluster.

While we do not offer the largely outmoded 'ftp' program on the cluster, we do offer the feature-rich and largely command compatible 'lftp'. From any login or compute node type 'man lftp' to see its usage and options.


What’s the best way to transfer my data?

See our 'Transferring data on the cluster' page for a list of options and best practices for data transfer within the cluster.

For transferring data to and from the cluster, see 'Transferring data externally'.


How do I access my cluster home directory from my laptop?

FASRC cluster home directories are available through SAMBA and so can be mounted as a network drive on Mac, Windows, and Linux computers. See the Access and Login page for specific instructions on how to mount the directory.


How do I check how much space I’ve used?

The standard linux tool du shows how much disk space is being used by individual files and directories. For example, the command:

First cd to your home directory cd ~

du -x --max-depth 1 .

will print how much space is used by each directory in your current working directory, plus a total at the end. This will take a few minutes if you have a lot of files, so be patient.

Note: With the legacy home directories, the command df showed your personal quota details; this does not work with our current configuration of home directories on the Isilon filesystem.


How much space do I have in my home directory?

You are given 100 GB in your home directory. This is twice as much as with the legacy home directories. This size limit is referred to as your quota.

Sorry, but we cannot increase this allotment. Please use disk shares associated with your lab or one of our scratch files systems if you require more space.

Please see our Storage document for more information.


I accidentally deleted my data, how do I get it back?

Your home directory has periodic snapshots taken. These snapshots are of your home directory files from various recent points in time. They are in a hidden directory named .snapshot, within every other directory in your home directory. The command ls -a will not show these, but you can ls .snapshot directly, and cd .snapshot to go into the directory.

In the .snapshot folder you will see “hourly” “daily” “monthly” folders with the date of the snapshots. Traverse (cd) to the snapshot folder corresponding to the period you wish to restore data from. From there you can simply copy the relevant files back into your home folder using your favorite file copy tool (rsync, cp, etc.)

Lab directory backups are for disaster recovery only, as they are handled separately and do not have snapshot capabilities. Additionally, these backups, given the size, run over a period of many days and are not 'up-to-the-minute' backups. As such, they are not intended to recover accidental file deletions. Please contact RC Help if you have any questions.

Please also see our Storage document for more info.

Last updated: August 21, 2018 at 11:39 am


Why are all my files executable?

You may notice that the x (execute) bit is set on all your files:

[username@boslogin01 ~]# ls -l myfile.txt
-rwxr--r-- 1 username groupname 3029 Aug 20 03:10 myfile.txt

Furthermore, chmod does not remove it:

[username@boslogin01 ~]# chmod u-x myfile.txt
[username@boslogin01 ~]# ls -l myfile.txt
-rwxr--r-- 1 username groupname 3029 Aug 20 03:10 myfile.txt

This is a feature, a result of the storage system doing mixed Unix-style and Windows-style permissions. If this is causing a problem for you, please contact rchelp@fas.harvard.edu.


Why does my UMASK not work?

You may also notice that your UMASK environment variable does not work as expected:

[username@boslogin01 ~]# umask 002
[username@boslogin01 ~]# touch newfile.txt
[username@boslogin01 ~]# ls -l newfile.txt
-rwx------ 1 username groupname 3029 Aug 20 03:10 newfile.txt

Normally, the outcome would be -rw-rw-r--. If this is causing a problem for you, please contact rchelp@fas.harvard.edu.


Is my home directory available as a network filesystem share?

Yes, your cluster home directory is available as a network filesystem share to which you can directly connect your own desktop or laptop. The technical protocol for this is called CIFS or Samba, so you will often hear us refer to it in that way. On Windows, this is also referred to as mapping a network drive, and on a Mac it is called connecting to a server.

In all cases, you need your RC username, password, server name, and path. Please see the Mounting Storage document for detailed information. (short link)



Billing FAQ