Policy: scratch space

December 4th, 2019: During the December 3rd maintenance, a new global variable called $SCRATCH will be introduced into the environment. This variable will allow scripts and jobs to point to the current preferred scratch path regardless of where it resides. Once enabled, this variable will point to /n/scratchlfs so, for example, one could use the path $SCRATCH/jharvard_lab/jsmithin a job script to point to /n/scratchlfs/jharvard_lab/jsmith. This will have the added benefit of allowing us to change scratch systems at any time without you having to modify your jobs/scripts.


RC maintains a large, shared temporary scratch filesystem for general use for high input/output jobs at /n/scratchlfs. This workspace is made available with the provision that each group use no more than 50TB, and that files older than 90 days will be deleted through a periodic purge process. This purge can run at any time, especially if scratch is getting full (df -h /n/scratchlfs to see current total use) and is also run at the start of the month during our maintenance period.

There is no charge for scratch, but please note that it intended as volatile, temporary scratch space intended for transient data and is not backed up. If your lab group needs a directory in scratch, please contact us.

Modifying file times (via touch or other process) to avoid retention is an abuse of the filesystem and will result in administrative action from RC.  You may initially modify the file date(s) on new data so that it is not in the past, but should not modify it further. These actions may include deleting data on scratchlfs owned by the offender and disabling the offending account.  If data on scratch needs to be kept longer, please contact us to discuss options.


To view your group's scratch quota and usage:
lfs quota -hg lab_name /n/scratchlfs


CC BY-NC-SA 4.0 This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Permissions beyond the scope of this license may be available at Attribution.