Purge Policy for Clusters

The filesystems on clusters is not for archiving purposes. As such it is subject to a periodic “purge” policy in order to ensure sufficient space remains available for ongoing cluster use. To accomplish this, larger older files will be periodically and automatically removed from the Lustre filesystem.

Do not expect data to be retained: please move any data you want to preserve to either your home directory, group directory or an external storage service.

Below is the policies for the different clusters being managed by RTS Team

Pegasus Lustre Filesystem (/lustre/groups)

The Lustre filesystem (/lustre/groups) on Pegasus is scratch storage only and is not for archiving purposes. As such it is subject to a periodic “purge” policy in order to ensure sufficient space remains available for ongoing cluster use. To accomplish this, larger older files will be periodically and automatically removed from the Lustre filesystem. Do not expect data to be retained: please move any data you want to preserve to either your home directory, group directory or an external storage service.

Implementation

An email notice will be sent out to the Pegasus Users list at least one week in advance of each pending purge cycle.
Names and information of files subject to purge will be placed in /lustre/lustre-purge/ .
On the first of the month (or the proceeding business day if the first falls on a weekend) purgeable data as listed in /lustre/lustre-purge will be permanently removed.

Thresholds

Files that have not been accessed in 60 days (as tracked by the filesystem’s ‘atime’ flag).
The Lustre partition fills up.

Penalties

Users found attempting to circumvent this by artificially changing ‘atime’ values or by other means are subject to account suspension and file deletion.

HTCondor Purge Policy

Cerberus Purge Policy

Data left on Cerberus will be permanently deleted 30 days following the date of the final exam. This includes /group/<courseID> and home directories. Data may be backed up to Box or Google Drive if data are needed after the course is completed. If you have questions or would like to request help egressing data please contact hpchelp@gwu.edu.