There are two main filesystems on Colonial One. The first is connected over NFS and used to store /home and /groups, and has 250TB of usable space. The second is a high-speed Lustre scratch filesystem, accessed as /lustre. Note that neither are meant for long-term storage. There is another filesystem, /import, which can be purchased for archival storage.
By default, you have access to three locations (with a fourth archival option available for purchase):
/home/$UserID/- your home directory. Default quota of 25GB. NOTE no jobs should be run against the /home partition, it is not designed for performance. Use /lustre instead./groups/$group/- shared group space, accessible by anyone in your group. (If you are unsure of what group you are in, thegroupscommand will tell you.) Default quota of 250GB. NOTE no jobs should be run against the /groups partition, it is not designed for performance. Use /lustre instead./lustre/groups/$group/- shared group scratch space/import/$group/- archival storage that can be purchased. NOTE no jobs should be run against the /import partition, it is for archival purposes only and is not designed for performance. Use /lustre instead.
Quotas are in place on /home and /groups spaces, and the current disk usage for your home directory and that of all the groups you belong to is displayed when you login. You can see the current status at any time using the quotareport tool. Please note that the statistics are delayed by up to five minutes. If you need additional space you must email hpchelp@gwu.edu with (1) an explanation of your space requirements, and (2) a short description of how your data is being transferred and maintained on permanent storage elsewhere.
Space on the Lustre filesystem is subject to removal under the conditions set in LustrePurgePolicy. This is to ensure sufficient space remains available for ongoing work on the system.
Warning: Backups are your responsibility. The /home and /groups directories are backed up on a monthly basis as a disaster-recovery mechanism only. The /lustre scratch space is never backed up. We cannot recover accidentally deleted files from any filesystem in Colonial One. Also note, neither filesystem is meant for long-term storage and is subject to requests of removal by support staff. . We recommend storing your source codes on Github.
File transfer
Globus is recommended for large-scale data transfers, both between your own systems (laptop, desktop) and the cluster, and between the cluster and other external HPC centers such as XSEDE.Colonial One's public endpoint is gw#colonialone. An overview of signing up and using Globus can be accessed here: GlobusSetup.
