Last Modified: Tuesday, 07-Aug-2012 10:32:24 EDT

CSB Filesystems and Disk Use

For practical information on filesystems, see CSB Unix Filesystems -- Description .

For policy on protecting and archiving data, see CSB Filesystems -- Data Backup and Recovery .

General Policies

Data storage is a finite resource. Therefore, it is necessary to define and enforce policies allowing for a fair distribution of this resource. To this end, quotas are placed on some filesystems, and time limits are placed on others.

Policies are enforced by user community cooperation, automatic system scripts and utilities, and direct intervention by the CSB staff. It is often easy to circumvent these mechanisms. We rely on the voluntary compliance of the user community for these policies to work. If the staff detects obvious attempts to circumvent the letter or the spirit of these policies, we may delete the offending files without notice, and/or suspend the account of user involved, and/or notify the offending user's PI that a problem exists. Failure of the staff to take such action in some instances is not a guarantee that such action will not be taken in the future.

Changes in definitions of this policy as it applies to various filesystems (eg: changes in time limits, etc) will be widely posted and open for discussion before implementation.

From time to time, the limits imposed by these policies become a major obstacle to legitimate work. In those cases, you are urged to discuss the situation with the system staff. Remedies can include: 1) finding temporary additional space for a specific project; 2) Adding group disk storage to the system; 3) discovering more efficient ways to use existing resources.

Policy approved by CSB PI's: 2 February, 1995

CSB Unix Filesystems -- Practical Guidelines

For information on specific filesystems, see CSB Unix Filesystems -- Description.

Hardware can be expected to fail. When hardware is reliable, people can be expected to make mistakes.

Data on /frames are subject to time limits. Data may be placed there only while it is actively being used. In addition, data on /frames should be considered volatile. It can disappear from hardware failure, system software error, operator (staff) error, and user (your) error. Any files you place on these filesystems should be capable of being regenerated from a backup medium or from files on a less volatile filesystem. You are responsible for making these backups, but the system staff will be happy to help you devise effective strategies. See separate memo on storage and processing of area detector data, and on general backup procedures.

Note: If you un-tar from a tape to /srv/frames/ your files may have a modification date older than 270 days and will be erased that night. Use the -m flag when restoring files using tar to avoid this problem.

Files on /srv/people are reasonably secure from hardware failure. However, human error (yours or ours) could still destroy some or all of your files. The staff will be backing up this filesystem periodically and occasional archives will be kept indefinitely. You might want to supplement this with your own backups. The staff can help you develop and implement a satisfactory strategy.

Last Modified: Tuesday, 07-Aug-2012 10:32:24 EDT