Part A: Preliminary Considerations
General Information about the System
The Silent Cubes long-term storage allows you to archive data on so-called Volumes via SMB/CIFS and NFS shares. The system is a true hardware WORM, which, unlike some other industry solutions, does not compromise on data storage. The following archiving functions make Silent Cubes unique:
Once data is stored in the Silent Cubes System, it cannot be altered or deleted afterwards.
Even though it is possible to remove shares, Volumes, and files after the lifespan has expired when using Volumes with lifespan (Compliance Option required), once allocated storage space on the Silent Cubes is not released.
Volumes can only be deleted when they are empty. For this, the retention period of all files stored on it must have expired. Only then can all files and thus the volume be deleted.
Data actually occupies storage space on the Silent Cubes. Files can therefore never be deleted. Approvals and Volumes, on the other hand, are references to this data and can, depending on the used Volume Types, also be removed from the system. Due to the special nature of storage (Content Addressed Storage), data cannot be interpreted without corresponding references (i.e., entries for files, approvals, and volumes). The 'deletion', for example, after the expiration of the lifespan is part of the certification for revision security.
Before you deploy the system, consider for what purpose data should be archived, whether it should not be retrievable for later use, and what settings need to be configured for this.
This decision aid is intended to assist you in setting up the Silent Cube long-term storage. The following chapters describe the core features of the system and answer the most important questions in advance.
Lifespan
In conjunction with the Compliance Option, a standard lifespan (Retention Period) can be defined for files stored in volumes created on Silent Cubes (see Chapter 3). The lifespan can be defined for all volume types except for the volume type WORM.
Please note:
Volumes themselves do not have a lifespan. The default setting refers to stored files from the time of archiving.
However, individually set Retention Dates are also supported, which can be set by software per Access timestamp for each file.
Retention Dates can be extended retroactively, but cannot be shortened.
The default setting can be changed at any time and will then apply to all files newly archived from that point onward.
Files can be deleted if the Access timestamp is in the past, meaning the lifespan (Retention Date) has expired.
Example 1: A file without a specific Access timestamp was archived on January 1. 2020 in a Volume with a set lifespan of 5 years. The file can be deleted starting from January 1. 2025. Via software, the Retention Date can be set to any date after January 1. 2025, but not before.
Example 2: A file received the access timestamp 1 from the archiving software. June 2035 and was transferred on January 1. 2020 to the volume (with lifespan, not triggered by RWRO) with a set lifespan of 15 years. The retention date is June 1, 2035. If, on the other hand, a lifespan of 10 years was set, the file receives the retention date of January 1, 2030.
WORM volume types
The Silent Cube storage system supports four different volume types, each performing a specific type of archiving:
WORM
WORM with versioning
WORM with lifespan
WORM with lifespan and versioning
WORM with lifespan (RWRO triggered)
WORM with lifespan and versioning (RWRO triggered)
WORM with lifespan (RWRO triggered, Autosave)
The Volume Type WORM is always available to you. The other three Volume Types can only be used if you have purchased the long-term storage system together with the Compliance Option. With these Volume Types, the Silent Cube long-term storage meets two essential Compliance Requirements: Administrators can demonstrate compliant storage. On the other hand, they can delete data after a specified period (Retention Date, see Chapter 2).
Volumes with lifespan can also be created as RWRO-triggered variants, where the actual archiving is initiated on the storage units by changing the file properties (from 'Read/Write' to 'Read Only').
For each volume type, numerous options are available to you.
Below you will learn which application area each of the four volume types is ideally suited for.
Please also refer to the user manual of your archiving software – it may require a specific volume type.
You can delete empty volumes, but the lifespan of all files stored on them must have expired. Only then can all files be deleted. To remove it, the volume must be completely empty (no directories remaining, all approvals removed).
Volume Type WORM
A WORM-type volume is particularly well-suited for archiving data that must be retained. The WORM volume acts like a classic WORM storage: the long-term storage system 'freezes' the data stored in it, archives it permanently, and protects it from any modification. Before the adjustable minimum age expires, the files can still be modified. Once archived, data cannot be modified or deleted by the administrator or other users of the system.
Accidentally or incorrectly archived files cannot be saved again under the same path and filename on the same share. Therefore, please verify the success of the copy process before the set minimum age is reached, and the data is thus permanently archived to the storage unit.
Volume Type WORM with Versioning
The volume of type WORM with versioning corresponds to the volume type WORM but additionally offers functionalities for versioning. This allows an already archived file to be virtually overwritten. In this process, revisions of a file are created, which can be accessed using special methods (Revert and Switch Revision). The FAST LTA Support Team provides a separate background paper on the topic of versioning upon request.
Volume Type WORM with Lifespan
Use the Volume Type WORM with Lifespan when data needs to be retained for a specific period. Subsequently, the data should or must be deletable.
The Volume Type WORM with Lifespan archives data stored in the volume just like the Volume Type WORM, but with one difference: Each stored file is assigned a lifespan (Retention Date). The lifespan of a file is determined by its Access timestamp or by the standard lifespan setting for the volume. Until this date, the data remains immutable in the filesystem and cannot be deleted. Only after the set lifespan has expired can users delete the corresponding files.
Files stored on this Volume Type have a lifespan that is at least equal to the set Default Value. If this Default Value is set to, for example, 5 years, each stored file remains unchangeable and undeletable for at least 5 years (from the respective Storage Time Point). If files with their own lifespan (Access timestamp) are stored that have a shorter lifespan than this default, the Access timestamp of these files will be increased to the Default Value. Files with a preset Access timestamp that is already greater than the Default Value will be retained unchanged.
The lifespan of each file can be extended at any time separately via the Access timestamp, but it can never be shortened!
The associated data will not be deleted by the storage units (but by the Head Unit). Internally, the long-term storage separates the connection between data and file system. Access to the stored data is then no longer possible.
Regardless of whether the files are deleted from the file system or not, the used storage space on the storage units remains occupied even after deletion. Volume Type WORM with lifespan and
Versioning
This volume type corresponds to Volume Type WORM with lifespan, but additionally offers functionalities forVersioning(see Chapter 3.2).
Volume Type WORM with Lifespan
(RWRO triggered)
The Volume Type WORM with lifespan (RWRO triggered) is based on the Volume Type WORM with lifespan, but does not use time-controlled archiving. The storage process only begins when the access rights of a file are set from 'Read Write' (RW) to 'Read Only' (RO), making the file immutable. The access timestamp of a file is retained unchanged.
Please note: To ensure the proper execution of the archiving process, you should not manually store data on Volume Type WORM with a lifespan (RWRO triggered), but instead use appropriate software for this purpose. Please select this option only if the application you are using, which is upstream of the long-term storage system, is capable of initiating the archiving process accordingly.
The trigger function of the software version you are using starts the archiving process on the Silent Cubes. During this process, the access rights of the file to be archived are set to 'Read Only' (manually in Windows: attrib +r <file>, Unix: chmod 0444 <file>, chmod 444 <file>, or chmod a-w <file>). If there are different users (both in Windows and Unix environments), they may only be able to write or modify their own files.
In contrast to the (untriggered) Volume Type WORM with lifespan, for this Volume Type, the default value for lifespan set in the user interface only affects files that do not have an explicitly specified lifespan (Access timestamp) up to that point. These files receive the specified lifespan. If files with an explicitly set lifespan (Access timestamp) are stored, this lifespan is adopted, regardless of whether it is shorter or longer than the default value.
Volume Type WORM with lifespan and versioning (RWRO triggered)
The Volume Type WORM with lifespan and versioning (RWRO triggered) does not use time-controlled archiving, just like the Volume Type WORM with lifespan (RWRO triggered), but becomes active only when access rights change. The access timestamp of a file remains unchanged. Additionally, this Volume Type supports storing different versions of a file. A preceding software can also access the various versions of the stored files individually with the appropriate connection.
Please note: To ensure a proper archiving process, you should not manually store data on Volume Type WORM with lifespan and versioning (RWRO triggered), but rather use suitable software for this purpose. Please select this option only if the application you are using, which is upstream of the long-term storage system, is capable of initiating the archiving process accordingly.
The trigger function of the software version you are using initiates the archiving process to the Silent Cubes. In this process, the access rights of the file to be archived are set to Read Only (Windows: attrib +r <file>, Unix: chmod 0444 <file>, chmod 444 <file>, or chmod a-w <file>). If there are different users (both in Windows and Unix environments), they may only be able to write or modify their own files.
In contrast to the (untriggered) Volume Type WORM with lifespan, for this Volume Type, the default value for lifespan set in the user interface only affects files that do not have an explicitly specified lifespan (Access timestamp) up to that point. These files receive the specified lifespan. If files with an explicitly set lifespan (Access timestamp) are stored, this lifespan is adopted, regardless of whether it is shorter or longer than the default value.
Volume Type WORM with Lifespan (RWRO triggered, Autosave)
The Volume Type WORM with Lifespan (RWRO triggered, Autosave) takes into account that some systems require temporary files, which change more frequently, to access the actual data. Only after switching from RW to RO are these files immutable.
With the previous volume types, these temporary files were only stored in the cache of the Head Unit and not on the secure Silent Cube before switching to RO. In the event of a failure or technical issue with the Head Unit, these temporary files could be lost, which could render the actual data unusable depending on the system.
This volume type now introduces the Auto-save function. Here, all data is always backed up on the Silent Cube. Data that is still marked as RW is 'modifiable' and 'deletable' - each of these actions results in a new file on the Silent Cube. Only when switching to RO are these files marked as immutable.
Even though a new 'version' of the file is created with each change or deletion of a temporary file before switching to RO, these 'versions' are not transparent - true versioning does not take place. Only the last version is always visible; after 'deletion', the file is removed.
Types of data transfer / Robocopy
The Silent Cubes long-term storage is unable to verify copy operations to the Head Unit. To detect faulty copy operations and to prevent incorrectly or incompletely stored data, we strongly recommend using software such as Robocopy, which can take on this task.
Through the adjustable minimum age for each Volume (at least 20 seconds, maximum 300 years), you can provide an appropriate time window for the verification process. Choose a sufficiently high minimum age depending on your software.
Important notes for the use of Robocopy in Windows environments: When the optimization function for Robocopy is active, which can be set separately for each Volume, the system ignores files with the following Modification-Time when archiving to the Silent Cubes:
01. January 1980 00:00:00 GMT to 02. January 1980 00:00:00 GMT.
Files with a Modification-Time within this time window are considered by Robocopy as not yet completed. Accordingly, the Head Unit does not pass these files to the Storage Units. They remain pending and can be deleted or overwritten if necessary.
In the event of a failed copy process with Robocopy, incompletely transferred files can thus be identified by a Modification-Time that falls within the specified time window. The reasons for a failed copy process can be varied:
Network interruption
Weak network (Time Out)
Missing parameters during the copy process with Robocopy that would have allowed for a resumption of the copy processes (Parameter /Z)
Robocopy can overwrite or delete such files marked as incomplete during a subsequent copy attempt by appropriately choosing the parameters before they are finally stored in the Silent Cubes.
More information about Robocopy and the parameters can be found online at http://en.wikipedia.org/wiki/Robocopy
Test operation
If you would like to familiarize yourself with the operation of the Silent Cube long-term storage in a relaxed and non-binding manner, simply do not connect any storage units to the Head Unit. Without connected storage units, so-called 'simulated Volumes' can be created in the user interface, which simulate the archiving to the storage units, meaning that no files are written to the storage units. Thus, you can try out the functions of the system and revert them at the end.
Benefits for you:
In contrast to normal operation, no data is archived during test operation. Thus, you exclude the possibility of accidentally placing data on the Silent Cubes.
All data in approvals, the approvals themselves, and also the simulated Volumes are always deletable.
No storage space is consumed. As a reminder: In normal operation, files that have been stored once cannot be modified or deleted afterwards.
Caution: The function to simulate archiving for test purposes is not the set default value in the dialog window when creating Volumes. To set up a test operation, please activate this function.
Delete the simulated data in the following order to prepare the Head Unit for product operation after the test operation:
Delete all files and directories
Remove all approvals
Remove the volumes