Understanding Crystal: A Simple Example

Crystal is devised to ease storage management in the cloud via Software-Define Storage models. To understand how easy it is to manage your OpenStack Swift cluster, we show a simple example policy as they are defined in the Crystal dashboard, as well as how it is enforce on object requests.

Creating a Static Policy from Scratch

Let us imagine that we are datacenter administrators in charge of a large OpenStack Swift deployment. Moreover, we deployed the Crystal components in our system, so we are in position of executing SDS services on storage flows. In what follows, we explain the entire process of executing SDS policies in Crystal, from an administrator’s perspective.

FOR TENANT:f0157f80fb07485888403600b039a1cc DO SET compression

Crystal enables to execute this kind of static policies. In this particular case, we want to execute compression on a tenant’s object requests, given that we realized that he/she is storing highly redundant data. This will achieve importante storage savings in our system.

As can be observed, in our policy we are using the term “compression” that stands to a storage filter that compresses data objects. However, by default Crystal has no filters installed. Instead, it has an architecture than enables system developers and administrators to install arbitrary filters at runtime to perform storage management. Let us show the process of installing a filter.

1) Develop the filter: In this example, we are going to develop a simple compression filter (check the link for browse source). As can be observed, the filter simply compresses the input stream of an object’s request resorting to standard compression libraries  (.gz, .lz4), and forwards the result of compression to the output stream to continue with the request flow. As simple as that; the compression filter can take less than 60 LoC in Java, being all the code embedded into a single, well-defined method (invoke). This gives an idea of how easy it is to implement new filters in Crystal.

2) Install and deploy the filter: The second step to take is to upload the new filter in the system via the Crystal’s dashboard. To this end, we just have to package compression filter as a .jar file and install it as a filter in Crystal. Moreover, in the installation form, we will be asked to provide a keyword for the filter (i.e., “compression”). This keyword will be the term used for defining policies.

3) Write the policy: As soon as the compression filter has been installed and registered with a unique keyword (“compression”), we can start writing any number of policies that use that filter. For instance, the policy that we have defined at the beginning of this example. In the dashboard, the policy would be defined as follows:

Immediately after the policy is compiled by Crystal, storage requests for the specified tenant will be compressed on PUT request, and decompressed on GET requests.