How Zesty Disk works

This topic describes the Zesty Disk process and how filesystem expanding and shrinking works.

The Zesty Disk continual optimization process

The following process takes place after Zesty Disk has been deployed:

1. Zesty creates a virtual disk for the storage filesystem which consists of several small storage volumes. Since Zesty Disk leverages the native cloud provider block storage devices (AWS EBS), your native tools, procedures, and SLAs are unchanged, while you remain the owner of your data and the only one that has access to it.

2. Zesty Disk continuously tracks usage metrics (capacity, IOPS, and read/write throughput) as well as instance and disk metadata (such as instance type, disk type, volume names) which are sent unidirectionally to the Zesty backend.

3. The usage and metadata metrics are processed by an AI model that generates a behavioral profile on the instance volume. It uses this profile to predict the usage patterns and fluctuations of the disk to ensure that it has optimal IOPS and throughput for any scenario.

4. The volumes on the filesystems are serialized so that one large volume of 100GB for example, is replaced with a filesystem of multiple smaller volumes, say, 50GB, 35GB, and 15GB.

5. On day one, the Zesty Agent identifies any excess provisioned storage and automatically shrinks that capacity down to just above usage levels. Providing immediate ROI from the get-go!

6. The Zesty backend issues an API command to the cloud provider with the shrink action. It then sends an update request to the Zesty Disk Agent on the instance to adjust capacity.

7. The following steps run in an iterative process:

   1. As the application needs more storage, the volume is extended or another small volume is added. Any new volume added is given the same tags as the original volume.
      When the application requires less storage, the small volume is detached, shrinking the available capacity.

   2. If a volume is removed, its data is moved over to other volumes in the filesystem before it’s moved out. This reshuffling occurs in run time, without requiring any restart or pausing availability.

   3. As the application needs more storage another small disk can be added or the disk can be extended.

   4. A buffer is always maintained above the needed capacity, so there is no concern of insufficient storage. This buffer is based on previous trends and is usually 10-15% of capacity.

   5. Every action is logged in the audit log and an alert can be sent to the environment’s Slack or Teams channel.

Expanding a filesystem

When Zesty Disk determines that extra capacity is needed, the filesystem is expanded immediately, either by adding more capacity to an existing volume or by adding a new volume.

The Zesty Disk Agent determines the need to expand by constantly evaluating the FS capacity, expected usage, and the minimum buffer required. If the current free capacity isn’t enough, an expansion operation is initiated.

The amount of the expansion, determined by the AI-predicted needs and the required buffer, will always be more capacity than is actually needed, for these reasons:

• to enhance system stability and not have to make changes too frequently, at the same time ensuring high utilization

• to comply with the AWS “at least 6 hour wait” limitation of EBS volume modification

• on Windows filesystems, always in multiples of 128 GB, as described in How Zesty Disk works - standalone instances

Shrinking a filesystem

When a Zesty Disk filesystem is shrunk, the data on the volumes to be removed is moved to other volumes. Because this process uses instance and volume resources, shrinking is performed only once every 12 hours, and only if the filesystem has either of these:

• current utilization can be significantly improved

• volumes can be consolidated to retain 3-5 volumes per FS (Linux only)

When shrinking, filesystem limitations require that volumes be removed only in their entirety; a volume cannot be partially shrunk.

On Linux instances, a shrink can take place in the following ways:

• If a volume exists of the size that needs to be shrunk, that volume can be removed.

• If no such volume exists, a current volume can be replaced as follows:

  1. Add a new, smaller volume.

  2. Move the data to the new volume.

  3. Remove the current volume.

On Windows instances, OS limitations require that only the last added volume can be removed.

For more information, see:

• How Zesty Disk works for standalone instances

• How Zesty Disk works for Kubernetes