- 4 Minutes to read
- Print
- PDF
How Zesty Disk works
- 4 Minutes to read
- Print
- PDF
This topic describes the Zesty Disk process and how filesystem expanding and shrinking works.
The Zesty Disk continual optimization process
The following process takes place after Zesty Disk has been deployed:
Zesty creates a virtual filesystem that consists of several small storage volumes. Since Zesty Disk leverages the native cloud provider block storage devices (AWS EBS), your native tools, procedures, and SLAs are unchanged, while you remain the owner of your data and the only one that has access to it.
Zesty Disk continuously tracks usage metrics (capacity, IOPS, and read/write throughput) as well as instance and disk metadata (such as instance type, disk type, volume names) which are sent unidirectionally to the Zesty backend.
The usage and metadata metrics are processed by an AI model that generates a behavioral profile on the instance filesystem. It uses this profile to predict the usage patterns and fluctuations of the filesystem to ensure that it has optimal IOPS and throughput for any scenario.
The volumes on the filesystem are serialized so that one large volume of 100GB for example, is replaced with a filesystem of multiple smaller volumes, say, 50GB, 35GB, and 15GB.
On day one, the Zesty Agent identifies any excess provisioned storage and automatically shrinks that capacity down to just above usage levels. Providing immediate ROI from the get-go!
The Zesty backend issues an API command to the cloud provider with the shrink action. It then sends an update request to the Zesty Disk Agent on the instance to adjust capacity.
The following steps run in an iterative process:
As the application needs more storage, the volume is extended or another small volume is added. Any new volume added is given the same tags as the original volume.
When the application requires less storage, the small volume is detached, shrinking the available capacity.If a volume is removed, its data is moved over to other volumes in the Zesty Disk filesystem before it’s moved out. This reshuffling occurs in run time, without requiring any restart and with no impact on the application performance.
As the application needs more storage another small disk can be added or the disk can be extended.
A buffer is always maintained above the needed capacity, so there is no concern of insufficient storage. This buffer is based on previous trends and is usually 5-15% of capacity.
Every action is logged in the audit log and an alert can be sent to the environment’s email, Slack, or Teams channel.
The following diagram illustrates how Zesty Disk works:
How Zesty Disk integrates with Kubernetes
Zesty Disk brings flexibility and automation to storage management in Kubernetes clusters.
By utilizing native Kubernetes features like Persistent Volume Claims (PVC) and Persistent Volumes (PV), Zesty Disk seamlessly integrates into the Kubernetes environment.
It delivers real-time volume autoscaling based on application needs. Large file systems volumes are broken into multiple smaller volumes, enabling Zesty Disk to add, remove, or extend persistent volumes without disrupting running pods. This ensures continuous operation and dynamic scalability, perfectly aligned with Kubernetes' flexible infrastructure.
For more information on Zesty Disk in K8s, see How Zesty Disk for Kubernetes works.
Expanding a filesystem
When Zesty Disk determines that extra capacity is needed, the filesystem is expanded immediately, either by adding more capacity to an existing volume or by adding a new volume.
The Zesty Disk Agent determines the need to expand by constantly evaluating the FS capacity, expected usage, and the minimum buffer required. If the current free capacity isn’t enough, an expansion operation is initiated.
The amount of the expansion, determined by the AI-predicted needs and the required buffer, will always be more capacity than is actually needed, for these reasons:
to enhance system stability and not have to make changes too frequently, at the same time ensuring high utilization
to comply with the AWS “at least 6 hour wait” limitation of EBS volume modification
on Windows filesystems, always in multiples of 128 GB, as described in How Zesty Disk works - standalone instances
Shrinking a filesystem
When a Zesty Disk filesystem is shrunk, the data on the volumes to be removed is moved to other volumes. Because this process uses instance and volume resources, shrinking is performed only once every 12 hours, and only if the filesystem has either of these:
current utilization can be significantly improved
volumes can be consolidated to retain 3-5 volumes per FS (Linux only)
When shrinking, filesystem limitations require that volumes be removed only in their entirety; a volume cannot be partially shrunk.
On Linux instances, a shrink can take place in the following ways:
If a volume exists of the size that needs to be shrunk, that volume can be removed.
If no such volume exists, a current volume can be replaced as follows:
Add a new, smaller volume.
Move the data to the new volume.
Remove the current volume.
On Windows instances, OS limitations require that only the last added volume can be removed.
For more information, see: