- 1 Minute to read
- Print
- PDF
Set up an EMR cluster
- 1 Minute to read
- Print
- PDF
This topic describes how to set up an EMR to be compatible with Zesty Disk.
Adjust the initial EBS size
Adjust the initial EBS size in the EbsBlockDeviceConfigs section of the Instance Group Config file.
To adjust the initial EBS size:
Open the Instance Group Config file.
Locate the EbsBlockDeviceConfigs section.
For example:"EbsConfiguration": {
"EbsOptimized": false,
"EbsBlockDeviceConfigs": [
{
"VolumeSpecification": {
"VolumeType": "gp3",
"SizeInGB": 250,
"Iops": 3000
},
"VolumesPerInstance": 1
}
]
Change the initial size to be large enough for 5-10 minutes of runtime.
From that point on, Zesty Disk will expand or shrink the filesystem size.
The minimum recommended size is 30 GB.
For example:"SizeInGB": 30,
Save the file.
Edit bootstrap actions
The bootstrap action file, located on an Amazon S3 bucket, is required to install the Zesty Disk Agent on the EMR cluster nodes. Bootstrap actions are scripts that run on EMR cluster nodes before any other application starts. They can be used to install additional software or configure settings.
To edit bootstrap options:
Create a bootstrap file (or edit an existing bootstrap file) based on the following:
#!/bin/bash
if [[ $# != 1 ]]; then
echo "API KEY wasn't provided" 2>&1
exit 1
fi
API_KEY=$1
sudo curl -s https://static.zesty.co/ZX-InfraStructure-Agent-release/install.sh | sudo bash -s apikey=$API_KEY --emr
sudo echo "Bootstrap script is DONE!"