Set up an EMR cluster

This topic describes how to set up an EMR  to be compatible with Zesty Disk.

Adjust the initial EBS size

Adjust the initial EBS size in the EbsBlockDeviceConfigs section of the Instance Group Config file.

To adjust the initial EBS size:

1. Open the Instance Group Config file.

2. Locate the EbsBlockDeviceConfigs section.
   For example:
   "EbsConfiguration": {

        "EbsOptimized": false,

        "EbsBlockDeviceConfigs": [

          {

            "VolumeSpecification": {

              "VolumeType": "gp3",

              "SizeInGB": 250,

              "Iops": 3000

            },

            "VolumesPerInstance": 1

          }

        ]

3. Change the initial size to be large enough for 5-10 minutes of runtime.
   From that point on, Zesty Disk will expand or shrink the filesystem size.
   The minimum recommended size is 30 GB.
   For example: "SizeInGB": 30,

4. Save the file.

Edit bootstrap actions

The bootstrap action file, located on an Amazon S3 bucket, is required to install the Zesty Disk Agent on the EMR cluster nodes. Bootstrap actions are scripts that run on EMR cluster nodes before any other application starts. They can be used to install additional software or configure settings.

To edit bootstrap options:

Create a bootstrap file (or edit an existing bootstrap file) based on the following:

#!/bin/bash

if [[ $# != 1 ]]; then
    echo "API KEY wasn't provided" 2>&1
    exit 1
fi

API_KEY=$1

sudo curl -s https://static.zesty.co/ZX-InfraStructure-Agent-release/install.sh | sudo bash -s apikey=$API_KEY --emr

sudo echo "Bootstrap script is DONE!"