This section explains how to deploy Cloud Native Qumulo (CNQ) by creating the persistent storage and the cluster compute and cache resources with Terraform. It also provides recommendations for Terraform deployments and information about post-deployment actions and optimization.

For an overview of CNQ on AWS, its prerequisites, and limits, see How Cloud Native Qumulo Works.

The aws-terraform-cnq.zip file contains comprehensive Terraform configurations that let you deploy S3 buckets and then create a CNQ cluster with 4 to 24 instances that adhere to the AWS Well-Architected Framework and have fully elastic compute and capacity.

Prerequisites

This section explains the prerequisites to deploying CNQ on AWS.

  • To allow your Qumulo instance to report metrics to Qumulo, your AWS VPC must have outbound Internet connectivity through a NAT gateway or a firewall. Your instance shares no file data during this process.

  • The following features require specific versions of Qumulo Core:

    Feature Minimum Qumulo Core Version
    • Adding S3 buckets to increase persistent storage capacity
    • Increasing the soft capacity limit for an existing CNQ cluster
    		 7.2.1.1
    7.2.0.2
    Creating persistent storage 7.1.3 with version 4.0 of this repository

  • Before you configure your Terraform environment, you must sign in to the AWS CLI.

    A custom IAM role or user must include the following AWS services:

    • cloudformation:*
    • ec2:*
    • elasticloadbalancing:*
    • iam:*
    • kms:*
    • lambda:*
    • logs:*
    • resource-groups:*
    • route53:*
    • s3:*
    • secretsmanager:*
    • sns:*
    • ssm:*
    • sts:*

How the CNQ Provisioner Works

The CNQ Provisioner is an m5.large EC2 instance that uses custom user data to configure your Qumulo cluster and any additional AWS environment requirements.

The Provisioner stores all necessary state information in the AWS Parameter Store and shuts down automatically when it completes any of its following major tasks:

Click to expand Qumulo Cluster Configuration
  • Forms the first quorum with specific Hot or Cold parameters
  • Adds nodes to the quorum (when expanding the cluster)
  • Assigns floating IP addresses to nodes in the cluster
  • Manages cluster replacement (new compute and cache resources) for changing instance sizes
  • Manages the addition of S3 buckets and soft capacity limit increases
  • Changes the administrative password
AWS Configuration
  • Checks for connectivity to Amazon S3
  • Checks for the presence of an S3 Gateway in the VPC (this is required for provisioning)
  • Checks that all S3 buckets are empty before forming quorum
  • Checks for connectivity to the public Internet running a curl command against api.missionq.qumulo.com/
  • Configures the throughput and IOPS for the EBS gp3 volume
  • Tags EBS volumes with deployment_unique_name and volume type
  • Tracks software versions, cluster IP addresses, instance IDs, and UUID in the AWS Parameter Store
  • Tracks the last-run-status for the Provisioner in the Parameter Store

Step 1: Deploying Cluster Persistent Storage

This section explains how to deploy the S3 buckets that act as persistent storage for your Qumulo cluster.

  1. Log in to Nexus, click Downloads > Deployment on AWS, and then download the Terraform configuration, Debian package, and host configuration file.

  2. In your S3 bucket, create the qumulo-core-install directory. Within this directory, create another directory with the Qumulo Core version as its name. The following is an example path:

    my-s3-bucket-name/my-s3-bucket-prefix/qumulo-core-install/7.2.3

  3. Copy qumulo-core.deb and host_configuration.tar.gz into the directory named after the Qumulo Core version (in this example, it is 7.2.3).

  4. Copy aws-terraform-cnq.zip to your Terraform environment and decompress it.

  5. Navigate to the aws-terraform-cnq directory and then run the terraform init command.

  6. Navigate to the persistent-storage directory and then take the following steps:

    1. Review the terraform.tfvars file.

      • Specify the correct aws_region for your cluster’s persistent storage.

      • Enter the soft_capacity_limit.

    2. Run the terraform apply command.

    Terraform creates each S3 bucket with a unique state for its deployment.

Step 2: Deploying Cluster Compute and Cache Resources

This section explains how to deploy compute and cache resources for a Qumulo cluster by using a Ubuntu AMI and the Qumulo Core .deb installer.

  1. Configure your VPC to use the gateway VPC endpoint for S3.

  2. Navigate to the aws-terraform-cnq directory.

  3. Choose config-standard.tfvars or config-advanced.tfvars and fill in the values for all required variables. For more information, see readme.pdf in aws-terraform-cnq.zip.

  4. To log in to your cluster’s Web UI, use the IP address from the Terraform output as the endpoint and the username and password that you have configured during deployment as the credentials.

    You can use the Web UI to create and manage NFS exports, SMB shares, snapshots, and continuous replication relationships You can also join your cluster to Active Directory, configure LDAP, and perform many other operations.

  5. Mount your Qumulo file system by using NFS or SMB and your cluster’s DNS name or IP address.

Step 3: Performing Post-Deployment Actions

This section describes the common actions you can perform on a CNQ cluster after deploying it.

Adding a Node to an Existing Cluster

  1. Edit terraform.tfvars and change the value of q_node_count to a new value.

  2. Run the terraform apply command.

  3. To ensure that the Provisioner shut downs automatically, review the /qumulo/my-deployment-name/last-run-status parameter in the AWS Parameter Store.
  4. To check that the cluster is healthy, log in to the Web UI.

Removing a Node from an Existing Cluster

Removing a node from an existing cluster is a two-step process. First, you remove the node from the cluster’s quorum. Next, you tidy up your AWS resources.

Step 1: Remove the Node from the Cluster’s Quorum

You must perform this step while the cluster is running.

  1. Copy the remove-nodes.sh script from the utilities directory to an AWS Linux 2 AMI running in your VPC.

  2. Run the remove-nodes.sh script and specify the AWS region, the unique deployment name, the current node count, and the final node count.

    In the following example, we reduce a cluster from 6 to 4 nodes.

    ./remove-nodes.sh \
  --region us-west-2 \
  --qstackname my-unique-deployment-name \
  --currentnodecount 6 \
  --finalnodecount 4
  3. When prompted, confirm the nodes’ removal.
  4. To check that the cluster is healthy, log in to the Web UI.

Step 2: Tidy Up Your AWS Resources

  1. Edit terraform.tfvars and change the value of q_node_count to a lower value (for example, 4).
  2. Run the terraform apply command.

  3. To monitor the provisioner’s status, you can watch the Terraform status posts in your terminal or in the AWS Parameter Store, under /qumulo/<my-deployment-name>/last-run-status.

    The node and the infrastructure associated with the node are removed.

  4. To check that the cluster is healthy, log in to the Web UI.

Changing the EC2 Instance Type for an Existing Cluster

Changing the EC2 instance type is a three-step process. First, you create a new deployment in a new Terraform workspace (this process ensures that the required instances are available) and join the new instances to a quorum. Next, you clean up your S3 bucket policies. Finally, you remove the existing instances.

Step 1: Create a New Deployment in a New Terraform Workspace

  1. To create a new Terraform workspace, run the terraform workspace new my-new-workspace-name command.
  2. To initialize the workspace, run the terraform init command.

  3. Use the existing deployment name or choose a new name.

  4. Edit the terraform.tfvars file and take the following steps:

    1. Specify the value for the q_instance_type variable.
    2. Set the value of the q_replacement_cluster variable to true.
    3. Set the value of the q_existing_deployment_unique_name variable to the current deployment’s name.
    4. (Optional) To change the number of nodes, specify the value for the q_node_count variable.

  5. Run the terraform apply command.

  6. To ensure that the Provisioner shut downs automatically, review the /qumulo/my-deployment-name/last-run-status parameter in the AWS Parameter Store.
  7. To perform future node addition or removal operations, edit the terraform.tfvars file and set the q_replacement_cluster variable to false.
  8. To check that the cluster is healthy, log in to the Web UI.

Step 2: Remove the Existing Instances

  1. To select the previous Terraform workspace (for example, default), run the terraform workspace select <default> command.
  2. To ensure that the correct workspace is selected, run the terraform workspace show command.

  3. To delete the previous instance in its entirety, run the terraform destroy command.

    The previous instances are deleted.

Step 3: Clean Up S3 Bucket Policies

  1. To select the new Terraform workspace, run the terraform workspace select <my-new-workspace-name> command.
  2. Edit the terraform.tfvars file and set the q_replacement_cluster variable to false.
  3. Run the terraform apply command. This ensures that the S3 bucket policies have least privilege.

Increasing the Soft Capacity Limit for an Existing Cluster

Increasing the soft capacity limit for an existing cluster is a two-step process. First, you set new persistent storage parameters. Next, you set new compute and cache deployment parameters.

Step 1: Set New Persistent Storage Parameters

  1. Edit the terraform.tfvars file in the persistent-storage directory and set the soft_capacity_limit variable to a higher value.

  2. Run the terraform apply command.

    Terraform creates new S3 buckets as necessary.

Step 2: Update Existing Compute and Cache Resource Deployment

  1. Navigate to the root directory of the aws-terraform-cnq repository.

  2. Run the terraform apply command.

    Terraform updates the necessary IAM roles and S3 bucket policies, adds S3 buckets to the persistent storage list for the cluster, and increases the soft capacity limit. When the Provisioner shuts down automatically, this process is complete.

Deleting an Existing Cluster

Deleting a cluster is a two-step process. First, you delete your Cloud Native Qumulo resources. Next you delete your persistent storage.

Step 1: To Delete Your Cluster’s Cloud Native Qumulo Resources

  1. After you back up your data safely, edit your terraform.tfvars file and set the term_protection variable to false.
  2. Run the terraform apply command.

  3. Run the terraform destroy command.

    Terraform deletes all of your cluster’s CNQ resources.

Step 2: To Delete Your Cluster’s Persistent Storage

  1. Navigate to the persistent-storage directory.
  2. Edit your terraform.tfvars file and set the prevent_destroy parameter to false.
  3. Run the terraform apply command.

  4. Run the terraform destroy command.

    Terraform deletes all of your cluster’s persistent storage.