This section explains how to create and manage credentials that S3 API actions in Qumulo Core require to access file system resources, such as access key pairs that sign requests.

In Qumulo Core 7.2.3.2 (and higher), system users can list, create, and delete S3 access keys for their identities.

Prerequisites

Administrative control over S3 access keys requires the following role-based access control (RBAC) privileges:

  • PRIVILEGE_S3_BUCKETS_WRITE: Create or delete any S3 access keys on a Qumulo cluster

  • PRIVILEGE_S3_BUCKETS_READ: List all S3 access keys on a Qumulo cluster

How S3 Access Keys Work in Qumulo Core

An identity is a single principal from an identity provider (IdP). Examples of identities include SMB security identifiers (SIDs), Active Directory user principal names (UPNs), POSIX user identifiers (UIDs), and local users in a Qumulo cluster.

An access key (or access key pair) is comprised of an S3 access key ID and an S3 secret access key.

  • The access key ID is the public component of an S3 access key pair. It identifies the user that performs an S3 request.

  • The secret access key (or secret key) is the private component of an S3 access key pair. The client uses the secret access key to sign requests. The server uses the secret access key to validate request signatures.

Qumulo Core creates an access key pair whenever an authorized user requests it. For more information, see Creating S3 Access Keys for a Qumulo Cluster.

The way in which Qumulo Core access keys let you access your Qumulo cluster makes the process similar to the way in which IAM Access Keys let you access Amazon S3 resources. For this reason, applications that access objects stored in a Qumulo cluster can use the Qumulo S3 API similarly to the native Amazon S3 API.

How S3 Access Keys work with Identities

An S3 access key doesn’t grant any additional permissions. It associates an S3 API request with a specific identity known to the Qumulo cluster.

When Qumulo Core processes a request, it evaluates permissions by using the Qumulo ACL (QACL) mechanism that operates like the access control list (ACL) mechanism that all file system protocols use. When the QACL grants or denies permissions to an associated identity, it also grants or denies the same permissions to the request being processed.

For more information, see Managing Access to S3 Buckets in a Qumulo Cluster.

How Qumulo Core Stores S3 Access Keys

To authenticate S3 API requests, Qumulo Core retrieves existing access key pairs that it stores securely as configuration metadata in your Qumulo cluster. Qumulo Core encrypts secret access keys on disk and holds decrypted secret access keys in memory (only while it processes a request).

S3 Access Key Lifecycle in Qumulo Core

Qumulo Core doesn’t limit how long you can use an access key pair after you create it. Your system administrators must take responsibility for using the Qumulo Core REST API or qq CLI to view the creation dates for access keys and revoke any pair at their discretion.

For more information, see Listing S3 Access Keys for a Qumulo Cluster.

Creating S3 Access Keys for a Qumulo Cluster

To make S3 API requests to a Qumulo cluster as a specific user, you must create an S3 access key pair for that user identity by using the Qumulo REST API or qq CLI.

To Create an Access Key for Your Own User Identity by Using the qq CLI

Run the qq s3_create_access_key command and use the --self flag. For example:

$ qq s3_create_access_key --self

The following is example output.

{
  "access_key_id": "AKIAIOSFODNN7EXAMPLE",
  "creation_time": "2022-12-12T21:37:53.553457928Z",
  "owner": {
    "auth_id": "501",
    "domain": "LOCAL",
    "gid": null,
    "name": "guest",
    "sid": "S-1-1-12-12345678-2345678901-1234567890-500",
    "uid": null
  },
  "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}

To Create an Access Key for Another User Identity by Using the qq CLI

To create S3 access keys for another user identity, you must have an administrator account or have the PRIVILEGE_S3_BUCKETS_WRITE privilege.

To create an S3 access key for a particular user , run the qq s3_create_access_key command and specify an identity. For example:

$ qq s3_create_access_key ANOTHER_IDENTITY

You can specify an identity by using:

  • A name, optionally qualified with a domain prefix:

    • ad:MY_NAME

    • AD\MY_NAME

    • local:MY_NAME

    • MY_NAME

  • An Active Directory Security Identifier. For example: SID:S-1-1-0

  • A Qumulo auth ID, Qumulo Core’s common representation for identities, in the form of a numeric identifier. For example: auth_id:513

The following is example output.

{
  "access_key_id": "AKIAJKSBDMSS8EXAMPLE",
  "creation_time": "2022-12-12T21:37:53.553457928Z",
  "owner": {
    "auth_id": "501",
    "domain": "LOCAL",
    "gid": null,
    "name": "guest",
    "sid": "S-1-1-12-12345678-2345678901-1234567890-500",
    "uid": null
  },
  "secret_access_key": "aGsfgDFdfEGWD/D5DWERF/sRsRdsXQEXAMPLEKEY"
}

To Create an S3 Access Key by Using the Qumulo Core REST API

Send a POST request to the /v1/s3/access-keys/ endpoint with the following body. You must include at least one of the following keys:

  • auth_id

  • sid

  • uid

For example:

{
  "user": {
    "sid": "S-1-1-12-12345678-2345678901-1234567890-500"
  }
}

The following is example output.

{
  "access_key_id": "AKIAJKSBDMSS8EXAMPLE",
  "creation_time": "2022-12-12T21:37:53.553457928Z",
  "owner": {
    "auth_id": "501",
    "domain": "LOCAL",
    "gid": null,
    "name": "guest",
    "sid": "S-1-1-12-12345678-2345678901-1234567890-500",
    "uid": null
  },
  "secret_access_key": "aGsfgDFdfEGWD/D5DWERF/sRsRdsXQEXAMPLEKEY"
}

Listing S3 Access Keys for a Qumulo Cluster

You can list every S3 access key known to your Qumulo cluster, along with the identities associated with the key and the key creation times, by using the Qumulo REST API or qq CLI. The PRIVILEGE_S3_BUCKETS_READ privilege is required.

To List S3 Access Keys for Your Own User Identity by Using the qq CLI

  • To list the S3 access keys for your user known to your Qumulo cluster, run the qq s3_list_access_keys :

    $ qq s3_list_access_keys  --self

    The following is example output. All times are in the UTC time zone.

    access_key_id         owner  creation_time
====================  =====  ==============================
000000000001fEXAMPLE  Guest  2022-12-12T21:37:53.553457928Z

  • For JSON output, use the --json flag.

    The following is example output. The command returns a single JSON object that contains the combined responses from calls to the /v1/s3/access-keys/ Qumulo Core REST API endpoint.

{
  "entries": [
    {
      "access_key_id": "AKIAIOSFODNN7EXAMPLE",
      "creation_time": "2022-12-12T21:37:53.553457928Z",
      "owner": {
        "auth_id": "501",
        "domain": null,
        "gid": null,
        "name": null,
        "sid": null,
        "uid": null
      }
    },
    ...
  ],
  "paging": {
    "next": null
  }
}

To List S3 Access Keys for All User Identities by Using the qq CLI

The PRIVILEGE_S3_BUCKETS_READ privilege is required.

  • To list all S3 access keys on your Qumulo cluster, run the qq s3_list_access_keys command.

    The following is example output. All times are in the UTC time zone.

    access_key_id         owner  creation_time
====================  =====  ==============================
000000000001fEXAMPLE  Guest  2022-12-12T21:37:53.553457928Z

  • For JSON output, use the --json flag.

    The following is example output. The command returns a single JSON object that contains the combined responses from calls to the /v1/s3/access-keys/ Qumulo Core REST API endpoint.

{
  "entries": [
    {
      "access_key_id": "AKIAIOSFODNN7EXAMPLE",
      "creation_time": "2022-12-12T21:37:53.553457928Z",
      "owner": {
        "auth_id": "501",
        "domain": null,
        "gid": null,
        "name": null,
        "sid": null,
        "uid": null
      }
    },
    ...
  ],
  "paging": {
    "next": null
  }
}

To List S3 Access Keys by Using the Qumulo Core REST API

To list all S3 access keys known to your Qumulo cluster, send a GET request to the /v1/s3/access-keys/ endpoint.

The following is example output. The entries list contains the access keys, limited to the first 10,000. The paging.next field contains the URI to which you can send a GET request to retrieve the next page of access keys. By making GET requests with all returned paging.next values, you can iterate over all of the access keys in the cluster.

{
  "entries": [
    {
      "access_key_id": "AKIAIOSFODNN7EXAMPLE",
      "creation_time": "2022-12-12T21:37:53.553457928Z",
      "owner": {
        "auth_id": "501",
        "domain": null,
        "gid": null,
        "name": null,
        "sid": null,
        "uid": null
      }
    },
    ...
  ],
  "paging": {
    "next": null
  }
}

Revoking S3 Access Keys for a Qumulo Cluster

To revoke an S3 access key, you must delete the access key from your Qumulo cluster by using the Qumulo REST API or qq CLI. The PRIVILEGE_S3_BUCKETS_WRITE privilege is required.

To Delete an S3 Access Key by Using the qq CLI

Run the qq s3_delete_access_key command and specify the access key ID. For example:

$ qq s3_delete_access_key \
  --id 000000000001fEXAMPLE

To Delete an S3 Access Key by Using the Qumulo Core REST API

Send a DELETE request to the /v1/s3/access-keys/<access-key-id> Qumulo Core REST API endpoint and specify the access key ID.

Configuring Active Directory (AD) for S3

For users that exist in an AD domain that has a trust relationship with the joined domain, you must append that domain’s base DN to the base DN in your Qumulo cluster’s AD configuration.

To append the trusted base DN to the base DN in use—with a semicolon (;) separating the two—use the Qumulo Core Web UI or the qq ad_reconfigure command. For example:

$ qq ad_reconfigure \
  --base-dn 'CN=Users, DC=joined_domain,DC=example,DC=com;\
  CN=Users,DC=trusted_domain,DC=example,DC=com'

For more information, see Configuring Cross-Domain Active Directory Trusts