Managing Access to S3 Buckets in a Qumulo Cluster

This section explains how to manage access to S3 buckets in a Qumulo cluster.

Managing user access to S3 buckets in a Qumulo cluster is very similar to managing access to SMB shares and NFS exports, with the following exceptions:

To let a user access S3 buckets in the cluster, you must assign an S3 access key to the user. Alternatively, you can create presigned URLs or enable read-only, anonymous access for the entire S3 bucket.
Because a Qumulo cluster restricts S3 actions based on file access control lists (ACLs), an S3 bucket might work differently or have more restrictive permissions than expected.

Note
To configure an S3 bucket in Qumulo Core to work more like an Amazon S3 bucket, use inheritable access control entries (ACEs) to imitate bucket-level permissions.

How S3 Bucket Permissions Work in Qumulo Core

To process an S3 API request, Qumulo Core performs one or more file system operations. Qumulo Core processes these operations by checking the user’s access against the access control lists (ACLs) for each file that is part of the request.

Note
To permit an action to be performed, the bucket policy and the object’s file system ACL must allow the action.

For authenticated requests signed with Amazon Signature Version 4, Qumulo Core maps the access key ID in the request to its corresponding auth ID, and then processes the request as that user. Qumulo Core processes unsigned, anonymous requests as the Guest user.

While Qumulo Core processes an S3 request, the ownership of any newly created files and directories belongs to the user that makes the request. These files and directories inherit access control entries (ACEs) from their parents (this process is the same for all protocols).

Granting Access to S3 Buckets by Using Presigned URLs

To let trusted users perform S3 API actions—such as GetObject or UploadPart—as if using your user account, you can generate a presigned URL (also known as query parameter authentication), associate the URL with specific API actions, and then share it with trusted users. Every presigned URLs has a configurable expiration time that ensures that the URL stops working at the configured time.

For more information, see Authenticating Requests: Using Query Parameters (AWS Signature Version 4) in the Amazon Simple Storage Service API Reference.

Note

Qumulo Core accepts only presigned requests that use the PUT, GET, HEAD, and DELETE HTTP methods. Qumulo Core rejects presigned requests for POST requests, such as the following:

AbortMultipartUpload
CompleteMultipartUpload
CreateMultipartUpload
DeleteObjects

To create a presigned URL, run the AWS CLI presign command. In the following example, the presigned URL expires in 10 minutes (600 seconds).

$ aws2 s3 presign s3://my-bucket/my-file.txt \
  --endpoint-url https://203.0.113.0:9000 \
  --profile my-qumulo-profile \
  --expires-in 600

The following is example output. The X-Amz-Expires header is set to 10 minutes.

https://203.0.113.0:9000/my-bucket/my-file.txt?
X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=00000000000003e88527%2F20230217%2Fus-east-1%2Fs3%2Faws4_request
&X-Amz-Date=20230217T205559Z
&X-Amz-Expires=600
&X-Amz-SignedHeaders=host
&X-Amz-Signature=141fa5b10caaa8575ba9c065d2270a24ce14b2ff58bb2c2e98382c76297b21ee

Enabling Anonymous Access for an S3 Bucket

In certain cases, it might be more practical to allow anonymous (unauthenticated) requests to access the contents of S3 buckets, for example, if you want to let users access objects from the S3 bucket by using a web browser or if the number of users who need read access is very large. When you enable anonymous access to an S3 bucket, your users can perform read-only S3 operations without authenticating their requests.

Important
Anonymous requests can never perform modifying operations. Qumulo Core requires all modifying operations on an S3 bucket to be authenticated.

When you enable anonymous access for an S3 bucket, Qumulo Core performs all anonymous requests as the Guest user. The Guest user is a member of the Everyone group, but not of the Users group.

To ensure that anonymous requests have permission to read files in a bucket, grant read permission to the Everyone group or to the Guest user. For more information, see Imitating Bucket-Level Read-Only Access.

Note
If a file’s ACL doesn’t allow reads for the Guest user, an anonymous request can’t read the file.

To view the current bucket configuration by using the Qumulo REST API or qq CLI, you need the PRIVILEGE_S3_BUCKETS_READ privilege.
To change the bucket configuration, you need the PRIVILEGE_S3_BUCKETS_WRITE privilege.

To Enable Anonymous Access to an S3 Bucket by Using the qq CLI

To view the current bucket configuration, run the qq s3_get_bucket command and specify the bucket name. For example:

$ qq s3_get_bucket \
  --name my-bucket

The following is example output. All times are in the UTC time zone.

{
  "anonymous_access_enabled": false,
  "creation_time": "2022-12-20T19:42:26.833076147Z",
  "name": "my-bucket",
  "path": "/buckets/my-bucket"
}

Anonymous access to S3 buckets is disabled by default. To enable anonymous access, run the qq s3_modify_bucket command, specify the bucket name, and use the --enable-anonymous-access flag.
To disable anonymous access, run the qq s3_modify_bucket command, specify the bucket name, and use the --disable-anonymous-access flag.

Using Inheritable ACEs to Imitate Bucket-Level Permissions

To grant multiple users access to all paths in a bucket and ensure that newly created directories inherit the correct permissions, use inheritable access control entries (ACEs).

In Amazon S3, permission to read objects from —and write objects to— an S3 bucket applies to the entire bucket. In Qumulo Core, each object key corresponds to a file path relative to a bucket’s root directory. Qumulo Core grants permissions for individual files and directories.

When users create objects in an S3 bucket in a Qumulo cluster, they might also create new directories. The user that creates these directories owns them. However, without the correct access control entries (ACEs) in your bucket, these directories might have restrictive permissions that prevent other users from creating objects with the same prefix.

How Permissions with Inheritable ACEs Work

Access control entries (ACEs) control the permissions that users have for files and directories in a Qumulo cluster. When you add ACEs to a directory and mark them as inheritable, all new files and directories created in that directory inherit those ACEs and pass them on.

You can use inheritable ACEs to:

Imitate bucket-level permissions by ensuring that any files and directories that your users create in an S3 bucket receive the same permissions.

To make all paths in an S3 bucket inherit the same set of ACEs, add the ACEs to the bucket’s root directory and mark them as inheritable.
Configure default permissions for newly created buckets.

To make a set of ACEs the default for buckets that your users create by using the S3 API, add the ACEs to the default bucket directory prefix.

To add ACEs to a directory, use the qq CLI or use the File Explorer on a Windows client with a mapped SMB share that contains the directory.

Note
Adding inheritable ACEs to a directory doesn’t affect any files that already exist in that directory. For more information, see To Recursively Add a New ACL with Multithreading.

Imitating Bucket-Level Permissions by Using the qq CLI

The following sections show how to use the qq CLI to imitate bucket-level permissions by adding inheritable ACEs.

Imitating Bucket-Level Read-Write Access

Run the qq fs_modify_acl command. In the following example, we add the access control entry (ACE) to the bucket whose root directory is /buckets/my-bucket for the user group MyWriters.

$ qq fs_modify_acl \
  --path /buckets/my-bucket add_entry \
  --trustee MyWriters \
  --type Allowed \
  --flags 'Container inherit' 'Object inherit' \
  --rights 'Delete child' 'Execute/Traverse' 'Read' 'Write file'

The ACE imitates bucket-level read-write access for a user or group of users.

Type     Flags                              Rights
=======  =================================  =====================================
Allowed  Object inherit, Container inherit  Delete child, Execute/Traverse, Read,
                                            Write file

Imitating Bucket-Level Read-Only Access

Run the qq fs_modify_acl command. In the following example, we add the access control entry (ACE) to the bucket whose root directory is /buckets/my-bucket for the user group MyReaders:

$ qq fs_modify_acl
  --path /buckets/my-bucket add_entry \
  --trustee MyReaders \
  --type Allowed \
  --flags 'Container inherit' 'Object inherit' \
  --rights 'Execute/Traverse' 'Read'

The ACE imitates bucket-level read-only access for a user or group of users.

Type     Flags                              Rights
=======  =================================  ======================
Allowed  Object inherit, Container inherit  Execute/Traverse, Read

Imitating Bucket-Level List-Only Access

Run the qq fs_modify_acl command. In the following example, we add two access control entries (ACEs) to the bucket whose root directory is /buckets/my-bucket for the user group MyListers.

$ qq fs_modify_acl
  --path /buckets/my-bucket add_entry \
  --trustee MyListers \
  --type Allowed \
  --flags 'Container inherit' \
  --rights 'Execute/Traverse' 'Read'

$ qq fs_modify_acl
  --path /buckets/my-bucket add_entry \
  --trustee MyListers \
  --type Allowed \
  --flags 'Object inherit' \
  --rights 'Read attr'

The two ACEs imitate bucket-level list-only access for a user or group of users:

Type     Flags              Rights
=======  =================  ======================
Allowed  Container inherit  Execute/Traverse, Read
Allowed  Object inherit     Read attr