This section documents Qumulo Core support for S3 API functionality and S3 API limits.

Supported S3 API Actions

The following table lists the S3 API actions that Qumulo Core supports and the version from which support begins. For the full list of S3 API actions, see Actions in the Amazon Simple Storage Service API Reference.

API Action Supported from Qumulo Core Version
AbortMultipartUpload 5.3.3
CompleteMultipartUpload 5.3.3
CopyObject 5.3.3
CreateBucket 5.2.3
CreateMultipartUpload 5.3.3
DeleteBucket 5.2.4
DeleteBucketVersioning 7.1.2
DeleteBucketPolicy 7.0.1.1
DeleteObject 5.2.1
DeleteObjects 5.2.2
DeleteObjectTagging 6.3.2
GetBucketAcl 6.1.1
GetBucketLocation 5.1.2
GetBucketVersioning 7.1.2
GetBucketPolicy 7.0.0.1
GetBucketVersioning 7.0.0.1
GetObject 5.0.4
GetObjectLockConfiguration 7.2.1.1
GetObjectRetention 7.2.1.1
GetObjectTagging 7.1.2
GetObjectAcl 6.1.1
GetObjectLockConfiguration 7.0.0.1
HeadBucket 5.1.2
HeadObject 5.0.4
ListBuckets 5.0.4
ListMultipartUploads 5.3.3
ListObjects 5.0.5
ListObjectsV2 5.0.4
ListParts 5.3.3
PutBucketPolicy 7.0.1.1
PutBucketVersioning 7.1.2
PutObject 5.2.1
PutObjectLockConfiguration 7.2.1.1
PutObjectRetention 7.2.1.1
PutObjectTagging 6.3.2
UploadPart 5.3.3
UploadPartCopy 6.0.2

Unsupported S3 Functionality

The following table lists some of the S3 API functionality that Qumulo Core doesn’t support.

Unsupported Feature Description
BitTorrent
Bucket ACLs For comparable functionality, use inheritable access control entries (ACEs).
Bucket lifecycle configurations
Bucket notifications
Control of server-side encryption All Qumulo Core data is encrypted at rest. You can’t control this functionality by using the S3 API.
Logging controls
Multi-chunk payload signing Qumulo Core doesn’t support the streaming version of Amazon Signature Version 4 (SigV4), only the single-chunk version.
Object locks
Signature Version 2 Qumulo Core supports only SigV4 signatures.
Storage classes Qumulo Core doesn’t use the storage class concept. All objects have the same storage class status.
Retention policies
Temporary access credentials
Virtual-hosted bucket addressing Qumulo Core supports only path-style bucket addressing.
Web hosting configuration

S3 API Limitations

This section describes the most important S3 API limitations in Qumulo Core.

Bucket Addressing Style

Because Qumulo Core supports only path-style bucket addressing, you must configure your client applications to use path-style addressing to send S3 API requests to a Qumulo cluster. For more information, see Configuring the AWS CLI for Use with Qumulo Core.

ETags

RESTful APIs, such as the S3 API, use HTTP ETags to identify different resource versions.

  • Qumulo Core uses a proprietary mechanism to generate an object’s ETag.

  • Amazon S3 uses the MD5 checksum of an object’s contents as its ETag.

Listing Objects

The S3 API supports listing objects in a bucket by using the ListObjects and ListObjectsV2 API actions.

Function Qumulo Core Amazon S3
Returning results Consistent but non-alphabetical order Alphabetical order, by object key
Arbitrary prefix Partial support for Prefix, only if Prefix is a path to a file or directory under the bucket root directory Prefix limits results to object keys that begin with the prefix
Arbitrary delimiter Only the slash (/) character can act as Delimiter Delimiter groups results into common prefixes

Request Authentication

Qumulo Core supports authenticating requests by using only Amazon Signature Version 4. Most S3 client applications support this authentication type.

If your application attempts to use a previous Amazon signature version, you receive a 400 Bad Request response with the error code AuthorizationHeaderMalformed.

Versioning

  • Object Version Limits: In Qumulo Core, S3 bucket versioning is consistent with that of Amazon S3, with the exception of individual object versions. Qumulo Core limits directories to approximately 4.3 billion child files. The approach that Qumulo Core takes to indexing files in a directory might cause object creation commands to output the QumuloDirectoryEntryLimitReached error when a directory gets close to its capacity. Because Qumulo Core gives object versions unique identifiers, it might be possible to retry the command successfully. However, if you begin to observe this error, we recommend removing previous object versions from your system.

  • Creating Empty Versioned Directories: Qumulo Core doesn’t support creating empty, versioned directories.

  • Deleting Versioned Objects: If you don’t specify an object version ID, the DeleteObject and DeleteObjects S3 API actions create a deletion marker for an object but don’t delete any file system data. Because currently Qumulo Core doesn’t support bucket lifecycle policies, the data remains accessible by using S3 API actions and the object version ID. To delete a specific object version permanently, specify its version ID when you use either of these API actions.

Comparison of Known Limits between S3 in Qumulo and Amazon

This section compares the Qumulo Core S3 API limits with native Amazon S3 limits.

Limits for S3 Buckets

Limit Qumulo Core Amazon S3
Maximum number of buckets 16,000 1,000
Maximum number of objects in one bucket Nominally unlimited Unlimited
Minimum bucket name length 3 characters
Maximum bucket name length 63 characters

Limits for S3 Objects

Limit Qumulo Core Amazon S3
Minimum object size 0 bytes
Maximum object size (by using PutObject) 5 GiB
Maximum object size (by using MultipartUpload) 48.8 TiB (10,000 * 5 GiB) 5 TiB
Minimum object key length 1 character
Maximum object key length 1,530 characters, if there are no slash (/) characters in the key 1,024 characters
Maximum object versions 4,294,967,296 (theoretical) Unlimited

Limits for S3 Multipart Uploads

Limit Qumulo Core Amazon S3
Minimum part ID 1
Maximum part ID 10,000
Minimum number of parts for each upload 1
Maximum number of parts for each upload 10,000
Minimum part size 5 MiB (except for the last part of an upload)
Maximum part size 5 GiB
Additional part size requirements Must be a multiple of 4 KiB (4,096 bytes), except for the last part of an upload

Limits for S3 API Requests

Maximum Limit Qumulo Core Amazon S3
Object keys that DeleteObjects specifies Nominally unlimited 1,000
Buckets that ListBuckets returns 16,000 1,000
Objects that ListObjects and ListObjectsV2 return 1,000
Parts that ListParts returns Unlimited 1,000
Uploads that ListMultipartUploads returns 1,000

In addition, the following API actions have the Qumulo-specific maximum payload size limit of 10 MiB.

  • CompleteMultipartUpload
  • CreateBucket
  • DeleteObjects