Managing User-Defined Metadata in Qumulo Core

This section explains how to create, retrieve, list, and delete user-defined metadata in Qumulo Core by using the qq CLI.

How User-Defined Metadata Works in Qumulo Core

Qumulo Core lets you add user-defined metadata to any file type stored in its file system. User-defined metadata comprises user-specified key-value pairs that have the following requirements:

• The key must be a Unicode string.
• The value must be a sequence of bytes.
• The total size of each key-value pair must be under 400 KB.

Keyspace Types and Functions

User-defined metadata in Qumulo Core is divided into GENERIC, S3, and WELL_KNOWN keyspaces. Keyspaces work like containers for key-value pairs. The S3 and WELL_KNOWN keyspaces primarily support the S3 API, which requires all files to have separate sets of metadata in separate keyspaces.

The GENERIC and S3 keyspaces can hold approximately 17 trillion key-value pairs and have the following requirements:

• All keys within a keyspace must be unique.
• The keyspace and key are required to create or access a user-defined metadata entry.

The WELL_KNOWN Keyspace

In Qumulo Core 7.10.2 (and higher), the WELL_KNOWN keyspace holds the standard object headers that the S3 API stores and returns for an object. The keyspace has the following requirements:

• The key must be one of the following:
  • Cache-Control
  • Content-Disposition
  • Content-Encoding
  • Content-Language
  • Content-Type
  • Expires
• The value:
  • Can be case-insensitive
  • Must contain UTF-8 text valid for the HTTP header field
  • Must not contain any control characters, except for the tab character
• The combined size of the key and value must not exceed 2,048 bytes.

Managing User-Defined Metadata by Using the qq CLI

This section explains how to create, retrieve, list, and delete user-defined metadata by using the qq CLI.

Prerequisites

Managing user-defined metadata requires the following privileges:

• READ_EA: Read the user-defined metadata from a file
• WRITE_EA: Write to, or delete, the user-defined metadata of a file

To Create a Generic User-Defined Metadata Entry for a File by Using the qq CLI

Run the qq fs_set_user_metadata command and specify the path to the file, the key, and the value. For example:

qq fs_set_user_metadata \
  --path my-file \
  --key my-key \
  --value my-value

To specify a non-text value for the user-defined metadata, use the --base64-value or --hex-value flag.

For more information, see qq fs_set_user_metadata in the Qumulo qq CLI Command Guide.

To Retrieve a Generic User-Defined Metadata Entry for a File by Using the qq CLI

Use qq fs_get_user_metadata command and specify the path to the file and the key. For example:

qq fs_get_user_metadata \
  --path my-file \
  --key my-key

• To specify a non-text value for the user-defined metadata, use the --base64-value or --hex-value flag.
• To access the user-defined metadata within a file snapshot, use the --snapshot flag and specify the shapshot ID.

For more information, see qq fs_get_user_metadata in the Qumulo qq CLI Command Guide.

To List All Generic User-Defined Metadata Entries for a File by Using the qq CLI

Run the qq fs_list_user_metadata command and specify the path to the file. For example:

qq fs_list_user_metadata \
  --path my-file

• To specify a non-text value for the user-defined metadata, use the --base64-value or --hex-value flag.
• To access the user-defined metadata within a file snapshot, use the --snapshot flag and specify the shapshot ID.

For more information, see qq fs_list_user_metadata in the Qumulo qq CLI Command Guide.

To Delete a Generic User-Defined Metadata Entry for a File by Using the qq CLI

Run the qq fs_delete_user_metadata command and specify the path to the file and the key. For example:

qq fs_delete_user_metadata \
  --path my-file \
  --key my-key

For more information, see qq fs_delete_user_metadata in the Qumulo qq CLI Command Guide.

To Change Standard Object Headers by Using the qq CLI

In Qumulo Core 7.10.2 (and higher), you can change the standard object headers that the GetObject and HeadObject S3 API actions returns for an object without rewriting the object.

Run the qq fs_set_user_metadata , specify the path to the object, the key and value for the headers, and the --well-known flag. For example:

qq fs_set_user_metadata \
 --path /buckets/my-bucket/my-object.json \
 --well-known \
 --key content-type \
 --value application/json

To List Standard Object Headers by Using the qq CLI

In Qumulo Core 7.10.2 (and higher), you can list the standard object headers that the S3 API returns for an object without rewriting the object.

Run the qq fs_list_user_metadata command, specify the path to the object, and use the --well-known flag. For example:

qq fs_list_user_metadata \
  --path /buckets/my-bucket/my-object.json \
  --well-known

Managing User-Defined Metadata by Using the S3 API

S3 categorizes metadata as follows: