S3

The S3 database engine exposes objects in Amazon S3 and compatible object storage as read-only tables.

The S3 database engine exposes objects in Amazon S3 and S3-compatible object storage as read-only tables. A table name is resolved as an object URL through the s3 table function.

Creating a database

CREATE DATABASE s3_data
ENGINE = S3([url[, NOSIGN | access_key_id, secret_access_key]]);

The engine can also take a named collection. Supported configuration keys are url, access_key_id, secret_access_key, no_sign_request, and use_environment_credentials.

Usage

With a base URL, table names are object paths relative to that URL:

CREATE DATABASE s3_data
ENGINE = S3('https://bucket.s3.amazonaws.com/data/', 'NOSIGN');

SELECT * FROM s3_data.`events.parquet`;

Without a base URL, use the full object URL as the table name. The schema and format are inferred in the same way as for the s3 table function. The database owns no table definitions and does not support table DDL or writes.

Access control

S3 URLs are checked against the server’s remote-host filter. Creating this database requires READ and WRITE source grants on S3, regardless of table_engines_require_grant, for example:

GRANT READ, WRITE ON S3 TO user_name;

See the SOURCES privileges for version and compatibility details.

See also

• s3 table function
• Using S3 with ClickHouse
• Filesystem database engine
• HDFS database engine