clickhousedb SQLAlchemy dialect on top of the core driver. The synchronous dialect supports SQLAlchemy 1.4.40 and later, including SQLAlchemy 2.x, with a focus on Core queries, ClickHouse DDL, reflection, and simple ORM inserts. The async dialect requires SQLAlchemy 2.0.44 or later.
Install the SQLAlchemy dependencies with the package extra:
Connect with SQLAlchemy
Create an engine with either theclickhousedb:// or clickhousedb+connect:// URL form:
ClickHouse session IDs
Each pooled connection in either the synchronous or async dialect generates a distinct ClickHouse session ID by default. When requests for that connection reach the same ClickHouse server process, settings changed withSET and temporary tables persist for that connection. Named-session state and same-session overlap checks are process-local. On one server process, an overlapping request for the same user and session ID is rejected immediately with server code 373 instead of being queued. If you configure a fixed session_id, use pool_size=1, max_overflow=0 or serialize access before requests reach ClickHouse. In ClickHouse Cloud or other load-balanced deployments, requests with the same session ID can reach different servers, so do not rely on a fixed session_id as distributed state or as a distributed mutex.
Async connections
The async dialect requires SQLAlchemy 2.0.44 or later and uses the native ClickHouse ConnectAsyncClient. Install its dependencies and create an async engine with the clickhousedb+async:// URL:
AsyncConnection.stream() raises InvalidRequestError. AsyncSession.stream() is accepted by SQLAlchemy, but the dialect buffers the full result before returning it. Use the native AsyncClient streaming methods for large results. The raw native client is available as driver_connection while its SQLAlchemy connection is checked out:
client.close() or any of its private lifecycle methods. SQLAlchemy’s pool owns connection concurrency. Each pooled connection owns one native async client and defaults its aiohttp connector limits to one connection and one connection per host. Set connector_limit, connector_limit_per_host, or keepalive_timeout in the URL or connect_args to override those transport settings. With pool_pre_ping=True, SQLAlchemy checks reused connections with SELECT 1 when a pooled connection is checked out.
Async SQLAlchemy executemany inserts currently send one HTTP request for each parameter set instead of using the driver’s Native bulk insert protocol. Use this path only for small batches. For bulk data, use the pool-owned driver_connection access pattern above and await client.insert() before returning the SQLAlchemy connection to the pool. Because async executemany uses query parameter binding, naive datetime values follow naive_datetime_binding, not the naive_datetime_insert setting used by synchronous Native executemany. Typed SQLAlchemy DateTime64 binds preserve fractional seconds with both client-side and server-side parameters. Untyped %s or %(name)s parameters passed to exec_driver_sql() retain the default whole-second formatting for naive datetime values. Use timezone-aware values for unambiguous timezone behavior. Use client.insert() for Native bulk semantics.
Create and dispose an async engine in the event loop where it is used. Return every checked-out connection, then await engine.dispose() during shutdown and before using the engine from another event loop. If the engine’s owning loop has already closed, await engine.dispose() in the current loop before reuse. aiohttp may still report an unclosed transport when cleanup begins only after the owning loop has closed, so dispose before transfer when possible. pool_pre_ping=True is not a replacement for disposal when moving a pooled async engine between event loops. To share one engine across event loops without retaining loop-bound connections, configure poolclass=NullPool. If disposal runs while a connection is still checked out, the dialect closes that connection when it is returned or garbage collected. Do not call engine.sync_engine.dispose() from synchronous code. SQLAlchemy cannot await async connection cleanup there and may log the error instead of closing pooled transports.
URL query parameters can contain ClickHouse settings, ClickHouse Connect client options such as compression, query_limit, and timeouts, or HTTP/TLS options such as ca_cert. Prefix a ClickHouse setting with ch_ to force it to be treated as a server setting when needed, for example ch_http_max_field_name_size=99999.
See Connection arguments and settings for the available client options.
Run synchronous SQLAlchemy helpers such as DDL and inspection through AsyncConnection.run_sync():
Per-query settings
Pass ClickHouse settings through SQLAlchemy execution options. Settings can be set on an engine, connection, or statement. A statement value takes precedence over a connection or engine value with the same key.Per-query read formats
Set ClickHouse read formats on an engine, connection, or statement through SQLAlchemy execution options withquery_formats, with statement formats applied first so they override matching connection or engine keys and wildcards.
Error handling
Errors raised by the driver through a SQLAlchemy connection use the DB-API classes exported fromclickhouse_connect.dbapi. They are the same class objects as the corresponding classes in clickhouse_connect.driver.exceptions, so SQLAlchemy wraps them in the matching sqlalchemy.exc.DBAPIError subclass. StreamFailureError is an OperationalError and is wrapped as sqlalchemy.exc.OperationalError.
If caller cancellation can interrupt an explicit AsyncConnection.invalidate(), run invalidation in an owned task and wait for that task before propagating cancellation. This lets SQLAlchemy finish its connection-record bookkeeping:
await connection.invalidate() is cancelled and connection.invalidated remains false, await connection.invalidate() again to finish cleanup before using or closing the connection.
Server-side parameters
SQLAlchemy normally renders client-side parameters. Opt in to ClickHouse server-side parameters when creating the engine:server_side_params=True argument with create_async_engine() for the async dialect.
In this mode every bound value must have a ClickHouse-compatible SQLAlchemy type. Supported IN lists become typed ClickHouse Array parameters. The compiler raises CompileError when it cannot derive a compatible type or safely process a bind.
Bind names must be ClickHouse ASCII BareWord names. Names that start and end with $ are rejected because the core driver reserves them for raw binary query parameters.
Core queries
The dialect supports SQLAlchemy CoreSELECT queries with joins, filters, ordering, limits and offsets, DISTINCT, and compound selects.
SQLAlchemy union(), intersect(), and except_() compile to ClickHouse UNION DISTINCT, INTERSECT DISTINCT, and EXCEPT DISTINCT. Their union_all(), intersect_all(), and except_all() counterparts compile to the corresponding ALL operators. This explicit mapping preserves SQLAlchemy’s duplicate semantics regardless of ClickHouse set-operation defaults.
DELETE is supported and requires an explicit WHERE clause:
Literal rendering
When SQLAlchemy inlines a bound value throughliteral_binds or literal_execute, the dialect uses ClickHouse quoting for generic string types and ClickHouse types. This also applies through TypeDecorator wrappers and with_variant() selections. String values retain percent signs and backslashes even when other bound parameters remain.
Python datetime values with a ClickHouse DateTime64 SQLAlchemy type retain their microseconds in client-side parameters and inline literals, including nullable values and values nested in arrays and tuples. ClickHouse applies the declared precision. Python datetime provides up to six fractional digits. Plain DateTime values retain whole-second formatting. For a text() statement, supply the type explicitly with bindparam("ts", type_=DateTime64(6)) to preserve fractional seconds.
SQLAlchemy column types must match the server schema. Declaring DateTime64 over a server DateTime column renders fractional seconds and can raise conversion errors on insert and in IN comparisons.
On SQLAlchemy 2.x, inline literals of generic sqlalchemy.ARRAY types containing ClickHouse Tuple items need dimensions=1, or the appropriate higher dimension count for nested arrays, so SQLAlchemy treats each tuple as one item. SQLAlchemy 1.4 does not support inline literals for generic ARRAY types.
If a named datetime parameter is reused, every occurrence needs a compatible DateTime64 bind type to preserve fractions. An untyped occurrence or a conflicting type keeps whole-second formatting. Set type_=DateTime64(6) on each bindparam, or use distinct parameter names with the appropriate types.
JSON type hints
Declare typed JSON paths with thetyped_paths mapping. A path type can be a ClickHouse SQLAlchemy type class, a configured instance, or a ClickHouse type name string. Type name strings support types without a SQLAlchemy constructor, such as Dynamic, and can still be used for complex configured type expressions. They preserve names in a named Tuple.
Type name strings can contain configured nested JSON types such as Array(JSON(`child` UInt32)). Recognized ClickHouse type names are case-insensitive in these strings and are emitted with their canonical capitalization. A string must contain one complete type expression. Trailing text and malformed nested JSON arguments are rejected.
An empty Tuple() is not supported as a JSON typed path because ClickHouse cannot serialize it through a JSON column’s Native format. The core driver supports Tuple() in query and insert columns at any position, including nested in positional or named tuples, inside Array, and as Nullable(Tuple()) where enabled by the server.
typed_paths, for example JSON(user_id=UInt32). Use typed_paths for dotted paths, spaces, backticks, %2E encoded dots, or names that match constructor options. A typed path named SKIP is supported through the mapping. Keys in typed_paths and values in skip_paths are decoded names. Leading or trailing backticks and double quotes are treated as literal path characters, not as pre-applied SQL quoting. Inside a raw type string, backticks and double quotes are ClickHouse identifier syntax.
Up to 1000 typed paths can be configured. max_dynamic_paths accepts 0 through 10000. max_dynamic_types accepts 0 through 254. These ranges also apply inside raw nested JSON type strings. Explicit server defaults of 1024 and 32 are omitted from generated DDL. Plain skip paths are deduplicated. Regular expression strings are not validated by Python because ClickHouse uses RE2 syntax. Duplicate regular expressions are preserved.
A plain skip path cannot be named exactly REGEXP because ClickHouse reserves that token for SKIP REGEXP. Names such as REGEXP_foo remain valid. In a raw JSON type string, a plain SKIP operand must be one ClickHouse identifier or a dot-separated compound identifier. An unquoted compound identifier cannot start with REGEXP; quote that first component when it is path data. SKIP REGEXP must have one single-quoted string literal. Quote identifier parts with backticks or double quotes when they contain spaces or punctuation. Raw JSON type hints support Variant(...); standalone Variant has no public SQLAlchemy constructor. Variant members are ordered and deduplicated by the same canonical names used by ClickHouse.
The constructor orders arguments in the same canonical form returned by ClickHouse. Reflected types, SQLAlchemy type copies, and Alembic autogeneration preserve the configuration.
JSON subcolumns
For a column declared or reflected as ClickHouseJSON, use square brackets to select one segment of a storage-backed subcolumn path at a time:
payload["severity"] compiles to ClickHouse dotted identifier syntax. Each part is quoted separately, for example `events`.`payload`.`severity`. It reads ClickHouse’s stored JSON subcolumn and does not call getSubcolumn. Chain [] or .subcolumn() once for each path segment. Each segment must be a non-empty string.
Passing type_ to .subcolumn() wraps the dotted path in a SQL CAST and assigns that type to the SQLAlchemy expression. Without type_, .subcolumn("segment") behaves like ["segment"].
An untyped path has ClickHouse’s Dynamic type. ClickHouse does not allow Dynamic values directly in ORDER BY or GROUP BY. Pass type_ when a subcolumn is used there.
For statically typed code, import json_subcolumn from clickhouse_connect.cc_sqlalchemy. The helper also takes one segment at a time and preserves the Python result type from type_:
request_id as ColumnElement[int].
Each segment is quoted independently, including names with spaces or backticks. Backticks do not make a dot literal to ClickHouse JSON path handling. When json_type_escape_dots_in_keys is enabled, use ClickHouse’s %2E encoding for literal dots in keys. Access a key named a.b as payload["a%2Eb"], not payload["a.b"].
ClickHouse query extensions
Importselect from clickhouse_connect.cc_sqlalchemy to expose typed ClickHouse methods to static type checkers. The standard sqlalchemy.select also has these methods at runtime.
Select methods are:
SQLAlchemy’s
Select.with_hint() is a table hint API. The ClickHouse dialect does not render table hints. An applicable wildcard or clickhousedb hint emits an SAWarning and leaves the generated SQL unchanged. Use final(), sample(), prewhere(), or limit_by() for those ClickHouse clauses.
Select.with_statement_hint() is a raw tail directive API. It appends the supplied text to the end of the SELECT without ClickHouse-specific validation. This remains available for trusted static SQL such as SETTINGS max_threads=1:
GLOBAL ANY LEFT JOIN can be chained without nesting a custom FromClause:
Lambda construct for ClickHouse higher-order functions:
values() construct compiles to ClickHouse’s VALUES table-function syntax, including when used in a common table expression. The CTE form requires SQLAlchemy 2.0.42 or later, where Values.cte() was added.
Materialized CTEs
By default ClickHouse inlines a common table expression, so a CTE referenced more than once has its body executed once per reference. Passmaterialized=True to .cte() to emit WITH <name> AS MATERIALIZED (...), which computes the body once:
enable_materialized_cte=1, and the analyzer is enabled. Set enable_materialized_cte on the statement, connection, or engine as shown in Per-query settings. The analyzer is enabled by default on every server that supports this feature, so setting enable_analyzer=1 explicitly is defensive. enable_materialized_cte is an experimental ClickHouse setting. With enable_materialized_cte=0 or enable_analyzer=0, the query succeeds and returns the same rows. ClickHouse silently ignores MATERIALIZED and inlines the CTE again, so a forgotten setting costs performance without raising anything. Materialized CTEs require ClickHouse 26.3 or later. Older servers reject the keyword as a syntax error.
For a statement built with the standard sqlalchemy.select, use the module-level cte() instead. It takes the statement as its first argument and otherwise mirrors Select.cte():
ValueError when recursive=True and materialized=True are both set.
DDL and reflection
ClickHouse Connect provides ClickHouse data types, table engines, dictionary constructs, database DDL, and table reflection. StandaloneVariant columns reflect through an internal SQLAlchemy type, and Alembic autogenerate preserves their canonical raw type names without repeated type changes. Geometry and MultiPoint columns reflect as public SQLAlchemy types.
server_default for DEFAULT expressions and dialect-specific attributes such as clickhouse_codec, clickhouse_ttl, clickhouse_materialized, and clickhouse_alias when present.
String values in DEFAULT, MATERIALIZED, ALIAS, and TTL clauses use ClickHouse string escaping. The same escaping applies to table, dictionary, and column comments, including comments emitted by Alembic.
MergeTree key arguments such as order_by, partition_by, primary_key, sample_by, and ttl accept SQLAlchemy column and SQL expressions as well as plain strings.
Memory(), Log(), StripeLog(), TinyLog(), Null(), and Set() accept zero arguments and round-trip through Alembic autogeneration. The existing dictionary argument remains supported. Use settings={...} to supply engine settings.
SummingMergeTree and ReplicatedSummingMergeTree accept an optional, keyword-only columns argument. Existing positional arguments keep their meaning, so SummingMergeTree("id") still sets ORDER BY id.
"delta" or "(delta, n_tx)". The server requires identifiers for these columns. Omit columns to let ClickHouse select the columns to sum. Reflection and Alembic autogeneration preserve an explicit column list.
Inserts and basic ORM use
Core inserts and simple ORM models are supported. For the synchronous dialect, prefer Core executemany inserts for compatible bulk data paths. For async bulk inserts, use the nativeAsyncClient.insert() path described in Async connections.
executemany inserts generated by the SQLAlchemy compiler use one Native bulk insert. Async executemany sends one request per parameter set, as described in Async connections. Raw SQL and inserts with expressions or other semantics that cannot be routed safely preserve the original SQL and execute once for each parameter set. If a later parameter set fails, rows written by earlier parameter sets remain committed.
Explicit multi-row insert(events).values([...]) statements work with dictionary rows, tuples in table column order, and per-row SQL expressions. Pandas to_sql(method="multi") uses this form. It inserts the rows but returns 0 because textual INSERT statements report a row count of 0 through the DB-API cursor. SQLAlchemy determines the column list from the first row. Extra dictionary keys in later rows and tuple values outside that selected column list are ignored. A later row missing a selected value fails compilation. Give every row the same columns.
With the default HTTP form limits in ClickHouse 26.4 and newer, server_side_params=True is suitable only for small explicit batches, below about 1000 bind values with headroom for other fields. Server configuration can raise this ceiling. For large plain batches with the synchronous dialect, pass rows as the second argument to execute() so the driver can use its Native bulk insert path. For async bulk data, await the native AsyncClient.insert() method.
Alembic migrations
ClickHouse Connect includes Alembic integration for ClickHouse schema migrations. Install it with:alembic.ini uses script_location = %(here)s/alembic. Keep that setting when the migration directory is named alembic, or update it to the directory passed to alembic init. Replace alembic/env.py with the checked-in async Alembic env.py example, then set sqlalchemy.url in alembic.ini.
Import clickhouse_connect.cc_sqlalchemy.alembic in Alembic’s env.py to register the dialect integration. Autogenerate supports common table evolution, including table creation and removal, column add/alter/drop, defaults, and comments. Use manual operations for table and column renames. Review every generated migration before applying it.
Alembic’s migration functions remain synchronous. An async environment creates an AsyncEngine, opens an AsyncConnection, and passes the synchronous migration function to await connection.run_sync(...). Offline migrations call context.configure(url=..., literal_binds=True, dialect_opts={"paramstyle": "named"}) directly and do not create an engine. The checked-in async Alembic env.py example includes both paths and reads the connection URL through Alembic’s standard sqlalchemy.url configuration. It keeps the ClickHouse Alembic hooks and options from the worked example, including include_object, make_include_name(...), clickhouse_writer, and version_table. Do not use engine.sync_engine to run or dispose async migrations.
ClickHouse-specific op.* helpers cover:
- Data skipping indexes, including add, materialize, and drop operations.
- Projections, including add, materialize, and drop operations.
- MergeTree table setting modification and reset.
- Materialized view creation and removal.
- Dictionary creation, removal, and reload.
Index, Column(index=True), op.create_index, and op.drop_index are rejected to avoid partial or incorrect DDL. Use op.add_clickhouse_index and op.drop_clickhouse_index.
See the complete Alembic worked example. Users migrating from clickhouse-sqlalchemy should also read the migration guide.
Scope and limitations
- ClickHouse does not provide traditional transactions through this HTTP dialect.
engine.begin()andSession.commit()organize Python-side work, but commit and rollback are no-ops on the server. UPDATE, two-phase transactions, sequences,RETURNING, and advanced isolation levels are not implemented by the dialect. Use explicit ClickHouse SQL for server mutations when needed.Column(..., primary_key=True)supplies SQLAlchemy object identity. It does not create a server-side uniqueness constraint. Define sorting and optional primary-key expressions through the table engine.- Traditional foreign-key, unique-constraint, and standard index metadata are not available because ClickHouse does not enforce those constraints.
- ORM relationship management, unit-of-work updates, cascades, and eager or lazy relationship loading are outside the supported ORM scope.