system.s3queue_metadata_cache

Contains in-memory state of S3Queue metadata and currently processed rows per file.

Description

Contains in-memory state of S3Queue metadata and currently processed rows per file.

Columns

• zookeeper_path (String) — Path in zookeeper to metadata
• file_path (String) — File path of a file which is being processed
• file_name (String) — File name of a file which is being processed
• rows_processed (UInt64) — Currently processed number of rows
• status (String) — Status of processing: Processed, Processing, Failed, or None for a file which the cache knows but has no state for, e.g. a file whose processing was reset. A non-null processing_observed_in_keeper_time means that the Processing status was not set by the processor which holds the file, but read from keeper
• processing_start_time (Nullable(DateTime)) — Time at which processing of the file started
• processing_end_time (Nullable(DateTime)) — Time at which processing of the file ended
• exception (String) — Exception which happened during processing. Always the one of the last attempt of this server, never the one which keeper stores for the file
• processing_observed_in_keeper_time (Nullable(DateTime)) — Time at which a processor of this server failed to take the file and read the Processing status from keeper instead of setting it itself. Usually the file is held by another server, but it can also be held by a processor of this server which had just taken it. In either case rows_processed, processing_start_time, processing_end_time and exception describe the last attempt of this server to process the file, not the processing which holds it now