Delete files contain the positions of rows that are deleted within a data file. Each data file can have its own delete file if any deletes are present for this data file.
| Column name | Column type | |
|---|---|---|
delete_file_id |
BIGINT |
Primary key |
table_id |
BIGINT |
|
begin_snapshot |
BIGINT |
|
end_snapshot |
BIGINT |
|
data_file_id |
BIGINT |
|
path |
VARCHAR |
|
path_is_relative |
BOOLEAN |
|
format |
VARCHAR |
|
delete_count |
BIGINT |
|
file_size_bytes |
BIGINT |
|
footer_size |
BIGINT |
|
encryption_key |
VARCHAR |
|
partial_max |
BIGINT |
|
row_group_count |
BIGINT |
delete_file_idis the numeric identifier of the delete file. It is a primary key.delete_file_idis incremented fromnext_file_idin theducklake_snapshottable.table_idrefers to atable_idfrom theducklake_tabletable.begin_snapshotrefers to asnapshot_idfrom theducklake_snapshottable. The delete file is part of the table starting with this snapshot id. For a partial delete file, this is the earliest deletion snapshot it contains.end_snapshotrefers to asnapshot_idfrom theducklake_snapshottable. The delete file is part of the table up to but not including this snapshot id. Ifend_snapshotisNULL, the delete file is currently part of the table.data_file_idrefers to adata_file_idfrom theducklake_data_filetable.pathis the file name of the delete file, e.g.,my_file-deletes.parquetfor a relative path.path_is_relativewhether thepathis relative to thepathof the table (true) or an absolute path (false).formatis the storage format of the delete file. Supported values areparquet(positional delete file) andpuffin(Iceberg V3 deletion vector, experimental).delete_countis the number of distinct deleted row positions at the latest snapshot represented in the file. For cumulative Puffin vectors, this is the latest vector's count, not the sum of the vectors.file_size_bytesis the size of the file in bytes.footer_sizeis the size of the file metadata footer, in the case of Parquet the Thrift data. This is an optimization that allows for faster reading of the file. For a Puffin container, it includes the JSON payload and the 16-byte footer structure. A bare deletion vector has no footer and stores0.encryption_keycontains the encryption for the file if encryption is enabled. Puffin files are written without encryption, even if this field is populated.partial_maxis the maximum snapshot id stored in a partial deletion file. When multiple deletes target the same data file across different snapshots, the deletion file is rewritten as a partial deletion file that tracks which rows were deleted in which snapshot. This column stores the highest snapshot id present in such a file. It isNULLfor standard (non-partial) delete files.row_group_countis the number of Parquet row groups in the delete file, orNULLfor Puffin files or when unknown.
DuckLake format version 1.1 supports Puffin files with cumulative deletion vectors for several snapshots. Each vector's ducklake-snapshot-id property identifies its snapshot. Readers select the vector with the greatest snapshot id that does not exceed the requested snapshot. A file for a single deletion snapshot can contain a bare deletion-vector-v1 blob, whose snapshot is given by begin_snapshot.
A replacement delete file preserves earlier deletions and their snapshot information. The old row is removed from ducklake_delete_file, and its file is added to ducklake_files_scheduled_for_deletion.