hn.today

DuckDB Ducklake

github.com25 points2 comments
Screenshot of DuckDB Ducklake

DuckLake is an open Lakehouse format that combines a SQL catalog with Parquet storage to provide ACID-aware table semantics on data lakes. Metadata is kept in a catalog database while actual data lives in Parquet files; a DuckDB extension enables direct read/write access from DuckDB. It supports standard SQL operations (CREATE, INSERT, UPDATE), schema evolution (ALTER TABLE ADD COLUMN), time travel queries by version, and a change-data feed function (table_changes) that returns snapshot and row-level change records. Examples in the usage show attaching a DuckLake catalog (ATTACH 'ducklake:metadata.ducklake' AS my_ducklake (DATA_PATH 'file_path/')), issuing queries through the attached namespace, and retrieving historical snapshots or incremental changes.

Installation and development are command-line driven: the extension can be installed inside DuckDB with INSTALL ducklake; or FORCE INSTALL ducklake FROM core_nightly for nightly builds. Building from source requires initializing submodules, running make (with options for parallel builds), and launching the bundled duckdb shell. A comprehensive test harness is provided with commands to run unit tests, run subset patterns, and exercise different catalog backends (Postgres, SQLite) or features like deletion vectors. Contributions target the main branch, the codebase is MIT-licensed, and the project includes documentation, examples, and CI/test configuration to support development and integration.

Read on github.com2 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Science

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.