TLDR Data · LAUNCHES & TOOLS

ASYNCHRONOUS I/O IN DUCKDB: WORK, THREAD, WORK

duckdb.org12 min read

Summary from the TLDR Data newsletter

DuckDB is adding asynchronous I/O so worker threads stop blocking on S3-backed Parquet and CSV reads. The design separates async and worker pools, adds read-ahead and memory governance, and shows some large performance wins: TPC-H Q6 on 22 GB Parquet fell from 8.23s to 2.84s, while a large CSV scan time improved by nearly 20x.