FROM DISK PAGES TO COLUMNAR MEMORY: WHY APACHE ARROW?

Summary from the TLDR Data newsletter

Apache Arrow addresses a different problem than Parquet: fast in-memory analytics rather than compact storage. By keeping data in contiguous typed buffers instead of heap objects, Arrow preserves column locality for cache-friendly scans and SIMD execution, and it also reduces repeated serialization between engines like Spark, pandas, and DuckDB.