A FILE FORMAT CLAIMS TO BE 100X FASTER THAN PARQUET

Summary from the TLDR Data newsletter

Parquet's scan-oriented design is showing strain under AI workloads that need fast random access to embeddings and multimodal data. Lance tackles that mismatch with a layout built for direct page access and finer-grained reads, reducing metadata overhead and memory pressure. The broader takeaway is that AI-era storage may need to optimize for point retrieval, not just analytical scans.