FROM S3 TO GPU IN ONE COPY: RETHINKING DATA LOADING FOR ML TRAINING (50 MINUTE VIDEO)

Summary from the TLDR Data newsletter

Vortex is an open-source columnar format designed to stream training data from S3 to GPUs with minimal copying. Cascading lightweight encodings, layout-aware pruning, and zero-copy memory paths target CPU and NVMe bottlenecks, giving ML platforms a way to keep expensive accelerators fed without converting everything upfront.