TLDR · PROGRAMMING, DESIGN & DATA SCIENCE

HOW AWS S3 SERVES 1 PETABYTE PER SECOND ON TOP OF SLOW HDDS

bigdata.2minutestreaming.com14 min read

Summary from the TLDR newsletter

AWS S3 is a scalable multi-tenant storage service with APIs to store and retrieve objects. It offers extremely high availability and durability at a relatively low cost. What started as a service optimized for backups and media storage for e-commerce websites has grown into the main storage system used for analytics and machine learning on massive data lakes. The growing trend now for entire data infrastructure projects to be based on S3, which gives them the benefit of stateless nodes while outsourcing difficult durability, replication, and load-balancing problems to S3 - this reduces cloud costs.