The Daily Index — Monday, September 28, 2026

GENERAL

TLDR Data

risk management, evaluation, and deployment. GET YOUR DIGITAL COPY

Summary unavailable

pages.awscloud.com

TLDR Data

ROBOTICS HARNESS OPTIMIZATION ON GRAPH-AS-POLICY

Robotics Harness Optimization used coding agents to evolve a Graph-as-Policy robot controller, boosting simulated throughput 5.27x and success from 67% to 96%. The strongest version only rewired existing skills, without editing them or using human demonstrations.

kael.tech.blog · 11 min

TLDR Data

PARTITION FINALIZATION IN PINTEREST'S NEXT-GENERATION DB INGESTION FRAMEWORK

Pinterest added a finalization layer to its Kafka, Flink, Spark, and Iceberg CDC pipeline so downstream jobs can tell when an hourly partition is safe to read. Flink checkpoints capture event-time statistics, which become Iceberg snapshot metadata and a non-regressing watermark. The reusable signal makes freshness-versus-completeness explicit without introducing another coordination service.

links.tldrnewsletter.com · 9 min

TLDR Data

THE GUEST JOURNEY, UPDATED IN REAL TIME: EXTENDING AIRBNB'S SEQUENCE RECOMMENDER WITH CHRONON

Airbnb extended Chronon to update guest-journey features from batch refreshes to event-driven processing, cutting feature staleness from roughly two days to under a minute. Push Mode and near-real-time model transforms feed embeddings into online ranking while preserving offline training consistency. The architecture illustrates how feature platforms can bridge streaming updates, experimentation, and low-latency serving.

airbnb.tech · 6 min

TLDR Data

SCALING AN ML INFERENCE PIPELINE FOR BATCH WORKLOADS

WHOOP cut a 15.8-million-task internal ML simulation from more than two months to six days by removing per-task HTTP calls, loading the model in workers, and tuning pod CPU. It also documents why Python fork deadlocked native libraries, how spawn fixed it, and how SQS chain state plus atomic S3 writes made retries idempotent.

engineering.whoop.com · 5 min

TLDR Data

WHAT REPACK (CONCURRENTLY) COSTS WHILE IT RUNS

PostgreSQL 19 adds built-in REPACK (CONCURRENTLY) for online table rewrites, avoiding extensions while generally running faster and generating less WAL than pg_repack or pg_squeeze. The tradeoffs are significant: it can delay VACUUM across the cluster, briefly block writes during the final swap, and currently has a ceiling of approximately 105 million updates/deletes per run.

boringsql.com · 15 min

TLDR Data

TEN YEARS OF POSTGRES LOGICAL REPLICATION

Postgres logical replication has steadily replaced custom data-movement plumbing with core SQL features: row filters, column lists, partitioned targets, streaming apply, parallel apply, and safer loop prevention. This retrospective connects those releases to practical migration and hub-and-worker scaling patterns. It is a useful checklist for teams revisiting replication slots, failover, write routing, and schema evolution.

tapoueh.org · 16 min

TLDR Data

BUILDING AN ULTRA-HIGH THROUGHPUT AI-SQL ENGINE

Quail is an open-source AI-SQL engine that jointly optimizes query planning and LLM inference, running 1.84x faster than tuned vLLM on average and up to 14x faster on some workloads. It reduces wasted KV-cache work and GPU scheduling overhead, making large-scale LLM filtering and joins much cheaper.

fsdatalab.github.io · 13 min

TLDR Data

APACHE ICEBERG VIEWS: PORTABLE VIEW METADATA ACROSS SQL ENGINES

Iceberg's View Spec makes metadata portable by versioning schemas, dependencies, properties, and dialect-tagged SQL text, but it does not standardize execution or authorization. Production portability still depends on REST-catalog support and compatible SQL dialects. Teams should test dependency integrity, concurrent replacement, and cross-engine result consistency before treating views as shared control-plane assets.

dremio.com · 16 min

TLDR Data

ALP: ADAPTIVE LOSSLESS FLOATING-POINT ENCODING IN APACHE PARQUET

Apache Parquet added ALP, a new lossless encoding for floating-point data that delivers compression similar to ZSTD while decoding around 10x faster. It also enables much faster random access and parallel decoding, making it especially useful for prices, coordinates, and scientific measurements.

parquet.apache.org · 7 min

TLDR Data

TRADING A CLOUD IDENTITY FOR YOUR OWN: WORKLOAD ATTESTATION ON MANAGED COMPUTE

Netflix describes how Spark jobs on managed compute exchange a cloud execution role for an internally trusted workload identity. A signed metadata payload and AWS identity proof are verified before short-lived certificates are issued for mutual TLS. The pattern separates cloud-provider authorization from application trust, giving the platform stronger workload identity and more auditable access boundaries.

netflixtechblog.com · 7 min

TLDR Data

THE CONTEXT GAP | HOW TO BUILD DATA ARCHITECTURE LIKE OPEN AI

OpenAI's AI data stack centralizes rich metadata to close the context gap, while a Rust rewrite reportedly cut Airflow scheduling latency 35x at 70,000+ concurrent tasks.

dataopsleadership.substack.com · 10 min