TLDR Index
TLDR Data

TLDR DATA 2026-08-06

5 sections14 entrieslinks blockcomplete parse

Parse notes

  • Subscription/account footer truncated before parsefooter_truncated

01DEEP DIVES

TIMESERIES TIERED STORAGE JOURNEY: KAFKA/FLINK STREAMS TO NATIVE CASSANDRA COLD READS

Netflix's TimeSeries Abstraction stores and queries multi-petabyte temporal datasets, tiering older, immutable slices from hot Cassandra into cheaper S3 storage. The earlier design used Kafka and Flink to stream Parquet to S3 and daily compaction, serving cold reads with roughly 500 ms p99 latency, but added operational cost. The later Cassandra-native cold tier reads directly from S3 backups, serves 15+ PB of compressed cold data, and improves p90 latency by about 30% while preserving the same query API.

netflixtechblog.medium.com6 min read

GEM TRAINING: HOW META DOUBLED THE EFFICIENCY OF ITS LLM-SCALE ADS FOUNDATION MODEL

Meta scaled its ads recommendation foundation model 4x in 12 months and doubled training efficiency to 20–25% MFU through five techniques: jagged flash attention to eliminate padding waste, a unified attention kernel with 2–3.5x speedup, MXFP8 precision training, SM-free collective communication that reclaims ~23 SMs, and a load-balancing shuffle that adds 4% efficiency at zero communication cost.

engineering.fb.com12 min read

LLMS FOR RELEVANCE: AUTOMATING HIGH-QUALITY PRODUCT RELEVANCE LABELING IN FLIPKART SEARCH

Flipkart's Product Analyser trains a relevance labeling model in two stages: SFT on millions of balanced query-product examples with generated reasoning traces, then GRPO alignment using an 8B reward model that scores explanation quality alongside label correctness. The automated NDCG report it produces differs from the manual one by under 1%, at 30% lower cost.

blog.flipkart.tech10 min read

02OPINIONS & ADVICE

PROTOTYPE ON A LAPTOP, SCALE TO 16 BILLION ROWS: ONE POLARS QUERY

Polars lets teams develop ETL pipelines locally on a representative data sample, then run the exact same LazyFrame queries across 16 billion rows in the cloud without rewriting the logic. The pipeline pre-aggregates raw Polymarket data into small Parquet artifacts in S3, allowing a Plotly Dash dashboard to remain fast without scanning the full dataset on each request.

pola.rs14 min read

THE MEAN MEANS NOTHING

A web service latency rollout shows the mean rising 9%, the median falling 46%, and p99 rising 119% within the same dataset, demonstrating how single-metric summaries can hide bimodal distributions. Further analysis like density plots, CDFs, shift functions, and ridgelines provide complementary views that together identify response size as the root cause.

fzakaria.com9 min read

03LAUNCHES & TOOLS

PYHCTSA: PYTHON TOOLKIT FOR HIGHLY COMPARATIVE TIME-SERIES ANALYSIS

pyhctsa is a Python toolkit for extracting hundreds of statistical and structural features from single or multiple time series, with support for custom feature sets and variable-length inputs. It also supports direct operation calls and parallel processing across local CPU cores for faster analysis.

github.comGitHub repository

PGGRAPH

pgGraph is a PostgreSQL extension that adds fast graph search, traversal, and shortest-path queries to existing relational tables without moving data into a separate graph database or learning a new query language. It builds a rebuildable, memory-efficient graph index while PostgreSQL remains the authoritative system of record.

github.comGitHub repository

SMEVALS - A SMALL EVAL SUITE FOR EVALUATING MODELS, PROMPTS, AND HARNESSES

Smevals is a lightweight Python CLI for running task-specific evals across multiple models to find the cheapest one that meets a quality bar. You define tasks, write YAML graders with custom checker scripts, run them against a set of model configs, and view scored results in a local dashboard or static site.

primeradiant.com9 min read

BRINGING DUCKLAKE TO DATAFUSION

DuckLake is now implemented for Apache DataFusion as a production-ready, open-source catalog backend, giving Hotdata transactional metadata, snapshots, and schema management for Parquet tables in object storage. The design supports multiple relational catalog backends and logical catalogs, enabling millions of isolated, ephemeral databases for AI agents without duplicating infrastructure. TPC-H benchmarks across 22 queries and three dataset sizes showed no measurable overhead versus direct Parquet access, with slightly better performance due to less metadata discovery during query planning.

ducklake.select4 min read

04MISCELLANEOUS

EVERYONE RECORDS THE MYSQL AUDIT LOG. NOBODY READS IT

DB Trail makes MySQL audit logs queryable by linking row changes to session identity, SQL statements, and before-images. It provides attributed, row-level answers for destructive actions and generates transaction-scoped reversal SQL to restore affected data.

blog.dbtrail.com4 min read

ENCODING OR COMPRESSION: WHY NOT BOTH?

Encoding improves analytical query execution, while compression mainly reduces storage. Dictionary encoding and frame-of-reference enable fast comparisons, pruning, and SIMD operations. CedarDB's results suggest always encoding, then adding zstd only when the disk savings justify it.

cedardb.com9 min read