The Daily Index — Monday, September 14, 2026

GENERAL

TLDR Data

branching

Summary unavailable

databricks.com

TLDR Data

Run analytics against the freshest transactional data

Summary unavailable

youtube.com

TLDR Data

EVOLVING PINTEREST'S EMBEDDING RETRIEVAL PLATFORM

Pinterest's Manas retrieval platform serves billions of embeddings across Home Feed, Search, Related Pins, Ads, and Notifications. Quantization is cutting serving cost by 20–30%, SSD-backed ANN experiments reduce memory and CPU needs, and multi-embedding retrieval moves beyond single-vector two-tower matching toward richer candidate scoring.

links.tldrnewsletter.com · 8 min

TLDR Data

DISKLESS KAFKA: WHAT HAPPENS WHEN BROKERS STOP OWNING THE DATA?

Diskless Kafka keeps the Kafka API while moving durable payload storage from broker-owned logs to object storage and strongly consistent metadata. The tradeoff is not just cheaper retention: batching, offset assignment, recovery, compaction, latency, and feature support all change, making diskless topics a per-workload storage choice rather than a universal Kafka replacement.

softwaremill.com · 16 min

TLDR Data

THE UNBEARABLE LIGHTNESS OF ONE MORE INDEX

Coding agents now produce competent PostgreSQL schemas, but they tend to stack indexes onto hot operational tables. In a measured support-ticket workload, extra indexes nearly doubled update time and WAL while increasing VACUUM work. Index review needs write-path evidence, HOT-update awareness, and usage counters, not just fast read plans.

boringsql.com · 11 min

TLDR Data

IT PASSED CI. IT PASSED YOUR EVALS. THE CUSTOMER STILL GOT THE WRONG ANSWER

AI agents can return successful, well-grounded responses that are still wrong because they retrieved invalid context or followed a faulty execution path. Debugging requires end-to-end traces of model calls, tool calls, retrieval inputs, and outputs, plus separate tests for deterministic system behavior and answer quality.

thenewstack.io · 11 min

TLDR Data

context. See exactly how it works.

Summary unavailable

chalk.ai

TLDR Data

118 MILLION QUERIES PER SECOND ON NEKI

PlanetScale set out to run 1 million queries per second against Neki, its new sharded Postgres, hit that on 5 shards almost immediately, then kept going until 512 shards were serving 118 million queries per second across 1.22 PiB of data. The workload is deliberately narrow: a single-shard point select fetching one row by primary key with no writes, joins, or cross-shard queries, which is what makes the scaling curve clean.

planetscale.com · 9 min

TLDR Data

NOW EVERYONE CAN PUT DATA TO WORK

ChatGPT Work's Data agent connects to approved company data and business definitions, letting employees investigate questions and build shareable dashboards using plain language. It respects existing permissions, works with major data and BI platforms, and can recommend or carry out approved follow-up actions.

links.tldrnewsletter.com · 3 min

TLDR Data

WHY DUCKDB 2.0 IS FASTER

DuckDB 2.0 speeds up S3 queries by 2-3x through asynchronous I/O, dramatically improves deep recursive queries, and makes semi-structured VARIANT data smaller and faster than JSON. It also adds practical SQL features including triggers, nested schemas, and data-modifying CTEs.

motherduck.com · 15 min

TLDR Data

YOUR AI ADOPTION LIFT IS A SELECTION EFFECT

Comparing AI adopters with non-adopters usually exaggerates impact because more engaged customers are also more likely to adopt. Instead, estimate causality using variation that customers did not choose, such as an eligibility threshold, while clearly limiting conclusions to the population and decision it measures.

towardsdatascience.com · 17 min

TLDR Data

POSTGRESQL CDC BACKFILLS AT SCALE: RUNNING A MULTI-DAY BACKFILL WHILE PRODUCTION KEEPS WRITING

Large PostgreSQL CDC backfills fail less from historical reads than from WAL retention, delayed slot acknowledgments, long-running transactions, and stale snapshot rows racing live changes. Safer designs keep consuming WAL during chunked reads, checkpoint progress, reconcile overlaps with watermarks and fences, and size chunking around real write rates.

estuary.dev · 15 min

TLDR Data

INVESTIGATING SPARK WASTE ACROSS THE STACK

Four production Spark cases cut runtime, compute, S3 operations, and costs by tracing hidden waste beyond top-level CPU and memory metrics.

blog.dataengineerthings.org · 10 min

TLDR Data

THE HALF-LIFE OF A QUERY

Tools like dbt turn fragile, undocumented analysis into reproducible systems, while AI often misses the constraints of real workplaces.

covertaltruism.substack.com · 7 min

TLDR Data

JETPACK: CONSENSUS MADE GENERALLY FAST (OSDI '26)

Jetpack adds a portable 1-RTT fast-commit path to existing consensus systems, cutting latency for write-heavy workloads by up to 60% while adding redundant work, delaying execution, and requiring careful recovery to remain safe during leader changes.

muratbuffalo.blogspot.com · 4 min