TLDR DATA 2026-11-02

Register free [4] · See the agenda

DO YOU EVEN BENCHMARK? HOW WE EVALUATE AGENTS AT SCALE

PostHog tests AI agents on real tasks, such as finding the bugs behind 33 past incidents, where its review agent caught 62%, or checking whether playback failures affected real users. It isolates each test so agents cannot copy earlier answers and is building resettable environments with fixed data to make comparisons fair.

I TESTED POLARS 2.1 AGAINST DUCKDB 2.0 (10 MINUTE VIDEO)

DuckDB 2.0 adds client-server and AI features, while Polars improves streaming and processing of larger-than-memory datasets. On a 23 GB dataset with 2 GB RAM, Polars 2.0 ran out of memory, but 2.1 finished in 438 seconds, nearly twice as fast as DuckDB's 837 seconds.

MOVE THE ELEPHANT, KEEP THE LIGHTS ON

Logical replication lets you upgrade or migrate PostgreSQL with only seconds of downtime: run the target cluster in parallel, stream changes, then cut over when ready. Its real advantage over in-place upgrades is rollback safety and cross-environment portability. Plan around what it does not move automatically, especially schema changes and sequences.

VECTOR SEARCH ON APACHE ICEBERG WITHOUT MOVING SOURCE DATA

StarTree adapts Pinot's HNSW vector index for Iceberg tables, keeping Parquet source records in object storage while storing and serving a derived index alongside them. This enables hybrid similarity and SQL-filtered queries without a second copy of the records. The trade-off: indexes are roughly as large as the embeddings, and freshness follows Iceberg snapshot synchronization rather than real-time ingestion.

PARTICLEDB 1.1.4

Jepsen found 96 issues in ParticleDB, including data loss, crashes, and incorrect SQL results, mostly without any server failures. Most were fixed during testing, but serious transaction bugs remained. Orkes says a newer release fixes them, though Jepsen has not verified this.

WHAT HAPPENED TO DATA PRODUCTS?

Data products outlasted data-mesh hype by becoming a delivery discipline rather than an enterprise reorganisation: a governed output, accountable owner, defined consumer, and evidence of value. AI agents raise the stakes because they cannot recover missing context through informal knowledge. Start with a business-critical workflow, then make meaning, quality, access, and permitted use machine-readable.

SEE IT →

NODAL ANALYTICS

Nodal is an open-source toolkit that turns analysts' business definitions into reusable context for AI analytics agents. It plans queries and checks results against approved metrics, filters, and joins, flagging uncertainty for expert review.

PRIMARY KEYS COME TO APACHE ICEBERG V4, AND MOST OF THEM START OUT UNENFORCED

Apache Iceberg V4 adds primary keys, unique rules, and checks, such as requiring positive order amounts, though most will initially be unenforced. Each snapshot records whether the data meets those rules, separately from whether writers must enforce them on new writes.

KAFGRES 0.2.0: 700 MB/S KAFKA-COMPATIBLE STREAMING INSIDE POSTGRES

Kafgres, a Kafka-compatible Postgres extension, reached about 700 MB/s after caching query plans, relaxing commit durability, and replacing timer-based polling with socket readiness checks. It could suit many workloads, but the single-broker setup lacks the resilience of a Kafka cluster spread across availability zones.

I TOLD AN AI AGENT THAT ITS WRONG ANSWERS WERE RIGHT. ONE MODEL BELIEVED ME

In most cases, flipping the reward labels in an agent's memory barely changes how it learns. This test hid a rule only the labels could reveal. With true labels, GPT-6 Sol scored 94.4 out of 100 on its last six trials, and with flipped labels 7.8. Claude Haiku 5.5 improved less, and Qwen3 4B barely moved.

ACTIVE-ACTIVE ORACLE GOLDENGATE ACROSS REGIONS: DECIDE WHO OWNS THE DATA BEFORE YOU RESOLVE CONFLICT

Active-active GoldenGate only works when data ownership and conflict policy are designed table by table before replication starts. Simulations showed that even convergent strategies like last-writer-wins can silently lose updates, while delta merging preserves each decrement but cannot prevent both regions from accepting a request against the same last unit before replication catches up. The lesson is blunt: if business rules cannot tolerate reconciliation, do not rely on multi-master replication to fix the problem later.

WHAT FIVE P99 CONF TALKS REVEAL ABOUT DATABASE TRADE-OFFS

A sponsored P99 CONF round-up exposes the costs behind several database designs: object storage trades low-cost durability for cold-read latency, read-optimized engines trade RAM and write amplification for tail latency, and vector search trades quantization and cache complexity for scale. The reported benchmarks are workload-specific, but the recurring lesson is: removing one bottleneck moves the constraint elsewhere.

UPGRADING MLFLOW PIPELINES ACROSS A PYTHON VERSION BOUNDARY

Moving Databricks from Python 3.10 to 3.12 broke Zalando's pickled MLflow models.

YOUR BACKUPS ARE GREEN. THAT DOES NOT MEAN YOU CAN RESTORE

Successful PostgreSQL backups can still fail to restore because of missing transaction logs, copied corruption, or lost encryption keys.