Daily Edition · Page 4

THE DAILY INDEX

Technology, AI, software, security, crypto and business

ARTICLES & TUTORIALS

TLDR Web Dev

THE ARCHITECTURE OF GRAB'S DATA LAKE

Grab manages its data using a data lake, using different storage formats for high and low throughput data. For high-throughput data, which is frequently updated, it uses Apache Avro with a Merge on Read (MOR) strategy, appending new data to log files for efficient writes and periodically compacting them for manageable reads. For low-throughput data with infrequent updates, it uses Parquet with Copy on Write (CoW), creating new file versions for each write.

PARSING JSON IS A MINEFIELD

Parsing JSON, despite its seemingly simple format, is actually quite challenging because there are so many specifications that can be interpreted in different ways. This has led to inconsistencies in how different JSON parsers handle edge cases, extreme values, and maliciously crafted payloads, resulting in potential bugs, crashes, and denial-of-service vulnerabilities. This article goes into detail about this issue with examples of different types of objects (arrays, objects, numbers, and strings) combined with different types of parsers (C Parsers, Regex, and more).

TLDR IndexPage 4