Parse notes
Sponsored
POST/CON 24 will be this year’s huge event to network and gain new skills! Connect with the global API community and tech leaders, all as you come together to: 1️⃣ Learn [5]: Get first-hand knowledge from Postman experts James Q Quick, Shweta, Palande, Kedasha Kerr, and other global tech leaders!
Elasticsearch was not scaling well for DoorDash’s search engine and lacked customization for the company’s needs in modeling complex data and search ranking, so DoorDash built its own search engine using Apache Lucene, a core search library, and designed its own indexer and searcher to meet its specific requirements. The indexer processes documents into easily searchable formats, while the searcher replicates index data and uses Lucene's search functionality to match queries and rank results.
When Google revealed its File System (GFS), it was a breakthrough in distributed storage. Designed for storing massive files across commodity hardware, GFS stored files in 64MB chunks replicated across machines for redundancy and performance. A central "master" managed metadata (file names, chunk locations, and versions) and coordinated reads/writes. To append data to files, the client would get write permission from the master, send data to multiple chunk servers, and have the designated "primary" server coordinate final writes.
Worker threads provide a way to create independent JavaScript execution threads that run in parallel. This post walks through an example of offloading CPU-intensive tasks to a worker thread in Node.js. Without offloading, tasks can block the event loop and prevent the server from processing other requests. The article shows an example of the server going from processing 7.3k requests to 0 tasks due to a CPU-intensive task.
Paul Graham, the founder of Y Combinator, gives advice to people who want to create the next Google. They should be makers and work on their own passion projects to learn fast. The best ideas come from projects you build for yourself and your friends to use. Something you truly need is a great sign. Great co-founders come from working together on things.
AI tools like Devin claim to replace software engineers. These startups face an uphill battle against established AI coding assistants like GitHub Copilot, so their claims need to be bold to get attention. For decades, there have been attempts to automate away the need for programmers, yet software engineers remain high in demand. The future likely lies in improved versions of existing AI coding assistants, not standalone "AI developers."
Meta’s build systems follow a two-step process. The first step generates instructions for what to build and the second step executes the build. This separation makes them inflexible, as you can't add new files without re-running the first step. "End-to-end" (E2E) systems combine both steps, allowing for dynamic target discovery. In these systems, cloud caching, early cutoff, suspending schedulers, and minimal builds are all features easier to implement correctly.
Sponsored
Web applications have critical vulnerabilities that only dynamic analysis can find. Rapidly find and fix runtime vulnerabilities with Veracode [18]: ✅ Scan hundreds of web apps and APIs simultaneously
Eta is a lightweight and fast embedded JavaScript templating engine that works inside Node, Deno, and the browser. It is written in TypeScript and emphasizes great performance, configurability, and small bundle size.
How you order elements in the `head` tag can have an effect on the perceived performance of the page. Capo.js helps identify which elements are out of order in your `head` tag on a web page.
Garnet is a remote cache store developed by Microsoft Research. It is compatible with standard Redis clients and features a wide range of APIs, custom server-side operations, a robust storage layer, security, and more.
This author built a Zig implementation of an Ethereum virtual machine with the aim of making it faster than the official Go version. A seemingly unrelated change to a benchmarking script made the Zig app slower. This was because commands in bash pipelines run in parallel, so a Zig app, when freshly compiled, has to wait for input from preceding commands. The author limited memory usage at compile time and used buffered reads to make his implementation faster than the official Go version.
An alleged AT&T data breach leaked data on 70 million customers, including sensitive information like SSNs. AT&T maintains the data did not originate from its systems, leaving the source of the breach unclear. The accuracy of the leaked data, which has now been dumped publicly, has been verified.
This author built a tech job board based on Hacker News threads using Node.js and the ChatGPT API. They found that ChatGPT was good at converting unstructured job post text into structured JSON data. The input size for the OpenAI API calls was optimized to stay within token limits and concurrent requests were made to speed up the entire process. They then used Node.js, TypeScript, and SQLite to track and store data. Most work in building AI apps is in building the infrastructure around it and handling error cases.
The median active developer merges 2 PRs per week. 48% of active developers are responsible for 80% of PRs merged since 2023.
jnv is designed for navigating JSON, offering an interactive JSON viewer and jq filter editor.
Bjarne Stroustrup, the creator of C++, disagrees with the White House's recommendation to avoid C++ due to memory safety concerns, as the language has improved its safety over time.
Java 22 has been officially released.