Parse notes
Sponsored
POST/CON 24 [5] is the place to be if you’re looking to hear from industry leaders, gain new skills, or engage with the global API community. And there’s a 50% discount if you sign up early [5]!
This post walks through how to build a search engine from scratch using Python. The project was motivated by the desire to address the "Small Website Discoverability Crisis" and make small websites more visible. The post walks through the key components of the search engine, including the crawler, inverted index, ranker, and interface, providing both theoretical descriptions and practical implementation details. The project is helpful for deepening one’s understanding of search engine mechanisms.
pg_analytics is a Postgres extension that significantly improves the analytical performance of Postgres databases by up to 94 times compared to traditional methods. It addresses the challenges faced by developers dealing with slow query times and poor data compression, offering a drop-in solution within Postgres itself. This post explains the technical workings of pg_analytics, explaining the introduction of deltalake tables and integration with Apache Arrow and Apache DataFusion.
We traditionally use a "public" folder to serve files in Node.js applications. However, this only works for static assets and doesn’t work for files like profile pictures, private files, or generated files - these files are served from an API and loaded into memory. This isn’t scalable when there are a lot of users. Instead, it’s useful to use a stream to return large files in chunks. This article looks at a couple of ways to achieve this using both Next.js API routes and also the more generic Route handlers.
Many developers have experienced “flow state”: your ideas are flowing, time and space fade away, and the path ahead of you is clear. Achieving flow state is not only great for productivity, but also enhances dopamine and endorphins, resulting in an increased feeling of motivation, thinking ability, and general well-being. To better reach flow state, it’s helpful to optimize your environment, map out your work ahead of time, and find joy in the work that you’re doing.
Once your app gets into an undesirable state, the best thing you can do is reset it and start from a fresh, well-known, and correct state. Erlang’s "let it crash" philosophy suggests resetting parts of an application upon failure. While effective, it requires developers to manually track and restore application states. Another approach, "durable execution," allows computation to run uninterrupted by storing minimal data for state reconstruction, making local variables resilient even through restarts.
The phrase "wherever you find podcasts” highlights a stark contrast between podcasting and closed media platforms like YouTube or TikTok. Podcasting's open format empowers creators to control their content and audience relationships, enabling portability and resisting surveillance-based advertising. There are challenges in podcast economics, but podcasting is a beacon for the open web's renaissance and a model for reclaiming control in online spaces. Podcasting is similar to RSS feeds for blogs and newsletters.
SQLite Web is a web-based SQLite database browser written in Python. It lets developers add, drop, or edit tables, columns, rows, and indexes. SQLite Web works with both existing SQLite databases or can be used to create new ones. It allows for the import and export of both JSON and CSV files.
Concrete.css is a simple, classless CSS framework. It offers styling for typography, sections, buttons, images, lists, forms, tables, blockquotes, and progress bars.
Meow is a CLI app helper. It can parse arguments, convert flags to camel case, negate flags with “--no-” prefixes, and output helpful descriptions and help text. Meow has no dependencies.
While working at startups and moving fast, it’s important to use existing tools before adding new ones to avoid complexity. Software often changes fast at startups, so optimize software for change and focus on iteration, testing, and implementing a modular monolith approach. Store data in databases for easy querying, utilize decision logs, and employ continuous deployment practices for efficient software delivery.
The ideal size for small pull requests (PRs) is around 50 lines long. Smaller PRs are reviewed and merged faster, less likely to be reverted, and attract more review comments per line changed. Data from various repositories suggests that PRs between 25-100 lines offer optimal benefits across speed, review quality, and total code output. Extremely small PRs (under 25 lines) tend to have a higher revert rate, while very large PRs (over 10,000 lines) may receive less engagement and feedback.
Google Cloud’s localllm enables developers to utilize LLMs on local CPUs without GPUs. Developers can use quantized models and Cloud Workstations for cheaper LLM usage and better privacy controls compared to online models like GPT-4.
Git has an auto-correct flag that replaces typos with a suggested corrected command.
React components for resizable panels, supporting horizontal, vertical, and persistent layouts.
jQuery 4.0 beta drops support for IE10 and older, removes deprecated APIs, and is now importable with ES modules.
The latest ChatGPT-4 system prompt can be used to help you understand and prompt the LLM better.