TLDR WEB DEV 2024-10-01

HOW WE BUILT NGROK'S DATA PLATFORM

Ngrok built its data platform with an engineering team of just one person, using open-source tools and self-hosting on Kubernetes. They had challenges like complex schemas and large data volumes, which were addressed through code generation, schema normalization, and scaling Apache Flink with Protobuf messages.

DATA SYNCHRONIZATION

React Query tackles the challenge of data synchronization by keeping the data displayed on the screen up-to-date with the server state. It uses a "Stale While Revalidate" strategy, where data is considered stale by default and refetched in the background when triggers like changes in query keys, new component mounts, window focus, or device going online occur. This allows for a smooth user experience with instant data delivery while still making sure the cache stays synchronized with the server.

PHRASE MATCHING IN MARGINALIA SEARCH

A search engine added phrase matching by transitioning from approximate word position storage to storing precise positions using compressed lists. This change required implementing a new way to store information about word occurrences within different document fields. The ranking algorithm was upgraded to take advantage of the new phrase matching capabilities, incorporating factors like phrase presence, keyword distance, and early occurrence.

WHEN TO DO WHAT YOU LOVE

If your main goal is to make money, you may have to compromise on your passion, as most people pay you for what they want, not what you want. However, if you are passionate about something that is also in high demand, like a niche market, you can potentially earn a living from it. Building a startup can be a viable option here, but mostly if it arises from your own passions and interests.

THE GRUG BRAINED DEVELOPER

This is a list of various pieces of advice on writing good software. For example, good devs fight against complexity, say "no" to unnecessary features, and embrace simple solutions. They are also patient and refactor thoughtfully.

DON'T BE AFRAID TO RE-INVENT THE WHEEL

Relying heavily on external codebases can create risks like security vulnerabilities, dependence on third-party maintenance, and ongoing integration work. It's fine to sometimes replicate external code from packages in internal codebases because this gives developers better control and security over that code.

all resources here

HUMANIFYJS

HumanifyJS is a tool that uses LLMs and Babel to deobfuscate, unminify, transpile, decompile, and unpack JavaScript code, making it more readable for humans.

FACAD

Facad is a colorful directory listing tool for the command line that features emoji-based file and directory icons, sorted output, support for symlinks and executables, and customizable column width.

EVOLUTION API

EvolutionAPI is an open-source WhatsApp integration API that provides a RESTful API service for controlling WhatsApp functions, allowing users to create multiservice chats, service bots, and other systems utilizing WhatsApp.

DO AI COMPANIES WORK?

AI companies face economic challenges due to the high costs of building and maintaining models. As newer and more advanced LLMs emerge, older ones quickly become obsolete, making it easy for a company to be left behind. Now, companies must either spend enormous amounts of money to stay ahead or risk falling behind and losing market share.

EPIC IS SUING GOOGLE — AGAIN — AND NOW SAMSUNG, TOO

Epic Games is suing Google and Samsung. The lawsuit focuses on Samsung's "Auto Blocker" feature, which makes it difficult for users to install apps from third-party stores. Epic argues that Auto Blocker is designed to protect Google and Samsung's app stores, not users, and that it is blocking the growth of the Epic Games Store.

NO SUCH THING AS EXACTLY-ONCE DELIVERY

The difference between delivery and processing in messaging systems is that delivery refers to the act of successfully transferring a message to a receiver, while processing covers the entire message lifecycle, including delivery, processing by the receiver, and acknowledgement. Sequin is a Postgres streaming library that prioritizes exactly-once processing by making sure that every message is delivered, processed, and acknowledged without duplication. Achieving exactly-once delivery is practically impossible due to the two-phase commit problem - confirming the simultaneous confirmation of message delivery across different locations.

BOP SPOTTER

Bop Spotter is an experimental project that is using a modified Shazam app attached to a pole to monitor music trends in San Francisco's Mission district.

ARE YOU SAYING THAT MY CODE IS BORING? THANK YOU!

“Boring” code is the best kind of code because it allows teams to focus on solving business problems without the distraction of technical complexities.

DON'T BELIEVE THE HYPE: AGI IS FAR FROM INEVITABLE

Despite claims from large tech companies, achieving artificial general intelligence (AGI) with human-level cognition is still far away due to the need of both further improvements in AI along with more compute power.

Y COMBINATOR TRADED PRESTIGE FOR GROWTH

Y Combinator is experiencing a decline due to its overemphasis on growth over prestige.