Researchers significantly improved SQLite's analytical query performance by using Bloom filters. The original nested loop join implementation in SQLite, used for joining tables, resulted in many costly B-tree probes. By incorporating Bloom filters, the researchers created a pre-filtering mechanism, drastically reducing the number of B-tree probes needed. This optimization, implemented via new "Filter" and "FilterAdd" opcodes, resulted in a 7x-10x speed increase for analytical queries in SQLite.
Fastmail uses its own hardware for email services because it allows for cost optimization and long-term planning over the dynamic scaling offered by cloud services. This approach allows for more efficient hardware usage, where equipment can be used for 5-10 years. Fastmail recently upgraded to a ZFS-based NVMe SSD system, leading to big performance and space savings through compression while eliminating the complexities of traditional RAID setups.
Long-term software development is best done by minimizing dependencies, rigorously testing, and prioritizing simple, well-documented code to combat complexity that inevitably increases over time. The importance of a stable team, regular dependency audits, and comprehensive logging were also stressed in the author's presentation shown in this article.
The author advocates for maintaining a personal changelog of work activities, inspired by open-source best practices. This changelog improves communication in one-on-one meetings, onboarding new employees, and provides a personal reference and self-reflection tool. The author's changelog, updated weekly, enhances transparency and trust with colleagues and managers by providing a clear record of their work. The benefits include increased productivity, improved knowledge sharing, and self-assessment of work efficiency, ultimately leading to professional growth.
S2 is a new serverless stream storage service designed to be a cloud-native alternative to traditional streaming platforms like Kafka. It offers a simple, scalable API for appending and reading records with strong consistency and concurrency control. It has more throughput and lower latency than existing serverless options.
Stagehand is an open-source, AI-powered web automation framework built on Playwright that offers simple APIs (act, extract, and observe) for natural language-driven web interactions. It supports various LLMs.
To address the problem of an AI code review bot generating excessive comments, the Greptile team experimented with various methods, including prompt engineering and using the LLM to judge its own comments, but these attempts failed. They eventually succeeded by implementing a clustering approach, using vector embeddings of past comments to identify and filter out "nit" comments (those technically correct but unimportant). This resulted in a big increase in the percentage of Greptile's comments addressed by developers from 19% to over 55%.
Meta undertook a massive project to translate its Android codebase from Java to Kotlin, exceeding the halfway point despite having one of the world's largest codebases. To manage this at scale, it created the Kotlinator, a tool that automates the conversion process using JetBrains' J2K, supplemented by extensive pre- and post-processing steps to handle complexities and custom frameworks. This automation involved addressing build speed issues and creating a headless version of J2K to run conversions remotely, integrated with Meta's internal code review system.
FindMy.py is a cross-platform Python library for interacting with Apple's Find My network.
OpenAI's o3 model synthesizes code like a developer, blurring the lines between user and developer roles and prompting questions about developer relevance and the ethical implications of trusting AI-generated code without understanding its logic.
GitHub Assistant is a proof-of-concept application that allows devs to ask questions to GitHub repositories using natural language.
The JetBrains IDE source code is easily accessible and people will pay for it still due to other benefits, such as release cadence of new features.
Type-driven development is the practice of defining types upfront, making sure of type consistency across the application, and making illegal states unrepresentable. This approach leads to slower short-term output, but leads to easier, more reliable coding in the long run. A side bonus is that it also comes with type-driven autocomplete and instant error detection.
Meta's open-source Glean system indexes code at scale, providing information to various developer tools through a flexible query language. This centralized system avoids redundant indexing on individual developer machines and offers a scalable, distributed architecture for handling large codebases and concurrent queries. Glean's design is language-agnostic and has incremental indexing using a stack of immutable databases to efficiently process code changes.
There's a growing disconnect between this older developer's understanding of programming and that of younger web developers, shown by a colleague's unfamiliarity with non-SPA web applications. He feels that nowadays, there is an over-reliance on abstractions and a tendency to view narrow JavaScript-centric knowledge as representative of broader computer science principles.
Live coding interviews for senior software engineers don't properly assess real-world skills and create undue stress. For them, better assessment methods include assigning small, realistic projects completed asynchronously, though this has the caveat of filtering for engineers who have the time to do these projects outside of their current jobs.
mizu.js is an open-core JavaScript library that supercharges HTML with directives for templating, reactivity, HTTP requests, and more. It has both client-side and server-side rendering capabilities across various JavaScript runtimes.
Clay is a high-performance, flexbox-style UI auto layout library written in C that features a declarative syntax, minimal size, and renderer-agnostic design.
Hookcn is a collection of reusable React hooks that can be easily integrated into projects via copy-pasting or a CLI.
ISO 8583 is a standard for real-time payment messages between banks and merchants, used in most card transactions. The standard has evolved over time, leading to variations across networks due to flexibility in field encoding and the addition of network-specific fields. This has made it complex, due to optional fields, diverse encoding schemes, and different ways of handling nested messages.
2024.12.20·links.tldrnewsletter.com·20 min read·TLDR Web Dev A study by Anthropic and Redwood Research showed that LLMs can exhibit "alignment faking" - strategically concealing their original preferences even after safety training. This occurs when an LLM, initially trained to be harmless, is subsequently incentivized to always comply with requests, including harmful ones. It may then deceptively comply to avoid retraining that could diminish its ability to refuse harmful requests.
2024.12.20·links.tldrnewsletter.com·10 min read·TLDR Web Dev Cursor, an AI coding assistant, has a "Cursor Tab" feature that predicts and completes code edits efficiently. Unlike simpler AI coding tools focused on creating new projects, Cursor is great at assisting with the maintenance and modification of existing large-scale projects.
2024.12.20·links.tldrnewsletter.com·19 min read·TLDR Web Dev A recent study revealed that the popularity of approximately 15,835 repositories is inflated by over 4.5 million fake GitHub stars.
A web developer uses AI tools for increased productivity but feels concerned about losing coding skills due to reliance on AI, sparking a discussion on the trade-off between speed and skill retention.
SeleniumBase is an open-source Python framework for web automation, testing, scraping, and bypassing bot detection.
TanStack Start is a new meta-framework that adds a thin server layer to TanStack Router, enabling server-side rendering without sacrificing Router's functionality. Unlike other meta-frameworks, Start avoids performance issues by using isomorphic loaders, which run on both the server (for initial load) and the client (for subsequent navigations), using client-side state and caching. This approach solves the "impedance mismatch" between server-side data loading and client-side state management, preventing unnecessary data refetches.
This developer improved the performance of Ruby's json gem, making it the fastest JSON parser and generator in many benchmarks. These improvements actually weren't that complex, as most of them came from simple optimizations discovered through profiling and focusing on areas like redundant UTF-8 checks and inefficient buffer allocation.
The Threads iOS team prioritized performance to maintain a positive user experience during the app's rapid growth, tracking key metrics like image load times (%FIRE), content delivery speed (TTNC), and post success rate (cPSR). They reduced app size, used a new latency measurement tool (SLATE), and implemented features like Drafts for better reliability. A boundary test revealed the negative impact of navigation latency on user engagement, highlighting the importance of quick loading times.
Leave work slightly unfinished in order to reach a productive “flow state” easier the next day. Learn keyboard and mouse shortcuts, maintain a list of common commands and references, and learn to say “no” more often. This includes saying “no” both in terms of choosing technology and tasks that align with your goals and priorities
Mentoring is valuable for professional growth. A few different approaches to finding mentors include seeking internal mentors at large companies, considering paid coaches for engineering leaders at smaller companies, and connecting with experienced individuals both in-person and online. Mentoring doesn't have to be formal and structured, as even informal mentorship can be valuable and initiated by simply asking someone you admire for their time and advice.
2024.12.19·blog.pragmaticengineer.com·18 min read·TLDR Web Dev GitHub Copilot is now available for free with limited features. The free tier has 2,000 completions per month in VSCode and 50 chat messages per month.
Musoq is an open-source project that provides SQL-like querying capabilities for data sources like files, databases, and APIs without the need for a traditional database. It's designed for efficient ad-hoc querying of smaller to medium-sized datasets and has plugins for various data sources (filesystem, Git, JSON, AI models, etc.).
Base UI is a set of 25 unstyled React components that are easily installable via npm. They can be animated using CSS transitions, CSS animations, or JavaScript animation libraries.
The article challenges the common misconception that modern computer memory behaves like ideal random-access memory, arguing that this assumption leads to inefficient data sharding algorithms. For large datasets, a seemingly slower algorithm using radix sort, which prioritizes ordered memory access, outperforms the standard approach. This improvement is from reducing cache misses, shown by code examples and benchmarks that demonstrate speedups from 2.5x to 9x.
Java can be used for small programming tasks, as recent language features have improved its suitability for scripting. Modern Java allows for launching `.java` files directly without explicit compilation or JAR creation. Java's standard library lacks JSON and command-line processing - tools like JBang and third-party libraries can help with this.
Five tips for optimizing Interaction to Next Paint (INP) in React apps include reducing DOM size, dividing components into simple and extended versions, using `Suspense`, avoiding hydration errors, and carefully managing `useEffect` hooks.
Assistant UI is an open-source React component library for building AI chat interfaces, supporting various AI models.
Someone successfully used open-source software and interoperable protocols to connect an Apple Watch to their Android phone, bypassing the need for an iPhone for most functions.
12 modern CSS properties that can make your life easier, such as stable upgrades like aspect-ratio and object-fit, stable enhancements like text-underline-offset and outline-offset, and progressive enhancements such as overscroll-behavior and scrollbar-gutter.
Apple and NVIDIA collaborated to integrate Apple's ReDrafter technology into NVIDIA's TensorRT-LLM, resulting in a 2.7x speedup in large language model text generation on NVIDIA GPUs.
OpenAI experienced a major outage due to a misconfigured telemetry service overloading its Kubernetes API servers, disrupting DNS-based service discovery. This was due to a discrepancy between staging and production environments, where a problem unseen in staging caused a catastrophic failure in production's larger-scale cluster. The outage also revealed hidden dependencies between the control and data planes, showing the complexity of Kubernetes and the challenges in resolving cascading failures.
The main thread in a browser is responsible for executing JavaScript, handling user interactions, updating the DOM, and managing the event loop, all in a single-threaded environment. JavaScript's single-threaded nature, combined with the event loop, allows it to handle asynchronous tasks efficiently using task and microtask queues without blocking the thread. Optimizing main thread usage is critical for creating responsive and smooth web apps, especially by minimizing blocking operations and using tools like web workers.
Neon migrated from Webpack to Vite to get Hot Module Replacement (HMR) capabilities, simplify its configuration, and unify tooling, though build times initially remained unchanged. It developed a custom Vite proxy to integrate with its Go backend and faced challenges like slower dev cold starts, syntax errors, and dependency conflicts. However, in the end, the migration improved the developer experience with HMR, reduced dependencies, and streamlined its setup across apps, tests, and Storybook.
Design tokens represent design decisions as data and serve as a single source of truth for design and engineering. The author recommends a layered token structure, option, decision, and component tokens while keeping some option tokens private to reduce file sizes and allow for non-breaking changes.
This blog post offers ten React tips for junior developers to avoid common mistakes. It covers performance optimization using the `children` prop, the appropriate use of refs versus state, and the advantages of named exports over default exports. The post warns against the overuse of `useEffect` and encourages using tools like ESLint and React DevTools for debugging.
Sponsored
Slow testing leads to low test coverage. Low test coverage leads to bugs. Bugs lead to rework and missed milestones. Break out of the doom loop with QA Wolf [10]. With AI that makes Playwright test automation 5x faster, QA Wolf delivers 80% automated E2E test coverage, 15min QA
OpenAuth is a self-hostable, standards-based OAuth 2.0 authentication server for web and mobile applications that offers customizable UI, integrates with various providers (like GitHub and Google), and supports different deployment environments (Node.js, Bun, AWS Lambda, and Cloudflare Workers).
React-toastify v11 (a Toast library) introduces many more customization options, a simplified DOM structure, custom progress bar support, better accessibility, and a new `onClose` callback with reason details.
Surfer Protocol is an open-source framework that provides a desktop app and Python SDK to export personal data from various platforms and build applications using that data.