Daily Edition · Page 3

THE DAILY INDEX

Technology, AI, software, security, crypto and business

HEADLINES & LAUNCHES

TLDR AI

THE TOP 100 [GEN AI] CONSUMER APPS

The Gen AI ecosystem is starting to stabilize. There were only 11 new names on Andreessen Horowitz's most recent Top 100 Gen AI Consumer Apps list, compared to 17 newcomers in the March ranking. There were significantly more mobile newcomers as app stores have cracked down on ChatGPT copycats, opening up room for more original apps. ChatGPT is still the leading general assistant, but Google, Grok, and Meta are closing the gap. More on the top Gen AI consumer apps is available in the article.

BUILDING AGENTS FOR SMALL LANGUAGE MODELS: A DEEP DIVE INTO LIGHTWEIGHT AI

Small language models are models ranging from 270M to 32B parameters that run efficiently on CPUs or modest GPUs. They offer immense potential: privacy through local deployment, predictable costs, and full control thanks to open weights. They also present unique challenges that require a shift in how agent architectures are designed. This article reviews lessons learned from hands-on experimentation, debugging, and optimizing of inference pipelines for small language models.

HOW CLOUDFLARE RUNS MORE AI MODELS ON FEWER GPUS: A TECHNICAL DEEP-DIVE

Omni is an internal platform at Cloudflare built for running and managing AI models on its edge nodes. It can spawn and manage multiple models on a single machine and GPU using lightweight isolation. Omni makes it easy and efficient to run many small and/or low-volume models. This article looks at how Cloudflare uses Omni to run more models on every node in its network, improving model availability, minimizing latency, and reducing power consumed by idle GPUs.

STOP “VIBE TESTING” YOUR LLMS. IT'S TIME FOR REAL EVALS

Stax is an experimental developer tool designed to streamline the LLM evaluation lifecycle. Evals give developers clear metrics to help understand what's actually better, so developers don't have to waste hours 'vibe testing' every time they try a new model or tweak a prompt. Stax empowers developers to rigorously test their AI stacks to help make data-driven decisions. It allows developers to easily define their own criteria and build a custom autorater.

OH GOODY, THE 'FIRST KNOWN AI-POWERED RANSOMWARE' HAS BEEN DISCOVERED, AND IT 'MAY EXFILTRATE DATA, ENCRYPT IT, OR POTENTIALLY DESTROY IT'

PromptLock is a newly discovered proof-of-concept ransomware that uses OpenAI's gpt-oss-20b model to generate code to exfiltrate data, encrypt it, and potentially destroy it. It uses the Ollama API to generate malicious Lua scripts on the fly, which it then executes. The ransomware runs locally, which allows it to avoid detection more easily as it doesn't use any online resources.

TLDR IndexPage 3