WORKING WITH LLMS: A FEW LESSONS

Summary from the TLDR newsletter

Large Language Models (LLMs) are inherently probabilistic, which means you have to add evaluation frameworks, human-in-the-loop processes, and design for graceful failure, using LLMs for probabilistic guidance rather than deterministic answers. Unlike with traditional software, there is no way to get better at using AI than using AI - there is no substitute for trial and error. This makes it hard to predict development and makes building future-proof technology almost impossible. While software is inexpensive to distribute once written, LLMs have continued costs.