EXPLORING LLM WEIRDNESS: HOW WE BUILT IT & WHAT I LEARNED

Summary from the TLDR Web Dev newsletter

Josh Watzman, a software engineer at Facebook, and his coworker built an LLM-powered quiz game that became popular online. They found LLM APIs simple to use and found that for a good user experience, it was best to immediately stream LLM responses. Since LLMs are functionally text prediction tools, they are great for language tasks, but not as good for interpreting non-textual information. Interestingly, immediate streaming of LLM responses is crucial for a positive user experience, even if it doesn't reduce overall response time.