CHAIN-OF-THOUGHT CAN REDUCE PERFORMANCE ON TASKS WHERE THINKING MAKES HUMANS WORSE

Summary from the TLDR Web Dev newsletter

Chain-of-thought (CoT) prompting LLMs reduces performance in cases where thinking also negatively affects human performance. Experiments across three task categories where thinking harms human performance (implicit statistical learning, visual recognition, and classification with exceptions) consistently demonstrate that CoT reduces model accuracy. This suggests that model reasoning, like human deliberation, can lead to detrimental outcomes when applied to certain tasks.