CAN AI DETECT USABILITY PROBLEMS?

Summary from the TLDR Design newsletter

AI can analyze usability test videos by sampling frames, transcribing audio, and generating plausible descriptions of user behavior — though this process is "lossy" and prone to inconsistency. In a test using a six-minute OpenTable usability session, ChatGPT identified seven specific problems, including a search field that dropped cuisine selections when the location was changed and results that surfaced non-sushi restaurants. Whether those findings hold up in terms of accuracy, reliability, and alignment with what human researchers would catch remains an open question to be addressed in follow-up research.