IS THERE A HALF-LIFE FOR THE SUCCESS RATES OF AI AGENTS?

Summary from the TLDR AI newsletter

The performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model - a constant rate of failing during each minute a human would take to do the task. This implies an exponentially declining success rate with the length of the task. The regularity allows for the success rate for an agent at different task lengths to be estimated. The underlying cause of failure on longer tasks is likely due to the increasingly large set of subtasks - failing any single one fails the whole task.