THE INFERENCE SHIFT

Summary from the TLDR newsletter

The most important aspect for agentic inference is memory. Agents need context, state, and history, which requires a memory hierarchy wrapped around a model. This implies a necessary trade-off of speed for capacity, but agentic tasks don't really need to be that fast if a human isn't involved.