TLDR AI 2024-11-19

Abstraction

→ Check out pgai on GitHub [5]: a framework for developing RAG, semantic search, and other AI applications directly in PostgreSQL

PIXTRAL LARGE

Pixtral Large is a 124B open-weights multimodal model built on top of Mistral Large 2. It is the second model in the multimodal family and demonstrates frontier-level image understanding. The model is able to understand documents, charts, and natural images while maintaining the leading text-only understanding of Mistral Large 2.

PERPLEXITY INTRODUCES A SHOPPING FEATURE

Perplexity has introduced a shopping feature for Pro users in the US, making it a one-stop solution where you can research and purchase products.

BEN AFFLECK TELLS ACTORS AND WRITERS NOT TO WORRY ABOUT AI

Ben Affleck reassured Hollywood actors and writers during an interview that AI poses little threat to their jobs due to its current limitations.

LLAVA-O1: REASONING VLMS

While this is not an actual replication of the process used to train o1, it is still a strong VLM trained on reasoning traces.

CASE STUDY OF SONNET 3.5 COMPUTER USE

This work aims to systematically test the ability of new autonomous computer use agents. It finds Claude quite powerful at traditional linear tasks.

TEACHING VIDEO MODELS TO UNDERSTAND TIME LIKE A STORY

This paper introduces NumPro, a creative solution that helps Video Large Language Models handle Video Temporal Grounding tasks.

LLAMA 3 405B INFERENCE

Cerebras has the largest Llama model, running 12x faster than comparable systems and 18x faster than some closed model API providers. It can run at almost 1k tokens per second. It will be available via API at the beginning of next year.

CLIP FOR SEMANTIC SEGMENTATION

Despite CLIP's success in open-vocabulary tasks, it struggles with semantic segmentation due to noisy features and limited resolution. Trident addresses the resolution issue with a training-free framework that combines CLIP and DINO features from sub-images and uses SAM's encoder for global feature aggregation.

CERTIFIED ROBUSTNESS

This work focuses on improving the certified robustness of smoothed classifiers by fine-tuning off-the-shelf models.

RECAPTURE

This paper from Google shows how you can change the camera viewpoint of an existing video.

NOUS RESEARCH FORGE

The Forge Reasoning API allows you to take any popular model and superpower it with a code interpreter and advanced reasoning capabilities. This has been shown to increase model performance.

WHERE'S MY ROBOT BUTLER?

Advancements in AI and robotics are accelerating the development of humanoid robots such as Atlas, Optimus, and Neo, robots aimed at fulfilling domestic tasks akin to Rosie from "The Jetsons." Challenges persist in creating cost-effective, safe, and efficient actuators. AI models are crucial for training robots to perform complex tasks autonomously. Despite progress, these humanoid robots are currently more suited for industrial settings. They may eventually transition into home environments if significant breakthroughs occur.

market data forecasting model. Registration is open.

GOOGLE'S HEAD OF RESEARCH ON WHETHER 'LEARN TO CODE' IS STILL GOOD ADVICE IN THE AGE OF AI

Despite AI handling some coding tasks, understanding coding basics is crucial and creates new opportunities across fields, including healthcare and education.

WHY ARE WE USING LLMS AS CALCULATORS?

Researchers are experimenting with LLMs' ability to solve math problems to assess their reasoning capabilities.

GPTS ARE MAXED OUT

OpenAI's next-gen model, internally known as Orion, reportedly underperforms compared to expectations set by Sam Altman, suggesting a potential limit to scalable growth in AI models.