TLDR Index
TLDR AI

TLDR AI 2024-10-07

6 sections18 entrieslinks blockcomplete parse

Parse notes

  • Subscription/account footer truncated before parsefooter_truncated

01GENERAL

Sponsored

DESTINATIONS

Would you spend one dollar if it meant you could travel the world?

app.dollarflightclub.comSponsor

02HEADLINES & LAUNCHES

META MOVIE GEN

Meta has trained a state-of-the-art movie model. It has 30B parameters and took 6,144 H100s to train. The model was trained on 1B images and 100m highly curated videos. Interestingly, it is a Temporal Autoencoder and Flow matching Llama. Meta released a 92 page paper, one of the most detailed research reports on the topic.

ai.meta.com6 min read

FINLLM LEADERBOARD

The Open FinLLM Leaderboard offers a specialized evaluation platform tailored for financial language models. It focuses on tasks crucial for finance, such as stock movement prediction, sentiment analysis, and information extraction from financial reports.

huggingface.co6 min read

OLLAMA IN BRAVE BROWSER

Nvidia's RTX-Acceleration combined with Ollama allows for running local models in the browser.

blogs.nvidia.com5 min read

03RESEARCH & INNOVATION

EMBERS OF AUTOREGRESSION FOR O1

Large language models have limitations due to next token prediction. However, OpenAI's o1 model was trained with a new objective over reasoning traces. Despite this change, o1 still shows signs of limitations due to next token prediction.

arxiv.org26 min read

CONTEXTUAL DOCUMENT EMBEDDINGS

This paper describes a method that is almost like a neutral TF/IDF in that it takes information from the entire corpus instead of just a single embedding document. It essentially encodes contextual information about neighboring documents. This method is state-of-the-art on MTEB.

arxiv.org18 min read

ENTP: ENCODER-ONLY NEXT TOKEN PREDICTION

This paper shows a novel design architecture for using encoder-only transformers as next token prediction models. This has been done in other ways before, but the researchers find a great way to prevent cheating. The models have hints of strong performance at scale and also are capable of in-context learning.

arxiv.org24 min read

04ENGINEERING & RESOURCES

INFINITE DATA LOADERS

New model testing is limited in the image domain by poor-quality small datasets like CIFAR10. This GitHub repository contains a tool that creates infinite complicated fractals as images or videos for testing.

github.comGitHub repository

AUTO JOBS APPLIER

An immensely viral repository that uses language models to automate the job application process and add an additional layer of personalization to the process.

github.comGitHub repository

PAIRWISE DISTILLATION FOR DENSE RETRIEVAL

This project proposes a new technique called Pairwise Relevance Distillation (PairDistill) to improve the accuracy of dense retrieval methods.

github.comGitHub repository

05MISCELLANEOUS

SOLVING COMPLEX PROBLEMS WITH AI

This study develops high-performance graph algorithms that help solve complex, large-scale problems like detecting fraud in financial networks or finding efficient delivery routes.

news.mit.edu4 min read

FACTUAL ACCURACY IN AI

Integrative Decoding is a method that enhances the factual accuracy of large language models, especially for open-ended tasks.

arxiv.org18 min read

LIQUID FOUNDATION MODELS: OUR FIRST SERIES OF GENERATIVE AI MODELS

Liquid AI has announced its first generation of Liquid Foundation Models (LFMs), which deliver state-of-the-art performance with reduced memory usage. The LFMs, optimized for various hardware, include a 1B, 3B, and 40B model and are already available on platforms like LIQUID PLAYGROUND and soon on Cerebras. These models excel in handling sequential data and offer innovations in efficiency and scaling across industries such as financial services and biotechnology.

liquid.ai11 min read