TLDR AI 2024-12-03

live demo

ADS MIGHT BE COMING TO CHATGPT

OpenAI is considering an advertising business model to help support its costly AI tool development. Despite no current active plans for ads, the idea remains on the table as a potential revenue source. CEO Sam Altman considers ads a last resort and has expressed discomfort at the possibility of combining ads and AI.

THE RACE IS ON TO MAKE AI AGENTS DO YOUR ONLINE SHOPPING FOR YOU

Tech companies are developing AI shopping agents to automate online purchases, potentially reshaping the retail industry. Perplexity's current model faces operational challenges. Companies like OpenAI, Google, and Amazon are also exploring AI purchasing capabilities. These developments aim to streamline shopping but raise questions about privacy, retailer interactions, and the evolving online shopping experience.

SALESFORCE CEO MARC BENIOFF HAS THOUGHTS ON AI AGENTS, AUTOMATION, AND THE FUTURE OF YOUR JOB

Salesforce CEO Marc Benioff envisions companies creating AI agents to handle customer service and sales tasks by leveraging their existing data and policies, positioning Salesforce as the key enabler of this transformation. He argues that AI-driven automation will enhance productivity rather than eliminate jobs, allowing businesses to operate more efficiently and expand without increasing human labor. Benioff highlights this shift as a critical moment in business evolution, providing a competitive edge against rivals and reshaping traditional workflows.

DEMO: DECOUPLED MOMENTUM OPTIMIZATION

2.5x faster, 100x less communication volume, and written by the original Adam author. This new optimizer boasts impressive performance for language model training with some strong improvements over the current dominant optimization schemes.

DEMO TRAINING RUN (LIVE DASHBOARD)

Nous is training a 15B distributed model using the DeMo optimizer. All of the training can be followed live at this link.

REWARD HACKING

Lilian Weng has another great blog post discussing the challenge of Reward Hacking within language model alignment. It is one of the major challenges preventing models from being deployed into production scenarios.

Kore.ai

FINE-TUNE MODELS WITH LORA-SB

LoRA-SB is a new method that brings full fine-tuning performance to low-rank adapters for large language models.

MAKING AI DATASETS MORE DIVERSE

Researchers introduced a new method called Diversity-driven EarlyLate Training (DELT) to improve dataset distillation for large-scale tasks.

OPENAI TARGETS 1BN USERS IN NEXT PHASE OF GROWTH

CREATE JSONL DATASET FROM API CHAT LOGS

A nice simple utility that allows you to create a JSONL dataset based on the messages sent back and forth between user and API.

GOOGLE'S PLAN TO KEEP AI OUT OF SEARCH TRIAL REMEDIES ISN'T GOING VERY WELL

US District Judge Amit Mehta suggests AI could play a significant role in developing remedies following the government's win in the Google search monopoly trial, potentially affecting Google's AI products. The DOJ has proposed remedies to prevent Google from using AI to maintain its market dominance, including restrictions on exclusive agreements and investments in AI. Microsoft is resisting Google's demands for confidential AI deal details, arguing relevance. OpenAI may be compelled to share data amidst these considerations.

swag! Learn more about ML at Jane Street

UV GUIDE TO PYTORCH

Documentation on how to use the new package manager UV to install PyTorch.

AI COMPANY MISTRAL IS LATEST EUROPEAN STARTUP TO EYE EXPANSION IN SILICON VALLEY

European AI startup Mistral, known for open-weight LLMs, is expanding to Palo Alto to access Silicon Valley talent and resources.

OPENAI GETS NEW $1.5 BILLION INVESTMENT FROM SOFTBANK, ALLOWING EMPLOYEES TO SELL SHARES IN A TENDER OFFER

OpenAI is allowing employees to sell $1.5 billion worth of shares to SoftBank, reflecting growing investor interest in AI.