Search the archive

APPLE WORKING ON ON-DEVICE LLM FOR GENERATIVE AI FEATURES

Apple's LLM will reportedly run entirely on-device rather than via the cloud like most existing AI services.

PHI 3

Phi 3 is a series of models, 3B-14B in size, which perform exceptionally well on modern benchmarks. The 3B model claims to outperform the original ChatGPT model. Weights have been released. There is a variant available with a 128k context length.

INTRODUCING SEED-X: MULTIMODAL FOUNDATION MODELS

INSTRUCTION HIERARCHY

OpenAI published research on giving system prompts stronger weighting, which dramatically improves model robustness to jailbreaks and adversarial attacks.

IMAGE GENERATION WITH MULTIBOOTH

MultiBooth introduces a two-phase process to enhance multi-concept image generation, overcoming the challenges of concept fidelity and high costs found in other methods.

EXPLORING LLAMA3'S PERFORMANCE IN LOW-BIT QUANTIZATION

Meta's LLaMA3, a leading large language model, is being tested for its efficiency in low-bit scenarios, often essential in systems with limited resources. This study, available on GitHub and Hugging Face, aims to refine and improve quantization strategies for future large language models.

INSTRUCTOR

Instructor is a Python library that makes it easy to work with structured outputs from large language models.

LOS ANGELES IS USING AI IN A PILOT PROGRAM TO TRY TO PREDICT HOMELESSNESS AND ALLOCATE AID

The Los Angeles County Department of Health Services is using predictive AI to prevent homelessness, identifying at-risk individuals to provide aid and successfully keeping 86% of participants housed. The program, initiated in 2021, has assisted nearly 800 households with over $4,000 in support each. Despite concerns over privacy and ethics, the AI initiative shows promise in addressing California's climbing homelessness crisis.

SELF-REASONING TOKENS, TEACHING MODELS TO THINK AHEAD

Recent experiments introduced "Reasoning Tokens" to improve the thinking process of language models like GPT-2, encouraging them to make calculations for future tokens. Early results show a 35% decrease in loss, indicating the models can indeed learn to anticipate future information. This approach could enhance the ability of language models to plan and reason in a self-supervised manner, potentially reducing the need for step-by-step explanations.

HOW DOES CHATGPT WORK? AS EXPLAINED BY THE CHATGPT TEAM

In this article, OpenAI's Evan Morikawa provides insights into ChatGPT's inner workings, covering input text processing and tokenization to prediction sampling using large language models. ChatGPT operates by turning tokens into numerical vectors (embeddings), multiplying them by a weight matrix of billions, and selecting the most probable next word. The tech is grounded in extensive pretraining to predict text based on vast internet data.

article

STARTUP USES AI TO EDIT HUMAN DATA

A Berkeley-based startup, Profluent, claims to have used generative AI to create a new gene editor called OpenCRISPR-1, which it has used to edit human DNA.

GENERATING 3D SCENES FROM SIX IMAGES

6Img-to-3D is a novel method that uses transformers to create 3D-consistent images from just six input photos.

PANJAYA (PRODUCT)

Localize or tailor video content for any audience in any language while preserving the speakers' natural voice and lip movements.

US AIR FORCE CONFIRMS FIRST SUCCESSFUL AI DOGFIGHT

DARPA's AI-controlled jet successfully engaged a human pilot in a dogfight, showcasing progress in autonomous aviation within the Air Force's ACE program. Tests will continue through 2024 amid discussions on military applications of AI for defense and drone enhancement.

INTEL COMPLETES ASSEMBLY OF FIRST COMMERCIAL HIGH-NA EUV CHIPMAKING TOOL — ADDRESSES COST CONCERNS, PREPS FOR 14A PROCESS DEVELOPMENT IN 2025

Intel Foundry has completed assembly of the first commercial High-NA EUV lithography machine, an advancement critical for its 14A process node R&D targeted for 2025. The High-NA tool, by ASML, enables smaller feature printing and up to 2.9X transistor density improvement. Intel's early adoption of this technology aims to overcome past delays and regain leadership in semiconductor process technology.

ADOBE PREVIEWS AI INNOVATIONS TO ADVANCE PROFESSIONAL VIDEO WORKFLOWS

Adobe is integrating generative AI video tools into Premiere Pro, including new capabilities for shot extension, object addition/removal, and text-to-video features, streamlined by its Firefly video model. The updates, including a technology preview and general availability of AI-powered audio workflows, aim to enhance video production efficiency and creativity.

BYTEDANCE'S DECOUPLEQ FOR MODEL COMPRESSION

DecoupleQ is a quantization approach that significantly enhances the accuracy of large models at ultra-low bit levels. This method restructures the quantization process by splitting model parameters into integer and floating-point parts that are then optimized using traditional methods.

MOVA: A NEW MULTIMODAL LANGUAGE MODELS

MoVA is a multimodal large language model (MLLM) that improves image content understanding by selectively integrating different vision encoders. It addresses the limitations of current encoders like CLIP by using a context-aware expert routing strategy and a mixture-of-vision-expert adapter to dynamically fuse expertise from various sources.

LIDAR-BASED MOVING OBJECT SEGMENTATION

Researchers have developed MambaMOS, a new approach for segmenting moving objects in LiDAR point clouds.

ENHANCED IMAGE HARMONIZATION

TF-GPH is a new method for Painterly Image Harmonization called that bypasses the need for training data or prompts by using an innovative "share-attention module".

CREDIT RISK PREDICTION

FinLangNet is a model designed to enhance risk prediction in the financial sector. By adopting natural language processing techniques, FinLangNet uniquely models credit loan trajectories akin to linguistic structures.

A PLATFORM FOR MULTI-AGENT EXPLORATION

The newly introduced MAexp platform supports multi-agent exploration by offering a high-efficiency, flexible environment that supports a wide array of scenarios and multi-agent reinforcement learning (MARL) algorithms.

INTRODUCTION TO TRITON (71 MINUTE VIDEO)

Triton is a high level language for writing low level CUDA kernels. It allows you to write in a Python style form and dramatically accelerate your AI model performance.

BUILDING RELIABLE SYSTEMS OUT OF UNRELIABLE AGENTS

AI agents might not be inherently reliable, but they can be harnessed to build reliable systems. Strategies include starting with simple prompts and creating an evaluation system for iterative improvement, deploying with observability, leveraging Retrieval Augmented Generation (RAG), considering model fine-tuning, and using complementary agents to cover each other's weaknesses and improve the system's overall reliability.

FOUNDATION CAPITAL SEES AI LEADING $4.6T OPPORTUNITY IN SERVICE-AS-SOFTWARE

Many VCs are talking about AI taking a bite out of the services business. Foundation Capital believes there is $4.6 trillion worth of work to be automated, thanks to AI: both for in-house functions and outsourced services. We're entering the era of Service-as-Software.

FINE-TUNE LLAMA3 WITH Q-LORA AND FSDP

Great blog covering a quick and efficient fine-tuning method using PyTorch on the recent Llama 3 model.

LEAST USEFUL LLAMA 3 LAYERS

Model pruning takes layers with extremely high similarity and removes them, saving costs with minimal accuracy reduction.

LLAMA 3 JAILBREAK

A trivial programmatic Llama 3 jailbreak.

* Explore the future of AI and the open web

* Learn how to mitigate the risks of AI tools * Get expert guidance on how AI is changing ecommerce * Discover why composability [4] is the foundation for AI transformation

See the full agenda—including guest speakers—and join for free

UPDATES FROM GOOGLE DEEPMIND ALIGNMENT RESEARCH

Following Anthropic, GDM has released some work from its alignment efforts. The main insightful piece here is using sparse autoencoders on Gemini Ultra. This is a substantial jump in interpretation size.

NVIDIA TO COLLABORATE WITH JAPAN ON THEIR CUTTING-EDGE ABCI-Q QUANTUM SUPERCOMPUTER

NVIDIA will power Japan's new quantum supercomputer, ABCI-Q, alongside Fujitsu, integrating 2,000 NVIDIA H100 AI GPUs and CUDA-Q platform for quantum-classical computing applications. The project aims to advance Japan's capabilities in quantum computing and AI. This collaboration is part of a broader technological partnership between NVIDIA and Japan.

BRAVE SEARCH IS ADOPTING AI TO ANSWER YOUR QUERIES

Brave Search has revamped its answer engine to include AI-synthesized responses worldwide, leveraging large language models and trusted data sources. The upgrade enables automatic or manual AI-enhanced answers to user queries. Brave acknowledges potential impacts on web publishers and plans to monitor and address the effects of AI-generated content on site traffic.

MOVING OBJECT SEGMENTATION

Segmenting objects across videos is challenging due to temporal consistency. This work presents the combination of a powerful image segmentation model and optical flow which combined gives compelling performance on this task.

YOUR LANGUAGE MODEL IS A Q-FUNCTION

A fairly technical RL paper that shows the theoretical underpinning of reward models and base models in language.

FINE-TUNE LLAMA 3 WITH ORPO

While this tune performs worse than Hermes, it is likely due to the small dataset used. Otherwise, this is a great walkthrough on how to tune these models to improve desired performance on certain tasks.

FINEWEB: 15T HIGH QUALITY WEB TOKENS (HUGGINGFACE HUB)

The newest Llama 3 models were trained on 15T tokens. This new dataset contains an extensively deduplicated corpus from common crawl and results in high quality models.

MISTRAL COMMON

Mistral has released its tokenizer utilities, chat templates, and more basic tools which it uses internally.

TRAINING-FREE CONTEXT EXTENSION

Researchers have developed a way to expand the context window of embedding models up to 32,000 tokens without additional training using strategies like position interpolation.

TEACHING MODELS TO THINK AHEAD

This article introduces "reasoning tokens" in language models, which generate additional tokens designed to predict future tokens rather than the immediate next one, thereby enhancing the model's ability to think ahead. Experiments demonstrate significant improvements in prediction accuracy, suggesting a potential for more complex reasoning without explicit step-by-step training.

LOOKING FOR AI USE-CASES

This article discusses the transformative potential and current limitations of generative AI like ChatGPT, noting that while it excels in tasks like coding and generating drafts, it struggles with complex tasks that require specific programming. It highlights the need for a vision that matches AI solutions with practical applications, emphasizing that identifying and integrating these into daily workflows remains a significant challenge.

A VISUAL GUIDE TO VISION TRANSFORMERS

A visual guide to Vision Transformers (ViTs), a class of deep learning models that have achieved state-of-the-art performance on image classification tasks.

LLAMA 3 IS NOT VERY CENSORED

The original Llama models had significantly more incorrect refusals. This brief blog shows some examples that previously failed.