Search the archive

THE OFFICIAL OPENAI PROMPT ENGINEERING GUIDE

A comprehensive step-by-step guide that shares strategies and tactics for getting better results from large language models like GPT-4.

GITHUB MAKES COPILOT CHAT GENERALLY AVAILABLE, LETTING DEVS ASK QUESTIONS ABOUT CODE

Copilot Chat allows developers to ask for real-time guidance with natural language.

SIMPLE ANALYTICS (PRODUCT)

Chat with your analytics.

TRANSITIONING FROM TORCH TO JAX

Thoughts on how to make the leap from Pytorch to Jax. This article presents 9 succinct concepts to grasp that make Jax powerful and different - each comes with a nice code example.

YOUR REFERRAL LINK - https://tldr.tech/ai?ref=2613930

SIKON, SONY, AND CANON FIGHT AI FAKES WITH NEW CAMERA TECH

Nikon, Sony Group, and Canon are developing camera technology to embed digital signatures in images, allowing differentiation from AI-generated fakes. Nikon's mirrorless cameras will include tamper-resistant digital signatures for professionals, providing details like date, time, and location. Sony plans to add an online tool called Verify to its mirrorless SLR cameras in spring 2024 to check images for digital signatures, highlighting any AI or tampered content. Canon will introduce similar technology for images and videos by 2024. These advancements address the growing challenge of distinguishing real images from sophisticated AI-generated fakes.

INTEL TO SPIN OUT AI SOFTWARE FIRM WITH OUTSIDE INVESTMENT

Intel is forming a new independent company called Articul8 AI focused on artificial intelligence software, with investment from DigitalBridge Group and other investors. Articul8, stemming from Intel's work with Boston Consulting Group, will have an independent board and will not be publicly traded, will focus on creating cost-effective, secure AI solutions for business use.

SELF-PLAY FINE-TUNING CONVERTS WEAK LANGUAGE MODELS TO STRONG LANGUAGE MODELS

One of the big drawbacks of current language models is that you need extremely expensive human preference data to make them work well. A big research direction recently has been figuring out if we can have the language models perform self-play to improve without collecting this data. SPIN is a new method that takes a step in that direction with just SFT data - it can dramatically improve a base model’s performance across various tasks.

LEARNING TO FIND BOUNDARIES WITH BOUNDARY ATTENTION

Finding contours and boundaries in images is a classic computer vision problem. However, when introducing noise, quality shifts, or out-of-distribution examples, many existing methods fail to perform well. This new learned method performs extremely well on sensor readings and is only 207k parameters. It uses a two-stage training process and greatly improves state of the art.

ENHANCING LOW-LIGHT PHOTOS WITH SELF-SUPERVISED ADAPTATION

This research leverages bracketing photography and a novel temporally modulated recurrent network (TMRNet) to significantly improve photo quality in low-light conditions. By training with synthetic data and adapting to real-world images, this approach outperforms existing multi-image processing methods.

GPT-FAST MIXTRAL SUPPORT

GPT-fast is a great push from the Pytorch team to speed up vanilla models. It can achieve an impressive 218 tokens per second across 8 GPUs.

TINYGPT-V

Efficient multimodal large language model with a small backbone.

GENERATING VR SIDE-BY-SIDE VIDEOS WITH MARIGOLD

By using state-of-the-art depth estimation and some classical video processing, you can now turn monocular videos into 3D viewable XR experiences! The code is MIT-licensed and available for commercial use.

TEXT-TO-AUDIO GENERATION WITH T2I MODEL ADAPTATION

The Auffusion system, inspired by Text-to-Image diffusion models, offers a breakthrough in Text-to-Audio (TTA) generation. It excels in creating high-quality audio from text, especially for complex inputs.

THE NYT VS OPENAI IS NOT JUST A LEGAL BATTLE

The New York Times' lawsuit against OpenAI highlights a deep-seated conflict between the advancement of technology and copyright infringement. While some argue for unrestricted AI progress as a boon to humanity, others stress the moral imperative to respect creators' rights and advocate for a balanced approach that values both technological advancement and the protection of intellectual property. This dispute underscores the complex relationship between morality and progress in the AI era.

WHAT WE LEARNED ABOUT AI AND EDUCATION IN 2023

Education is adapting to the integration of AI by exploring collaborative approaches to AI-assisted tutoring, writing instruction, and assessment.

Start with a quick demo

COLBERT USAGE EXAMPLES (COLAB NOTEBOOK)

ColBERT is a state-of-the-art method for retrieval and generation. Its authors have provided a notebook to help people get up to speed and try the method out.

CHATGPT UNWRAPPED (PRODUCT)

Create a year in review for your ChatGPT usage.

EFFICIENT LLM INFERENCE ON CPUS

Intel has released an optimized toolkit for inferencing Transformers on the CPU.

YOUR REFERRAL LINK - https://tldr.tech/ai?ref=2613930

MICROSOFT’S COPILOT APP IS NOW AVAILABLE ON IOS

STUFF WE FIGURED OUT ABOUT AI IN 2023

This article attempts to round up the highlights in AI development over 2023. It covers how to run LLMs on personal devices, fine-tuning models, the problem of gullibility, LLM applications, and more. LLMs can provide people who use them effectively with a significant boost to their quality of life. They're actually quite easy to build, but there's still a lot unknown about them and they're still unreliable for many applications.

HOW IBM SEES AI CHANGING THE GAME

AI technology is transforming businesses in various industries. IBM’s VP of Software and Technology Raj Datta and Director of Startups Kylie Rutherford share in this article how AI is changing the game for companies of all sizes, presenting some use cases of various products.

MOSAICBERT: A BIDIRECTIONAL ENCODER OPTIMIZED FOR FAST PRETRAINING

BERT is an Encoder-only Transformer. This means it is often used to represent natural language in the form of embeddings and isn’t usually used to generate text. Importantly, it was one of the first Transformers used in search because it can represent both the query and the information to be retrieved. The team at Mosaic has integrated FlashAttention, GLU, and many other innovations to dramatically improve pretraining speed (1 hour, $20) while matching the performance of much larger traditional BERT models.

IMPROVING TEXT EMBEDDINGS WITH LARGE LANGUAGE MODELS

Researchers at Microsoft used synthetic data to train a decoder-only transformer based on Mistral for embeddings. It is best in class. Interestingly, they use a two-step prompting strategy with GPT-4 to generate the synthetic retrieval training data.

IMAGES ALTERED TO TRICK MACHINE VISION CAN INFLUENCE HUMANS TOO

Recent research reveals that subtle changes made to digital images to deceive AI vision systems can also influence human perception. While humans and AI perceive images differently, the study found that humans, under controlled conditions, can be systematically biased by adversarial perturbations designed to mislead AI.

LLM COURSE

A modern look at hot topics in the LLM space like merging, GGUF, quantization, DPO, and more. It is designed for beginners, scientists, and engineers to dive into the field and get up to speed quickly.

LLAMA FILE ONE-LINERS

Llamafile is a project that embeds the model and inference code into a single portable executable. This blog explores how to pipe command line output into the language model for further processing.

UNVEILING GEMINI'S POTENTIAL: A STUDY OF MULTIMODAL COMMONSENSE REASONING

This project introduces an in-depth analysis of Google's Gemini, a Multimodal Large Language Model, assessing its performance in common sense reasoning across various tasks. The study, which compares Gemini with other models, reveals its competitive abilities in integrating knowledge across modalities.

LLMS AND PROGRAMMING IN THE FIRST DAYS OF 2024

In 2023, Large Language Models (LLMs) have become invaluable for programmers, significantly accelerating code writing and understanding complex APIs or frameworks. While they have limitations in complex system programming, LLMs excel in high-level Python coding and mundane tasks, serving as an efficient tool for enhancing productivity and as a condensed form of documentation.

SINGLE IMAGE SUPER-RESOLUTION WITH IMPROVED STABILITY

Researchers have developed a new method that improves single image super-resolution by focusing on two key aspects: the optimal centroid of potential high-resolution images and the inherent noise that affects image quality.

A RECAP OF CONSUMER AI IN 2023

This thread provides a recap of AI products and trends that hit the mainstream in 2023. It goes through product launches month by month, providing user numbers and other relevant metrics. Highlights include ChatGPT reaching 100 million monthly active users in January, Balenciaga Pope going viral with 28 million views in March, AI covers taking off in June, users getting roasted by TikTok's AI memes in August, and X's Grok launching in December.

AI-CREATED “VIRTUAL INFLUENCERS” ARE STEALING BUSINESS FROM HUMANS

Companies are hiring virtual influencers and reporting significant results and decreases in costs.

CAPGO (PRODUCT)

Fastest market research in spreadsheet.

PYKOI

Pykoi is an open-source Python library for improving LLMs with RLHF.

YOUR REFERRAL LINK - https://tldr.tech/ai?ref=2613930

OPENAI’S ANNUALIZED REVENUE REPORTEDLY TOPS $1.6B

OpenAI’s annualized revenue has topped $1.6B just two months after reaching the $1.3B mark.

GOOGLE'S VIDEOPOET PAPER RELEASED: A CLOSER LOOK

APPLE’S LLM GAP IS REAL. IT MIGHT NOT LAST MUCH LONGER

Apple, after developing the "Ferret" Large Language Model with Cornell University, is expected to reveal its approach to integrating LLMs at the next Worldwide Developers Conference, focusing on user privacy and simplicity.

ANYTEXT VISUAL TEXT GENERATION AND EDITING

AnyText is a multilingual visual text generation and editing tool. It features a diffusion pipeline with an auxiliary latent module and a text embedding module for generating and editing text. The tool employs OCR models and tokenizer embeddings to blend texts seamlessly into backgrounds. AnyText stands out in accuracy for both Chinese and English text generation. The tool offers an online demo, installation instructions, and methodology details on its GitHub page.

A TRAINING-FREE APPROACH HIGH-RESOLUTION IMAGE INPAINTING

HD-Painter marks a significant advancement in text-guided image inpainting by introducing a training-free method that ensures high-resolution results and better alignment with user prompts. It utilizes innovative techniques like the Prompt-Aware Introverted Attention layer and Reweighting Attention Score Guidance, allowing for detailed inpainting up to 2K resolution.

PAINT3D: GENERATE HIGH-RESOLUTION TEXTURES FOR 3D MESHES

Paint3D introduces an approach to generating high-quality, 2K UV texture maps for 3D meshes without embedded lighting, allowing for easy re-lighting and editing.

DIFFUSERS JS

A reimplementation of key HuggingFace Diffusers functionality in JavaScript to be run in the browser. Includes examples of running popular models.

MIXTRAL OFFLOADING

Mixtral offloading is a project focused on efficient inference of Mixtral-8x7B models on platforms like Colab or consumer desktops. It employs mixed quantization with HQQ and a MoE offloading strategy to fit the model into combined GPU and CPU memory. The project includes a demo notebook and offers online execution via Colab. It is still under development, with features like other quantization methods and speculative expert prefetching planned.

IMAGE SEGMENTATION WITH COMMON SENSE KNOWLEDGE

ProLab introduces a unique approach for improving image segmentation models that uses property-level labels based on common sense knowledge. By combining Large Language Models with structured prompts and a description embedding model, ProLab not only performs better on classic benchmarks but also shows promise in segmenting unknown categories, offering a more interpretable and scalable solution.

A CASE FOR AI ALIGNMENT BEING DIFFICULT

This analysis explores the challenges of aligning AGI with human values. It suggests that while integrating human values into AGI is complex due to differences in mind architecture and evolutionary influences, understanding the human brain and enhancing human capabilities could be key to developing AGIs that align with human values, offering potential yet challenging paths forward.

AI IS NOT CONSCIOUS

A group of 19 researchers suggested a checklist to assess AI consciousness and determined that no existing AI systems possess consciousness yet - but the creation of potentially conscious AI systems remains a technical possibility.

HOW LEADING AI STARTUP INVESTORS APPROACHED ARTIFICIAL INTELLIGENCE IN 2023

Most investors took a wait-and-see approach, knowing full well that the big, big disruptions are coming in the form of new silicon and solutions that go beyond fun “parlor tricks”. While funds may say they have seen this coming for a long time, no one can say with a straight face how big the impact would be on the general zeitgeist.