Full-text search across article titles and summaries. Refine with sector, year, content type, and reading-time filters.
Pixar turned to AI to solve a unique animation challenge for its new movie 'Elemental', using AI to animate a character made of fire. Paul Kanyuk, a technical supervisor at Pixar, applied Neural Style Transfer (NST) technology to animate the fiery character, Ember. Despite the hefty computational demand, the team achieved a 20X speed-up by leveraging GPUs, heralding a new intersection of art and technology in animation.
Textbooks may be all you need for extreme AI performance. By using a highly curated set of data and synthetic textbooks with exercises, a 1.3B parameter model achieves 50+ on HumanEval, a challenging code benchmark. For context, this rivals ChatGPT and far surpasses the original 12B Codex model. This paper follows the Tiny Stories line of work and has lots of nice tidbits.
Language models often hallucinate/confabulate information. This is likely because they are trained to imitate expert sequences which in turn causes them to treat their own (potentially wrong) generations as ground truth. Therefore, as sequence lengths grow, models are more likely to make stuff up. However, if you treat it as a Reinforcement Learning problem and find the next most probable action - while including a backspace action - you can dramatically reduce these hallucination issues.
As Transformers have started to gain more and more traction, they are being served far more often than trained. This can be quite slow and expensive, but with some clever optimizations around memory layout on GPU, they get many multiples faster.
The repo presents Wanda, a unique method for reducing the complexity of large language models by selectively dropping less significant network weights. Unlike other methods, Wanda doesn't require retraining or intensive computation, providing effective model pruning while maintaining performance.
This repo introduces OpenSTL, a benchmark for models that predict future patterns based on past frames. It categorizes approaches and evaluates them on various datasets.
Jack Ma-backed Ant Group Co. is developing large-language model technology that will power ChatGPT-style services.
Sponsored
You’ll need more than an OpenAI API key to ship AI products that are fast, accurate, and unique
OpenAI plans to launch a marketplace that will allow developers to sell their AI models built on top of its own AI technology.
PhotoBook is an interactive game in which two players find common images from their own private sets, pushing machines to learn human-like communication around visual context. The approach, based on the DeBERTa model and CLIPScore features, bypasses the need for reference chains, directly predicting whether an image is shared.
This research introduces LOw-Memory Optimization (LOMO), an optimizer designed to dramatically decrease the resources needed for Large Language Models (LLMs) training, a process that currently requires extensive GPU resources. LOMO's innovative approach allows the fine-tuning of a 65 billion parameter model using just a single machine with 8 RTX 3090 GPUs, each with 24GB memory, reducing memory usage to just 10.8% of the standard method.
The Clip2Protect project introduces an innovative two-step approach to ensure facial privacy, especially in an age of powerful face recognition systems. It creates 'makeup' faces that look natural, yet fool facial recognition software, showing significantly better performance than previous privacy-enhancing methods.
Autolabel is a Python library to label, clean and enrich text datasets with any Large Language Models (LLM) of your choice.
Institutional investors are increasing pressure on technology companies to take responsibility for the potential misuse of artificial intelligence as they become concerned about the liability for human rights issues linked to the software.
The quality of the training mix for AI models is key to getting good performance. In code models, deduplication of data tends to matter quite a bit. It's not clear if it matters as much for text. However, in any case, this blog presents a great description of the techniques used to deduplicate large data sets of text-like data.
AI should be used as a tool to enhance productivity, imagination, and creativity, and not as a means of automation. This article provides insights on how to use AI without sacrificing quality.
Dump all your files and thoughts into your GenerativeAI Second Brain and chat with it.
The European Parliament has passed a draft law for regulating artificial intelligence (AI). The law restricts the use of AI in certain high-risk areas, such as in the legal system. It would also require companies to disclose more information about the data they use to create AI systems.
Synthesia announced a $90M Series C funding round led by Accel. The funds will boost their mission to make video creation easy for everyone, driving their AI research and enhancing their collaborative video editing platform with better avatar technology, real-time collaboration, and enhanced sharing functionality.
AMD announced its latest AI-dedicated GPU, the MI300X, set to ship later this year. The chip poses a robust challenge to Nvidia's dominance in the AI chip market. This launch aligns with AMD's strategic focus on AI, a sector forecasted to grow from $30 billion to over $150 billion by 2027. Designed for large language models, the MI300X, with its superior memory capacity and support for ROCm software, could drive down costs of AI applications and tap into Nvidia's market share.
Multivariate time series anomaly detection (MTAD) is important for many real-world applications like smart surveillance and risk management. In this study, the authors create a new system called Coupled Attention-based Neural Network (CAN) to better detect these anomalies. It uses a special network that can adapt to changing data, making it more accurate in different scenarios.
This research presents SoViT, a vision transformer that has been optimized for size and shape using advanced scaling methods, resulting in performance competitive with much larger models, but with less computation. Through tests across various tasks like image classification, captioning, visual question answering, and zero-shot transfer, SoViT demonstrates its effectiveness, indicating a new direction for optimizing, rather than simply scaling up vision models.
In this research, the authors created a better way to detect objects in combined color-depth (RGB-D) images using a new training method that requires less supervision and separates different kinds of data effectively.
OmniEvent is an open-source toolkit designed for event extraction, encompassing both event detection and argument extraction processes. The toolkit offers a comprehensive approach for various paradigms, providing fair and unified evaluations on commonly-used English and Chinese datasets. Its modular design ensures a high degree of extensibility.
Closed models are strong because they are actually code models disguised as language models. A lot of the open language models fall short because they don't gain reasoning skills from massive code pretraining. Here, we have Starcoder models instruction tuned on Evol instructions and achieving an astonishing 57 on the HumanEval Benchmarks. Models and code are available.
Audiocraft is a PyTorch library for deep learning research on audio generation. At the moment, it contains the code for MusicGen, a state-of-the-art controllable text-to-music model.
Senators Josh Hawley and Richard Blumenthal want to clarify that the internet's bedrock liability law does not apply to generative AI, per a new bill introduced Wednesday that was shared exclusively with Axios.
Researchers have warned that AI models that are trained on AI-generated content can collapse, meaning they forget the true underlying data distribution and produce erroneous responses. This can lead to problems such as discrimination and a distorted perception of reality.
Microsoft's CEO, Satya Nadella, has led the company to an impressive resurgence in AI tech by integrating OpenAI's language model into Bing via a chatbot named Sydney and launching Copilot, an AI tool that automates aspects of coding. Despite initial hiccups, Microsoft is becoming a heavyweight in the AI arena. The company sees its $10B investment in OpenAI as a key strategic move. Nadella views AI as an empowerment tool for humans and emphasizes the importance of aligning AI with core human values while continuing to explore its potential and manage unintended consequences.
Google is bringing generative AI technology to shopping, aiming to get a jump on e-commerce sites like Amazon.com Inc.
Google will have to postpone starting its artificial intelligence chatbot Bard in the European Union after its main data regulator in the bloc raised privacy concerns.
Doctors are using ChatGPT to improve their bedside manner. ChatGPT can help doctors communicate more compassionately with patients by providing them with words to break bad news, express concerns about a patient's suffering, and more clearly explain medical recommendations.
WordPress has released a new AI writing tool called Jetpack AI. The tool can be used to generate blog posts, social media posts, and other types of content. Jetpack AI is powered by GPT-3, a large language model developed by OpenAI. The tool is still in beta, but it has the potential to make it easier for people to create content for their websites.
OpenAI, Google DeepMind, and Anthropic have committed to provide “early or priority access” to their AI models to the UK to support research into evaluation and safety.
LlamaIndex, an innovative platform aiming to connect LLMs and private data, has secured $8.5 million in seed funding in a round led by Greylock. The company’s solution aims to make it easier for developers to leverage the reasoning capabilities of LLMs over their own data. The funding is set to expand LlamaIndex’s open-source data framework and tackle data-related challenges for enterprises at scale.
The recent hype in language models has led to statements like "our model is preferred to ChatGPT N% of the time". This statement usually hides the fact that GPT4 was the model doing the preferring. Wrapped up in that is all sorts of questions about calibration, reliability, and the plain fact that GPT4 prefers models that output more tokens per response. This blog dives into these details and the Open LLM leaderboard.
Understanding how AI language models react to slight changes in their inputs is vital for ensuring fairness and usefulness, but it's not an easy task. A new method called Contrastive Input Decoding (CID) has been introduced to shed light on these subtle differences by generating text that reflects the unique characteristics of two slightly different inputs, thus making the AI's responses more understandable and manageable.
RL4F is a new method that uses a smaller, critique-generating AI to help a much larger AI like GPT-3 fix its mistakes, much like how humans improve from feedback. By studying different tasks like planning, summarizing, and alphabetizing, researchers found that this approach improved the big AI's performance by about 5% on average.
DataDM is your private data assistant - a conversational interface for your data where you can load, clean, transform, and visualize without a single line of code. DataDM is open source and can be run entirely locally, keeping your data secrets fully private.
This paper introduces FinGPT, an open-source artificial intelligence model designed for the finance sector. It aims to provide an accessible tool for financial research and development, offering features like automated data management and the potential for applications such as robo-advising and algorithmic trading.