Search the archive

APPLE FINALLY LAUNCHES HIGH-YIELD APPLE SAVINGS ACCOUNT

OPENAI’S CEO SAYS THE AGE OF GIANT AI MODELS IS ALREADY OVER

OpenAI CEO Sam Altman says that the research strategy that birthed ChatGPT is played out and that further progress will not come from making models bigger. There are diminishing returns on scaling up model size and physical limits on how many data centers the company can build and maintain. It is unclear what research strategies or techniques OpenAI will use instead for its next-generation models. New model designs, architectures, and further tuning based on human feedback are promising directions that many researchers are already exploring.

THE FUTURE OF FERTILITY

Some scientists believe that sex may no longer be the method by which most people will make babies within the next few decades. While life expectancy has increased over the century, the age at which women lose their fertility has stayed the same. As populations are aging and fewer people are having children at a young age, most countries can no longer take a passive approach to fertility issues. This article takes a deep look at a technology called in-vitro gametogenesis which can create gametes outside of the body using non-reproductive cells.

PRETTY TYPESCRIPT ERRORS

REDPAJAMA-DATA

AI, NIL, AND ZERO TRUST AUTHENTICITY

HEGEMONY

WHAT IS A PROTOCOL AND WHY DOES IT MATTER?

PHONE

ADOBE TEASES GENERATIVE AI VIDEO TOOLS

ALIEN ARTEFACTS

GOOGLE CEO SUNDAR PICHAI WARNS SOCIETY TO BRACE FOR IMPACT OF A.I. ACCELERATION, SAYS ‘IT’S NOT FOR A COMPANY TO DECIDE'

SPACEX SCRUBS FIRST LAUNCH ATTEMPT OF ITS STARSHIP LAUNCH SYSTEM

TRUTH-SEEKING AI’

DATASET RELEASED TO REPRODUCE LLAMA TRAINING

The Llama models from Meta are powerful but unusable for many commercial applications due to licensing restrictions. However, because they’re so good, lots of folks have been building on top of them. It starts to get tricky when people go to deploy them, however, and often it is a complete impossibility. In the Llama paper they describe the training mix proportions which have now been reproduced as a 1.2 Trillion token dataset. Open models are coming soon.

AI READINESS REPORT

56%-72% of companies have accelerated their AI plans because of the recent generative AI progress. Many of these groups prefer to use open source technology, while OpenAI's APIs are the dominant closed models being used. The report outlines industries adopting AI technologies and good things to know to help cut through the hype.

WITH TEXT

Multimodal C4 (mmc4) is a publicly available dataset that augments the text-only c4 corpus with interleaved images, supporting in-context vision and language models like Flamingo. The dataset, created using a linear assignment algorithm with CLIP features, covers everyday topics and contains 103M documents, 585M images, and 43B English tokens, with 90% of images being topically relevant.

INSTRUCTION TUNING WITH GPT-4

Utilizing GPT-4-generated instruction-following data for LLaMA finetuning, researchers achieve superior zero-shot performance on new tasks in both English and Chinese, surpassing previous state-of-the-art models. This breakthrough is accompanied by comprehensive evaluation and reward model training, with the data and codebase made publicly available.

CEREBRAS-GPT: EFFICIENT, SCALABLE LANGUAGE MODELS WITH OPEN COMPUTE

Cerebras-GPT, a family of compute-optimal language models, demonstrates state-of-the-art training efficiency on pre-training and downstream objectives by leveraging advances in efficient pretraining, scaling, and Maximal Update Parameterization. As the first open and reproducible work comparing compute-optimal model scaling, pre-trained models and code are publicly released.

INPAINT ANYTHING

Inpaint Anything (IA) is a new mask-free image inpainting paradigm that combines various models to create a powerful and user-friendly solution for inpainting tasks. IA offers three main features: Remove Anything, Fill Anything, and Replace Anything, allowing users to edit images using clicks and text-based prompts seamlessly.

SOUNDINI: SOUND-GUIDED DIFFUSION FOR NATURAL VIDEO EDITING

MULTILINGUAL TEXT2IMAGE MODEL

A nice new text to image model that can handle multiple languages. The output is high quality as it was trained on LAION HighRes and an internal proprietary dataset of images. It doesn’t work with diffusers, but they have an online demo. It seems to have more parameters than stable diffusion but might be smaller than the new XL model in the works.

DATA SCIENTISTS CLONES HIS GROUP CHAT

DOLLY IS A “REALLY BIG DEAL”

HYPE GROWS OVER “AUTONOMOUS” AI AGENTS

AI FUNCTIONS

An easy-to-use implementation of AI functions using OpenAI's GPT-4 (or any other model version) to perform various tasks.

DOMINANCE

ETHGPT: A CHATBOT FOR ETHEREUM CODEBASE AND DOCS

ETHGPT is a GPT-powered ChatBot designed to provide accurate Ethereum information. It compiles resources from various sources, including go-ethereum and ethresear.ch, to keep users updated on the latest developments. By generating documentation and incorporating resources, ETHGPT streamlines access to Ethereum-related information for users and developers. The ChatBot overcomes limitations of ChatGPT's knowledge, which is only up-to-date through September 2021, by using new data and resources to provide accurate answers on recent Ethereum updates.

GATING DAPPS USING ANY ON-CHAIN DATA OR TRANSACTION

TOP CRYPTO PODCASTS TO KEEP UP WITH ALL THE ALPHA

SPOTTING CRYTPO PUMPS LED BY KOREA EARLY ON

GROWTH IN EUROPE

CRYPTO OVERREACH

BREAKING DOWN ONE OF SOLANA’S MOST HYPED GAMES FOR ITS LAUNCH

CONCERNS

BANK OF ENGLAND OFFICIAL SAYS STABLECOIN USE MAY NEED LIMITS

Bank of England Deputy Governor Jon Cunliffe suggested regulators may need to limit the use of stablecoins for payments to maintain financial stability. He claimed that rapid innovation in payment systems could introduce risks for customers and financial markets, and stablecoins currently fall outside of existing regulatory frameworks.

CLOSURES

DINOV2: STATE-OF-THE-ART COMPUTER VISION MODELS WITH SELF-SUPERVISED LEARNING

Meta AI has built DINOv2, a new method for training high-performance computer vision models using self-supervision. DINOv2 delivers strong performance and does not require fine-tuning, making it suitable for use as a backbone for many different computer vision tasks. DINOv2 can learn from any collection of images and also learn features, such as depth estimation, that the current standard approach cannot. Meta has released a demo and open-sourced the model on GitHub.

KV.JS

KV.JS is a fast, in-memory data store written in pure JavaScript, heavily inspired by Redis. It is capable of handling multiple data types, including strings, lists, sets, sorted sets, hashes, and geospatial indexes.

KUBERNETES CHATGPT BOT

A ChatGPT bot for Kubernetes issues. Ask AI how to solve your Prometheus alerts, get pithy responses.

A FORTY-YEAR CAREER

A career usually lasts 40 years, but that’s not always thought about in Silicon Valley’s entrepreneurial culture. What does a career look like if focused on growth and engagement, and if eras were measured not in equity and IPOs but instead in decades? The author argues for focusing on a small handful of things that build together, with each making the others more impactful as they compound over time. These things are pace, people, prestige, profit, and learning.

SOME THOUGHTS ON OPENSSH VERSUS SSH

OpenSSH is the dominant SSH implementation, but not the only one. While SSH is a protocol with multiple implementations, OpenSSH is the main project moving the protocol forward with its extensions for certificates. Other implementations still matter, as they are used in important and relevant places, even if OpenSSH is the only one evolving the protocol.

HOW THE METAVERSE CAN TRANSFORM EDUCATION

REACT CONTEXT: THE DETAILED GUIDE

Context can be used as an alternative to passing data using props. This guide gives you all the information you need to understand what React Context is and how you can implement it in your React code.

BROWSER, AND IT’S VERY IMPRESSIVE

Web LLM has been a hit since its debut. It allows you to run LLMs locally in your browser, instead of relying on an API. Simon Willison tries it out with many examples in this blog and expresses his thoughts on why Web LLM matters a lot.

ENGINE ON GALAXY PHONES

REDPAJAMA, A PROJECT TO CREATE LEADING OPEN-SOURCE MODELS, STARTS BY REPRODUCING LLAMA TRAINING DATASET OF OVER 1.2 TRILLION TOKENS

Foundation models such as GPT-4 have driven rapid improvement in AI. However, the most powerful models are closed commercial models or only partially open. RedPajama is a project to create a set of leading, fully open-source models. They have announced the completion of the first step of this project: the reproduction of the LLaMA training dataset of over 1.2 trillion tokens.