Full-text search across article titles and summaries. Refine with sector, year, content type, and reading-time filters.
Data efficiency is one of the challenges of many reinforcement learning algorithms. It usually takes millions of environmental interactions before the agent starts to learn. This is often infeasible with real world robotic systems. This work is one example of a system that is more data efficient.
The paper proposes PixMIM, a simple and effective method for Masked Image Modeling (MIM) that addresses two previously overlooked bottlenecks. PixMIM filters high-frequency components from the reconstruction target and adopts a conservative data transform strategy to alleviate the problem of missing foreground in MIM training. The method can be easily integrated into most existing pixel-based MIM approaches, improving three MIM approaches consistently across various downstream tasks without additional computation. PixMIM can serve as a strong baseline for self-supervised learning and provide insights for future improvements of the MIM framework.
By taking advantage of the new Apple neural computing system, you can run the llama models on m1 and m2 computers. You can even get many tokens per second for the smallest model. This points at an exciting future of personalized AI running on consumer grade hardware.
Powered by the above code, this project packages things up nicely and allows you to install and run the llama models with familiar web tools like npx.
Parameter efficient fine tuning (PEFT) is a way to update only a small fraction of the weights of a model. Reinforcement learning from human feedback (RLHF) is a similar step to instruction tuning that improves model generation quality and model alignment. By combining the two, you can tune large models up to 20 billion parameters on a relatively small GPU with 24GB of on chip memory. This process would usually require multiple A100s.
This article dives into how Large Language Models are exploding in innovation and popularity similar to the explosion in generative AI following Stable Diffusions release in August 2022.
This GitHub Repo is a mix of machine learning guides and tools.
Dune has launched DuneSQL, an attempt to make the fastest, most powerful and convenient crypto data querying experience on the planet. It features many Dune-specific improvements such as using other queries as views and querying byte arrays directly. Postgres-based v1 platform is almost fully sunset and Spark SQL will be sunset in July 2023. Wizards can use an automated tool to help make query changes and get help from the team if they are having difficulties. There is also a tool that will help Wizards migrate their existing queries to DuneSQL.
In this article, Arthur Hayes discusses a new design for a stablecoin called Satoshi Nakamoto Dollar (NUSD). The goal of the NUSD is to create a token that is worth 1 USD but does not require the services of the fiat banking system. It works by depositing 1 BTC on a derivatives exchange and shorting one XBTUSD swap. The Bitcoin value of the swap increases exponentially as the USD price of Bitcoin falls, and approaches zero as the USD price of Bitcoin rises. In theory, each side of the pair should equally offset the other. There are some risks that come into play, and there are still aspects of centralization, but the article attempts to provide some solutions to those potential problems.
The concept of Community-as-a-Service (CaaS) focuses on providing access to a particular group of people that can be considered a marketable product. Examples of CaaS include Mobalytics for League of Legends, Aim Trainers for FPS games, and mods for modern video games. In DeFi, Convex was created to serve the community using Curve.fi, and Delegate.cash services NFT holders. CaaS differs from the current competition-driven crypto landscape, as its goal is to serve the community of an established project, like Tribally did for Axie Infinity by building a new website. The article posits that more value will be able to be created by catering to specific communities as crypto continues to mature rather than attempting to disrupt the status quo.
Sponsored
Fervo Energy claims it can create flexible geothermal power plants that can increase or decrease their electricity output as needed. The power plants would serve as a power source in periods of low production and act as a battery. While there are still outstanding questions about whether the system is scalable, affordable, and entirely safe, the technology offers a way to make it cheaper and easier to eliminate greenhouse-gas emissions from electricity systems.
UpTrain is a tool that aids machine learning (ML) practitioners in observing and improving ML models for optimal performance. It is a data-secure tool that easily integrates with existing production pipelines. UpTrain's features include data drift checks, data integrity checks, customizable metrics, real-time dashboards to monitor the model's health, edge case signals, AI explainability, and special dashboards to understand model-inferred embeddings. Users can get alerts on Slack when using UpTrain.
Trance Stack is a Remix stack built for AWS Lambda, providing various features such as authentication, security, internationalization, feature flags, and analytics. It is intended to serve as a starting point for Remix projects. The stack offers production-ready Remix applications and allows for customization and modification in accordance with users' preferences.
Sponsored
Tech companies have turned to mass layoffs in order to make cuts. Many employees are at risk of losing their jobs due to the failure of long-term projects. Unlike previous years, employees can no longer slip into another position at their company if a project fails. Employees are now much more vulnerable to layoffs. Technology projects that were previously championed by former CEOs, but may be years from profitability, are the most affected by layoffs in recent months.
Rabois says that Google and Meta brought on too many staff to fulfill the vanity metric of hiring.
Statisticians observed a surprising increase in the number of babies born last year.