Parse notes
Sponsored
Natural language interfaces can simplify data exploration. But the data needs to be accessible and trustworthy — which is where the semantic layer comes in. Join this AtScale Tech Talk
Meta has released Code Llama, a Llama 2-based large language model for coding. The 7B, 13B, and 34B parameter models are now available. They performed strongly on HumanEval (53.7) and MBPP (56.2). Code Llama models can generate and debug code, are instruction-tuned, and have extremely long context - as much as 100k tokens.
Hugging Face has raised $235 million in a Series D funding round, with participation from Google, Amazon, Nvidia, Intel, AMD, Qualcomm, IBM, Salesforce, and Sound Ventures. The startup is now valued at $4.5 billion.
The White House, in collaboration with tech giants like Google and OpenAI, has initiated the "AI Cyber Challenge" to develop AI that can spot and fix software vulnerabilities. The competition is offering $20 million in prizes. Winning teams will be encouraged to open-source their solutions.
A pretty large and collaborative effort at constructing a benchmark for measuring language models' ability to reason in a legal framework. Watch out for contamination in already trained models.
Storytelling via automatic video generation is an emerging area of research. The tasks of story continuation, story remixing, and others are tough to measure. Hopefully, this great new Google benchmark helps the field make progress.
Most computer vision tools have trouble tracking 3D objects using LiDAR points because of distractions or not noticing long-term movements. MTM-Tracker mixes two methods and works in two stages to solve these problems.
Graphcore released an excellent Pytorch library that makes training in fp8 super easy and stable. A single model wrapper and torch compile is all you need.
Instead of using big and resource-heavy AI models, Prompt2Model lets you describe what you want in simple words and then makes a smaller, efficient AI model for that task. It can make models that work better than some popular big models.
Recent models often struggle with complex vision-language tasks due to limitations in understanding mixed image-text context. The I4 benchmark was introduced to evaluate these tasks. Results showed a flaw in the Visual Prompt Generator's attention. To fix this, researchers developed Cheetah, a model with a unique module and training strategy that excels in understanding complex interleaved instructions. Cheetah achieved top performance on the I4 benchmark.
Human feedback is key to improving language model alignment and overall performance. However, the process of training with Proximal Policy Optimization has some challenges. Recent work showed that you can optimize directly against human preference and bypass the reward model. With Text-based Reinforcement Learning, you can do this on state-of-the-art open models.
An exploration into why the current crop of LLMs are unlikely to lead to AGI and why physics itself will prevent any sudden leaps from where we are now to AGI.
While AI is very good, it’s not yet good enough to provide the automation that the economy needs.
A cool tool that automates the new gpt-3.5-turbo tuning processes with local files from disk.
A web app for training Machine Learning pipelines with MLJAR AutoML.