Parse notes
Language models take text queries as input and output text responses. The length of both query and output are referred to as the context. Language models have limited context length (ChatGPT is 8k 'words'). With some cool recent algorithmic changes, MosaicML has released a model that can operate in 65k 'words'. This is enough to fit the entire novel Great Gatsby in context and have the model write an epilogue.
We live in a somewhat divisive world and it's hard to have meaningful conversations where understanding is fostered. Interestingly, if you have an AI chatbot intermediate conversations about difficult topics - both parties report improved outcomes and understanding.
Let's say we have two different models of the same architecture trained on different tasks. Is it possible to merge them into a single model that works well on both tasks? Current methods like rebasin or model soup don't work because of the presence of different task-specific features. This new work proposes a way forward with the two step process which improves as much as 60% over previous algorithms.
OpenAI has quietly been working on text to 3D with a few releases. This new work is a fun addition to that line of research. It works fairly well in standard objects or objects with a few compositional ideas. This model can also create 3D objects from 2D images, although that doesn't seem to work quite as well.
Encoding long chunks of text is a tough task for natural language processing (NLP) models. This study improves the process by using graphs and a new attention mechanism, which helps the model understand relationships between sentences and perform better on longer sequences.
Today's language models can still make silly mistakes when it comes to commonsense knowledge. VERA is a new model that estimates the plausibility of statements based on commonsense, helping to filter out incorrect information and improve performance in real-world settings.
This study introduces a new method, called MIMIC-IT, for improving how models like Otter understand and follow instructions in different situations, including images and text. By making these models more accessible, researchers can use them more easily to create better AI systems.
Sponsored