GATEON: A NEW METHOD FOR BETTER CONTINUAL LEARNING

Summary from the TLDR AI newsletter

GateON is a new technique that helps neural networks learn a series of tasks without forgetting previous ones, a common problem known as 'catastrophic forgetting'. It achieves this by creating overlapping connections between tasks and protecting important knowledge from being overwritten. GateOn has a mechanism to re-activate 'fixed' neurons, making it a powerful tool for large-scale learning tasks, whether it's identifying hand-written numbers or understanding natural language.