Researchers have improved deep learning by selecting the most efficient overall path to the output, leading to a more effective AI without added layers.
Like climbing a mountain via the shortest possible path, improving classification tasks can be achieved by choosing the most influential path to the output, and not just by learning with deeper networks.
Deep Learning (DL) performs classification tasks using a series of layers. To effectively execute these tasks, local decisions are performed progressively along the layers. But can we perform an all-encompassing decision by choosing the most influential path to the output rather than performing these decisions locally?
In an article published today (August 31) in the journal Scientific Reports, researchers from Bar-Ilan University in Israel answer this question with a resounding “yes.” Pre-existing deep architectures have been improved by updating the most influential paths to the output.
Like climbing a mountain via the shortest possible path, improving classification tasks can be achieved by training the most influential path to the output, and not just by learning with deeper networks. Credit: Prof. Ido Kanter, Bar-Ilan University
Analogy and Implications
“One can think of it as two children who wish to climb a mountain with many twists and turns. One of them chooses the fastest local route at every intersection while the other uses binoculars to see the entire path ahead and picks the shortest and most significant route, just like Google Maps or Waze. The first child might get a head start, but the second will end up winning,” said Prof. Ido Kanter, of Bar-Ilan’s Department of Physics and Gonda (Goldschmied) Multidisciplinary Brain Research Center, who led the research.
“This discovery can pave the way for better enhanced AI learning, by choosing the most significant route to the top,” added Yarden Tzach, a PhD student and one of the key contributors to this work.
Bridging Biology and Machine Learning
This exploration of a deeper comprehension of AI systems by Prof. Kanter and his experimental research team, led by Dr. Roni Vardi, aims to bridge between the biological world and machine learning, thereby creating an improved, advanced AI system. To date they have discovered evidence for efficient dendritic adaptation using neuronal cultures, as well as how to implement those findings in machine learning, showing how shallow networks can compete with deep ones, and finding the mechanism underlying successful deep learning.
Enhancing existing architectures using global decisions can pave the way for improved AI, which can improve its classification tasks without the need for additional layers.
Reference: “Enhancing the accuracies by performing pooling decisions adjacent to the output layer” 31 August 2023, Scientific Reports.
DOI: 10.1038/s41598-023-40566-y