Modern artificial neural networks are impressive: they recognize images, understand language and even write code. But, as a rule, they are trained according to rules invented by humans. Neuroscientists from the Massachusetts Institute of Technology (MIT) asked the question: is it possible to build a network that organizes its own learning, without an external "conductor"? The results were published in the journal PNAS.
In ordinary neural networks, an engineer specifies how to change the strength of connections between artificial neurons in response to errors. This works, but it hardly resembles how a living brain learns. "The brain has no external designer," explains Qianli Liao, one of the authors of the work. "It must organize itself."
The researchers created a network consisting of several layers of randomly connected neuron-like elements. Each element obeys the same simple rules: for example, if two connected neurons fire simultaneously, the connection between them strengthens - a principle known as "neurons that fire together, wire together." The network receives no other instructions.
It turned out that such a self-organizing system is capable of learning to recognize patterns and classify images. During learning, stable structures arise in it - motifs that serve as building blocks for functional circuits. These motifs are rearranged using an optimization strategy close to gradient descent - a method in which connections are gradually adjusted taking errors into account. In most modern AI, gradient descent is implemented through the backpropagation algorithm. However, neuroscientists believe that the brain is unlikely to use this algorithm.
The team compared its self-organizing network with networks trained using backpropagation and found that they are comparable in effectiveness. This does not prove that the brain learns in the same way, but it offers a plausible biological alternative.
"If the synaptic motifs we propose are indeed used by the brain for what backpropagation does in artificial networks, then we will obtain a strong link at the level of fundamental mechanisms between machines and the brain," says Professor Tomaso Poggio.
The authors emphasize: self-organization is only the first step toward biological plausibility of models. They do not claim that the network learns exactly like a living brain, but they believe that the capacity for self-organization should become an important criterion in evaluating neural networks. Understanding these principles may also be useful for creating neural interfaces - systems that will help restore speech or control prostheses.
Although the scientists' main goal is to better understand how the brain works, the knowledge gained may in the future also improve machine learning, making artificial intelligence more efficient and more human-like.