The History of Neural Networks: Three Neuro revolutions

Did you know that the principle of neural networks was already formulated over the last century?

Even before that, they started to explore the capabilities of machines and compare these capabilities to the human brain: Leonardo da Vinci designed a mechanical calculator in 1500! Sure, compared with modern technological achievements, the discoveries from past periods might seem naive and trivial. Nevetheless, without those discoveries, modern innovation would not exist.

The first neural network revolution

For the very first time, the concept of a neural network was put forward in the 20th century by Warren McCulloch and Walter Pitts. Although creating a neural network in the 20th century was virtually impossible because of a lack of technological progress and a better understanding of how the human brain could be replicated and even revolutionized. Just a few years later, Warren's discovery was developed by Donald Hebb, who suggested algorithms for training artificial neurons. However, his model turned out to be far too complex to implement.

As a result, the basis for the work of modern neural networks was a method of learning that required far fewer resources than others: the gradient descent method. This method, for example, was applied in the perceptron, an artificial neuron created by the eminent scientist Frank Rosenblatt.

The Second Neural Network revolution

Even though the discovery of the concept of a neural network was a revolution in itself, at the initial stages, neural network capabilities were far from what they are today. Although the first computers already existed in the 20th century, which limited their true potential: the computers from the past lacked sufficient processing power for neural networks and simply could not handle a large amount of data necessary for their operation.

However, the excitement after discovering the fundamental neural network principles and creating artificial neurons rapidly waned. For many years, scientists were marking time, and the complete application of neural networks seemed like an impossible task straight from the category of science fiction. Nevertheless, scientists did not abandon their efforts to develop this concept by the mid-sixties. In the 20th century, the situation changed dramatically: An idea emerged that it was possible to train individual neurons and entire neural networks simultaneously through the gradient descent method.

In the 1980s, algorithms for training neural networks were developed, which in theory allowed it to solve many problems. Even though, unfortunately, until now, it was only a theory: technical capabilities still prevented the implementation of these solutions in practice.

The third neural network revolution which is still happening today

Eventually, they started to train neural networks using nuclear (kernel) and probabilistic approaches (Bayesian approaches). Finally, and fairly recently, the training of neural networks has reached a whole new level: at the end of the 2000s, the development of technologies finally made it possible to put into practice all the solutions which previously existed only in theory.

Today, the development and implementation of neural networks are affected by several factors, not just the availability of the necessary computing power or data. It is also influenced by mathematical ideas, the popularity of scientific and technological advances in society, and the need for commercial organizations with neural network capabilities. Mainly, the latter factor will determine the principal tasks of modern neural networks.

Now training neural networks enables you to do what scientists from the last century could only dream of: neural networks independently recognize speech or images, certain effects, modify and create content, etc. In addition, neural networks started to play a leading role in the development of artificial intelligence. Thanks to neural networks, it has come close to performing the same professional tasks (for example, writing journalistic texts) humans can perform.

Modern neural networks are more powerful than ever before. Still, it will take a lot of time and effort before they can reach their full potential and turn our lives into Cyberpunk or Blade Runner. For example, the tasks of neural networks that are associated with computer vision, have yet to be solved. However, their progress is still impressive.

In everyday life, modern neural networks can solve almost any daily task: editing photos or videos, voice recognition, producing certain reactions to words, or performing entire algorithms of actions. All of this brings us much closer to solving other ambitious tasks. For example, in Elon Musk's Neuralink project, neural networks let you directly transmit signals from the brain to a computer! What will happen when neural networks become even more advanced is a mystery. The solution to which is worth waiting for.