The most popular types of artificial intelligence are inspired by different biological processes, such as the functioning of nervous systems or adaptive immunity. Here, I make a short dive into the history of AI to track how bio-inspired models of AI replace other models and discuss which model could be the next one.
Today, AI is a hot topic. Since 2024 started, at least three magazines (among those I usually read) have released special issues on AI. The German magazine Spektrum lies before me as a hard copy, while Scientific American and Skeptic are opened in other tabs to browse. Right now, I am writing an article for the fourth AI-focused special issue. It corroborates the feeling that people all around are talking about AI — and this is the case, in some measure. AI now surrounds us embodied in smart devices, recommendation engines and new scientific advances powered by it.
All these discussions keep an air of mystery about AI. Some people adore it, some people are afraid of it but, when you come right down to it, it is based on relatively simple models of very familiar biological processes – and it has stayed on these grounds since its origin.
Neural networks are conquering the world
Neural networks were the first bio-inspired species of AI. Their history began in 1943 when Warren McCulloch and Walter Pitts published the article ‘A logical calculus of the ideas immanent in nervous activity’ in The Bulletin of Mathematical Biophysics. The idea of using artificial neural networks for applied computational tasks was not discussed – it was just the paper on mathematical modelling of the functioning of a single neuron. A little bit later, Donald Hebb formulated its famous principle which was further paraphrased as ‘Neurons that fire together wire together’, which was just the physiological concept at that time, but later became the keystone of teaching artificial neural networks.
The first attempt to use the neural network model of human perception was Perceptron constructed by Frank Rosenblatt in the 1950s. This device could recognize some handwritten symbols, which was a great achievement at that time. This determined the further development of neuronal networks for years – new models were totally bio-inspired, and their key features were inherited from human visual perception.
In the 1960s, David Hubel and Torsten Wiesel started to decipher the structure of the human visual cortex – the part of the brain which transforms the unordered set of blots and dabs we see into recognizable pictures with forms, faces and objects. Due to their studies, we know that the visual cortex has seven zones. In each of them, the neurons fire on the specific patterns in the input from the previous zone, and these patterns are more and more complex with each next zone (Figure 1).
Seven zones of visual cortex and objects recognized in each zone. Image credit:Anastasiia Samoukina/Biomolekula, with the kind permission of the Author and the Publisher.
Seven zones of visual cortex and objects recognized in each zone. Image credit:Anastasiia Samoukina/Biomolekula, with the kind permission of the Author and the Publisher.
Hubel and Wiesel were rewarded with a Nobel Prize in 1981, and a year earlier, Kunihiko Fukushima created Neocognitron, the first computer based on a convolutional neural network. This network scans the image for the local patterns with a small virtual ‘window’, then the next layer scans the result with a similar ‘window’ and this layered scanning enables the network to recognize the patterns like the human visual cortex does (Figure 2). To implement this network algorithmically, computer scientists use a mathematical operation called convolution which has given the name to this family of neural networks.
The principle of work of convolutional neural networks when recognizing images. Image credit:A Convolutional Neural Network for Impact Detection and Characterization of Complex Composite Structures/Sensors/MDPI, CC BY 4.0.
The principle of work of convolutional neural networks when recognizing images. Image credit:A Convolutional Neural Network for Impact Detection and Characterization of Complex Composite Structures/Sensors/MDPI, CC BY 4.0.
This class of neural networks appeared to be promising in a wide variety of pattern recognition tasks and went far beyond image recognition. As a medical translator, I use DeepL when I need a neural network as an assistant – and there is a consensus opinion among translators that it is better than Google Translate, despite DeepL being based on a convolutional neural network, and Google Translate being based on a transformer neural network, the representative of the next network generation.
The point is that no sooner was the first network created than coders had all the essential mathematical and programmatical toolkit required to create new ones. This made further bio-inspiration dispensable and powered the burst of neural networks with quite new structures which cannot be observed in a living brain, such as capsule neural networks, echo state networks and, finally, transformers. For example, recreating a transformer on a hypothetical brain basis would require introducing an astrocyte – a glial cell with just supportive and trophic functions – in a computational scheme and building synapses between the astrocyte and neurons (Figure 3). There is no evidence that such structures really exist but this doesn’t hamper the use of transformers in a computer environment.
There is no way to create a transformer from living neurons only – an astrocyte, a glial cell, is also required. Such computational function is at least not characteristic for astrocytes. Image credit:Building transformers from neurons and astrocytes/PNAS/CC BY 4.0.
There is no way to create a transformer from living neurons only – an astrocyte, a glial cell, is also required. Such computational function is at least not characteristic for astrocytes. Image credit:Building transformers from neurons and astrocytes/PNAS/CC BY 4.0.
The latest type of networks is considered optimal for language processing – but the example of DeepL suggests that the specific type is not crucial. In the world of neural networks, transformers can fold proteins, and convolutional neural networks (inspired by visual cortex) can process speech. But this generalization has its own limits, because all modern neural networks descend from a digital mathematical model of neurons and it is not fully exact and efficient.
Spike transformer’s guns
Modern neural networks are very expensive in terms of computational resources, while our brain is quite economical. The matter is that all neural network models were implemented to ‘live’ on a digital 0-or-1 basis while the human nervous system uses an analogue coding system.
Each neuron fires with an ‘all-or-nothing’ action potential when incited, but the summation of voltages is an analogue process (Figure 4). Any digital-based activation functions appear to be just a crude approximation of a real neuron firing. A recent paper in PNAS by Cramer et al. addresses this problem and describes a chip whose elements can fire with spikes, like real neurons. If the use of such chips becomes widespread, they could be the most bio-inspired computers in history. The computers that mimic principles of nervous system function are usually called neuromorphic, but such spiking chips could be called biomorphic instead – because the spike-like action potential is a common feature in almost all multicellular eukaryotes. Plants and fungi have no nervous system, but their tissues generate action potentials resembling human ones in terms of shape. Thus, new developments could mimic not only neurobiology, but a common cell physiology and biochemistry of life, to say nothing about immunology.
Neuronal action potential appears as a voltage spike. Created with BioRender.com.
Neuronal action potential appears as a voltage spike. Created with BioRender.com.
Artificial immune system: think with lymphocytes
When I am asked to give a definition for intelligence, I usually say that this is the ability to react to the changing environmental conditions by new responses not inherited either genetically or culturally, but I always clarify that this definition must not include adaptive immunity.
This philosophical question shows that adaptive immunity is technically similar to intelligence, but on a new basis. If neurons must wire together, lymphocytes – the cells of adaptive immunity – must undergo clonal selection (Figure 5). This means that only the cells which recognize specific antigens efficiently will proliferate and form a clone with identical type of antigen receptors. This evolution in our own body helps us to solve the most difficult microbial problems. None of our ancestors faced SARS-CoV-2, but after a few years of its existence we have developed a significant immunity to it – after a vaccination or a disease (or both). If lymphocytes are so efficient, why couldn’t AI be based on artificial lymphocytes instead of artificial neurons?
Such a type of AI does exist and is called an artificial immune system. Like their living counterparts, virtual lymphocytes must recognize something presented as an antigen. This principle makes artificial immune systems good antiviruses, anti-spams and classifiers, but this mechanism has nothing to do with generative tasks. This explains why neural networks still dominate – in contrast to artificial immune systems, they could create a picture or a video for you. Or this text for me, for example.
But an artificial immune system described in Frontiers in Human Neuroscience raises a question on the neurological perspective of this type of AI. Scientists have trained an AI to recognize limb movement by electroencephalogram patterns. This opens a very promising prospect of creating new Neuralink-like devices and could potentially change the AI landscape.
Maybe, generating pictures with AI will soon become boring due to AI’s creative limitations, but neurocomputer interfaces are now rapidly developing. What if the next one will be based on an artificial immune system? This will be an interesting case of returning an immunity-inspired AI in biology... but in the sphere of neurobiology! This could make neural networks less popular routine solutions, and we will speak about them as ‘good old-fashioned AI’. This remains to be seen!
Further reading
McCulloch, W. S., and Pitts, W. (1943) A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115-133. doi: 10.1007/BF02478259
Shatz, C.J. (1992) The developing brain. Sci. Am. 267, 60–67. doi: 10.1038/scientificamerican0992-60
Rosenblatt, F. (1957) The Perceptron — a perceiving and recognizing automaton. Technical report 85-460-1. Cornell Aeronautical Laboratory.
Fukushima, K. (1979) 位置ずれに影響されないパターン認識機構の神経回路のモデル --- ネオコグニトロン - [Neural network model for a mechanism of pattern recognition unaffected by shift in position — Neocognitron]. Trans. IECE (in Japanese). J62-A (10): 658–665
Kozachkov, L., Kastanenka, K.V., and Krotov, D. (2023) Building transformers from neurons and astrocytes. Proc. Natl. Acad. Sci.120, e2219150120. doi: 10.1073/pnas.2219150120
Cramer, B., Billaudelle, S., Kanya, S., et al. (2022) Surrogate gradients for analog neuromorphic computing. Proc. Natl. Acad. Sci., 119, e2109194119. doi: 10.1073/pnas.2109194119
Timmis, J. (2011). Artificial Immune Systems. In: Encyclopedia of Machine Learning (Sammut, C., Webb, G.I. (eds)). Springer, Boston, MA. doi: 10.1007/978-0-387-30164-8_33
Rashid, N., Iqbal, J., Mahmood, F., et al. (2018) Artificial immune system–negative selection classification algorithm (NSCA) for four class electroencephalogram (EEG) signals. Front. Hum. Neurosci., 12, 439. doi: 10.3389/fnhum.2018.00439
Whitten, A. (2022) AI Overcomes Stumbling Block on Brain-Inspired Hardware. Quanta Magazine. https://www.quantamagazine.org/ai-overcomes-stumbling-block-on-brain-inspired-hardware-20220217/
Ananthaswamy, A. (2021) Artificial Neural Nets Finally Yield Clues to How Brains Learn. Quanta Magazine. https://www.quantamagazine.org/artificial-neural-nets-finally-yield-clues-to-how-brains-learn-20210218
Author information
Georgy Kurakin, MRSB, is a biochemist and science communicator. His area of expertise covers evolution of multicellularity and cross-kingdom host jumps in pathogenic bacteria. He is a regular contributor to The Skeptic magazine as well as different science media in Russia. Twitter: @KurakinEgor Email: [email protected].