Essential biology and electrochemistry for Deep Learning

Some basic statistics of human brain

The human brain is a wrinkled soft wet organ. Made up of more than 100 billion neuron cells and over 10 times more of other types of cells (glial cells), is the centre of all thought. Enclosed in the skull, it has the same general structure as the brains of other mammals but is over three times as large as the brain of a typical mammal with an equivalent body size.

The brain along with the spinal cord forms the central nervous system. Weighing about 3 pounds (1.4 kilograms for humans), it consists of three main structures: the cerebrum, the cerebellum and the brainstem. Human brain weighs approximately 2% of the body weight but consumes 20% (in adults) of the body’s oxygen. It can stay alive only for 4 to 6 minutes without oxygen. Approximately 20% of the blood flowing from the heart is pumped to the brain. There are 100,000 miles of blood vessels in the brain. Your brain is 60% white matter and 40% gray matter. The brain's gray matter is made up of neurons and white matter is made up of dendrites and axons. Humans have the largest brain to body weight ratio. Water constitutes 75% of brain.

Cerebrum and Cerebellum

The cerebrum is a large part (85% in weight) of the brain containing the cerebral cortex (of the two cerebral hemispheres), as well as several subcortical structures, including the hippocampus, basal ganglia, and olfactory bulb. In the human brain, the cerebrum is the uppermost region of the central nervous system.

 

Figure-1:

Brain Anatomy

 

Figure-1 illustrates an exposed view of the human brain. 

The cerebellum ("little brain") is a major feature of the hindbrain of all vertebrates. In humans, the cerebellum plays an important role in motor control, and it may also be involved in some cognitive functions such as attention and language as well as in regulating fear and pleasure responses. Its movement-related functions are the most solidly established. The human cerebellum does not initiate movement, but contributes to coordination, precision, and accurate timing.  Although usually smaller than the cerebrum, in some animals it may be as large as or even larger.

The cerebrum and cerebellum have thin outer surface layers of gray matter and larger inner regions of white matter.

Figure-2:

 

 

The cerebrum is covered by a continuous layer of gray matter that wraps around either side of the forebrain—the cerebral cortex. The grey matter contains most of the neuronal cell bodies or soma. While the grey matter is mainly located on the surface of the brain, white matter is found buried in the inner layer of the brain's cortex. White matter areas of the brain mainly consist of myelinated axons, which are long relays that extend out from the soma,

The outer surface is wrinkled and consists of ridges (gyri) and valleys (sulci).  The grey matter cortex has a layer thickness of 3-4 mm. Due to the wrinkles on the surface the cortical area is many time larger than that of the skull. When unfolded the surface area of each hemisphere is approximately 1.3 sq.ft.    

 
Neuron and Glial Cells.

Although extremely complex, the brain is largely made up of only two principal cell types: neurons and glial   cells (which serve to support and protect the neurons). There are over 100 billion neurons in the brain and an even greater number of glial cells. The glial to neuron ratio is 10:1. Each neuron may be connected to up to 10,000 other neurons, passing signals to each other via as many as 1,000 trillion synaptic connections, equivalent by some estimates to a computer with a 1 trillion bit per second processor. Estimates of the human brain’s memory capacity is close to 2.5 petabytes (1 petabytes = 1024 terabytes). Newer estimates indicate even higher capacity.

Neurons

Neurons are the basic units used for computations in the brain. They are involved in receiving, processing and transmitting information through their highly specialized structure. Neurons consist of a cell body and two types of projections – the dendrites and an axon.

Neurons use their highly specialized structure to both send and receive signals. Individual neurons receive information from thousands of other neurons, and in turn send information to large number of neurons connected to its axon terminals. Information is passed from one neuron to another via neurotransmission. This is an indirect process that takes place in the area between the nerve ending (nerve terminal) and the next cell body. This area is called the synaptic cleft or synapse. Most neurons have many dendrites, but only one axon.

 
Physiology of a Biological Neuron.

Neurons are cells that have a nucleus and the related cellular metabolic apparatus. They are the information-processing/computational cells of the brain. The generic neuron is modeled after spinal motor neurons. The image of a biological neuron seen through an electron microscope is shown below.

 

Figure-3:

Neuron through Scanning Electron Microscope

 

Electron micrograph of a motor neuron. S = soma, or cell body; P = podite or cell extension; D = dendrite; arrow is at the axon

 

 

Neurons exists in various shapes and sizes. Some neurons are as small as red blood cells (6-8 μm), others can be larger and long approximately a meter in length.

 

Figure-4:

Neuron supported by Glial cells

 

 

Each neuron has four basic elements

1. A  soma (cell body) which contains the cell’s nucleus and other vital components called organelles that perform specialized tasks.
2. A set of dendrites which form a tree like structure spreads out from the cell body. The neuron receives its input electrical signals from the dendrites. A dendrite starts out from the soma and then form a very dense structure as the distance increases by repeatedly branching. Dendrites may emerge from different regions of the soma.
3. A single axon which is a tubular extension from the cell soma, carries an electrical signal away from the soma to other neurons. Axons may repeatedly branch to form an axonal tree. It has diameter of appr. 1μm, but length varies from a few millimetres to as long a meter. When axons reach their final destination, they branch again. This is called terminal arborization.
4. At the ends, specialized structures called synapses connects the axon to the dendrites of other neurons or muscles. Synapses are the "places" or "gaps" where neurons communicate with another. Synapses are classified in two, electrical and chemical which depends up on the mechanism involved in the transmission of signals from presynaptic terminal to post synaptic terminal.

Figure-5:

Synaptic cleft.

 

 

Figure-6:

Interconnected neurons

 

 

Cell Electrochemistry

 

All animal cells are surrounded by a plasma membrane composed of a lipid bilayer. The plasma membrane possesses remarkable properties. It encloses the neuron and has a two layered structure about 90 angstroms thick (1angstrom = 10^-10meter). This membrane is primarily composed of lipids and protein molecules. Each molecule has a hydrophilic head and hydrophobic tail. The hydrophilic head face towards the surrounding water medium and hydrophobic tails are oriented away from the water. The intracellular and extracellular medium contains water molecules. Therefore lipids molecules are arranged in a bilayer membrane in which the proteins embed themselves. The tail parts of lipid molecules are oriented away from the aqueous medium. The lipid component of the membrane is a diffusion barrier.

 

Figure-7:

Membrane activity

 

 

The membranes have continuous passages or pores through which mobile ions can flow in and out. However the pores can change their conformation (properties) under either electrical or chemical control so that the ion flow through them can be regulated. That is, the permeability of the membrane is controlled by the electrical and chemical environment. The fluid (aqueous medium) on both sides of the membrane contains large number of mobile ions; of which sodium (Na⁺), potassium (K⁺), chloride (Cl^–), and calcium (Ca²⁺) are the most important. The electrical potential of both mediums is mainly controlled by the number of Na⁺ ions. The concentration of Na+ ions is 10 times stronger outside the cell than that at the inside. Therefore the inside medium is electrically more negative than the outside medium.

 

Resting Potential

Due to the difference in the concentration of Na+ ions, when a neuron is at rest the potential of the medium inside the cell body is at appr. -70mV w.r.t the external medium. This causes an electrical field across the membrane of the order of 100,000 V per cm. At the resting potential the ion channels are closed and hence cell membrane of the neuron is impenetrable for the Na⁺ ions.

 

Figure-8:

 

Reasons for a membrane’s resting potential:

The membrane has protein (or enzyme) channels, or gaps, which forms a transmembrane pump which are closed.

The Sodium-Potassium pumps use energy-storing molecules called adenosine triphosphate (ATP). ATP actively pumps 3 Na⁺ ions out of the cell, at the same time pumping 2 K⁺ into the cell. After a while, a ionic concentration gradient is generated across the membrane, whereby more Na⁺ ions are outside and more K⁺ are inside

Because of diffusion, the tendency is for Na⁺ ions to travel back to the inside, and vice versa for K⁺ ions

Figure-9:

Sodium and Potassium diffusion

There are nongated Na⁺ and K⁺ channels in the membrane that permit the passage of some Na⁺ ions back into the neuron, and K⁺ ions out of the neuron (using diffusion to achieve a concentration equilibrium), however, the membrane is not very permeable to Na⁺ ions. Ions whose size and charge “fit” the pore can diffuse through it, allowing these proteins to serve as ion channels. Hence many more K⁺ ions leave the cell than Na⁺ ions enter. This causes an excess of negative charge inside the cell.

The K⁺ ions continue to leak out until there is an equilibrium reached between the concentration gradient and the electric potential gradient (i.e., the attraction of K⁺ positive ions back to the negatively charged intracellular fluid).

At equilibrium there is no ion exchange through the membrane it is in a state of equilibrium or rest.  The voltage difference reaches a value of –70 mV.

Firing of the neuron

Every neuron constantly receives inputs from other neurons along their dendrites at points of contact (synapses). Inputs from other neurons can raise the interior potential of the neuron.  These inputs take the form of low voltage electrical disturbances called postsynaptic potentials (PSPs). The low voltage PSPs occur asynchronously in space and time at various points along the dendrites. The postsynaptic potentials can be positive or negative voltage pulses. The cell body (soma) sums up the signals from the numerous synapses. Therefore the cell potential keeps varying in time.

 

At around a threshold potential difference of -50 mV the cell membrane suddenly loses its impenetrability. Opening of ion channels at any point lets the Na⁺ ions to flow inside through the membrane. This causes electric current to flow rapidly to other points in the membrane producing a local change in the membrane potential, and a sudden enormous increase in voltage across the membrane occurs. The potential difference becomes positive by tens of millivolts for about half a millisecond.

 

This sudden change of potential difference called action potential or spike causes the neuron to fire. An action potential is an electric pulse that travels down the axon until it reaches the synapses, where it then causes the release of neurotransmitters. During this short-lasting action potential, the membrane potential very rapidly undergoes a large reversal of its sign and drops backs to appr. -70mV. A series of potential spikes are propagated to the output of the neuron in the form of ion currents. This electric current is transmitted down its axon to other neurons. 

 

 

   

Figure-10:

 

 

The firing frequency of a neuron varies from 50 to 100 pulses per second.  A weak stimulus will cause a lower rate of fire than a strong stimulus. It is not the amplitude of the action potential that is important, but the number of times a neuron fires for a given time period. However, it has been shown in experiments that the rate of fire of a neuron is directly related to the depolarizing current applied to that neuron. This fact will be important for explaining the neural model.

 

 

 Figure-11:

Important features of the neuronal action potential.

Different phases of the neuronal action potential are emphasized in this figure. 0, resting state; 1, depolarization to threshold and beyond; 2, overshoot; 3, peak of the action potential; 4, repolarization; and 5, hyperpolarization.

 
Neuron’s Learning method

Input connections that receives higher frequency of pulses are strengthened and those with lesser frequencies are weakened. The strength of input connections gets continuously modified. This method of continuous modification of connection strengths results in the complex process of learning.

Natural Intelligence of Brain vs Machine Intelligence

Neuroscientists have estimated that the human brain is made of approximately 100 trillion synapses (connections) and is therefore massively parallel. All mammalian brains (wetware processors) are massively parallel. The structure of information processing is different compared to that of powerful hardware processors on which machine intelligence is implemented.   Despite the similarity in names the components of natural intelligence and artificial intelligence of machines differs in many aspects.

Potential of Human Brain

A lone biological neuron keeps receiving large of neurotransmitter signals through its dendrites. From time to time, it spontaneously unleashes a wave of electric current that travels down its axon. The signals appear as spikes of voltages at its output. The neuron may respond with extra output spikes of voltage if inputs are frequent. With various neurotransmitters the strength and timing of its electrical waves can be altered. On its own, the single biological neuron can’t do much. But join together 302 neurons, and they become a nervous system that can keep the 1 mm transparent nematode worm Caenorhabditis elegans alive, sensing the animal’s surroundings, making decisions and issuing commands to the worm’s body.

 

Figure-12:

C. Elegan worm

 

 

Join together 100 billion neurons—with 100 trillion connections—you have a human brain, which should be capable of much, much and even much more. Artificial systems are nowhere comparable to this capability of natural biological world.

Robustness in presence of noise and uncertainty

A two-year toddler would be able to recognize a cat’s image on a piece of crumpled paper, Natural brains are robust in presence of noise, uncertainty and sparse data. Humans routinely, and often effortlessly, sort through probabilities and act on the likeliest, even with relatively little prior experience.

Figure-13:

Broken cookies

 
AI is data hungry

As of the current state of the art machine learning and artificial intelligence algorithms are data hungry and needs large amounts of data to derive reasonably accurately inferences and results. Consider the control of highly complex combustion process in gas turbines, where air and gas flow into a chamber, ignite, and burn at temperatures as high as 1,600 degrees Celsius. The volume of emissions created and ultimately how long the turbine will continue to operate depends on the interplay of numerous factors, from the quality of the gas to air flow and internal and external temperature. Using bottom-up machine learning methods, the gas turbine would have to run for a century before producing enough data to begin training a monitoring system that resulted makes fine adjustments that optimize how the turbines run in terms of emissions and wear, continuously seeking the best solution in real time, much like an expert knowledgeably twirling multiple knobs in concert.

Machines are now being taught to mimic such reasoning through the application of Gaussian processes—probabilistic models that can deal with extensive uncertainty, act on sparse data, and learn from experience. 

Wetware and Hardware

The architecture of brain and standard hardware processors are different. In the following figure an electric pulse information is sent from neuron-1 to neuron-2. The synapse at neuron-2 evaluates the signal using its own previous state and its synaptic strength (weight) with neuron-1. Two pieces of information flow into neuron-2 body through its dendrites. Due to the electrochemical properties when inputs are received by the time they reach the neuron soma computations are complete and the soma receives a signal value.   Storage of weight values and computations happen at the same time and in the same place.

Figure-14:

Hardware vs Wetware

In contrast in a hardware processor synaptic the strengths (weight values) of all neuron synapses are stored in memory. To change the state of the synapse a signal must be received by the processor from some input, the memory location of the synaptic weight must be accessed and transferred to processor to perform the computation and stored back at its appropriate location.

Reconfiguring and Redundancy

The natural brain has remarkable capability to recover from physical problems, reconfiguring its connections to adapt after an injury or a stroke.  It can continually adapt to changing environmental demands. Such adaptation spans multiple time scales, from seconds during task performance to days and weeks during motor or cognitive training. The brain is so impressive that patients with severe medical conditions can have a major of their brain damaged but recover normal cognitive and physical functions. Compare this with a  computer with as much its circuits removed.

 

 Figure-15:

Cerebral Infarction – CT scan

Millions of neurons drive the activity of hundreds of muscles, meaning many different neural population activity patterns could generate the same movement. Studies have suggested that these redundant (i.e. behaviourally equivalent) activity patterns may be beneficial for neural computation. However, it is unknown what constraints may limit the selection of different redundant activity patterns. When you swing a tennis racket, muscles in your arm contract in a specific sequence. For this to happen, millions of neurons in your brain and spinal cord must fire to make those muscles contract. If you swing the racket a second time, the same muscles in your arm will contract again. But the firing pattern of the underlying neurons will probably be different. This phenomenon, in which different patterns of neural activity generate the same outcome, is called neural redundancy. Neural redundancy allows a set of neurons to perform multiple tasks at once. For example, the same neurons may drive an arm movement while simultaneously planning the next activity.

Power efficiency

Even the most intelligence human brain would require around 20W of power to function which is astonishingly small compared to 1 MW power of IBM Deep Blue that defeated world chess champion Garry Kasparov. 

 

References:

1. Neural Networks and Learning Machines, 2^nd Edition, Simon O. Haykin, McMaster University, Ontario Canada
2. The central nervous system, opentextbc.ca
3. Neuroscience and Artificial Intelligence can help improve each other theconversation.com
4. 100 trillion connections join together 100 billion neurons, capable of much much more, scientificamerican.com
5. MoNETA: A Mind Made from Memristors, DARPA’s new memristor-based approach to AI consists of a chip that mimics how neurons process information, By Massimiliano Versace and Ben Chandler, IEEE Spectrum, Nov 2010
6. The future of ai will be about less data not more, hbr.org
7. Artificial_and_Biological_Intelligence_Hardware_vs_Wetware, Piero Chiarelli., Frontiers in Cognitive Psychology, researchgate.net.
8. 11-times-ai-beat-humans-at-games-art-law-and-everything-in-between, interestingengineering.com/
9. nature.com/articles/d41586-019-02212-4
10. disabled-world.com/health/neurology/brain
11. Electrical signaling by neurons, neupsykey.com
12. Overview of neuron structure and function, khanacademy.org
13. Neuronal action potential important features, physiologyweb.com
14. Constraints on neural redundancy, Jay A Hennig, Matthew D Golub, Peter J Lund, Patrick T Sadtler,  Emily R Oby, Kristin M Quick, Stephen I Ryu, Elizabeth C Tyler-Kabara,  Aaron P Batista, Byron M Yu, Steven M Chase, eLife 2018;7:e36774 DOI: 10.7554/eLife.36774

 
Image Credits:

Figure-1: image.slidesharecdn.com

Figure-2: wikimedia.org

Figure-3: faculty.fortlewis.edu

Figure-4: ninds.nih.gov

Figure-5: eprojects.isucomm.iastate.edu

Figure-6: familyphysiotherapy.com

Figure-7: wikimedia.org

Figure-8: Reference -1

Figure-9: wikimedia.org

Figure-10: Reference -1

Figure-11: physiologyweb.com

Figure-12: images.theconversation.com

Figure-13: st.depositphotos.com

Figure-14: spectrum.ieee.org

Figure-15: myheart.net