Neural Ne
twor
ks
The human brain is constructed of a very large number (~ 10
11
) of relatively slow and simple processing elements called neurons. The response time of a neuron (i.e., the time between successive signals) is of the order of a tenth to one hundredth of a second. In computing terms this is equivalent to a 'clock speed' of 0.01 to 0.1 kHz, very slow compared with the processor speeds of commonly used personal computers (2, 2.5 and now > 3 GHz). So what is it that makes man so smart? The answer lies in the fact that the brain contains a large number of processing elements which are working all the time; this is parallel processing on a grand scale, and the brain in computing terms is a massively parallel device. The other important feature of biological intelligence is the highly complex 'wiring' which joins the neurons together; a single neuron may be connected to as many as 100,000 other neurons.
So what do these processing elements do? Even a cursory examination of a textbook of neurobiology will show the complexity of the biochemical processes which take place in the brain. Various compounds (neurotransmitters) are involved in the passage of signals between neurons, and the functions of the neurons themselves are regulated by a variety of control processes. Ignoring the complexity of these systems the functions of a neuron can be summarized as follows.
This highly simplified description of how a biological neuron functions may not be a good model for the real thing but it serves as the basis for the construction of ANN. Intelligence in living biological systems appears to reside in the way that neurons are connected together and the 'strength' of these connections. Indeed, the creation of connections and the modification of connection weights is thought to be part of the processes involved in our development, i.e. learning and memory. It may not be clear where in the brain signals arise and which pathways they follow, although for certain regions of the brain, such as the sensory organs, it is more obvious. The eyes, for example, produce nervous signals in response to light and these are passed to the visual cortex. Some preprocessing of the information received by the eyes is carried out by sets of neurons which are organized in particular structures, e.g., layers. It is these three functions of biological neurons and their physical organization and connectivity which forms the basis of the construction of ANN.
ANN, like their biological counterparts, are built up from basic processing units as shown in the figure. This artificial neuron receives one or more input signals, applies some kind of transformation function to the summed signal and produces an output signal (equal to the transformation) to be passed on to other neurons. A network usually receives one or more input signals and the input neurons, one for each input signal, behave somewhat differently in that they usually do not do any processing but simply act as distributors to deliver the signal to other neurons in the network. There are many ways in which the neurons in an ANN can be connected together, often referred to as the ANN 'architecture', but one of the most common is in the form of layers as shown in the figure. This is known as a 'feed-forward' network since information enters the network at the input layer and is fed forward through the hidden layer(s) until it reaches the output layer. Each neuron in the input layer is connected to every neuron in the hidden layer and each hidden layer neuron in turn is connected to every neuron in the next layer (hidden or output). The strengths or weights of the connections between each pair of neurons are adjustable and it is the adjustment of these connection weights that constitutes training of a network. The neurons, or processing elements, in the hidden layer(s) and output layer apply a non-linear function (transfer function) to their summed inputs. The combination of a non-linear transfer function and the large number of connections allows an ANN to model complex relationships between its inputs and outputs.
At first, ANN were only of interest to researchers in AI, but progress in their development has been dramatic and these tools have moved on from being oddities used by specialists to general purpose algorithms for data analysis and pattern recognition tasks.
As with all new techniques, the adoption of ANN by scientists in other fields had a slow start which soon accelerated. Their use in chemistry, for example, has been documented by the number of papers; 3(1988), 5(1989), 20(1990), 105(1991), 290(1992),….927(1997).
The workings of the human mind have long held a fascination for man, and we have constantly sought to explain how processes such as memory and reasoning operate, how the senses are connected to the brain, what different physical parts of the brain do, and so on. Almost inevitable consequences of this interest in our own minds are attempts to construct devices which will imitate some or all of the functions of the brain, if not artificial 'life' then at least artificial intelligence. It might be thought that we have already achieved this goal when some of the awesome computing tasks, such as weather forecasting, that are now carried out quite routinely (and surprisingly accurately) are considered. Nowadays, for example, even a simple electrical appliance like the humble toaster is likely to contain a microchip 'brain'. Computers, of course, have revolutionized artificial intelligence (AI) research, so much so that devices are now being built which are models, albeit limited, of the physical organization and 'wiring' of the brain. These systems are known as artificial neural networks (ANN) and they have proved to be so remarkably successful that they have found application in a very diverse set of fields as shown to the right.
