A Logical Neuron

McCulloch and Pitts

Warren McCulloch was a psychiatrist, computer scientist, neurophysiologist, poet, and philosopher. He worked with Norbert Wiener to pioneer the new field of cybernetics, and is sometimes credited as a founder of artificial intelligence.  He was also an accomplished experimental physiologist. He was a colorful and controversial figure by design. He had a spectacular beard. His papers often had wonderful titles, like "Why the mind is in the head", or "What is a number, that a man may know it, and a man, that he may know a number".  A collection of his works, entitled "Embodiments of Mind" is still available from MIT Press.  Containing several of his most influential experimental and theoretical papers, philosophical essays, and even some sonnets, it remains an interesting read more than 50 years after his death in 1969.

Walter Pitts was an eccentric genius who ran away from home in 1938 at age 15, not to join the circus, but to attend lectures by Bertrand Russell at the University of Chicago.  Pitts had read Russell and Whitehead's Principia Mathematica at age 12, and had developed an interest in logic and mathematics.  After Russell went home to England, Pitts hung around the University of Chicago, doing odd jobs and unofficially studying with Rudolph Carnap, one of the world's leading figures in symbolic logic at the time. In 1942 Pitts met McCulloch, who was working on a formulation of neuronal function as binary logic. McCulloch and his wife invited Pitts to live with them, and the two began their collaboration.  Pitts was 18 years old and had no academic credentials at the time his most famous paper was published.  Later, McCulloch and Pitts would both move to MIT, where Pitts would enroll as a graduate student under the supervision of Norbert Wiener, but would never graduate.

The McCulloch and Pitts Neuron

It was 1943.  A lot of what we know about neurons had not yet been discovered.  What was known about the action potential came from studies of isolated peripheral nerve, not central neurons. Renshaw, Forbes, and Morrison published their first description of central nervous system single unit extracellular recordings using micro-electrodes only 3 years previously, and the method had not yet seen much use.  Of course, there were no intracellular recordings from central neurons and synaptic potentials had not yet been discovered.  What was known about synapses was inferred from studies of the timing of monosynaptic reflexes.  Given the state of knowledge at the time, McCulloch and Pitts had to make some assumptions about how neurons worked.

Some of their assumptions seem familiar. They assumed that the cell has a fixed voltage threshold, and that synapses produced subthreshold changes in membrane potential. They concluded that it must take multiple excitatory synapses acting together to make the cell fire. Not knowing the action potential threshold or the sizes of synaptic potentials, they specified the threshold as the number of simultaneous excitatory synapses required to trigger a spike. They also included inhibition.  Inhibition was known to exist at that time, but its mechanism would not be established until the intracellular recording studies by Brock, Coombs and Eccles, almost 10 years later. In the McCulloch and Pitts model, inhibition held a veto; a single active inhibitory synapse could prevent a neuron from spiking.  

Some of their other assumptions are less familiar. They assumed a synaptic delay, but synaptic delay to McCulloch and Pitts didn't mean the same thing that it does now.  We distinguish between the synaptic delay proper (time for release and action of chemical transmitter) and the integration time (from the onset of a synaptic current to postsynaptic action potential generation).  They combined these ideas in the synaptic delay and gave it a minimum value, saying "Between the arrival of impulses upon a neuron and its own propagated impulse there is a synaptic delay of > 0.5 ms".   They assumed that the synaptic delay was the same for all connections in a circuit.

The also assumed a fixed window for synaptic summation, which they called "integration time". This is the maximum time interval that can elapse between inputs that can cooperate to trigger a postsynaptic action potential.  This is a current idea, and closely related to our current definition of integration time. But they thought it to be extremely short, and give it a maximum value of 0.25 ms.  They say "Observed temporal summation of impulses at greater intervals  is impossible for single neurons..." In their view, neurons make the decision of whether or not to fire in a quarter of a millisecond.  Decisions by neurons are made very fast, and when they say inputs are simultaneous they really mean it.

Their strangest assumption was the one about axon conduction times. They assumed that the conduction times are the same for all axons in their network.  This means that if two neurons fire simultaneously, their effects on all postsynaptic neurons will also be simultaneous.

The combination of these timing rules imply that all neuronal firing decisions occur in discrete moments in time, no longer than 0.25 ms in duration and occurring at intervals equal to the synaptic delay, which is about twice as long. Because conduction times are all the same, the decision times of all neurons occur simultaneously, and are followed by a brief period in which no decisions are made. This allows McCulloch and Pitts to divide time into discrete slices, with durations equal to the synaptic delay.  On every time slice, each input on each cell is either active or not. The integration time allows for a tiny bit of slop (about 0.25 ms) in the synaptic delays between cells, but not enough for inputs to cooperate across time slice boundaries.  On every time slice, neurons sample the activity of their inputs.  If the number of excitatory ones exceeds threshold and there are no inhibitory inputs active, the cell will be activated. Its firing will be felt by its postsynaptic targets one synaptic delay later. Their neurons also include a refractory period, which prevents cells from firing twice in response to inputs arriving in the same time slice.

No references required

Where did they get these ideas about neurons?  You can't tell by reading their paper. It is remarkable for not making any attempt to justify its biological assumptions by reference to published experimental measurements.  There are exactly zero references to any papers on the biology of neurons.  The entire reference list, consisting of only 3 entries, are classics in the field of symbolic logic.  But there is no doubt the timing properties of the McCulloch and Pitts neuron come from a series of papers on hypoglossal reflexes published by Raphael Lorente de Nó between 1935 and 1939. For several years Lorente de Nó had been studying reflex responses of the hypoglossal nucleus. In his 1939 review he set a value of 0.5 ms for the minimum delay, saying "The synaptic delay cannot be reduced below a minimal value of about 0.5 msec.; this minimal interval is observed even when the effective impulses are delivered to motoneurons for which the transmission has been facilitated by the arrival of other impulses."  He also argues that the duration of synaptic responses is brief compared to the refractory period, so that it is impossible for a cell to fire twice on a single synaptic excitation, and he measures the window for synaptic summation at 0.25 ms.  Lorente de Nó's work was highly respected and cited at the time, and he  was a participant in the first of the famous Macy conferences in 1946, entitled "Feedback Mechanisms and Circular Causal Systems in Biological and Social Systems", chaired by McCulloch. McCulloch also cited his work extensively in other papers, for example his paper with Dusser de Baron in 1939. Given the state of knowledge at the time, Lorente de Nó's numbers were a reasonable basis for the calculations, but certainly not the only one.  

The temporal logical proposition

The McCulloch and Pitts neuron is a logical element.  In the language used by McCulloch and Pitts, each neuron has a logical proposition that it asserts – a statement about the activity of its inputs. When it fires an action potential, the neuron asserts the truth of its logical proposition. The goal of the paper was to develop a method to express the function of each neuron in symbolic form.  They used symbolic logic to write out an equation for the function of a network of cells.

The logical notation they used is the one employed by Russell and Whitehead in their classic Principia Mathematica, with modification to include the effect of time. Because of the synaptic delay, the firing of a neuron actually asserts a proposition about its pattern of activity in its inputs in the previous time slice. The result is a sequence of logical states propagating through the network in time. The sequence is represented using a logical formula they called the temporal logical proposition. McCulloch and Pitts used two notations for neuronal networks, one diagrammatic, and one using their time-specific variant of symbolic logic notation.  Their diagrammatic notation represents the neuron as an arrowhead, with the axon exiting from the base. Excitatory synapses were indicated by axon terminals apposed to the neuron, as is often done today.  Inhibitory input was drawn with the axon making a loop at the apical vertex of the cell.  They don't show any example with more than one inhibitory input, but I suppose if there were, the loops could be stacked.  Each neuron has a threshold.  In all the examples in their figure 1, the threshold is 2, meaning that the cell is active in time slice ti+1 whenever two or more of its excitatory inputs are active in time slice ti.  If an afferent makes two synapses on a neuron, then it can activate the cell on its own.  If it makes only 1, it must cooperate with at least one other afferent.  Activation of neuron 1 in time slice t is indicated as N1(t). Thus if there are only two excitatory afferents (1 and  2), each making one synapse on a neuron, the cell will detect the conjunction of its inputs. In symbolic notation, conjunction is indicated with a decimal point, as N1(t).N2(t).  If the two inputs each make two synapses on the postsynaptic cell, then it will be activated in time t+1 by the disjunction, when either 1 or 2 is activated at time t, indicated as N1(t) v N2(t).  If there is an inhibitory input from afferent 3, then the cell is only eligible to fire when 3 is not active, indicated as ~N3(t).