¿Artificial? Intelligence

Last week my friend David Krakauer presented three lectures on Intelligence -- c.f. Cognitive Ubiquity -- in the SFI Ulam Lecture series. I thought the slides were online someplace but I can't find them; however the videos should be posted at santafe.edu sometime soon. He made some good, and some arguable, points and was quite entertaining in the process.

One of the good points is that what we call intelligence, if we can even define it, goes much deeper than the human cortex. He showed a video clip of a white blood cell "chasing" a bacterium through a forest of red cells where the white cell appeared to be behaving quite smartly in it's search-and-destroy mission. He then made the point that the low level components of computerized artificial intelligence have none of the characteristics of that "simple" white cell, e.g.: NAND gates don't adapt.

I think this is not an apt comparison. Where transistors are atoms, NAND gates are more comparable to simple molecules. Large Scale Integrated circuits -- memory chips and the like -- might measure up to the capabilities of a complex organic molecule, and micro-controllers could be compared to one or two neurons. To support my claim I present you with three series-connected neurons: Each neuron might (conservatively) have 1000 synapses which gives the whole system one-billion possible binary states. Show me a microchip that does that. Then realize that there are about 10¹¹ neurons in the human brain and another (hand-waving-estimate) 10¹⁰ elsewhere in the body.

This is the scale of the problem we have.

But Wait! There's More!

Getting back to the hand-waving-estimate thing... A year or so ago I tried, unsuccessfully, to estimate the Shannon Information content of our nervous system in order to have a reasonable retort when folks asked me why my robots behaved so stupidly. I was not successful because I found it almost impossible to get good estimates of three -- to me -- important values:

1. The number of Sensor Inputs;
2. The number of Motor Outputs;
3. The resolution of a "Synaptic Connection".

I did dig up swagish values for the number of Inputs, and finally settled on the number of muscles as a stand-in for the Output count. But I could not get anyone to hazard a guess at #3 -- no one seems to know how much you can vary a synaptic connection weight: the putative mechanism for learning and adaption. Everywhere I asked I got some run-around about how it doesn't really work that way or other long-circuit "I don't know". As a geek this was surprising because some of the first things one wants to know about a computer program are how much input and output and what resolution, accuracy, and speed is required.

Anyway, I put together a cheat sheet of what I found: here. And just so you don't have to follow -- and make sense of -- that link, here's the chase:

    Input:          10^8 eye sensors; 10^7 touch, hearing, taste, and smell
      Sight:         5*10^6 cones + 1.3*10^8 rods ~= 1.4*10^8 sensors
      Touch:        (swag) 3*10^6 sensors
      Hearing:    8.8*10^2 sensor neurons
      Taste:        (swag) 1*10^6 sensors
      Smell:        1.2*10^6 sensors
    Output:       Estimate, 300-700 muscles in a human body

I also guessed at 8 bits -- for convenience -- of synaptic weight, and put the neural firing rate at 50 per second, with each synapse doing a scale and each neuron doing a sum operation. That gave me, for the brain only:

• 7*10^14 bytes or 5.6 petaBit of state
• 3.5*10^16 or 35 petaFlop/second of calculation

-- This is the scale of the problem we have --

It is also interesting to note that the number of touch sensors is in the same order of magnitude as the number of cones in the eye. Until now, much of the interest in neural signal processing has been in the visual cortex. But the motor cortex may have as many inputs and probably many more outputs. The visual system is pretty good at linear algebra, but the motor system solves simultaneous differential equations each time you toss a wad of paper at the trash can. So literally putting a robot out in the field may be a very fruitful line of research after all.