Hi Ron,
Thanks for your detailed and
well-thought out response. I appreciate your taking the time to read the podcast transcription.
ron2
I
finally finished reading the podcast interview with Gallistel. The compelling
interview you speak of speaks more of the desire of Gallistel to switch
learning theory to that of computational theory, ala the flip-flop circuits
that form memory in a computer. And essentially processor that links to the
binary states of chain molecules or neurons to abstract shapes and concepts
such as smells, sounds, sights. Fair enough and I hinted as much with the
reference to "Do Robots Dream of Electric Sheep?" ... In the
interview, without explanation, he wishes to change the thinking of the
scientific community away from "pavlovian response" or words to that
effect in favor of computational theory of learning or brain activity. Why? I
don't know.
For those who haven't read the interview, and would like to be
brought up to speed, here are the relevant passages:
---------------
Dr. Campbell: Randy, could you go over what some
of the key assumptions are that must be challenged, based on these experiments?
Dr. Gallistel: Well, perhaps the most important
thing is that when people are thinking about the nature of memory, they should
stop thinking about the kind of Pavlovian conditioning experiments which have
dominated neurobiological thinking about memory, and think about the demands on
memory that are made by dead reckoning, and by this food caching behavior, and
by innumerable other behavioral examples of the same kind.
None of these examples is mysterious
if you grant the animal read/write memory. All of these examples are not
computationally mysterious. It wouldn't be difficult to simulate these
performances in a conventional computer. But it's very important to understand
that the reason it would not be difficult is that we understand very well how
to represent locations, we understand very well how to encode food types, and
so on.
Most importantly, we understand how
to encode that information physically in a computationally accessible memory in
a conventional computer. There's no mystery about it. But if you try to do the
same thing in a neural net, if you throw away the baby in the bath-namely, the
read/write memory mechanism that is so fundamentally important in a modern
computer-then trying to build a machine that can do what the jays are doing, or
do what the ant is doing when it dead reckons, then it becomes extremely
difficult.
So, the most important thing is to
focus our experimental and theoretical efforts on trying to imagine what the
physical realization of such a mechanism-that is, a read/write memory-must be
in nervous systems. We argue in the book that this mechanism is as basic to
computation as genes are to life. Understanding this memory mechanism-the
read/write memory mechanism-until we understand that mechanism, we have no hope
of understanding how the brain computes, because that kind of memory is central
to computation.
This is the central argument of the
book. And at the moment we do not know what that mechanism is in the brain. We
argue that there's good reason to think that it will prove to be universal,
just as DNA is universal. That is, that it performs a basic, simple,
foundational function-namely, carrying information forward in time. There's no
reason why a mechanism that works in one domain, or one context, wouldn't work
in any other just as well. And therefore, there's no reason to assume that it's
not a universal mechanism.
Dr. Campbell: But you think we need to look
beyond the synapse.
Dr. Gallistel: Well, the question arises, so
people are persuaded that it must be changes in synaptic conductance: could
this mechanism be mediated by changes in synaptic conductance? Well, yes, it
could be. But, for reasons we go into at great lengths in the book, it doesn't
seem to us very likely that it is. It seems to us that once you grasp what the
essential nature of the mechanism must be, then changes in synaptic conductance
don't look very plausible or very attractive.
It's not impossible. A
change in synaptic conductance can be viewed as a change in the setting of a
switch, or potentiometer. And that's what you need to implement any memory
system. You need something that has more than one physically stable state. But
we point out that synapses are way more complex than is actually required. I
mean individual molecules have exactly that property.
Like the rhodopsin molecule in
vision. It's a perfect molecular switch. It has two different settings-an off
setting, which is the normal setting, and an on setting, which is produced when
a photon is absorbed by the molecule. Then that literally changes the physical
configuration of the molecule-it isomerizes the molecule, in more technical
language-and the isomerized molecule is now enzymatically active. That is, it's
readable; it affects other physical processes within the cell.
Another thing we try to get people
to grasp is the difference in size between cellular-level structures, like
synapses, and molecular-level structures, like rhodopsin. Because, if you
implement memory function at the molecular level, then you can put gigabytes
worth of information into single cells; whereas, if you implement it at the
synaptic level, you're going to talking about kilobytes, at most. So, we argue
that there are a variety of reasons-this isn't really the most important-but
there are a variety of considerations that make changes in synaptic conductance
not a very appealing story, once you realize that what we're looking for here
is a read/write memory mechanism.
----------
ron2
I
don't see it as contradictory. In fact, I would think, he would see the learned
response (associative) and the repeated learned response (operant) as
supporting the information carried forward in time thing. He speaks of the bird
experiment where birds, and others, can place a cache of resources, being there
one time, and come back to exactly that same spot later in time to retrieve the
items. That is a complex behavior chain but is a reward-based mechanism. How
so? He who remembers where the cache is gets to eat and survive. That is, the
ability to learn from the environment and have memory separates the living from
the dead.
Okay, so first of all, I don't think
Gallistel and others are trying to get rid of Pavlov's work. There's no
question that this type of conditioning exists. The question is whether
temporal contiguity is a necessary part of explaining how and why it works. And that's a pretty big question. (See: "Temporal maps and
informativeness in associative learning," Balsam and Gallistel,Trends
in Neurosciences, Volume 32, Issue 2, 73-78, 12 January 2009.)
There's also no question that having
the ability to remember things (like where you cached your food supply) is
important to an animal's survival. The question is, how does this kind of
behavior happen, what are the cognitive processing mechanisms?
You seem to have a pretty good
working understanding of how operant and classical conditioning work. And from
what you tell us, you're using behavioral science techniques in training
Shadow. So how many times would you say you've seen conditioning work to create
new behaviors without the necessity for any repetition? And how can a
behavior be learned via a reward-based mechanism, if said behavior is learned
before it's even been
rewarded? Remember, the blue jays come back to their cache before they get the reward of finding and
eating it. Plus, I would say that this behavior seems pretty simple, and not
at all part of a complex behavior chain, as you stated.
So I think this brings us back into
the territory of pattern recognition as opposed to after-the-fact, reward-based
learning. I don't know of any types of conditioning that operate independently
of the idea that behaviors are reinforced by positive feedback (reward)
for their consequences. Yet you yourself said that this was an example of a
"the information carried forward in time thing." How, in your view, would this feedforward type of learning gibe with operant conditioning?
ron2
Gallistel
admits that his book is ... an attempt to get future students to change their
paradigm. Their [King and Gallistel's] initial viewpoint, which is not based on
science or theories on evidence collected but is instead based on a
pre-established theory and hopes that future students can then find the data to
support the theory.
I didn't see any evidence of that at
all. Where did you get that idea?
ron2
Dopamine might even turn out to be a chemical that is
simply an aid to neural behavior, which would explain why it is present in both
reward and non-reward instances. That is, because it was present in earlier
reward research, it was assumed at the time to be part of the reward sequence.
Current research suggests that dopamine plays a key role in enabling and motivating us to pay close attention to, and clearly remembering, certain key events,
positive or negative.
ron2
And
does not invalidate OC.
No, but it calls one of its basic
principles into question, i.e., the idea that there needs to be a strong temporal relationship between a behavior
and its subsequent consequences in order for such a behavior to become learned.
ron2
Gallistel wants to put a stop to pavlovian thought. That is, in fact, throwing
out the baby, the bath water, the whole durned bathtub.
Again, I think that's a
mischaracterization. What Gallistel actually said was, that when it comes to
understanding how memory works, people need to stop thinking about the idea
that there has to be a temporal relationship between an US and a CR. That's
all. (And he's not suggesting throwing the baby out with the bath water; he's
just pointing out that there are some holes in learning theory that aren't
being addressed.)
Thanks again,
LCK