Dog Community

OC, Clicker Training, Learning Theory

ron2

Posted : 9/9/2007 9:20:17 AM

My knowledge and understanding of Operant Conditioning, Clicker Training, and Learning Theory is being called into question.

Be mindful that I have 1 dog, 1 cat, I am not a certified or mentor-trained dog trainer nor do I have a degree in psychology, behavioral or otherwise. I am an electrician and the largest part of my experience and education support that. We do have at least one person here with all the proper creds for dog training, including decades of experience with dogs of all natures.

Anyway, to the point of a particular point in a debate is the notion that satiation decreases learning and deprivation increases learning and that this is supposed to be a flaw in clicker training. Now, it may simply be that this disagreement is based on personality. But to truly debate it, there must be something somewhere that gives rise to this particular thought.

Here are some samples of what I have read and learned in the process of clicker training.

To get a truly reliable behavior, there's only one way to do it. Practice with intent. Generalize the behavior. Practice in the conditions in which you need the behavior reliable. Work on latency. Keep records and train until you've achieved the level of reliability you need, whether it's nine of ten or 999 of 1000.
You determine which cues are the most reliable and have immediate responses by training them that way. But never fool yourself into thinking training, no matter how severe the aversive used, overcomes free will.
"The click must be followed by a food treat."
False. The click must be followed by a reinforcer—something the dog is willing to work to obtain. You have a variety of reinforcers available. Some of the most commonly used include:

Food
Toys
Praise, attention
Opportunity to do something the dog wants to do
Opportunity to perform a well-known behavior
In a formal [link>http://forum.dog.com/glossary/lterm260]training session[/link] you want to get as many repetitions as possible. Food is an excellent reinforcer because it can be cut into tiny pieces and eaten quickly. Toys are also a good reinforcer, but playing with the toy takes time, meaning you get fewer repetitions in a session. Praise and attention are wonderful additions to food or toys, but are often not desired enough by the dog to use alone, particularly in distracting situations. The opportunity to do something else is sometimes the most powerful reinforcer you have.
The most important thing to remember is that the dog determines what is and isn't a reinforcer in a certain situation. If the dog doesn't want what you're offering, it's not a reinforcer.
"Clicker training won't always work because food isn't a strong enough reinforcer."
AND
"Instinctive drives and self-rewarding behaviors are so powerful that you must use corrections to ensure reliability."
As mentioned in the last section, food isn't the only reinforcer available. No matter what reinforcer you choose, consider its relative value. One food might be worth more than another food. Your dog's tug toy might be worth more to your dog than food in certain situations. The opportunity to greet another dog may be the best of all! It all depends on your dog and the particular situation.
One of the commonly sited concerns is that there are things the dog wants in the environment that he can't have. Or that the dog has a powerful, instinctive drive to perform a certain behavior, and the trainer can't find a positive reinforcer more powerful than that drive. Guess what—you may not! So that's when clicker trainers move beyond operant conditioning and employ techniques based in [link>http://forum.dog.com/glossary/lterm204]classical conditioning[/link], such as [link>http://forum.dog.com/glossary/lterm218]desensitization[/link] and redirection. Desensitization lowers the strength of the animal's response to particular stimuli. Redirection can be used to actually transfer the focus of the drive from an undesired outlet to a different—even unrelated—desired one.

Cautions in using Positive Punishment
Behaviors are usually motivated by the expectation for some reward, and even with a punishment, the motivation of the reward is often still there. For example, a predator must face some considerable risk and pain in order to catch food. A wild dog must run over rough ground and through bushes, and face the hooves, claws, teeth, and/or horns of their prey animals. They might be painfully injured in their pursuit. In spite of this, they continue to pursue prey. In this case, the motivation and the reward far outweigh the punishments, even when they are dramatic.
The timing of a positive punishment must be exquisite. It must correspond exactly with the behavior for it to have an effect. (If a conditioned punisher is used, the CP+ must occur precisely with the behavior). If you catch your dog chewing on the furniture and you hit him when he comes to you, you are suppressing coming to you. The dog will not make the connection between the punishment and the chewing (no matter how much you point at the furniture).
The aversive must be sufficient to stop the behavior in its tracks - and must be greater than the reward. The more experience the animal has with a rewarding consequence for the behavior, the greater the aversive has to be to stop or decrease the behavior. If you start with a small aversive (mild electric shock or a stern talking-to) and build up to a greater one (strong shock or full-on yelling), your trainee may become adjusted to the aversive and it will not have any greater effect.

Punishments may become associated with the person supplying them. The dog who was hit after chewing on the furniture may still chew on the furniture, but he certainly won't do it when you're around!
Physical punishments can cause physical damage, and mental punishments can cause mental damage. You should only apply as much of an aversive as it takes to stop the behavior. If you find you have to apply a punishment more than three times for one behavior, without any decrease in the behavior, you are not "reducing the behavior", you are harassing (or abusing) the trainee.
[linkhttp://www.wagntrain.com/OC/]http://www.wagntrain.com/OC/[/link]

To the debated point, I haven't encountered yet that satiation decreases learning. I'll try and find a source to this idea or maybe someone can provide a source to this idea, rather than just a proclaimed "axiom" floating out there conveniently.

IMLE, my dog hasn't decreased learning based upon satiation. For one thing, I follow, as best I can, the rules of clicker training. Keeping lessons short and fun. It is more important to practice correctly than to practice for a long time in a session. Quality beats quantity. I also try to end on a successful note. With my one dog being food motivated and using food rewards, I might run the risk of him getting full. The side effect of being full is that he lays down for a nap. I think a lot of dogs are like that. Blood rushes to the digestive system to process the load and takes blood away from the brain. That effect happens to me, as well. My nickname for it is "stun time", from a science fiction story. I can zonk out after a full meal.

But I still haven't seen where he quits learning, even after a sandwhich bag full of jackpot meat. Which introduces the concept of high value treats. I have, for myself, defined two levels of treats. 'Great" and "I must be in heaven."

I do not view withholding jackpots for a while as deprivation. And I also don't think the learning process itself has been hindered if he does get full. My one dog cues me at times for training. Let's say that he does it because he is hungry, even though he just ate an hour or two ago. Fine, and he's willing to work for some more food. Exactly then, how is the training effect of clicker training and +R diminished from him getting more food? I just don't see it happening with my one dog.

My BIL has two dogs. Yes, a multi-dog owner. Yay, whoopee, hallelujah. One of them is a female Blue Merle Aussie named Cassie. She lives to herd and cut a ball as if it were a sheep. This means more to her than a food treat, which she also likes. I have seen her, exhausted and panting and needing a rest in the humid heat of south Louisiana, get up and run one more time because our nephew kicked the ball one more time. In fact SIL had to get nephew to quit so that Cassie would rest. Cassie's reward is different than Jessie's, a geriatric Cocker Spaniel.

I'm not saying the satiation principle doesn't exist, I just haven't found it yet, either in theory or in practice.

DPU

Posted : 9/9/2007 10:46:08 AM

ORIGINAL: ron2

My knowledge and understanding of Operant Conditioning, Clicker Training, and Learning Theory is being called into question.

Be mindful that I have 1 dog, 1 cat, I am not a certified or mentor-trained dog trainer nor do I have a degree in psychology, behavioral or otherwise. I am an electrician and the largest part of my experience and education support that. We do have at least one person here with all the proper creds for dog training, including decades of experience with dogs of all natures.

Anyway, to the point of a particular point in a debate is the notion that satiation decreases learning and deprivation increases learning and that this is supposed to be a flaw in clicker training. Now, it may simply be that this disagreement is based on personality. But to truly debate it, there must be something somewhere that gives rise to this particular thought.

Once again the OP is referencing me and I must say that the man from the south is overly sensitive. Instead of characterizing a simple discussion as an attack, being personal, or questioning one's comprehension...it should be view as maybe the understanding is not complete.

Anyway, we were simply discussing the values of the rewards. I made the statement that highly prized rewards disminish over time in favor of another reward that is also present. The OP stated it does not diminish. We were just discussing.

Ron2, then I take it you reject the statement "deprivation increases learning and satiation decreases learning".

I also can confirm if you google the phrase, you get nothing. I tried to find my source. I do recall I came across this phrase doing research on how to rehab an emaciated dog. I accepted this as "common sense" in learning theory and it has helped me examine and understand training methods and which method would be best to apply to the dog in front of me. That is why when I consider training methods like Clicker Training I see withholding a treat as deprivation and having an emotional effect on the dog such as repeated teasing. So for a dog with true SA, a psychological disorder, the mechanics needs to be modified. Same is true for a dog pulling, the treat or reward value gets satisfied and the dog will look for a higher value one and pull to get it.

I will continue to look for the source...but it seems such a common sense thing to me.

snownose

Posted : 9/9/2007 12:59:00 PM

My knowledge and understanding of Operant Conditioning, Clicker Training, and Learning Theory is being called into question.

Dear Ron.....if you feel I have questioned you, then I apologize, maybe you should take it up with a certain member who keeps throwing your name in the ring on a regular basis, who has cleverly made you out to be his/her little protoge.....the only comment I made was in reference to owning more than one dog, perhaps 4 like me and 6 just a little while ago. I mentioned that it changes somewhat when more than one dog is involved....you gotta at least give me that, man[

] Owning one or two is a cakewalk ....

Liesje

Posted : 9/9/2007 1:09:47 PM

How does having more than one pet affect operant conditioning? I only have one dog at the moment (and several cats, one of which is clicker trained) and when I do clicker or any sort of "operant" method I take that animal out by itself so it is not distracted by the environment or other animals while we are learning the new behavior. Kenya's previous home was with a trainer who has always used operant conditioning. She is a professional trainer with lots of letters behind her name and her own training facility based on operant conditioning. She has like 7 dogs of her own and as far as I know, the amount of dogs has never been a problem. I don't think operant conditioning works that well if you are trying to train more than one dog at the same time, but when shaping new behaviors isn't the dog being trained usually isolated? I'm not trying to be argumentative b/c like I said I only have one dog, but I don't get why it would be a problem. When I work with the dog, we go out in the yard and when I work with the cat, we go into the room where he is most comfortable and least distracted (the bedroom) and shut the other animals out.

snownose

Posted : 9/9/2007 1:17:46 PM

How does having more than one pet affect operant conditioning? I only have one dog at the moment (and several cats, one of which is clicker trained) and when I do clicker or any sort of "operant" method I take that animal out by itself so it is not distracted by the environment or other animals while we are learning the new behavior. Kenya's previous home was with a trainer who has always used operant conditioning.

Show me where I wrote the part about multiple pets and operant conditioning[

]......but, here is a question for you.....you say Kenya was with a trainer who always used operant conditioning, so I am assuming that has been going on for years, so, at this point you still have to use a clicker and food as reward?

DPU

Posted : 9/9/2007 1:30:45 PM

Ron2, I am still looking and trying to retrace my steps in doing the research. Here is the start that kept me looking and satisfying my inquisitive mind. From this site :

[linkhttp://en.wikipedia.org/wiki/Reinforcement]http://en.wikipedia.org/wiki/Reinforcement[/link]

"Often primary reinforcers shift their reinforcing value temporarily through satiation and deprivation. Food, for example, will cease to be effective in increasing response strength once a certain amount has been consumed (satiation). After a period during which it does not receive any of the primary reinforcer (deprivation), however, the primary reinforcer will once more be effective in increasing response strength.

A secondary reinforcer, sometimes called a conditioned reinforcer, is a stimulus or situation that has acquired reinforcing power after being associated with a primary reinforcer or an earlier conditioned reinforcer (such as money). An example of a secondary reinforcer would be a clicker, as used in [linkhttp://en.wikipedia.org/wiki/Clicker_training]clicker training[/link]. A dog associates the clicker with praise or treats, and now the clicker is reinforcing. As with primary reinforcers, an organism can experience satiation and deprivation with secondary reinforcers."

Liesje

Posted : 9/9/2007 1:51:37 PM

Show me where I wrote the part about multiple pets and operant conditioning......but, here is a question for you.....you say Kenya was with a trainer who always used operant conditioning, so I am assuming that has been going on for years, so, at this point you still have to use a clicker and food as reward?

I wasn't quoting you when I asked the question. In past threads, DPU has mentioned having difficulties with a clicker b/c he has so many dogs.

Yes, Kenya has always been trained with operant conditioning. I only click to shape new behaviors. My understanding of operant conditioning is that it is for learning, not reinforcing. Right now we are clicking/marking for crawling, a down with her head down ("nose down!"), "finish!", and the correct drive through the weaves. I am also clicking for Shirley Chong's retrieve b/c Kenya's current retrieve is not accurate or reliable enough so we are simply starting over. I do not click/mark for sits, paws, downs, stays, waits, recalls, leave its, heels, or any agility obstacle other than the correct drive through weaves. We only click for things she is just learning and doesn't yet know what is the right way or the wrong way. We don't click for commands she already knows b/c they don't need to be reinforced. I proof them in various environments with various distractions, but don't click, just praise when done as expected and "eh eh" or ignore when not done correctly.

snownose

Posted : 9/9/2007 2:00:28 PM

I proof them in various environments with various distractions, but don't click, just praise when done as expected and "eh eh" or ignore when not done correctly.

Heyyyyy, now we are thinking alike....that is exactly what I do even from the beginning, I praise or give a good rub to let the dog know he did good......I have never had to use a clicker to accomplish any training......and when you have as man dogs as I have having to hunt for a clicker while the dog is not fully clicker weaned is crazy to me.

Liesje

Posted : 9/9/2007 2:12:43 PM

Yeah I don't see anything wrong with that. Operant conditioning is operant conditioning. If you are marking to shape new behaviors, it's operant conditioning no matter what the "marker" is and what the "reward" is. I really only click with Kenya when I want her to stop after a certain behavior is offered. For example, last week I was working on her jumping technique and attention to me after each obstacle, so I wanted her to do ONE jump at a time, not run on and do the rest of the course like usual and pay no attention to me. I used the clicker to mark the jump b/c then she knows a reward is coming and stops for the treat rather than continuing on. Same with the "finish!" b/c she needs to complete that skill and STOP doing everything otherwise it's not exactly a finish if she heels, sits, but then gets up and leaves. I use the click-reward as much to indicate that the behavior is DONE as I do to reinforce it. For off-leash heeling, I have not been using a clicker b/c usually I will praise or treat every 5 steps or so but we keep going. If I click, she will actually stop and sit to take the reward and I need her to understand that "heel!" is not a one-time command, we keep going until I give a new command.

She was actually never trained on clicker until I got her. I have trouble using a noise (I tihnk you said that you snap....see I can't snap with either hand!) or word consistently with the same tone of voice, so I chose the clicker. Her trainers simply said "yes!". But the clicker is just one method and I only use it on a trick-by-trick basis. Sometimes she picks up on things so quickly it's a waste of time to "charge" up a clicker and use that when she already gets what behavior I want.

For the cat, I use his own kibbles as the treats. I will admit, he is very stubborn and will not do what I ask unless I am clicking and treating. When he sees his target wand come out, he's all business, but he won't "touch!" anything else if he knows I don't have the clicker and the treats. The cat is much more sensitive to timing and consistency since he is not exactly easily trainable like a GSD. He's a far greater challenge!

DPU

Posted : 9/9/2007 2:12:56 PM

ORIGINAL: ron2

I also try to end on a successful note.

Karen Pryor suggest not to end a training session abruptly because this may be a downer for some dogs. She suggest you transition out of the session. Of course if you are not observant, you may not notice this. I did and I wrote her, but I got no response and then I read the fix in her newsletter.

Liesje

Posted : 9/9/2007 2:16:32 PM

Karen Pryor suggest not to end a training session abruptly because this may be a downer for some dogs. She suggest you transition out of the session.

She also talks about "jackpots" so that's how I end my clicker sections. When I want to be done, the last click is followed by a treat that might take them a while to chew (the dog will get a whole treat rather than the little bits I use for training), or for the cat, he gets the whole bowl with the rest of his dinner (I use his meal kibbles for his rewards). It's not a transition, but it's sure not a downer for them!!

DPU

Posted : 9/9/2007 2:19:50 PM

ORIGINAL: Liesje

It's not a transition, but it's sure not a downer for them!!

Ok, whatever you see is fine but I see what you do as a transition and satiation in play.

Kim_MacMillan

Posted : 9/9/2007 2:48:47 PM

My knowledge and understanding of Operant Conditioning, Clicker Training, and Learning Theory is being called into question.

I'm not sure who called your knowledge into question, but I think you're very knowledgable considering you are completely self-taught in this field. I don't necessarily think your knowledge is in question, but simply rather how you portray it?

There is a lot lot of literature that discusses the fact that deprivation can lead to learning faster than satiation can. After all, if you've just finished a turkey dinner at Thanksgiving, are you going to necessarily find another turkey dinner as a reward for cleaning up all that reinforcing? [

] You'd probably find it quite punishing, actually (in that case the second turkey dinner isn't a R+ after all!)

Some articles on food deprivation use in learning:

[linkhttp://www.the-scientist.com/news/display/23132/]http://www.the-scientist.com/news/display/23132/[/link]
Some research has found evidence that the neurohormone ghrelin, which affects both appetite stimulation, also affects memory formation:

Diano and her colleagues then tested the effects of ghrelin administration in several learning and memory tasks. First, injecting rats peripherally with ghrelin or a ghrelin-receptor agonist improved their performance on a maze task that depends on hippocampal function. Next, injecting ghrelin into the cerebral ventricles after training on an avoidance task improved task learning not only in wild-type mice but also in mice that display the pathological and cognitive symptoms of Alzheimer?s disease. Lastly, the authors found that ghrelin-knockout mice fare worse during object recognition tasks that employ the hippocampus, but improve after peripheral ghrelin administration.
...Learning and memory may be enhanced by high levels of ghrelin during food deprivation because animals need increased cognitive skills to track down food sources, Diano told The Scientist.

Hughes, J.E., H. Amyx, J.L. Howard, K.P. Nanry, and G.T. Pollard (1994). Health effects of water restriction to motivate lever-pressing in rats. Laboratory Animal Science 44(2): 135-140.
NAL call number: 410.9 P94

Abstract: The objectives of this study were to determine the degree of water restriction necessary and sufficient to motivate acquisition and performance of a lever-press response in rats and the physiologic and general health effects of chronic daily restriction. Parameters examined included: hematological and clinical chemistry, body weight, gross necropsy, clinical examination.

[linkHashimoto>http://www.ncbi.nlm.nih.gov/sites/entrez?Db=PubMed&Cmd=Search&Term=%22Hashimoto%20T%22%5BAuthor%5D&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVAbstractPlus]Hashimoto T[/link], [linkWatanabe>http://www.ncbi.nlm.nih.gov/sites/entrez?Db=PubMed&Cmd=Search&Term=%22Watanabe%20S%22%5BAuthor%5D&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVAbstractPlus]Watanabe S[/link]. Chronic food restriction enhances memory in mice--analysis with matched drive levels.
PMID: 15973161 [PubMed - indexed for MEDLINE]

We compared the effects of chronic and acute food deprivation on learning and memory using a dry-type water maze, active avoidance and passive avoidance in C57BL/6L mice. The drive level of the animals--under acute and chronic food deprivation--was matched by a progressive ratio schedule. Both deprivations led to a high degree of activity in the animals; however, the animals on an acute dietary restriction did not exhibit a significantly better performance than those on ad libitum feeding, while those on a chronic food deprivation exhibited memory enhancement. These effects were subtle and were found at a later stage of learning. These findings suggest that chronic food restriction induces memory consolidation or resistance to memory reduction in addition to increased activity.

Deprivation is commonly used to enhance learning in many situations. In lab rats and mice, it's a huge example. In dogs it is commonly used as well, depending on the training. In a lot of police dogs social deprivation is put into place as a normal protocol to enhance the time spent with their handler, and to make them more "drivey". There are many competition dogs that are also food-deprived, toy-deprived, and socially isolated in order to increase their "game" in other areas. While it does enhance learning, and might have its place, it's not something I would personally do.

On another note, without going to the extremes of true food deprivation, you can take advantage of the primary reinforcement power of hunger and feed before meals. Or use their meal as a reward. Because they are "hungry" (as anybody is hungry before a normal meal!) they are a lot more focused and it has direct relevance to their homeostasis, so they may remember it better, or more quickly. For some dogs. Some dogs are so revved up over food that trying to teach them on an empty stomach causes very sporadic concentration (along the lines of trying to concentrate on learning in class, when all you can think about is what you're eating later. Another problem with hunger is that it can potentially inhibit learning as well if you can't concentrate as well, as you aren't sufficiently energized). So it goes two ways really.

Also, there is a basis for satiation, in some circumstances, reducing learning because of the simple physiological effects of satiation. After you're done eating the parasympathetic nervous system is in full gear (the "rest and digest", rather than "fight or flight" aspect). It causes digestion, relaxation, all those things that eating a meal and becoming full entails. If you've just eaten a large meal, would you be up to going out and running the marathon? Not likely. [

] Even if you were an avid runner, it still wouldn't be very appetizing, even if you were offered $100. Because your body is telling you that it needs to digest its meal. That meal will provide later energy you may want to burn off, but not likely right after you finish eating it.

When I work with my guys, I try to work right before a meal and use a combination of part of their meal along with other high-value rewards, or I'll work between meals at a time when they are suitably interested in working (for instance with my guys I don't tend to work them in mid-afternoon because that is their afternoon down-time, when they rest. I would likely work them morning or evening when they are most active, alert, and engaged). But in some circumstances it's not going to make a big difference in the large realm of things. There are ways to enhance learning, by simply working with your dog's individual schedule and time-clock, but in the grand scheme of things, for most people, it's pretty irrelevant. I can take my guys out right after a meal because usually they aren't full, since we split our meals into to per day, so they are still relatively active anyhow and just the idea OF clicker teaching gets them quite drivey and alert.

In terms of reliability and the use of corrections, I think that's a relatively moot point because a) they didn't define their use of "correction", so it's hard to argue against it, and b) if you use the right reinforcers and are working WITH the animals instinctive drift (some animals are subject to instinctive drift despite the highest reinforcers OR punishers - Re: Breland and Brelands paper "The Misbehaviour of Organisms"), you can sometimes teach some of the most reliable behaviours without the use of corrections at all. For instance, if it was feasible, you could use sex (for males, since most females only copulate during certain times of the year, whereas males in many species can copulate at any time) as one of the most powerful reinforcers. But because it's not feasible to do, it's not often used (they do use access to a receptive female a lot in rat experiments as a reinforcer, and you should SEE some of the things those rats will do to get that reinforcement).

Does that help clarify anything? I'm not supremely experienced in using food deprivation in learning, but because I'm a psych major and a biology major I have a lot of exposure to the use of using deprivation of different sorts to enhance learning and memory.

Kim_MacMillan

Posted : 9/9/2007 2:51:27 PM

I wasn't quoting you when I asked the question. In past threads, DPU has mentioned having difficulties with a clicker b/c he has so many dogs.

On a side note. I've got 14 dogs under one roof at the moment, and I have no problems whatsoever using a clicker to teach what it is that I need. [

] For public interest, if it helps.

snownose

Posted : 9/9/2007 3:20:08 PM

On a side note. I've got 14 dogs under one roof at the moment, and I have no problems whatsoever using a clicker to teach what it is that I need. For public interest, if it helps.

What kind of dogs do you have?