Tearm paper
Mesfer Alotaibi
lecture_7_chapter_7_1.pptOperant Applications
Chapter 7
Animal Care and Training
Operant procedures are used to facilitate veterinary care of captive animals.
Restraints and aversives were used in the past.
Currently, operant learning procedures are used to handle captive animals.
Wilkes (1994) use of shaping with positive reinforcement to handle aggressive elephants.
Punishment and negative reinforcement rarely are used to handle animals today.
B.F. Skinner was –the first to note that environment shapes, or selects, behavior in much the same as it does with species characteristics. This chapter tries to answer the following questions: does operant learning provide insight into complex behavior and does it offer practical solutions to important problems?
Firstly, we look at animal care and training to answer these questions. In comparison to past methods, positive reinforcement is a more humane and less risky method for training and caring for animals.
Traditional methods are dangerous for the animal and caretaker.
A shaping procedure was used by Wilkes (1994) to smooth elephants’ calluses. A large steel gate was built at one end of a cage. The gate had a large hole through which an elephant’s foot could fit. The animal had to be shaped to walk up to the gate, put its leg through the hole and allow people to groom away the calluses. They first paired a clicking noise with food so that the clicking was a conditioned reinforcer and used successive approximations to groom the elephant. Before long, it took very few clicks for the elephant to adhere to the process accordingly. In addition, the elephant subsequently was not aggressive.
Self-Awareness
Self-awareness involves being aware of one’s own behavior. Are animals capable of such behavior?
Gallup (1970; 1979) was the first to demonstrate evidence of self-awareness in chimpanzees.
The study does not provide evidence for self-awareness of thoughts and moods but demonstrates that animals can carefully observe their own body.
Humans learn to be self-aware from other people, which involves many operant procedures.
Self-awareness is not considered to be a separate process from the awareness we have of others. It just may be more detailed because you have the advantage of internal reflection. However, judgments of the self are derived in the same manner as judgments of others. We observe others’ behaviors because we are reinforced for it. For example, we observe that Mary is in a good mood, which leads us to approach her; her bad mood leads us to avoid her (positive and negative reinforcement, respectively). We also observe our own behavior to receive reinforcement. If you are in a bad mood, you may avoid seeing certain people who might worsen the mood. When you have a flu, you seek out medication to alleviate the symptoms. When you graduate from University, you celebrate.
Gallup (1970; 1979) Procedure – It was once thought that only humans had self awareness. Gallup was the first to demonstrate self-awareness in animals. He first exposed chimpanzees to mirrors for several days. They first reacted to the image as if it was another chimp but eventually realized that it was themselves and began inspecting themselves in the mirror. For example, the groomed areas of their bodies that otherwise they could not see, picked food from their teeth, and made faces at themselves. Then he anesthetized the animals and placed a red dot over their eyebrow and on their ear. He first observed them for 30 minutes without and then with the mirror. Without the mirror, they did not notice the red dots on their bodies, but investigated the red dots quickly after the mirror appeared. A control group that went through the same processes without mirrors did not show any signs of self awareness when presented with mirrors at the end of the procedures. These findings demonstrate self awareness at a physical level, not a demonstration of self awareness of moods and thoughts. However, the findings do show that self-awareness can be learned.
Self-awareness in Humans – Humans learn self awareness through observing others. Skinner (1953) pointed out that we teach children things like “that hurts” or “that tickles” when we observe behavior that typically accompany such experiences. By observing and commenting on behavior suggestive of certain experiences, we teach the child to observe those private events. Skinner also suggested that children are taught to make predictions from observations through asking questions like “what are you doing?”, “Why do you do that?”, “How do you feel?, “DO you want to play?”. These questions direct children to observe thir own internal thoughts and feelings and then use these thoughts and feeling s to direct behavior. Correct/accurate observations will be reinforced, incorrect/inaccurate punished.
Self Control
Self Control – Acting in one’s own best long-term interest through choice.There are several methods used for self control, which are learned:
Physical Restraint
Distancing
Distraction
Deprivation and Satiation
Inform Others and Monitoring
Methods for exercising self control, all of which are learned:
Physical Restraint – This means to physically prevent the occurrence of a behavior. Some examples include giving keys to friend so you do not drive drunk, using gloves to prevent nail biting, lock the booze cabinet and give someone the key, etc.
Distancing – Sometimes, troublesome behavior only occurs in certain situations. In this case, avoidance of the situation will reduce the frequency of the target behavior. Examples include avoiding fast food restaurants, avoiding places where a drug problem may have occurred, etc.
Distraction – To prevent an outburst at a party when you are annoyed with a conversation you may change the topic, or begin talking to someone else. Another example would be exercise when you feel a craving for a cigarette.
Deprivation and Satiation – For example, if you want to eat less at a party, you may eat a small meal before the party is attended.
Inform Others and Monitoring – Other people are an important source of reinforcement and informing them of goals will ensure reinforcement.This may not seem like self-control, but others are an important component of the environment and self-control is the act of controlling the environment. Monitoring behavior is also effective for determining frequency, and often this leads to decreasing the behavior.
Self Control and Age – Bandura and Mischel (1965) demonstrated that young children take immediate small reinforcement, while older kids delay gratification to receive better reinforcement later on. It also was found that training self-control techniques leads to reduction of undesirable behaviors, such as aggression in adolescent boys.
Verbal Behavior
The traditional view of verbal behavior involves the following procedures:
An idea is encoded into words by the sender.
These words are sent to a recipient, known as the receiver, in the form of speech or writing.
The receiver decodes the message to achieve understanding.
This approach suggests that ideas are sent from head to head.
Other views of verbal behavior are viable.
Verbal Behavior
Skinner (1957) rejected the traditional approach to verbal behavior.
Verbal behavior is not different from any other behavior. It has a functional relationship with the environment (consequences).
Therefore, the social environment and behavior of other people shapes and maintains verbal behavior.
There is strong evidence for language development being dependant upon operant learning (Verplanck, 1955).
Skinner (1957) rejected the encoding/decoding explanation. He suggested words are behavior. Verbal behavior is no different from other behavior, whereby it can be understood based on its functional relationship with the environment. Verbal behavior is a function of its consequences (the social environment shapes verbal behavior). This shaping begins at birth, whereby certain approximations to language are selected by the parents and reinforced. The closer the approximation to the actual word, the more reward is given to the child by the parent and previous “less accurate” approximations are no longer reinforced. We learn to speak because speaking leads to reliable reinforcement.
Greenspoon (1955) asked college students to say as many individual words as possible. The positive reinforcement group received “Mmmhmm” after each plural noun. The punishment group received “Huh-uh” after each plural noun, and the control group received no comments. The frequency of plural nouns increased for reward, decreased for punishment, and there was no relationship for the control group.
Verplanck (1955) engaged people in a conversation for 10 minutes and recorded the number of times the person began sentences with expressions of opinion, such as “I believe that.” Then for the next 10 minutes, the experimenter reinforced expressions of opinion. Finally, the experimenter engaged in another 10 minute conversation without reinforcing opinion expressions. The results suggested that opinion expressions only significantly increased during the reinforcement phase.
Verbal Behavior
Quay (1959) and psychotherapy patients: uniquely important events or a simple matter of reinforcement?
Why are completing word associations such as black-white, up-down, etc. so predictable?
Much of our verbal behavior is learned without awareness.
Quay (1959) asked whether psychotic patients talk about their family experiences because they are important, or because patients were reinforced for talking about these events by the therapist. He asked college students to recall childhood experiences. One group were given verbal reinforcement for talking about family experiences. The other group were rewarded for talking about any experience except family experiences. The tendency to report family experiences increased with reinforcement.
Word associations are predictable because of our history with being reinforced to respond in specific ways. An example of a word association is black-white. You are more likely to receive reinforcement for replying white when presented the word black. Actually, replying with another word, such as hairy, may even lead to punishment.
All of the studies discussed regarding verbal behavior have confirmed that often it is learned without awareness (people in the studies did not realize their behavior was being shaped).
Who cares? These studies provide strong EMPIRICAL support for verbal behavior being learned based on operant procedures.
Insightful Problem Solving
Traditionally, problem solving has been viewed as ‘a mystery of the mind.’
Based on operant learning, a problem is a situation in which reinforcement is available but behavior required to receive reinforcement is not readily apparent.
Many problems are solved by trial and error.
When the solution of a problem comes to you like an epiphany, this is known as insightful problem solving.
Often, the behavior currently is not in the organisms repertoire. Think of Thorndike’s cats where they learned a behavior that they had never produced before – mystery of the mind? The cats used trial and error to escape the box. The cats scratched and pawed at various portions of the environment until they stumbled onto the solution. Most problems are solved using a trial and error process.
Describe trial and error learning based on Thorndike’s cats.
Insightful Problem Solving
Kohler (1927) and Sultan the Ape – Could this insight be driven by operant learning?
Peckstein and Brown (1939) challenged Kohler’s (1927) position of non-reinforcement of behavior.
Further evidence of operant learning regarding problem solving (Harlow, 1949).
Kohler’s (1927) insightful ape versus Epstein’s (1984) operantly conditioned pigeon.
Kohler (1927) gave sultan 2 bamboo rods, which could be connected to create a large rod. Outside of the cage was food that only could be obtained using the long rod. Sultan tried unsuccessfully for an hour to get the food. Then, Sultan sat down and seemed to be thinking while examining the sticks. Sultan seemed to spontaneously discover the solution to the problem, and this was suggested as evidence for insightful problem solving in animals. The response occurred suddenly without reinforcement – could it be operant learning?
Peckstein and Brown (1939) replicated Kohler’s study and found no evidence for insightful problem solving. The animals had to first learn to retrieve food with a single stick, then to put the 2 sticks together for play purposes, and then to combine the sticks to obtain food. Reinforcement and shaping, and not miraculous insight, led to the solution.
Harlow (1949) presented monkeys with 2 different lids where food could be found only under 1 lid (for example, food was always placed under the large lid, or the red lid, etc.). Learning to select the appropriate lid was slow. Performance improvement depended on reinforcement. These results suggest that Kohler’s ape’s prior learning/reinforcement history likely is the culprit of immediate learning.
Kohler (1927) suspended fruit from the ceiling. Chimpanzees could not reach the fruit. A large box that could be used to reach the fruit was placed in the cage. Sultan seemed suddenly to use the box to reach the fruit. The question is “before the fruit experiment, did Sultan have experience with using boxes to reach objects?”
Epstein (1984) taught pigeons to move a box toward a green light. Then the pigeons were to climb onto another box that was situated under a hanging banana. The bird was then required to peck the banana. Once this task was learned, a box was placed in the cage and a banana was hung on the ceiling. The animal quickly moved the box to where the banana was and completed the task. The bird was never trained to move the box toward the banana; this may explain Kohler’s experiments.
Creativity
To be creative is to be unique or novel or to produce unique/novel products/items. Does creativity defy scientific analysis?
Is creativity dependant on reinforcement? Is this an oxymoron?
How can you have a reinforcement history for completely novel thoughts and actions?
Reinforcing Novel Behavior (Pryor, 1991)
Reinforcing Creativity in Children (Glover & Gary, 1976)
Does reinforcement mute creativity? Not if creativity is reinforced!
If creativity is defined as novel behavior, how can we have a past reinforcement history for novel behavior? Pyor (1991) was an animal trainer in the 1960s. She ran porpoise shows and noticed that the show was not as entertaining as it once was. The trainers decided to demonstrate to the audience training techniques to spice up the show. They used Malia, a star performer and the show was a success. Once, Malia engaged in a complex trick that was never rehearsed or reinforced in the past. They reinforced this novel trick and found the she continued to engage in novel tricks for reinforcement. They replicated these findings with other porpoises and pigeons.
Glover and Gary (1976) asked forth and fifth graders to think of uses for various items, including a brick, can, pencil, etc. The children worked in teams and earned points for generating ideas. When they varied the number of points allotted to different items, for example, by increasing the number of points allotted to unusual items, the number of ideas for uses of the unusual items increased.
Creativity potentially is reduced by reinforcement, but this most likely is a confound in the research design. For example, if you ask people to perform a task for a reward and compare their performance to another group asked to do the same task but without any reward, the reward group’s results are less creative. The problem is that these studies do not make reward contingent on creativity. If you told subjects that the greater the creativity the better the reward, for example, they likely would work harder to produce creative results. Eisenberger and Armeli (1997) found that in reality, the surest way to receive reward is to act in conventional ways, such as the word association task previously discussed. However, they found that rewarding creative behavior results in more creative behavior. For example, if you were hired to paint the shutters on a house and thought that other trim work would also look nice in the same color as the shutters, you would still not proceed with painting anything but the shutter in fear of not being paid more for your work or that the home owner might get upset. Therefore, in this case, your best bet is to act conventionally, rather than creatively, to be rewarded.
Superstition
What happens if reinforcement is not contingent on behavior, or is coincidental?
Skinner (1948) taught pigeons to engage in bizarre ritualistic behavior, which he called it superstitious behavior, to receive reinforcement simply by rewarding coincidental behavior.
Superstitious behavior in university students (Ono, 1987)
Training superstitious behavior in pigeons (Hernstein, 1966)
Is this why we avoid floor 13 or throw salt over our shoulder?
Skinner (1948) put pigeons in a box and reinforced them every 15 seconds regardless of their behavior. Many of the birds did all what Skinner called superstitious behavior such as turn counter-clockwise and head bobbing because they seemed to think that t was those behaviors that led to reinforcement.
Ono (1987) sat students at a table with 3 levers. At the back of the table was a signal light and a counter (to record points). The objective was to earn as many points as they could, but points were not contingent on behavior. Periodically the light would shine and a point would be rewarded. The students developed superstitious behavior, such as stroking the levers.
Hernstein (1966) suggested that superstitious behavior is a by-product of training. A behavior that leads to reinforcement usually involves other behaviors. These other behaviors inadvertently become reinforced due to their association with the essential behavior. He suggests that the uniqueness of handwriting are superstitious behavior. To make a lower case t it is necessary to have a long vertical line and a short horizontal line. Why do people write with various non-essential forms, such as loops and tails? Hernstein (1966) trained pigeons to peck a disk to receive reward after the passing of 11 seconds. Then, the bird was given reward every 11 seconds regardless of the behavior emitted. Inadvertent/adventitious reinforcement was enough to maintain disk pecking behavior. Does this explain superstitious behavior, such as a wearing rabbits foot, in humans? The fundamental point is that we can protect ourselves from superstitious behavior using the scientific method.
Learned Helplessness
Seligman (1967) conducted the most widely cited experiments on learned helplessness.
The repeated inability to escape an aversive event leads to listless-type behavior, known as learned helplessness.
Learned helplessness occurs despite saliency of escape.
Learned helplessness has been supported empirically in studies using several types of species, including humans.
Learned Helplessness – Overmeier and Seligman (1967) strapped a dog into a harness and presented a tone followed by a shock. The shock always followed the tone no matter what the dog did to try to escape the shock. Then the dog was placed in a different box that contained two chambers, one that elicited shock (signaled by a tone) and one to where the dog could escape the shock. In spite of having an escape chamber and a tone to signal the shock, the dog did not try to escape the shock. Actually, the barrier between the chambers was removed so that the dog could walk to the safe side, but the result was the same. Seligman even stood in the safe zone and presented the hungry dog with salami to coax it to come to the safe chamber, but the result was the same. The dog learned to be helpless.
Learned Helplessness
Learned helplessness may help to explain human depression (Seligman, 1967):
Depression usually coincides with sadness and general inactivity.
Can learning prevent helplessness? Yes, according to Seligman and Maier (1967).
Immunization Training (Volpicelli, 1983)
It was suggested that this phenomena may explain human depression. Depression is characterized by inactivity. Depressed people do not engage in many activities and tend to be inactive when faced with problems/stress. Essentially, depressed people act like Seligman’s dogs.
Can learning experiences prevent helplessness? Seligman found that when dogs were given the opportunity to escape shock before the restrained/helplessness trial, the dogs did not learn to be helpless.
Immunization training – Volpicelli (1983) trained rats to escape shock using a lever and another group received inescapable shock. Both groups of rats then were placed in a shuttle box, but the shock could not be escaped because both chambers were electrified. Naïve rats tried to escape at first, but eventually became helpless. The inescapable rat group seemed helpless from the start. However, the group of rats who learned to escape shock continued to shuttle/escape between the chambers for the entire session (the rate refused to give up).
Delusions and Hallucinations
Delusions (false beliefs) and hallucinations (false sensations) often have an organic base but their frequency of occurrence tends to depend on the frequency of reinforcement.
Alford (1986) and the ‘haggly old witch’ – Reinforcing the delusions led to the patient’s problem.
However, bizarre behavior often tends to occur without known reinforcements.
Validating bizarre behavior requires engaging in the behavior even when reinforcement is absent (Goldiamond, 1975).
Alford (1986) worked with a schizophrenic who believed that he was being followed by a “haggly old witch.” The patient was asked to record the frequency of witch sightings and the degree to which the patient believed the witch was real. Alford began reinforcing patient statements of doubting the existence of the witch. The degree of confidence in the reality of the witch steadily declined. Could the patient be failing to admit his belief? The patient often took tranquilizers to remain calm (usually agitated by a witch sighting). The amount of medication consumed by the patient reduced throughout the therapy sessions, which suggests that the patient was starting to doubt the existence of the witch.
Goldiamond (1975) described a woman paralyzed in fear of the thought of cockroaches. She refused to get out of bed. Her husband began to give her extra attention and sympathy. Could attention be the reinforcer of the phobia even when she remained in bed without the husband being present? If people act in a bizarre fashion only when people are present to witness such behavior, people will likely catch on. Therefore, the bizarre behavior must persist even in the absence of reinforcement.
Self-Injurious Behavior
The most disturbing problems that clinicians face is self-injurious behavior, mildly severe being common.
This problem originally was dealt with by using restraints.
Punishment is an effective treatment for this behavior but aversive treatments are used only when other methods fail.
Self-injurious behavior often is accompanied by negative reinforcement, which led to creating non-aversive treatments.
Was Jim’s scratching a wicked case of invisible fleas or a behavior maintained by parental attention?
Self-injurious behavior, such as punching, pinching, and gouging oneself, often occurs in people with autism or mental retardation. This problem use to dealt with using physical restraints.
Lovaas (1960) found that punishment could quickly eliminate self-injurious behavior. A boy who hit himself 300 times every 10 minutes was given a painful shock immediately following the punch. It took 4 contingent shocks to eliminate the behavior. These methods are used only when other methods fail. This is because it was discovered that self-injurious behavior may not simply be a symptom of a disorder; the behavior may be being maintained by a reinforcement contingency.
Wolf (1967) found that this behavior tended to follow requests for performance in a school setting. When a teacher asked a student to do something, the child hit themselves. When the teacher stopped the request, the child stopped the physical abuse. The self-injurious behavior was being maintained by negative reinforcement (engaging in the behavior removed the punishment of having to work).
Carr and McDowell (1980) studied Jim. Jim began scratching after a poison oak incident. Lim continued to scratch long after the poison oak reaction healed. He came to therapy severely scarred and full of sores. All of Jim’s scratching appeared to occur at home and was reinforced by parental attention. When the parents were instructed to systematically withhold and provide attention for scratching, Jim’s scratching behavior increased with attention and decreased when attention was withheld. Extinction could not be used because of the severity of the case. The parents were instructed to use punishment in the form of a time-out. As well, Jim was to be rewarded for non-scratching behavior. This technique eliminated the unwanted behavior.
It is apparent that operant learning maintains most behaviors, good or bad.
