It all began - so the story goes - when a great Russian physiologist walked into his laboratory and a dog salivated. Most of us would merely have been flattered and continued with our physiology. But Ivan Petrovich Pavlov was not like most of us. He recognized this reaction as an important phenomenon. Before this incident, Pavlov had been awarded a Nobel Prize for his work in physiology. Most of us would have been content with that. But Ivan Petrovich Pavlov was not like most of us. He began a 40-year study to discover the secret of the saliva and thus laid one of the cornerstones of psychology [PAVLOV]. Have you ever heard of Twitmyer? He was a graduate student in the United States when he stumbled on the same phenomenon before Pavlov, considered it footnote-worthy to his doctoral thesis, and went no further. Twitmyer was like most of us.

Environment affects behavior. This statement is true but trivial. Pavlov suggested how it may be made more precise and thus more meaningful. Representing a dog or a person or whatever organism as a rather unflattering empty box, we could consider environment as a set of stimuli acting on it and behavior as a set of responses produced by it. Now we can substitute the precise statement "Stimulus X elicits response Y" for the vague statement "Environment affects behavior". We all know that an organism can come to behave differently in the same environment. That is, it can learn. How does it learn? Or, more precisely, how can stimulus X, which was previously neutral, come to elicit response Y?

Pavlov begins his answer by pointing out that, at birth, some stimuli are already capable of eliciting certain responses. If I tap you sharply below your knee, then you will raise your lower leg. The tap (stimulus) is prewired to the raising of the lower leg (response). No experience necessary. Such a prewired link between a stimulus and a response is called an unconditioned reflex (UCR).

If I blow a whistle, you will not raise your lower leg. However, if I were to blow the whistle, tap below your knee, blow the whistle, tap below your knee, blow the whistle, tap below your knee, and so on and on, then eventually you would raise your lower leg to the whistle alone. Such an acquired link between a stimulus and response is called a conditioned reflex (CR). It is acquired by the operation of presenting a stimulus that was originally neutral - the conditioned stimulus (CS) - together with a stimulus that is already wired to the response - unconditioned stimulus (UCS). This operation is called classical conditioning.

Pavlov continued to explore B - the process of "undoing" conditioning by presenting the conditioned stimulus without the unconditioned stimulus, generalization - dogs conditioning to a whistle of one tone are also conditioned to a lesser extent to nearby tones, and differentiation - dogs can be taught to differentiate between two tones.^₂

Psychology was originally considered as the study of consciousness, but a dynamic young man swept onto the psychological stage and transformed it into the study of behavior. He left as abruptly as he arrived - into the world of commerce, where he worked as a door-to-door salesman and finally became vice-president of the company. In his foray into psychology, however, John B. Watson left a permanent mark. As a psychologist, he was a good salesman. He demonstrated that sometimes an overstatement is more valuable than a true statement. He wrote the manifesto of behaviorism [WATSON JO], and much of psychology since has been an extended debate about his thesis. Most psychologists even today could be considered as behaviorists, neo-behaviorists, post-neo-behaviorists or anti-behaviorists.

Watson argued that consciousness is unobservable and hence irrelevant to science. We must focus on observables - stimuli impinging on an organism and responses elicited from the organism - and find the functional relationships between them. He had to demonstrate how certain stimuli, previously neutral, came to elicit certain responses. He stumbled upon the work of Pavlov and, thus, Pavlov was adopted as the reluctant grandfather of behaviorism.

Watson extended Pavlov's model of classical conditioning to explain not only the learning of simple, local responses like salivation, but also complex, whole-body responses like fear. We have many fears, ranging from specific things - like the number 13 (triskaidekaphobia) and being stuck in chimney pots (Santaclaustrophobia) - to general things - like everything (panaphobia) or fear itself (phobophobia). We were not born with those fears. How did we learn them? By classical conditioning, said Watson, and he proceeded to demonstrate just how.

He introduced an 11-month-old infant named Albert to a white rat - whose age and name were not recorded [WATSON JO & RAYNER]. Albert made the appropriate 11-month-old responses to the rat - he reached for it and cooed at it. Albert liked white rats. Then Watson presented the rat a number of times, fiendishly arranging for his assistant, Rosalie Rayner, to make a terrifying noise behind Albert each time. The noise frightened Albert and made him cry. After a few repetitions of the rat and the noise together, Albert began to cry at the appearance of the white rat alone. Moreover, he began to cry at the appearance of a white rabbit, a ball of cotton wool, a false beard, a man with a beard, a man who had accompanied a man with a beard - that is, at anything white and fluffy or at anything associated with anything white and fluffy.^₃

Behavior, however, does not consist of isolated responses, whether small or large, but a stream of responses. Watson explained that stream of responses which we call a habit as a chain of conditioned reflexes (chain reflex). As each response is made, a stimulus is fed back to the brain to inform it that the response has been made. The links in the chain are formed as each such fedback stimulus becomes classically conditioned to the next response.^₄

A certain subset of habits, involving the muscles of the larynx and throat, is the basis for speech. Talking is the moving of the muscles of the throat, just as walking is the moving of the muscles of the legs. Since talking to oneself out loud is frowned up, we do it in a small inner voice. Thinking is simply talking to oneself so that no one else can hear. Thus Watson attempted to explain all behavior in terms of classical conditioning. He was, of course, only partly right. He grasped some truth but not the whole truth. Classical conditioning determines some behavior but not all behavior. Let us turn to another kind of conditioning which also determines some behavior.

It is necessary to reset the stage for the next character in our cast. The theory of evolution placed us where we belong - with the other animals on the same phylogenetic scale. This discovery has two implications: humans are seen as more animal-like and animals are seen as more human-like. The violent repercussions of the first implication are very familiar - Thomas Huxley versus Bishop Wilberforce, Scopes versus State of Tennessee. Let's briefly consider here the less familiar repercussions of the second implication.

Certain scholars begin to attribute human qualities to animals. They soberly collected anecdotes from retired colonels, minister's wives and other animal-lovers that demonstrated how ingenious animals were in solving problems. A typical anecdote describes how a field mouse got honey out of a narrow-necked jar by squatting on the rim, dipping its tail into the honey, and licking its tail. G. J. Romanes, the leader of this movement, the father of comparative psychology, the gossip columnist of the animal world, collected those stories in his book Animal Intelligence [ROMANES]. He concluded, on the basis of this anecdotal evidence, that animals are very intelligent.

Enter Edward L. Thorndike, a graduate student at Harvard University, arguing: Such anecdotes describe the behavior of an animal after it has learned. If one were to study the process rather than the product of learning, the animals would perhaps not appear so intelligent. Thorndike set out to study the process of learning in animals, by collecting a motley menagerie in his squalid room in a run-down boarding-house.

Enter Thorndike's landlady, the first villain of our story, lacking sympathy for the scientific spirit and throwing Thorndike and his animals out into the street. Enter William James, one of the greatest and kindest characters in the story of psychology, coming to the rescue by housing the menagerie in the basement of his own home and arranging for Thorndike to continue his research at Columbia University.

The rest of the story is history. Thorndike rounded up stray cats from the back alleys of New York City. He built a box with a door that could be opened by pressing a lever. Inside the box he placed a cat; outside the box he placed things that the cat liked (typically one or more of his famous three Fs - fish, friends, and freedom). The problem was to get out, and the solution was to press the lever.

When first put in the puzzle box, the cat went through its repertoire of responses: clawing at bars, hissing, arching its back, spitting and snarling, smiling at Thorndike, purring and meowing, and so and so on, more or less at random. Finally, by chance, it hit on the Thorndike-ordained correct response. Each time it was put back into the box, it took less and less time to get out because it spent more and more time closer and closer to the lever and was thus more likely to trigger it by chance. Eventually it went immediately to the lever and pressed it.

If Romanes had entered Thorndike's laboratory at this point, and watched the cat strolling nonchalantly over to the lever and casually pressing it, he would have run off to write yet another anecdote to show how very intelligent animals are. Thorndike, who had observed the mechanical process by which this apparently insightful product was established, knew otherwise. Romanes had written a book called Animal Intelligence describing how smart animals are; Thorndike now wrote his book also called Animal Intelligence demonstrating how stupid animals are [THORNDIKE].

In that book, Thorndike presented his now-famous description of trial-and-error learning. His theory of learning was somewhat analogous to the survival-of-the-fittest principle which is central to Darwin's theory of evolution. In a population of organisms, some are fitter to survive in a given environment; in a repertoire of responses, some are fitter to survive in a given situation. The fittest response is the one leading to reward.

When a response is followed by a reward, it is more likely to occur again (law of effect). In other words, the link between the stimulus situation and this reward-followed response is strengthened, and, since the total probability of all possible responses must add up to 1, the links between the stimulus situation and all the other responses are thereby weakened. Thus the fittest response survives and the other responses die.

Since all the responses are eventually followed by the reward, the law of effect must be supplemented by the gradient of reinforcement: the closer in time the reward to the response, the greater the strengthening effect. This principle implies that rats will learn the last turns in a maze first and the first turns last. Indeed they do. This principle implies that rats will run faster and faster in a straight runway as they approach the goal box at the end. Indeed they do.

Thus, we have a second answer to the question "How does a particular stimulus, previously neutral, come to elicit a particular response? " The response is a means of gaining access to a stimulus that already elicits some response that is intrinsically rewarding. This is instrumental conditioning. The two types of conditioning are seen in contrast in Figure 2-2.

As the most famous exponent of pure behaviorism, B. F. Skinner is seen by many lay people as a sinister ogre scheming to manipulate their behavior. They see him as Big Brother watching them. The publication of his novel, Walden Two, enhanced this reputation [SKINNER 1960]. He described a utopian society, a sort of benevolent Brave New World - based on his conditioning techniques. In today-the-community-tomorrow-the-world fashion, he has argued convincingly in a later book, Beyond Freedom and Dignity [SKINNER 1972], that the problems of our planet can be alleviated by the appropriate application of those conditioning techniques.^₅

Skinner transformed Thorndike's puzzle box into what has come to be called, in his honor and to his dismay, the Skinner box. It contains a lever and a tray arranged so that pressing the lever permits a food pellet to drop into the tray. The index of learning in the Skinner box is the number of times the lever is pressed per unit of time, rather than, as in the puzzle box, the time to get out of the box.

Originally the rewards were dispensed and the responses were recorded by Skinner himself. Now, however, the Skinner box has become completely automated, and the rat can run its own experiment without the aid of an experimenter. When the lever is pressed, two metals surfaces make contact, a circuit is completed, a disk turns, and a pellet drops into the tray. Thus the rewards are dispensed. When the bar is pressed, a pen pressed against a tape moves up one notch. Since the tape is moving horizontally at a constant speed, the pen leaves a cumulative record of the number of bar presses per unit of time. Thus the responses are recorded.

Total automation is prevented only by the fact that the rat must be taught to press the lever. This is done by a process called shaping, using the method of successive approximations. The rat glances toward the lever. Give it a pellet. It looks at the lever. Give it a pellet. It takes a step toward the lever. Give it a pellet. It sniffs the lever. Give it a pellet. It raises its paw in the direction of the lever. Give it a pellet. Each response that is a successively closer approximation to the desired response is rewarded, until the rat is pressing the lever and supplying its own pellets.

At first Skinner arranged for the rat to get a pellet every time it pressed the lever (total reinforcement). However, he got tired of making so many pellets and decided to give the rat a pellet only some of the times it pressed the lever (partial reinforcement). In this way he stumbled inadvertently into a more true-to-life situation. The fisherman does not get a bite every time he casts his line, the saleswoman does not make a sale every time she delivers her sales pitch, and the suitor does not get a date every time he asks. We live in a world of partial reinforcement.

Schedules of reinforcement may be ratio schedules or interval schedules. That is, reward may be a function of response or of time - a pellet may drop after every 20 bar presses or after every 20 seconds. Schedules of reinforcement may also be fixed schedules or variable schedules. That is, a pellet may drop after every 20 presses or every 20 seconds, or after, on the average, every 20 presses or every 20 seconds. Ratio schedules tend to produce a higher rate of responding than do interval schedules, and variable schedules tend to produce a higher rate of responding than do fixed schedules. Thus rats work better on piecework than on salary and when they are paid sporadically rather than regularly. The most powerful schedule of all - the variable ratio schedule - is used in gambling casinos to produce a high rate of feeding coins into one-armed bandits and in homes to produce a high rate of crying in babies.

Skinner boxes have been adapted to an number of purposes. There is the gigantic Skinner box to contain all Skinner boxes the utopian society portrayed by Skinner in Walden Two. There are Skinner boxes for babies - in which they can be raised in a well-regulated environment (untouched, add Skinner's critics, by human hands). There are Skinner boxes for schizophrenics. A bare room is fitted with a lever and a cup so that, on pressing the lever, a reward (cigarette, candy, or whatever the patient likes) falls into the cup. Hopeless schizophrenics, who have not done anything for themselves for decades, will work hard for such goodies and thus take a small step back toward caring for themselves again. There are Skinner boxes for students - the much-discussed teaching machines. Since none of us are babies, few of us are schizophrenics, but all of us are students, let's focus on the teaching machine. Let's focus, more specifically, on the teaching program, since the teaching machine is merely a mechanical device for presenting, in order, the set of frames of which the program is composed. We are interested in the radio script rather than in the radio. The program consists of a series of statements and questions to which the students make some response. They then turn to the next frame to check whether the answer is correct and read the next set of statements and questions.

The situation is analogous to that of the rat in the Skinner box. The response is writing the answer rather than pressing the lever. The reward is learning that the response is correct rather than receiving a food pellet. The various principles in designing a program are derived from work with rats in the laboratory. The principle that the reward should follow as soon as possible after the response is a direct application of the gradient of reinforcement. The principle that each frame should go only a little beyond the previous frame is a direct application of the method of successive approximations.

B. F. Skinner once found himself sitting at a banquet next to the great philosopher Alfred North Whitehead and launched into an enthusiastic exposition of his project to explain all behavior in terms of conditioning. The calm old philosopher listened benignly to the brash young scientist. He conceded that non-verbal behavior may possibly be explained in terms of conditioning but not verbal behavior. By way of example, he challenged Skinner to explain, in his terms, why Whitehead chose at that moment to say "No black scorpion is falling on this table ".

The next morning, Skinner began his book Verbal Behavior [SKINNER 1957], in which he presented the following response to Whitehead's challenge. Verbal behavior is behavior reinforced through the mediation of other people. There are two ways in which Skinner could have got the salt at that famous banquet - reaching for it himself (non-verbal behavior) or by asking Whitehead to pass it to him (verbal behavior). We use words then to gain reinforcement through the mediation of other people. This answer would appear to be very far from meeting Whitehead's challenge. Skinner argues, however, that the scientist is not required to explain each specific event within the domain of the science, but only the general principles underlying the specific events. The physicist is not expected to predict the order in which leaves will fall from a tree and the pattern they will form on the ground, but only to provide the general laws governing falling bodies.

Thus Skinner attempts to explain all behavior in terms of instrumental conditioning, just as Watson tried to explain all behavior in terms of classical conditioning. He, like Watson, is only partly right. He too has grasped some truth but not the whole truth. Instrumental conditioning determines some behavior but not all behavior.

3   Watson and Rayner did not extinguish this fear response. Does anyone know a twitchy old man called Albert?

4   This theory may sound preposterous. However, many of our habits are this mechanical. Am I the only person who has gone into his bedroom to change his shoes, mechanically gone through the undressing chain reflex started with the response of taking off the socks, and found himself in bed? Am I the only person who can't type "ratio" without putting an "n" at the end? At first, I had a Freudian explanation about growing up living on "rations" during the war. I now realize that the sequence of responses a t i o is almost invariably followed by n.