Sebastian Grossman
on electrical brain stimulation...

Reinforcing Effects of Electrical Brain Stimulation

Introduction

In 1954, Olds and Milner reported an observation that arguably represents the most intriguing and enigmatic phenomenon biopsychologists have yet encountered. They noted that a rat preferred the region of the test apparatus where it received electrical brain stimulation, and inferred that it might find the experience pleasurable. When the experimenters restructured their experimental paradigm to test this possibility, it was determined that the rat would learn and execute novel behaviors in order to obtain brief pulses of brain stimulation. Olds and Milner plausibly concluded that they had discovered brain mechanisms responsible for "reward."

It is difficult to exaggerate the implications of this observation. Complex biological organisms could not have survived on earth had they not developed the ability to learn from experience - i.e., to repeat behaviors that have positive consequences such as finding food, water, or a mate, and to eliminate behaviors that have negative consequences such as exposure to predators, or extremes of heat or cold. Learning implies a fundamental reorganization of the relationship between the organism and its environment. Olds and Milner's discovery promised insights into the basic neural mechanisms that are responsible for our capacity to achieve that reorganization.

Before we can meaningfully ask what has become of this promise, we must take a small detour into the domain of the philosopher. If we agree that the behavior of complex organisms is the result of learning, it follows that the conditions that promote learning are at the very heart of psychology. Philosophers have recognized this imperative by debating whether all behavior should be considered to be determined by a desire to maximize pleasure and minimize pain. A good many of them are uncomfortable with the implication of this dictum that humans might not exercise voluntary, discretionary control over their behavior. For some, this concern can be assuaged by definitions of reward and punishment that are expanded to include such acquired elements as satisfaction with one's accomplishments and disappointment at one's failures. Others will insist that humans are endowed with rational capabilities that can supersede the basic relationship between behavior and its consequences however broadly defined.

Scientific psychology must insist on a strict definition of cause-and-effect relationships. This was formally recognized by Thorndike's statement of the law of effect around the turn of the Century which stated, in essence that rewarded behavior is "stamped in" (i.e. learned) whereas punished behavior is eliminated from the organism's repertoire of responses (Thorndike, 1898). This idea provides the cornerstone of many subsequent theories of behavior that attribute a pivotal role to reward and punishment. B.F. Skinner stated the principle most succinctly in his influential book:"The Behavior of Organisms" as the now famous dictum that: "behavior is controlled by its consequences" (Skinner, 1938).

Few contemporary biopsychologists would quarrel with Skinner's conclusion, particularly if some of its bite is removed by amending it to state that behavior is controlled by anticipation of its consequences. However, Skinner's insistence that the intimate relationship between behavior and its consequences can be understood without recourse to intervening variables (such as the notions of pleasure and pain that were intrinsic in the original formulations) has been the subject of much debate. Because the issue has obscured some of the fundamental implications of the research that originated with Olds and Milner's historic observation, we need to bring it to the forefront of our discussion before proceeding.

Skinner postulated that behavior is strengthened by reinforcers. Food, water, or a desirable mate are positive reinforcers. Painful electric shock is a negative reinforcer. Whether a particular stimulus is neutral, a positive reinforcer, or a negative reinforcers in a particular situation can only be determined by experimental analysis. The data that are thus collected permit a listing of environmental conditions that predict the status of a stimulus. For instances, food maybe neutral when the animal is well-fed; a positive reinforcer after prolonged food deprivation, and a negative reinforcer after repeated association of a particular food with illness. There are no ambiguities in this definition, and this is its appeal of Skinner's argument if one can put up with its circularity.

Contemporary alternative explanations require recourse to intervening variables (Hebb, 1955; Stellar, 1954). They postulate that stimuli are positive reinforcers if they elicit physiological responses (i.e., brain activity) that gives rise to subjective sensations that humans experience as rewarding. We refer to that state as "pleasure" or positive affect. The status of a particular stimulus at any point in time is defined, not by features of the external environment, but by the prevailing condition of the organism. When the body has ample energy reserves, there is no perception of hunger, and food is a neutral stimulus. Depletion of the body's energy stores gives rise to hunger sensations that motivate the search for, and ingestion of food which thus is a positive reinforcer. After a particular food has been associated repeatedly with illness, a conditioned aversion has been learned that gives rise to negative affect. This motivates avoidance of the food which thus is a negative reinforcer.

It is easy to dismiss this distinction as a largely semantic quarrel between behavioral scientists. It is often suggested that one can reconcile the two positions by agreeing to the following: "A positive reinforcer, such as food, (a) strengthens the behavior that produces it and (b) results in positive affect. A negative reinforcer, such as painful electrical shock, (a) strengthens the behavior that avoids or terminates it and (b) results in negative affect." It is not necessary (although convenient for some purposes) to assume that "a" and "b" are causally related." Unfortunately, this not only misses the theoretical point of the issue but also runs afoul of reality. Although positive reinforcers are generally associated with positive affect, the connection may be quite indirect. You may study the material in this chapter in the hope of someday obtain a good grade (or avoiding a bad one) but the actual payoff is uncertain and the grade has itself only a tenuous relationship to positive affect. We might learn to live with this ambiguity for the sake of harmony but matters are even more complex when we consider negative reinforcements. It is well established that animals as well as humans learn and perform behaviors that produce consequences associated with neutral or even negative affect. Rats, for instance, can be trained to press a lever to receive brief bursts of sound or light that have no obvious hedonic quality, and may even work to obtain painful electric shocks under some circumstances. Humans similarly engage in "titillating" risk-taking behaviors that expose them to the threat of physical or psychological harm, scuba diving, mountain climbing, and experimenting with psychotropic drugs (discussed below) being some rather obvious examples.

It is not surprising that the problems we have just touched on should be an issue in the area of research we are about to discuss. What is rarely appreciated, however, is how pervasive their influence has been. Milner has recently reviewed the progress of the field he co-founded more than 40 years ago and commented that: "During most of the first half of this century psychologists knew what they wanted to do but had no idea how to do it, and during the second half they have, for the most part, been so preoccupied with how to do it that they have forgotten what they wanted to do." (Milner, 1991). It is, indeed, sad to see that the initial enthusiasm that greeted Olds and Milner's discovery has gradually given way to resigned acceptance of our inability to agree on its implications. The topic is not currently "fashionable" but an appreciation of the progress that has, in fact, been made in the past 40 years will assist us in our search for an understanding of the intricate relationship of behavior and its consequences.

Brain Stimulation Reinforcement in Animals

Literally tens of thousands of research papers were published on the topic of brain stimulation reinforcement in the first two decades after Olds and Milner's seminal observations (Wauquier&Rolls, 1976). Brain stimulation reinforcement has been demonstrated in all species examined, including humans (Sem-Jacobsen, 1976) ; monkeys (Lilly, 1958) ; cats (Roberts, 1958) ; teleosts (Boyd&Gardner, 1962) , and snails (Balaban&Maksimova, 1993). Even the activity of single brain cells has been shown to be modifiable by electrical brain stimulation (EBS) (Olds, 1965). One can safely conclude that the generality of the phenomenon is not in question.

Relationship to Natural Rewards

Much of the early research was aimed at demonstrating similarities and/or differences between EBS and natural rewards such as food. A consistent finding is that the reinforcing effects of brain stimulation can be far more powerful than those produced by food or water. In one experiment, rats pressed a lever for brain stimulation reinforcement almost without pause for 20 days, averaging 29.2 responses per minute (Valenstein&Beer, 1964) ! When the reinforcing effects of brain stimulation are pitted against those of natural rewards, hungry rats (Routtenberg&Lindy, 1965) as well as humans (Bishop et al., 1963) disregard palatable foods in order to work for brain stimulation reinforcement. Yet, the reinforcing effects of brain stimulation are not, in every way like those of natural rewards. In many instances, animals do not work for intermittent brain stimulation reinforcements on low density reward schedules (e.g., when many responses are required to obtain a single reinforcement) (Gallistel, 1964; Sidman et al., 1955). This may be related to the fact that behavior maintained by brain stimulation reinforcement is subject to rapid extinction when the behavior is no longer reinforced (extinction refers to the usually gradual disappearance of learned behavior that is no longer reinforced). Indeed, in many cases well-trained animals will not resume EBS reinforced behavior even when there is no opportunity to emit non-reinforced responses during an enforced pause. The apparently extinguished behavior can be quickly reinstated by one or more free "priming" stimulation (Howarth&Deutsch, 1962)

The need for priming has been a hotly debated and much researched subject because it implies that brain stimulation may not only reinforce the behavior that produced it but also provides the motivation to emit the behavior again (Gallistel, 1969; Wise, 1982). Sated rats, that are trained to work for large and very palatable natural rewards (such as a goodly amount of chocolate milk) behave quite similarly (Panksepp&Trowill, 1967). This has led to the suggestion (Trowill et al., 1969) that animals may not work for brain stimulation reinforcements because they are "driven," but because the incentive value of the reinforcement is very high. (Just as we might eat potato chips or nuts at a party even although we are not hungry.) It is also possible that the "anomalous" effects of brain stimulation reinforcement may reflect ambivalent reactions to an undoubtedly unnatural activation of brain pathways. Such an interpretation suggests that EBS may elicit both positive and negative affect and thus creates an approach-avoidance conflict. If the reinforcing effects are initially more potent than the aversive consequences (as they presumably must be in order to support self-stimulation), but decay more rapidly (as the effects of positive natural reinforces tend to do), one would predict rapid extinction and a need for priming. A number of investigators have, in fact, reported that animals can be trained to press one lever to obtain brain stimulation and press another to turn it off (Bower&Miller, 1958; Valenstein&Valenstein, 1964). Moreover, in one study, some animals that did not require priming stimulation in order to initiate EBS-reinforced behavior, did so when each lever press that resulted in reinforcing brain stimulation also produced painful tail shock (Kent&Grossman, 1969) In spite of much research on the subject, it is not entirely clear whether the need for priming is, indeed, a defining characteristic of all EBS reinforcement

Relationship to Natural Drives

Neither humans nor animals have an innate desire to have their brains stimulated. Why, then, do we do it and what brain functions are activated by reinforcing EBS? Does it stimulate neural pathways specifically related to positive affect (or the processes of positive reinforcement), or should one search for a direct connection with naturally occurring motivational states? At some electrode sites, the rate of self-stimulation increases dramatically as a result of food deprivation and decreases promptly after a meal or after intragastric or intravenous injections of nutrients (Hoebel, 1968; Hoebel&Teitelbaum, 1962; Olds, 1958a). At other sites (which may be in the same animal), the rate of self-stimulation varies as a function of water deprivation (Brady, 1961), castration (Olds, 1958b), sex hormone replacement therapy (Caggiula, 1970), or changes in ambient temperature (Bloomfield&Mrosovsky, 1974).

The interaction between hunger and the reinforcing properties of lateral hypothalamic stimulation has been examined in great detail. We now know that at some electrode sites, any experimental treatment that modifies hunger also affects the reinforcing effects of EBS, and does so in a lawful manner. Self-stimulation increases after ventromedial hypothalamic (VMH) lesions (Hoebel&Teitelbaum, 1962) or insulin injections (Hoebel, 1968) (both increase food intake), and decreases after vagotomy (which reduces food intake) (Ball, 1972), and after injections of the satiety-related hormone glucagon (Balagura&Hoebel, 1967) or appetite depressant drugs such as amphetamine (Mogenson et al., 1969). We have less complete data on the relationship of brain stimulation reinforcement to other basic motivational states. However, the available evidence leaves little doubt that the extensive and close relationship between brain stimulation reinforcement and hunger is a good model for other basic drives as well.

This conclusion is supported by another set of intriguing experimental observatons: Reinforcing brain stimulation often elicits behaviors such as eating (Margules&Olds, 1962)., drinking (Mendelson, 1967), or copulation (Caggiula&Hoebel, 1966). Stimulation at such electrode sites is more reinforcing when an appropriate natural reward is available and consumatory behavior is permitted (Coons&Cruce, 1968; Mendelson, 1967). On closer analysis, it can be demonstrated that the intensity threshold for eliciting consumatory behaviors at a given electrode site is often different from the threshold for brain stimulation reinforcement, suggesting that different types of neurons may be activated (Coons&Cruce, 1968; Cruce&Coons, 1974). Other changes in the physical properties of reinforcing electrical stimulation (e.g., the frequency or duration of pulses) also produce major changes in the apparent magnitude of its reinforcing properties. This provides additional evidence that more than one type of neural pathway may be stimulated because larger and/or more heavily myelinated neurons can respond to higher stimulation frequencies than smaller cells (Gallistel, 1983; Mark&Gallistel, 1993; Yeomans, 1975).

We should note, at this point, that rate of responding (e.g., lever-pressing) is not a good measure of the reinforcing effects of brain stimulation (Gallistel, 1983). We tend to think that "better" stimulation might make an animal work harder but it is just as plausible that at some point, the stimulation becomes so very good that a little bit goes a long way - rate of responding would then drop off even though each stimulation produces more reward. Although the notion that more may not always be better is intuitively appealing, this point has been difficult to demonstrate experimentally. We now have many complex psychophysical procedures capable of demonstrating that the correlation between rate and reinforcement is, in fact low (Shizgal&Murray, 1989; Yeomans et al., 1979; Yeomans, 1975). Most contemporary investigators therefore rely on rate-free measures. Some use simple tests, such as the place-preference procedure employed in the first experiment by Olds and Milner - the rat is simply rewarded for going into or staying in a particular part of the test apparatus. Others have developed sophisticated analyses of lever pressing behavior that permit a critical assessment of the relationship between the reinforcing properties of EBS and the physical properties of the affected neurons (such as their refractory periods, axon diameter, etc.) From such an analysis, one can predict, with some success whether increased stimulation intensity does, in fact, increase its reinforcing properties (Forgie&Shizgal, 1993; Gallistel&Leon, 1991a; Gallistel et al., 1991b).

The results of these experiments indicate that reinforcing brain stimulation may have two distinct effects: (a) it activates pathways related to natural drives, and (b) it stimulates reinforcement pathways normally activated by natural rewards. The empirical observations seem to contradict classic "drive-reduction" theories of reinforcement (reinforcement appears to be associated with increased drive in the EBS paradigm). However, it is not difficult to construct a plausible alternate hypothesis: Animals may self-stimulate because the stimulation provides the experience of an intense drive that is instantly reduced due to the concurrent activation of related reward neurons. This interpretation accounts neatly for many of the apparent paradoxes we have already encountered. Priming is necessary, according to this interpretation, because EBS reinforcement not only activates reward pathways but also provides the reason why that should be pleasurable (Deutsch, 1976). (This also accounts for rapid extinction, as well as the decreased efficacy of intermittent reinforcement.) The hypothesis assumes that the reinforcing properties of EBS are determined by the degree of activation of related motivational systems. It therefore accounts readily for the observed interactions between the reinforcing properties of a stimulus and various experimental conditions that affect related primary drives such as hunger. When there is little endogenous activity, for instance immediately after a meal, the stimulation elicits only a small amount of drive-related activity. Concurrent activation of related reward circuits therefore can produce only a small reinforcement effect. When hunger-related neural pathways are already active because of deprivation, the same stimulation elicits more drive and hence more reinforcement. Indeed, it may arouse the drive system sufficiently to elicit consumatory behavior that further potentiates the reinforcing effects of the electrical stimulation.

It is important to emphasize, at this point, that most of what we have said about the relationship between brain stimulation reinforcement and natural rewards and drives applies only to electrode sites in the hypothalamus and adjacent areas of the brainstem and forebrain. This region has been the target of most investigations because these placements are far more effective than electrode sites in other regions of the brain (See our discussion of neuroanatomical observations, below). Animals and humans work to obtain stimulation of many other areas of the brain but the acquisition of EBS-reinforced behavior is usually slow and rates of responding typically low. There is little or no evidence that stimulation rates (or more sophisticated measures of the reinforcing properties of the stimulation) covary with endogenously generated drive states. This raises interesting questions about the possible relationship of brain stimulation reinforcement and motivational states that may be less readily identified and manipulated than hunger or thirst. There are, as yet, few answers because such questions are difficult to translate into animal experiments. (If one predicts that EBS may be reinforcing because it satisfies a need to be loved or appreciated, how does one design an animal experiment to test the hypothesis?) Since primary drive states are not represented in many of the cerebral and cerebellar regions that sustain EBS reinforced behavior, it must be assumed that the simple model we have just described may not pertain to other brain regions. Indeed, the limited data we have on human responses to brain stimulation (below) indicate that EBS may owe its reinforcing properties to a wide variety of reasons - a not altogether surprising conclusion in view of the multitude of motivations humans can cite for their behavior.

Conclusions

The reinforcing effects of electrical brain stimulation are (a) ubiquitous (they have been reported in every species tested to date); (b) powerful (humans, as well as animals, may voluntarily starve when given the option to eat or work for EBS); and (c) effervescent (often there is no indication of satiation but also no persisting desire to obtain EBS). The reinforcing properties of EBS are often related to natural drives and rewards - at least in the most extensively studied regions of the brain. Stimulation is often more reinforcing when a natural drive, such as hunger, and a natural reward, such as food, are present. Indeed, reinforcing EBS often elicits consummatory behavior such as eating or mating.

Brain Stimulation Reinforcement in Humans

When psychiatrists and neurologists learned of the reinforcing effects of brain stimulation in animals, they immediately saw a potential application of the technique in psychotic patients that were unresponsive to psychiatric treatment, at least in part because they failed to respond to normal social rewards such as praise or disapproval. At a time when the frontal lobotomies that Moniz had introduced in the 1930s began to loose favor, a new neurological miracle seemed to become available. In the United States and many European countries, the enthusiasm for clinical applications of rewarding brain stimulation lasted barely two decades. We nonetheless have published data on several thousand patients that were trained to push a button to obtain electrical brain stimulation and often worked avidly for hours on end to receive it. (Delgado, 1976; Sem-Jacobsen, 1968). Could we not simply ask them how it feels or why they work for it? The questions have obviously been asked many times but the answers are ambiguous and complex. (That should, of course, not be surprising. Could one expect to elicit coherent affective experiences that can be communicated in terms of their similarity to normal emotions, by a procedure that causes grossly unphysiological activation of some brain region and is not, in any meaningful way related to ongoing mental activity? It is, indeed, remarkable is the fact that EBS does, in fact, produce sufficiently positive reactions in humans to sustain effortful behavior.)

Due to obvious ethical concerns, electrodes are not implanted into the brains of humans unless they are afflicted with a severe and debilitating illness. Indeed, we currently have a moratorium on most kinds of "psycho-surgery" (brain surgery not related to life-threatening physical conditions such as tumor growth or stroke) in the United States. Permanent electrode implantations are permitted only for the control of intractable pain (e.g., in terminal cancer patients). Electrical brain stimulation can be applied briefly to verify the placement of electrodes that are used to make lesions for the control of epilepsy or other severe neurological disorders but this rarely provides information that would be helpful in the context of our discussion. This effectively restricts our data base to early clinical trials of rewarding electrical brain stimulation in humans, conducted mostly before many of our contemporary questions about the phenomenon had taken shape. The usefulness of these data is further limited by practical considerations. Neurosurgeons did not want to incur unnecessary risk of vascular damage, particularly in deep brain structures known to control basic biological functions. Yet, they wanted to optimize their chances for success by implanting as many electrodes as possible for future study (some implanted arrays of 50 electrodes). Combined with a lingering predilection for assigning complex affective reactions to the cerebral hemispheres, these considerations dictated predominantly cortical electrode placements. Only few were directed at such limbic system structures as the septal area and amygdala, and almost none at the hypothalamus which provides more than 90 % of the data from animal laboratories (Heath, 1963; Sem-Jacobsen, 1968).

Some patients respond to electrical brain stimulation by spontaneous and often abrupt expressions of general pleasure, well-being, fondness of the interviewer, and general approval of their present situation (Higgins et al., 1956). Others report "pleasant" sensations in various parts of their body that are not amenable to definition in terms of normal human experience (Heath, 1963). Although casual reports of sexual arousal during brain stimulation abound in the clinical literature, one careful examination of human responses to stimulation of over 2000 electrode sites revealed only 2 that had unambiguous sexual content (Sem-Jacobsen&Styri, 1972). Most common among clearly positive behavioral reactions to brain stimulation in human patients is what psychiatrists call a positive change in mood. This may include frequent laughter and expressions of positive feelings towards the treatment in general. The effect often includes an increase in the patients' willingness to discuss his problems with the interviewer and to be friendly and cooperative (Sem-Jacobsen, 1968). This can be viewed as a very positive outcome in many mental patients but provides little insight into the nature of their experiential reaction to the stimulation. Patients typically cannot identify the reasons for their apparent well-being. One of the pioneers in this field has suggested that: "curiosity about the strange sensations that are aroused by brain stimulation, rather than pleasure per se, may be the dominant cause of human self-stimulation" (Sem-Jacobsen&Styri, 1972)

It is interesting to note that while the animal literature suggests that brain stimulation has positive, reinforcing effects, the human literature indicates that relief of anxiety, depression and other unpleasant affective conditions may be the most common "reward" of electrical brain stimulation in humans. Patients with electrodes in the septum, thalamus, and periventricular gray of the midbrain often express euphoria because the stimulation seems to reduce existing negative affective reactions (even intractable pain appears to loose its affective impact). However, many psychiatrists caution that this may not reflect an activation of a basic reward mechanism (Delgado, 1976; Heath et al., 1968). Relief from chronic anxiety has been reported during and even long after stimulation of frontal cortex. Again, the experiential response appears to be relief rather than reward per se (Crow&Cooper, 1972).

We might briefly note that psychiatrists have reported apparently pleasurable affective reactions to reinforcing brain stimulation in primates other than humans. Lilly, for instance, conduced a detailed analysis of hundreds of electrode sites in monkeys and concluded that his subjects showed clear evidence of "contentment, increased interest, reduction of anxiety, improved cooperation with the observer, improved appetitive, etc." His animals initially worked for EBS reinforcements until exhausted (emitting as many as 200,000 responses before stopping) before eventually settling down to a regular rhythm of about 16 hours of "work" and 8 hours of sleep (Lilly, 1958).

Anatomical Distribution of Brain Stimulation Reinforcement

Lateral Hypothalamus

The highest rates of responding for brain stimulation reinforcement have consistently been found in the lateral hypothalamus (Olds et al., 1960; Olds&Olds, 1963). The region is traversed by the medial forebrain bundle (MFB), a major fiber system that interconnects the brainstem with most areas of the cerebrum that have also been implicated in EBS reinforcement. It is thus tempting to agree with Olds, who conducted many of the seminal mapping experiments in this field and concluded that the efficacy of lateral hypothalamic stimulation might be due to an activation of the densely concentrated fibers of the MFB (Olds et al., 1960).

The lateral hypothalamus does contain so-called pathneurons that are in contact with many fibers of passage and are thus in a position to monitor the ascending and descending information flow in the MFB (Millhouse, 1969). Olds has argued that this arrangement provides an ideal anatomical substrate for the complex interactions between basic biological drive states (e.g., hunger and thirst) and reinforcing brain stimulation that we have discussed above. (Olds, 1977). The possible contribution of lateral hypothalamic cell bodies to the reinforcing effect of electrical stimulation in the region has been the subject of debate, although evidence of dendritic sprouting after reinforcing stimulation in the region indicates significant persisting local effects (Bindu&Desiraju, 1990). Neurotoxin lesions in the area of the stimulating electrode (which destroy nerve cell bodies but not fibers of passage) have been reported to produce significant impairments (Velley et al., 1983) but it has been suggested that this might be due to axonal damage in the immediate vicinity of the neurotoxin injection (Stellar et al., 1991).

The most consistent body of evidence for a lateral hypothalamic focus for the reinforcing effects of EBS comes from electrophysiological studies. In the rat and monkey, neurons in the lateral hypothalamus respond to reinforcing stimulation of electrode sites in the posterior LH as well as other regions of the brain (Olds, 1974; Rolls, 1974b). Some of the hypothalamic cells that respond to reinforcing brain stimulation also respond to the taste or sight of food or water and the effect can be remarkably specific. For instance, a cell has been isolated that responded to the taste of glucose (but not other foods or fluids) and did so only when the animal was food deprived. Other hypothalamic cells responded to the sight of a peanut when food deprived but not to other foods or other visual stimuli (Rolls, 1976). Stimuli that do not affect neural activity in the LH may come to do so after they have been repeatedly associated with food-reward (Olds, 1973). The convergence of signals for unconditioned as well as conditioned natural reinforcers, such as food and reinforcing electrical brain stimulation, has been interpreted as strong evidence that the lateral hypothalamus may play a major role in the mediation of reward (Rolls, 1976). It is only fair, however, to point out that the direction of information flow is not unidirectional. Reinforcing stimulation of the lateral hypothalamus also modulates the electrical activity of neurons in other regions of the brain (e.g., the ventral tegmental area and frontal cortex) that are also EBS reinforcement sites (Rolls, 1971a; Rolls, 1971b; Rolls&Cooper, 1974c). Moreover, neurons in other regions of the brain (e.g., the nucleus accumbens) are activated, apparently selectively, by reinforcing brain stimulation (Wolske et al., 1993).

Medial Forebrain Bundle

Many of the pioneering studies in this field were essentially mapping studies, designed to describe the anatomical substrate of EBS reinforcement in detail. A large data base accumulated rapidly indicating that the reinforcing effect was most pronounced in brain regions that received afferents from, or contributed efferents to the medial forebrain bundle (German&Bowden, 1974; Olds et al., 1960; Wauquier&Rolls, 1976).(Fig. XXX) More recent studies, using autoradiographic labeling to detect neuronal activation, have supported this conclusion. In these experiments radioactive 2-deoxy-D-glucose (2-DG) was systemically administered to rats before permitting them to self-administer EBS at electrodes in the medial forebrain bundle. (2-DG is incorporated into active neurons just like glucose but cannot be metabolized. It thus remains concentrated in active neurons and its radioactive label can later be detected by a variety of quantitative techniques.) These autoradiographic studies demonstrated that reinforcing stimulation at many different MFB sites produced a common, widespread pattern of neural activation, whereas stimulation at electrodes in prefrontal cortex (which may not be part of the MFB circuit) resulted in a quite different pattern. (Gallistel et al., 1985; Porrino et al., 1990; Yadin et al., 1983).(Fig. XXX) Most contemporary investigators have accepted the view that components of the medial forebrain bundle are responsible for the reinforcing effects of electrical stimulation in many areas of the brain (Milner, 1991; Mora&Cobo, 1990; Phillips&Fibiger, 1989a).

That the MFB is probably not the sole anatomical substrate of reinforcing brain stimulation is indicated by the fact that EBS reinforcement can be obtained from regions of the brain that have no direct projections to or from the MFB (discussed below). Some investigators have, in fact, suggested that the MFB might not play a very important role in EBS reinforcement at all. This minority opinion is based mainly on the disappointingly small and transient effects of MFB lesions. Many investigators have reported transient impairments after large MFB lesions. However most also find significant, and often complete recovery. In a few investigations, MFB lesions produced little or no effects at all. Valenstein's review of this extensive literature (Valenstein, 1966) concluded that the reinforcing effects of EBS at two of the most positive reinforcement sites (the lateral hypothalamus and septal area) does not depend on the integrity of the MFB. This conclusion receives support from a series of studies on thalamic rats whose cortex and forebrain had been removed. Although incapable of lever-pressing, these animals learn simple operant responses such as head turning or tail-lifting to obtain hypothalamic stimulation even though most ascending and descending components of the MFB are undoubtedly transected and their target tissues missing (Huston&Borbely, 1973).

The negative results of the lesion studies are almost certainly influenced by the fact that some MFB components course through portions of the diencephalon (e.g., the cerebral peduncle and the medial hypothalamus) that are not affected by MFB lesions. (The thalamic rat also has intact interconnections between the hypothalamus and midbrain.) The fact remains, however, that EBS reinforcement can survive massive damage to the MFB. This leaves us with a choice of unpalatable conclusions: either the MFB is, indeed, not as important to EBS reinforcement as is generally believed, or the system is so redundant that a small percentage of its components can support its role in EBS reinforcement. We shall return to this conundrum in our discussion of neurochemical mechanisms where similar problems have been raised (below).

Before we briefly turn to other regions of the brain that appear to be related to EBS reinforcement, it is interesting to consider the direction of information flow in the MFB. Two different lines of evidence supported the initial conclusion that reinforcement related information ascends to the forebrain in the MFB: (a) The reinforcing effects of EBS were discovered at about the time that the attention of neuroscientists was drawn to small groups of brainstem neurons that project axons into the forebrain which use catecholamines (i.e., dopamine and norepinephrine) as neurotransmitters. These pathways were soon implicated in sleep and arousal as well as the mood-altering effects of many drugs and EBS reinforcement itself (discussed below). (b) Lesions caudal to an EBS reinforcement site in the hypothalamus typically produce much more severe and persisting effects of EBS reinforced responding than comparable lesions rostral to the stimulation site (Valenstein, 1966).

Contemporary neuropharmacological studies, (discussed below) have strongly implicated catecholaminergic pathways in EBS reinforcement. However, the results of psychophysical studies consistently indicate that the reinforcing effects of hypothalamic stimulation are mediated by descending components of the MFB. The details of these studies are very complex but their rationale is simple: An analysis of changes in the reinforcing properties of distinct pulses of electricity provides data for estimating the duration of the refractory periods of the neurons that are activated. This, in turn, permits inferences about the physical properties of the affected axons as well as the direction and velocity of the action potentials they propagate.

Refractory period estimates provide consistent evidence that the reinforcing effects of electrical stimulation in the lateral hypothalamus and adjacent ventral tegmental area are related to one or perhaps two populations of neurons that have common physical properties (Bielajew et al., 1982; Gratton&Wise, 1988a; Yeomans et al., 1979). The results of so-called "directionality" studies indicate that reinforcing EBS produces action potentials that always travel from the lateral hypothalamus to the ventral tegmentum. (Directionality studies involve an analysis of the interactions of orthodromically and antidromically conducted action potentials. Electrical stimulation of an axon causes action potentials that travel not only orthodromically - i.e., towards the axon terminal - but also antidromically - i.e., towards the cell body.) (Bielajew&Shizgal, 1986; Durivage&Miliaressis, 1987; Gratton&Wise, 1988b).(Fig. XXX) Related experiments have provided evidence of a similar relationship between the ventral tegmentum and EBS reinforcement sites in the midbrain and pons (Boyce&Rompré, 1987). These studies do not indicate a direct interaction between the lateral hypothalamus and the midbrain and pons (Bielajew et al., 1981). This suggests that the VTA may be a focal point in the EBS reinforcement system that collects both descending influences from the hypothalamus and ascending influences from the lower brainstem.

Other Areas of the Brain

Many of the pioneering studies in this field concentrated on electrode sites in the hypothalamus and adjacent forebrain regions, including the septal area, because animals rapidly learned to self-administer stimulation in this region and worked very hard to obtain it afterwards (Brady, 1961). Early clinical studies of the phenomenon also focused on the septal area and adjacent forebrain regions because tumor growth in the area had been related to psychiatric disturbances (Heath, 1963). Most investigators paid little attention to the region after Olds concluded that even higher responses rates (and presumably stronger reinforcing effects) could be obtained from the lateral hypothalamus (Olds et al., 1960). The forebrain region just anterior to the hypothalamus and preoptic region reappeared prominently in the EBS reinforcement literature when it was demonstrated that the nucleus accumbens, adjacent to the septal area, was a major target for dopaminergic projections from the brainstem that have been strongly implicated in EBS reinforcement (see below).

Other brain regions that support high rates of responding for EBS reinforcements include: the prefrontal cortex (Rolls&Cooper, 1974a; Rolls&Cooper, 1974c; Routtenberg&Sloan, 1972), amygdala, cingulate gyrus, entorhinal cortex, and hippocampus (Kane et al., 1991; Rolls, 1974b), and ventral tegmental area of the midbrain (Rompré&Miliaressis, 1985). Regions that support lower rates of EBS reinforced behavior include: lower portions of the brainstem (pons and medulla), the cerebellum, thalamus and striatum (Olds, 1977; Sem-Jacobsen, 1976; Wauquier&Rolls, 1976). Many of the structures that support EBS reinforced behavior have been implicated in positive or negative affect on the basis of lesion studies and/or clinical observations but some (e.g., the pons, medulla, cerebellum and striatum) have not. Even some primary sensory pathways, including the olfactory bulb (Phillips&Mogenson, 1969), pontine trigeminal nuclei (Corbett&Wise, 1979; Van der Kooy&Phillips, 1979) and pontine taste nuclei (Carter&Phillips, 1975) support EBS reinforcement. The major "silent" component of the brain appears to the neocortex (Olds et al., 1960) although experimental as well as clinical studies have reported a few positive sites in frontal and temporal lobe neocortex (Bishop et al., 1963; Rolls, 1974b; Sem-Jacobsen, 1968).

Psychophysical studies indicate that the physical properties of the neurons that mediate the reinforcing effects of frontal cortex stimulation are quite different from those observed in the ventral tegmentum and lateral hypothalamus (Schenk&Shizgal, 1982). Investigations of the basal forebrain (preoptic area and nucleus accumbens) have obtain refractory period estimates that were intermediate between those obtained from LH electrodes and those recorded from cortical sites. The data from these studies are compatible with the hypothesis that two different neural reinforcement systems may overlap in this transitional area of the brain (Bielajew et al., 1987; Fouriezos et al., 1987).

Summary

The complex data we have just reviewed show that EBS reinforcement can be obtained from most major regions of the brain except for the neocortex. The diversity of the EBS reinforcement sites suggests that the subjective sensations associated with EBS reinforcement are also probably diverse. Human instrospection supports that conclusion. Stimuli of all sensory modalities are capable of eliciting positive as well as negative affect. In some (relatively rare) instances, the associations are innate (humans, at least, have little difficulty identifying pleasant or unpleasant odors, tastes, sights and sounds, and respond positively to light touch and negatively to deep pressure.) More typically, the associations between specific stimuli and their affective consequences are learned.

Most recent reviews of the literature that describes the anatomical basis of EBS reinforcement conclude that there are probably several anatomically distinct and perhaps functionally independent circuits: The most prominent MFB system may itself encompass two or more distinct pathways, including separate mesolimbic and mesocortical dopamine pathways (discussed below). There is also evidence for a separate cortical reinforcement system that originates in the prefrontal cortex (where it interfaces with dopaminergic pathways) and projects to limbic cortex in the cingulate gyrus and in the entorhinal region of the temporal lobe. In addition, some investigators have proposed a distinct hind brain reinforcement system that may be specifically related to taste- and gustatory sensations that play a major role in survival, as well as mating and maternal behavior, in many mammalian species.

Neurochemistry of Brain stimulation Reinforcement

Shortly after the brain stimulation reinforcement phenomenon was discovered, neuroanatomist learned to stain catecholamine neurotransmitters in the brain. This led to the discovery that several major noradrenergic and dopaminergic pathways originated the midbrain and lower brainstem and projected diffusely to the hypothalamus, medial forebrain, and limbic system. Their trajectory followed the medial forebrain bundle and overlapped extensively with maps of brain stimulation reinforcement sites. Biopsychologists therefore initiated a concerted effort to determine whether noradrenergic or dopaminergic pathways provide the anatomical substrate of brain stimulation reinforcement.

We now know that there are two principal noradrenergic pathways. The ventral tegmental tract arises from several nuclei in the lower brainstem and projects preferentially to the hypothalamus. The dorsal tegmental tract arises mainly from the nucleus locus coeruleus in the midbrain and projects diffusely to the medial forebrain and limbic system components of the cerebrum, as well as the cerebellum. There are three major dopamine pathways. The nigro-striatal bundle arises from the substantia nigra (SN) in the ventrolateral midbrain and projects exclusively to the striatum. The mesolimbic pathway originates in the ventral tegmental area (VTA) just medial to the substantia nigra. It projects mainly to the medial forebrain (nucleus accumbens, lateral septum) and amygdala. The mesocortical pathway also originates in the VTA and projects diffusely to limbic system cortex (Fallon, 1988; Moore&Bloom, 1979).

Noradrenergic Reinforcement Pathways

By the early 1960s, we knew that drugs that increase the release, or block the re-uptake or metabolic destruction of catecholamines, increase the rate of EBS reinforced behavior. Drugs that reduce the availability of catecholamines in the brain, or block their postsynaptic receptors, decrease EBS reinforced behavior (Olds, 1959; Stein, 1962; Stein, 1968).

The drugs that were available for this pioneering research affected both noradrenergic and dopaminergic transmitter mechanisms. Stein nonetheless proposed a norepinephrine "theory of reward" that played a major role in shaping subsequent research in this field (Stein, 1968; Stein et al., 1976). He selected norepinephrine (NE) rather than dopamine (DA) as the critical "reinforcement transmitter" mainly because of two sets of empirical findings. Firstly, there were the results of numerous mapping studies indicating that there were many good EBS reinforcement sites in the brainstem below the level of the substantia nigra and ventral tegmental area, that received no dopaminergic innervation. The positive sites were most prominent in regions (such as the locus coeruleus) that were known to give rise to the major noradrenergic projections to the forebrain (Ritter&Stein, 1973). Secondly, psychopharmacological studies showed that drugs that block the conversion of dopamine into norepinephrine in the brain (the only way NE can be synthesized in noradrenergic neurons), reduced or abolished self-stimulation and this effect was blocked by NE but not DA injections into the ventricles (Stein et al., 1976). Since these NE synthesis blockers did not affect dopaminergic neurons in the brain, Stein's conclusion seemed well founded. Although contemporary interest is focused on precisely those dopaminergic pathways (below), there is some contemporary research suggesting that noradrenergic pathways may, indeed, play a role in EBS reinforcement. For instance, microdialysis studies have shown that reinforcing brain stimulation at some electrode sites releases NE, but not DA, from the nucleus accumbens and medial frontal cortex (Cenci et al., 1992). Electrophysiological studies have demonstrated that stimulation of the locus coeruleus stimulates alpha-1 noradrenergic receptors on neurons in the ventral tegmentum and thus increases their activity (Grenhoff et al., 1993).

A conceptual problem has plagued Stein's noradrenergic theory from the start: The brainstem NE pathways are strongly implicated in cortical as well as behavioral arousal (Jones, 1990). If one finds that drugs or lesions that interfere with the functions of NE pathways, inhibit EBS reinforced behavior, could this not be explained, most parsimoniously, in terms of a general decrease in reactivity to all stimuli? Stein, and other advocates of a noradrenergic basis of EBS reinforcement, tried to circumvent this problem by demonstrating preferential effects on EBS reinforced behavior, but that has proven to be an extremely difficult task. One might, in fact, argue that we would expect a general inhibition of most, if not all behavior, when reinforcement-related pathways are blocked (if, as is generally accepted, reinforcement is an integral part of all learned behavior and nearly all behavior is learned). As we shall see shortly, similar problems plague contemporary dopamine theories of reinforcement.

In the 1970s, more evidence for dopaminergic involvement in EBS reinforcement became available. At first, this led to the hypothesis that whereas "reward" itself might be a product of noradrenergic pathways, the "incentive motivation" for brain stimulation might be due to an activation of dopaminergic neurons (Crow, 1972). More recently, the prevailing opinion has favored explanations that propose a central role for dopaminergic pathways in EBS reinforcement itself (Self&Stein, 1992; Wise et al., 1992). This does not, of course, exclude a significant role of noradrenergic pathways in EBS reinforcement, and some models of interacting NE-DA reinforcement systems have been proposed (Stellar&Rice, 1989).

Dopaminergic Reinforcement Pathways

Systemic injections of dopamine antagonists (e.g., receptor blockers) reduce the reinforcing effects of EBS in the ventral tegmentum and hypothalamus (Gallistel et al., 1982; Stellar et al., 1983; Stellar&Rice, 1989) as well as medial frontal cortex (Corbett, 1990.; Duvauchelle&Ettenberg, 1991). Dopamine antagonists also block the effects of natural reinforcers, such as food or water, as one might expect if the EBS reinforcement phenomenon is as fundamental to behavior as many biopsychologists assume (Rolls et al., 1974d; Spyraki et al., 1982). Dopamine antagonists inhibit the reinforcing effects of EBS not only in portions of the brain that are innervated by DA pathways but also in lower brainstem regions that have no dopaminergic innervation. This suggests that dopaminergic neurons may be "in series" with other (possibly noradrenergic) pathways. (Rompré&Boyce, 1989a; Rompré&Wise, 1989b).

Dopamine, like many other neurotransmitters, acts on several different receptor sites that interact differently with various dopamine agonists and antagonists. Some early studies demonstrated inhibitory effects on EBS reinforcement of dopamine D2 receptor blockers (Gallistel&Davis, 1983), but more recent reports have specifically implicated D1 receptors (Miller et al., 1990; Nakajima, 1986; Sabater et al., 1993). (D 2 receptors may exert facilitatory effects on EBS reinforcement that can only be expressed after D1 receptors have been activated) (Nakajima et al., 1993; Nakajima&OÕRegan, 1991).

Systemic injections of dopamine agonists, (mainly drugs that release dopamine or inhibit its reuptake) such as cocaine or amphetamine, enhance the reinforcing effect of EBS in the ventral tegmental area (Frank et al., 1992), medial forebrain bundle (Gallistel&Karras, 1984; Kornetsky&Esposito, 1981), and frontal cortex (Corbett, 1991; McGregor et al., 1992). (Fig. XXX) In some cases, the effect is only small, possibly because the drugs themselves produce reinforcing effects that are independent of the animals' behavior in these studies (discussed below). Dopamine agonists increase locomotor activity and there have been many attempts to dissociate their general stimulant effects from a more specific facilitation of EBS reinforcement. For instance, one such study demonstrated that the locomotor effects of amphetamine become more prominent with repeated administration, whereas the drug's facilitating effect on EBS reinforced behavior is largest the first time the drug is administered (Wise&Munn, 1993).

The nucleus accumbens, as well as the prefrontal cortex, have been implicated in the effects of dopamine agonists. Rats avidly self-administer amphetamine into either of these regions (Hoebel et al., 1983; Stein&Belluzzi, 1989). Experimenter-controlled microinjections of dopamine and amphetamine into both regions also modulate EBS reinforcement elsewhere, but the pattern of the behavioral effects is quite different in nucleus accumbens than in prefrontal cortex, suggesting that the two regions may exert different dopaminergic influences on EBS reinforcement (Olds, 1990). Drug discrimination experiments have shown that reinforcing electrical stimulation of the VTA has subjective effects similar to those of systemic amphetamine (Druhan et al., 1990).

Microdialysis studies consistently show that reinforcing electrical stimulation of the ventral tegmental area (Fiorino et al., 1993; Phillips et al., 1989b) or medial forebrain bundle (Nakahara et al., 1989) releases dopamine from the nucleus accumbens. Systemic injections of dopamine agonists, such as amphetamine or cocaine, have similar effects. (Hernandez&Hoebel, 1988; Moghaddam&Bunney, 1989). Consumatory behaviors such as eating, drinking, or mating, have also been shown to release dopamine from the nucleus accumbens and prefrontal cortex (Damsma et al., 1992; Hernandez&Hoebel, 1988; Hernandez&Hoebel, 1990; Mark et al., 1989).

On balance, these data provide strong empirical support for the general conclusion that dopaminergic pathways play a very important role in EBS reinforcement, and a dopamine theory of reward is now widely accepted. However, we also have numerous observations indicating that DA pathways may be only one link in a complex system. The psychophysical experiments we have briefly discussed above show that the reinforcing effects of EBS are probably rarely due to the direct activation of dopamine (or norepinephrine) fibers. Refractory period estimates consistently indicate that reinforcement neurons in the MFB have refractory periods that are typically shorter than 1 msec. and rarely longer than 1.4 msec. (Gratton&Wise, 1985; Rompré&Miliaressis, 1987; Yeomans, 1975). Most catecholamine fibers, one the other hand, have refractory periods greater than 2.0 msec (Foote et al., 1983; Wang, 1981). Moreover, directionality studies have shown that reinforcement related signals typically descend in the MFB (dopaminergic fibers only ascend) (Bielajew&Shizgal, 1986). (There is some evidence that excitatory amino acid receptors on VTA dopamine neurons may mediate the reinforcing effects of EBS in the hypothalamus and forebrain (Herberg&Rose, 1990).)

Mapping studies demonstrate that the reinforcement pathways extend into brain areas that do not contain dopamine cells or their projections (Rompré&Boyce, 1989a; Rompré&Miliaressis, 1985). Yet, a review of the pharmacological literature indicates that a blockade of brain dopamine reduces EBS reinforced behavior at all electrode sites (MacConell et al., 1992; Simon et al., 1979). On the other hand, near-complete depletion of brain dopamine has been reported to produce only minor disturbances in some rate-free measures of EBS reinforcement (Colle&Wise, 1987; Fibiger et al., 1987). There are, moreover, some experimental findings that indicate that activation of dopaminergic components of the nucleus accumbens or frontal cortex does not always imply activation of reinforcement-related functions. For instance, chronic stress, which reduces the reinforcing effects of EBS, has been shown to release dopamine from the nucleus accumbens just as positive natural reinforcers, such as food or water, do (Stamford et al., 1991). Microdialysis studies have also demonstrated that changes in the physical parameters of VTA stimulation can produce major changes in the amount of dopamine that is released from nucleus accumbens even though they do not affect the reinforcing property of the stimulation as measured by rate-free measures (Miliaressis et al., 1991).

Proponents of a dopaminergic theory of EBS reinforcement have found it very difficult to distinguish drug- and lesion- effects on reinforcement from changes in other functions that are mediated by brain dopamine pathways. The most obvious problem concerns the well-documented involvement of DA in motor functions and arousal. Electrolytic as well as neurotoxin-induced lesions in the ventral tegmentum typically produce somnolence and severe sensory-motor disturbances due to the destruction of dopaminergic nigro-striatal projections (see our discussion of hunger for detail) (Ungerstedt, 1971). Complex, learned behavior remains severely impaired long after simple voluntary behaviors have recovered (Kent&Grossman, 1973). This suggests that an interference with reinforcement-related pathways may also occur (Grossman, 1976). Lesions restricted to medial portions of the VTA, as well as systemic injections of DA antagonists, can have less severe sensory-motor dysfunctions, but a decrease in EBS reinforced behavior is nonetheless difficult to interpret. Dopamine agonists, such as cocaine and amphetamine, have well-known psychomotor stimulant effects and there is considerable evidence that electrical stimulation of DA brain sites can produce similar effects. Rate-free measures of the reinforcing effects of EBS address this problem but cannot entirely assuage one's concerns. There is, in fact, no entirely satisfactory solution to this problem (just as there is none for the involvement of NE in arousal). If EBS reinforcement is, in fact, related to natural reward, and reinforcement is, in fact, essential for learned behavior, one should not expect to be able to separate behavior from reinforcement (any more than one can separate behavior from the intervening variable reward).

Conclusions

There is consensus among contemporary specialists in this area that EBS reinforcement is a far more complex phenomenon than originally believed. Although connectivity studies have discovered direct interconnections between some of the key areas, reinforcement-related brain pathways are undoubtedly multi-synaptic. Since dopamine pathways do not project to other dopamine pathways, non-dopaminergic components undoubtedly occur in the reinforcement system(s) of the brain. Some of these may well be noradrenergic. Most contemporary investigators agree that dopamine pathways are part of the neural substrate of reinforcement. However, many reject the stronger hypothesis that reinforcement is uniquely associated only with dopaminergic pathways. The notion of a single neural pathway for all types of reinforcement processes seems no longer tenable even if one modifies the original idea to include a number of non-catecholaminergic stages connected "in series" with a dopaminergic link.

Section Review

Brain-stimulation reinforcement is, arguably, the most intriguing and enigmatic phenomenon biopsychologists have yet encountered. The mere fact that the administration of random electrical pulses that cannot bear any meaningful relation to ongoing neural processes should be rewarding seems to defy our basic concepts of how the brain works. The fact that in most cases some "priming" stimulation must be administered deepens the mystery. Surely the animal has not forgotten that the stimulation was rewarding or how it could be obtained? Does prolonged and unnatural activation of the reinforcement system cause functional changes (such as decreased transmitter production or release or decreased receptor sensitivity) that makes it more difficult to be activated by natural rewards? Is this a sufficiently negative experience to keep the animal away from brain stimulation reinforced behavior unless it is in some way "hooked" again by the administration of some free "priming" stimuli?

The relationship between brain stimulation reinforcement and natural reward also remains a puzzle. Some studies have shown that experimental treatments that facilitate brain stimulation reinforced behavior also affect natural rewards. Others fail to find a reliable interaction, perhaps because different areas of the brain (and different aspects of the postulated reinforcement system) are involved. Such a conclusion is supported by the observation that some neurons respond to brain stimulation as well as natural rewards, but others are responsive only to one or other.

Lastly, we need to know more about the nature of the experiential response to reinforcing brain stimulation. Is it so difficult to describe because there is no "reward" experience per se? Or are there numerous different experiences depending upon the motivational aspects of the situation? Can we define a special experience when we receive a natural reward such as money for running an errand or praise from the boss for a job well done? If there is no clear-cut experiential component, why do animals (and humans) avidly press a lever to obtain brain stimulation? If it were just curiosity about the unusual experiences elicited by the stimulation (as some have suggested), surely animals would not forgo nourishment and sleep in order to work furiously for days on end?

Sebastian P. Grossman