Punishment prediction by dopaminergic neurons in Drosophila

The temporal pairing of a neutral stimulus with a reinforcer (reward or punishment) can lead to classical conditioning, a simple form of learning in which the animal assigns a value (positive or negative) to the formerly neutral stimulus. Olfactory classical conditioning in Drosophila is a prime model for the analysis of the molecular and neuronal substrate of this type of learning and memory. Neuronal correlates of associative plasticity have been identified in several regions of the insect brain. In particular, the mushroom bodies have been shown to be necessary for aversive olfactory memory formation. However, little is known about which neurons mediate the reinforcing stimulus. Using functional optical imaging, we now show that dopaminergic projections to the mushroom-body lobes are weakly activated by odor stimuli but respond strongly to electric shocks. However, after one of two odors is paired several times with an electric shock, odor-evoked activity is significantly prolonged only for the "punished" odor. Whereas dopaminergic neurons mediate rewarding reinforcement in mammals, our data suggest a role for aversive reinforcement in Drosophila. However, the dopaminergic neurons' capability of mediating and predicting a reinforcing stimulus appears to be conserved between Drosophila and mammals.     

The Role of Dopamine in Drosophila Larval Classical Olfactory Conditioning

Learning and memory is not an attribute of higher animals. Even Drosophila larvae are able to form and recall an association of a given odor with an aversive or appetitive gustatory reinforcer. As the Drosophila larva has turned into a particularly simple model for studying odor processing, a detailed neuronal and functional map of the olfactory pathway is available up to the third order neurons in the mushroom bodies. At this point, a convergence of olfactory processing and gustatory reinforcement is suggested to underlie associative memory formation. The dopaminergic system was shown to be involved in mammalian and insect olfactory conditioning. To analyze the anatomy and function of the larval dopaminergic system, we first characterize dopaminergic neurons immunohistochemically up to the single cell level and subsequent test for the effects of distortions in the dopamine system upon aversive (odor-salt) as well as appetitive (odor-sugar) associative learning. Single cell analysis suggests that dopaminergic neurons do not directly connect gustatory input in the larval suboesophageal ganglion to olfactory information in the mushroom bodies. However, a number of dopaminergic neurons innervate different regions of the brain, including protocerebra, mushroom bodies and suboesophageal ganglion. We found that dopamine receptors are highly enriched in the mushroom bodies and that aversive and appetitive olfactory learning is strongly impaired in dopamine receptor mutants. Genetically interfering with dopaminergic signaling supports this finding, although our data do not exclude on naïve odor and sugar preferences of the larvae. Our data suggest that dopaminergic neurons provide input to different brain regions including protocerebra, suboesophageal ganglion and mushroom bodies by more than one route. We therefore propose that different types of dopaminergic neurons might be involved in different types of signaling necessary for aversive and appetitive olfactory memory formation respectively, or for the retrieval of these memory traces. Future studies of the dopaminergic system need to take into account such cellular dissociations in function in order to be meaningful.     

Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in an insect

Background  

In insect classical conditioning, octopamine (the invertebrate counterpart of noradrenaline) or dopamine has been suggested to mediate reinforcing properties of appetitive or aversive unconditioned stimulus, respectively. However, the roles of octopaminergic and dopaminergic neurons in memory recall have remained unclear.     

Appetitive and aversive learning in Spodoptera littoralis larvae

Adult Lepidoptera are capable of associative learning. This helps them to forage flowers or to find suitable oviposition sites. Larval learning has never been seriously considered because they have limited foraging capabilities and usually depend on adults as concerns their food choices. We tested if Spodoptera littoralis larvae can learn to associate an odor with a tastant using a new classical conditioning paradigm. Groups of larvae were exposed to an unconditioned stimulus (US: fructose or quinine mixed with agar) paired with a conditioned stimulus (CS: hexanol, geraniol or pentyl acetate) in a petri dish. Their reaction to CS was subsequently tested in a petri dish at different time intervals after conditioning. Trained larvae showed a significant preference or avoidance to CS when paired with US depending on the reinforcer used. The training was more efficient when larvae were given a choice between an area where CS-US was paired and an area with no CS (or another odor). In these conditions, the memory formed could be recalled at least 24 h after pairing with an aversive stimulus and only 5 min after pairing with an appetitive stimulus. This learning was specific to CS because trained larvae were able to discriminate CS from another odor that was present during the training but unrewarded. These results suggest that Lepidoptera larvae exhibit more behavioral plasticity than previously appreciated.     

桔小实蝇雄成虫联系性嗅觉学习

【目的】为了探究桔小实蝇 Bactrocera dorsalis (Hendel)雄成虫的嗅觉学习能力。【方法】本研究采用经典性嗅觉条件反射训练法（classical olfactory conditioning）在室内对固定的羽化后14-17日龄的桔小实蝇雄成虫进行气味与食物的联合学习训练, 即薄荷精油和10%蔗糖溶液联合的奖赏性训练（appetitive conditioning）以及甲基丁香酚(methyl eugenol, ME)和饱和盐溶液联合的惩罚性训练（aversive conditioning）,并以伸喙反射行为（proboscis extension reflex, PER）作为学习与否的判定标准。【结果】经过奖赏性训练后,桔小实蝇雄成虫对薄荷精油的伸喙反射率可从0%增加至68%;而经过惩罚性训练后,桔小实蝇对甲基丁香酚的伸喙反射率可从100%降低至36.54%,且这种伸喙反射率的变化是通过气味条件刺激（conditioned stimulus）和食物非条件刺激（unconditioned stimulus）的对称性联合而产生的。【结论】结果表明,桔小实蝇雄性成虫具有较强的联系性嗅觉学习能力,并且两种刺激的联合是形成学习记忆的必要条件。     

Experimental evolution of olfactory memory in Drosophila melanogaster

In order to address the nature of genetic variation in learning performance, we investigated the response to classical olfactory conditioning in "high-learning" Drosophila melanogaster lines previously subject to selection for the ability to learn an association between the flavor of an oviposition medium and bitter taste. In a T-maze choice test, the seven high-learning lines were better at avoiding an odor previously associated with aversive mechanical shock than were five unselected "low-learning" lines originating from the same natural population. Thus, the evolved improvement in learning ability of high-learning lines generalized to another aversion learning task involving a different aversive stimulus (shock instead of bitter taste) and a different behavioral context than that used to impose selection. In this olfactory shock task, the high-learning lines showed improvements in the learning rate as well as in two forms of consolidated memory: anesthesia-resistant memory and long-term memory. Thus, genetic variation underlying the experimental evolution of learning performance in the high-learning lines affected several phases of memory formation in the course of olfactory aversive learning. However, the two forms of consolidated memory were negatively correlated among replicate high-learning lines, which is consistent with a recent hypothesis that these two forms of consolidated memory are antagonistic.     

Analysis of associative learning in the terrestrial mollusc Limax maximus. II. Appetitive learning

Summary The odor and taste processing systems of the terrestrial mollusc Limax maximus have been shown capable of a number of complex computations. Most of the complex higher-order features of Limax learning have been demonstrated using differential aversive conditioning. The present experiments probe the appetitive learning ability of Limax. In the first experiment a differential appetitive classical conditioning procedure was used. An aversive CS⁺ odor was paired with an attractive taste while a CS^– odor was explicitly unpaired with the attractive taste. This appetitive conditioning procedure dramatically increased the preference for the CS⁺ odor. Further experiments determined the time course of acquisition, the effect of an extinction procedure and long-term retention of the appetitive conditioning. Now that Limax has been shown capable of appetitive conditioning, the neural network simulation of Limax learning, called LIMAX, can be examined for its ability to display appetitive conditioning.Abbreviations AA amyl acetate - CS conditioned stimulus - ISI interstimulus interval - ITI intertrial interval - MCH methylcyclohexanol - US unconditioned stimulus     

Parallel processing of appetitive short- and long-term memories in Drosophila

It is broadly accepted that long-term memory (LTM) is formed sequentially after learning and short-term memory (STM) formation, but the nature of the relationship between early and late memory traces remains heavily debated [1-5]. To shed light on this issue, we used an olfactory appetitive conditioning in Drosophila, wherein starved flies learned to associate an odor with the presence of sugar [6]. We took advantage of the fact that both STM and LTM are generated after a unique conditioning cycle [7, 8] to demonstrate that appetitive LTM is able to form independently of STM. More specifically, we show that (1) STM retrieval involves output from γ neurons of the mushroom body (MB), i.e., the olfactory memory center [9, 10], whereas LTM retrieval involves output from αβ MB neurons; (2) STM information is not transferred from γ neurons to αβ neurons for LTM formation; and (3) the adenylyl cyclase RUT, which is thought to operate as a coincidence detector between the olfactory stimulus and the sugar stimulus [11-14], is required independently in γ neurons to form appetitive STM and in αβ neurons to form LTM. Taken together, these results demonstrate that appetitive short- and long-term memories are formed and processed in parallel.     

Altered representation of the spatial code for odors after olfactory classical conditioning; memory trace formation by synaptic recruitment

In the olfactory bulb of vertebrates or the homologous antennal lobe of insects, odor quality is represented by stereotyped patterns of neuronal activity that are reproducible within and between individuals. Using optical imaging to monitor synaptic activity in the Drosophila antennal lobe, we show here that classical conditioning rapidly alters the neural code representing the learned odor by recruiting new synapses into that code. Pairing of an odor-conditioned stimulus with an electric shock-unconditioned stimulus causes new projection neuron synapses to respond to the odor along with those normally activated prior to conditioning. Different odors recruit different groups of projection neurons into the spatial code. The change in odor representation after conditioning appears to be intrinsic to projection neurons. The rapid recruitment by conditioning of new synapses into the representation of sensory information may be a general mechanism underlying many forms of short-term memory.     

Effect of Flumethrin on Survival and Olfactory Learning in Honeybees

Flumethrin has been widely used as an acaricide for the control of Varroa mites in commercial honeybee keeping throughout the world for many years. Here we test the mortality of the Asian honeybee Apis cerana cerana after treatment with flumethrin. We also ask (1) how bees react to the odor of flumethrin, (2) whether its odor induces an innate avoidance response, (3) whether its taste transmits an aversive reinforcing component in olfactory learning, and (4) whether its odor or taste can be associated with reward in classical conditioning. Our results show that flumethrin has a negative effect on Apis ceranàs lifespan, induces an innate avoidance response, acts as a punishing reinforcer in olfactory learning, and interferes with the association of an appetitive conditioned stimulus. Furthermore flumethrin uptake within the colony reduces olfactory learning over an extended period of time.     

Distinct traces for appetitive versus aversive olfactory memories in DPM neurons of Drosophila

The global logic used by the brain for differentially encoding positive and negative experiences remains unknown along with how such experiences are represented by collections of memory traces at the cellular level. Here we contrast the cellular memory traces that form in the dorsal paired medial (DPM) neurons of Drosophila after conditioning flies with odors associated with aversive or appetitive unconditioned stimuli (US). Our results show that the appetitive DPM neuron trace is distinguished from the aversive in three fundamental ways: (1) The DPM neurons do not respond to an appetitive US of sucrose by itself, in contrast to their robust response to an aversive US. (2) The appetitive trace persists for twice as long as the aversive trace. (3) The appetitive trace is expressed in both neurite branches of the neuron, rather than being confined to a single branch like the aversive trace. In addition, we demonstrate that training flies with nonnutritive sugars that elicit a behavioral memory that decays within 24 hr generates, like aversive conditioning, a short-lived and branch-restricted memory trace. These results indicate that the persistence and breadth of the DPM neuron memory trace influences the duration of behavioral memory.     

Classical reward conditioning in Drosophila melanogaster

Negatively reinforced olfactory conditioning has been widely employed to identify learning and memory genes, signal transduction pathways and neural circuitry in Drosophila. To delineate the molecular and cellular processes underlying reward-mediated learning and memory, we developed a novel assay system for positively reinforced olfactory conditioning. In this assay, flies were involuntarily exposed to the appetitive unconditioned stimulus sucrose along with a conditioned stimulus odour during training and their preference for the odour previously associated with sucrose was measured to assess learning and memory capacities. After one training session, wild-type Canton S flies displayed reliable performance, which was enhanced after two training cycles with 1-min or 15-min inter-training intervals. Higher performance scores were also obtained with increasing sucrose concentration. Memory in Canton S flies decayed slowly when measured at 30 min, 1 h and 3 h after training; whereas, it had declined significantly at 6 h and 12 h post-training. When learning mutant t beta h flies, which are deficient in octopamine, were challenged, they exhibited poor performance, validating the utility of this assay. As the Drosophila model offers vast genetic and transgenic resources, the new appetitive conditioning described here provides a useful tool with which to elucidate the molecular and cellular underpinnings of reward learning and memory. Similar to negatively reinforced conditioning, this reward conditioning represents classical olfactory conditioning. Thus, comparative analyses of learning and memory mutants in two assays may help identify the molecular and cellular components that are specific to the unconditioned stimulus information used in conditioning.     

Drosophila Adult Olfactory Shock Learning

Drosophila have been used in classical conditioning experiments for over 40 years, thus greatly facilitating our understanding of memory, including the elucidation of the molecular mechanisms involved in cognitive diseases^1-7. Learning and memory can be assayed in larvae to study the effect of neurodevelopmental genes^8-10 and in flies to measure the contribution of adult plasticity genes^1-7. Furthermore, the short lifespan of Drosophila facilitates the analysis of genes mediating age-related memory impairment^5,11-13. The availability of many inducible promoters that subdivide the Drosophila nervous system makes it possible to determine when and where a gene of interest is required for normal memory as well as relay of different aspects of the reinforcement signal^3,4,14,16.Studying memory in adult Drosophila allows for a detailed analysis of the behavior and circuitry involved and a measurement of long-term memory^15-17. The length of the adult stage accommodates longer-term genetic, behavioral, dietary and pharmacological manipulations of memory, in addition to determining the effect of aging and neurodegenerative disease on memory^{3-6,11-13,15-21}.Classical conditioning is induced by the simultaneous presentation of a neutral odor cue (conditioned stimulus, CS⁺) and a reinforcement stimulus, e.g., an electric shock or sucrose, (unconditioned stimulus, US), that become associated with one another by the animal^1,16. A second conditioned stimulus (CS^-) is subsequently presented without the US. During the testing phase, Drosophila are simultaneously presented with CS+ and CS- odors. After the Drosophila are provided time to choose between the odors, the distribution of the animals is recorded. This procedure allows associative aversive or appetitive conditioning to be reliably measured without a bias introduced by the innate preference for either of the conditioned stimuli. Various control experiments are also performed to test whether all genotypes respond normally to odor and reinforcement alone.     

Aversive learning in honeybees revealed by the olfactory conditioning of the sting extension reflex

Invertebrates have contributed greatly to our understanding of associative learning because they allow learning protocols to be combined with experimental access to the nervous system. The honeybee Apis mellifera constitutes a standard model for the study of appetitive learning and memory since it was shown, almost a century ago, that bees learn to associate different sensory cues with a reward of sugar solution. However, up to now, no study has explored aversive learning in bees in such a way that simultaneous access to its neural bases is granted. Using odorants paired with electric shocks, we conditioned the sting extension reflex, which is exhibited by harnessed bees when subjected to a noxious stimulation. We show that this response can be conditioned so that bees learn to extend their sting in response to the odorant previously punished. Bees also learn to extend the proboscis to one odorant paired with sugar solution and the sting to a different odorant paired with electric shock, thus showing that they can master both appetitive and aversive associations simultaneously. Responding to the appropriate odorant with the appropriate response is possible because two different biogenic amines, octopamine and dopamine subserve appetitive and aversive reinforcement, respectively. While octopamine has been previously shown to substitute for appetitive reinforcement, we demonstrate that blocking of dopaminergic, but not octopaminergic, receptors suppresses aversive learning. Therefore, aversive learning in honeybees can now be accessed both at the behavioral and neural levels, thus opening new research avenues for understanding basic mechanisms of learning and memory.     

Parametric and genetic analysis of Drosophila appetitive long‐term memory and sugar motivation

Distinct forms of memory can be highlighted using different training protocols. In Drosophila olfactory aversive learning, one conditioning session triggers memory formation independently of protein synthesis, while five spaced conditioning sessions lead to the formation of long‐term memory (LTM), a long‐lasting memory dependent on de novo protein synthesis. In contrast, one session of odour–sugar association appeared sufficient for the fly to form LTM. We designed and tuned an apparatus that facilitates repeated discriminative conditioning by alternate presentations of two odours, one being associated with sugar, as well as a new paradigm to test sugar responsiveness (SR). Our results show that both SR and short‐term memory (STM) scores increase with starvation length before conditioning. The protein dependency of appetitive LTM is independent of the repetition and the spacing of training sessions, on the starvation duration and on the strength of the unconditioned stimulus. In contrast to a recent report, our test measures an abnormal SR of radish mutant flies, which might initiate their STM and LTM phenotypes. In addition, our work shows that crammer and tequila mutants, which are deficient for aversive LTM, present both an SR and an appetitive STM defect. Using the MB247‐P[switch] system, we further show that tequila is required in the adult mushroom bodies for normal sugar motivation.     

Genotypic Influence on Aversive Conditioning in Honeybees,Using a Novel Thermal Reinforcement Procedure

In Pavlovian conditioning, animals learn to associate initially neutral stimuli with positive or negative outcomes, leading to appetitive and aversive learning respectively. The honeybee (Apis mellifera) is a prominent invertebrate model for studying both versions of olfactory learning and for unraveling the influence of genotype. As a queen bee mates with about 15 males, her worker offspring belong to as many, genetically-different patrilines. While the genetic dependency of appetitive learning is well established in bees, it is not the case for aversive learning, as a robust protocol was only developed recently. In the original conditioning of the sting extension response (SER), bees learn to associate an odor (conditioned stimulus - CS) with an electric shock (unconditioned stimulus - US). This US is however not a natural stimulus for bees, which may represent a potential caveat for dissecting the genetics underlying aversive learning. We thus first tested heat as a potential new US for SER conditioning. We show that thermal stimulation of several sensory structures on the bee’s body triggers the SER, in a temperature-dependent manner. Moreover, heat applied to the antennae, mouthparts or legs is an efficient US for SER conditioning. Then, using microsatellite analysis, we analyzed heat sensitivity and aversive learning performances in ten worker patrilines issued from a naturally inseminated queen. We demonstrate a strong influence of genotype on aversive learning, possibly indicating the existence of a genetic determinism of this capacity. Such determinism could be instrumental for efficient task partitioning within the hive.     

Gamma neurons mediate dopaminergic input during aversive olfactory memory formation in Drosophila

Mushroom body (MB)-dependent olfactory learning in Drosophila provides a powerful model to investigate memory mechanisms. MBs integrate olfactory conditioned stimulus (CS) inputs with neuromodulatory reinforcement (unconditioned stimuli, US), which for aversive learning is thought to rely on dopaminergic (DA) signaling to DopR, a D1-like dopamine receptor expressed in MBs. A wealth of evidence suggests the conclusion that parallel and independent signaling occurs downstream of DopR within two MB neuron cell types, with each supporting half of memory performance. For instance, expression of the Rutabaga (Rut) adenylyl cyclase in γ neurons is sufficient to restore normal learning to rut mutants, whereas expression of Neurofibromatosis 1 (NF1) in α/β neurons is sufficient to rescue NF1 mutants. DopR mutations are the only case where memory performance is fully eliminated, consistent with the hypothesis that DopR receives the US inputs for both γ and α/β lobe traces. We demonstrate, however, that DopR expression in γ neurons is sufficient to fully support short- and long-term memory. We argue that DA-mediated CS-US association is formed in γ neurons followed by communication between γ and α/β neurons to drive consolidation.     

Activity of neurons in the pedunculopontine nucleus in conditioned instrumental appetitive reflex

Activity of 91 neurons in the compact and diffuse parts of the pedunculopontine tegmental nucleus (PPTg) was recorded in freely moving rabbits during execution of instrumental appetitive conditioning. Of the recorded neurons, 37.4% changed their activity in response to a conditioned stimulus, which is the evidence for the PPTg involvement in instrumental conditioning. Excitatory conditioned neuronal responses to the conditioned stimulus and food reinforcement significantly prevailed over inhibitory reactions. Neuronal responses to the conditioned stimulus were classified in several basic patterns reflecting stimulus effects, structure of the behavioral act, and reinforcement properties. These reactions indicate the involvement of the PPTg in attention, motor learning, and reinforcement. The revealed differences in associative reactive properties of the compact and diffuse parts of the PPTg to the conditioned stimulus and reinforcement point to the functional heterogeneity of this structure and suggest the leading role of the cholinergic part of the PPTg in instrumental appetitive conditioning and reinforcement as well as the essential contribution of the diffuse part into classical appetitive conditioning.     

Sequential use of mushroom body neuron subsets during drosophila odor memory processing

Drosophila mushroom bodies (MB) are bilaterally symmetric multilobed brain structures required for olfactory memory. Previous studies suggested that neurotransmission from MB neurons is only required for memory retrieval. Our unexpected observation that Dorsal Paired Medial (DPM) neurons, which project only to MB neurons, are required during memory storage but not during acquisition or retrieval, led us to revisit the role of MB neurons in memory processing. We show that neurotransmission from the alpha'beta' subset of MB neurons is required to acquire and stabilize aversive and appetitive odor memory, but is dispensable during memory retrieval. In contrast, neurotransmission from MB alphabeta neurons is only required for memory retrieval. These data suggest a dynamic requirement for the different subsets of MB neurons in memory and are consistent with the notion that recurrent activity in an MB alpha'beta' neuron-DPM neuron loop is required to stabilize memories formed in the MB alphabeta neurons.     

Olfactory learning

The olfactory nervous systems of insects and mammals exhibit many similarities, suggesting that the mechanisms for olfactory learning may be shared. Neural correlates of olfactory memory are distributed among many neurons within the olfactory nervous system. Perceptual olfactory learning may be mediated by alterations in the odorant receptive fields of second and/or third order olfactory neurons, and by increases in the coherency of activity among ensembles of second order neurons. Operant olfactory conditioning is associated with an increase in the coherent population activity of these neurons. Olfactory classical conditioning increases the odor responsiveness and synaptic activity of second and perhaps third order neurons. Operant and classical conditioning both produce an increased responsiveness to conditioned odors in neurons of the basolateral amygdala. Molecular genetic studies of olfactory learning in Drosophila have revealed numerous molecules that function within the third order olfactory neurons for normal olfactory learning.