An event-related brain potential study of visual selective attention to conjunctions of color and shape

What cognitive processes underlie event-related brain potential (ERP) effects related to visual multidimensional selective attention and how are these processes organized? We recorded ERPs when participants attended to one conjunction of color, global shape and local shape and ignored other conjunctions of these attributes in three discriminability conditions. Attending to color and shape produced three ERP effects: frontal selection positivity (FSP), central negativity (N2b), and posterior selection negativity (SN). The results suggested that the processes underlying SN and N2b perform independent within-dimension selections, whereas the process underlying the FSP performs hierarchical between-dimension selections. At posterior electrodes, manipulation of discriminability changed the ERPs to the relevant but not to the irrelevant stimuli, suggesting that the SN does not concern the selection process itself but rather a cognitive process initiated after selection is finished. Other findings suggested that selection of multiple visual attributes occurs in parallel.     

The role of inhibition for working memory processes: ERP evidence from a short‐term storage task

Human working memory is the central unit for short‐term storage of information. In addition to the selection and adequate storage of relevant information, the suppression of irrelevant stimuli from the environment seems to be of importance for working memory processes. To learn more about the interplay of information uptake and inhibition of irrelevant information, the present study used ERP measures and a short‐term storage and retrieval task, in which pairs of either numbers or letters had to be compared. Random sequences of four stimuli (two numbers and two letters) were presented, with either the numbers or the letters being relevant for comparison. The analysis of ERPs to each of the four stimuli indicated more pronounced P2 and P3b amplitudes for relevant than irrelevant stimuli. In contrast, the N2 (reflecting inhibitory control) was only elicited by irrelevant stimuli. Moreover, the N2 amplitude of the second irrelevant stimulus was associated with behavioral performance, indicating the importance of inhibition of task‐irrelevant stimuli for working memory processes. In sum, the findings demonstrate the role of cognitive control mechanisms for protecting relevant contents in working memory against irrelevant information.     

Contralateral delay activity tracks the storage of visually presented letters and words

Electrophysiological studies have demonstrated that the maintenance of items in visual working memory (VWM) is indexed by the contralateral delay activity (CDA), which increases in amplitude as the number of objects to remember increases, plateauing at VWM capacity. Previous work has primarily utilized simple visual items, such as colored squares or picture stimuli. Despite the frequent use of verbal stimuli in seminal investigations of visual attention and memory, it is unknown whether temporary storage of letters and words also elicit a typical load‐sensitive CDA. Given their close associations with language and phonological codes, it is possible that participants store these stimuli phonologically, and not visually. Participants completed a standard visual change‐detection task while their ERPs were recorded. Experiment 1 compared the CDA elicited by colored squares compared to uppercase consonants, and Experiment 2 compared the CDA elicited by words compared to colored bars. Behavioral accuracy of change detection decreased with increasing set size for colored squares, letters, and words. We found that a capacity‐limited CDA was present for colored squares, letters, and word arrays, suggesting that the visual codes for letters and words were maintained in VWM, despite the potential for transfer to verbal working memory. These results suggest that, despite their verbal associations, letters and words elicit the electrophysiological marker of VWM encoding and storage.     

Modulation of similarity on the distraction resistance of visual working memory representation

Controversy exists regarding the distraction resistance of priority items in visual working memory (VWM). The protection account proposes that high-priority items resist distraction more than low-priority items, while the vulnerability account proposes that distractors impair high-priority items more. We proposed another available resource threshold account: distraction will not impair items if available resources exceed their threshold needed for sufficient processing. Using a change-detection paradigm, we manipulated item priority by color similarity and inserted distractors during VWM retention. We investigated the effect of similarity on distraction resistance of relevant (color) and irrelevant (shape) feature representations (Experiments 1–2), and the neural mechanism of this effect using event-related potentials (ERPs; Experiment 3). Behavioral results showed distractors impaired the accuracy of dissimilar items when relevant features were memorized and of similar items when irrelevant features were memorized under simultaneous presentation of similar and dissimilar items. Moreover, distractors impaired the accuracy of dissimilar items when relevant features were memorized and of both similar and dissimilar items when irrelevant features were memorized under separate presentation of items. ERP results showed a smaller negative slow wave and P2 but larger N2 under the similar condition. Similarity protected relevant features of similar items against distraction by reducing memory load, decreasing attentional resources allocated to distractors, and strengthening inhibition of distractors. However, similarity did not protect irrelevant features of similar items. Our results support the available resource threshold account, suggesting that VWM is a flexible and intelligent system despite its limited capacity.     

Electrocortical and ocular indices of attention to fearful and neutral faces presented under high and low working memory load

Working memory load reduces the late positive potential (LPP), consistent with the notion that functional activation of the DLPFC attenuates neural indices of sustained attention. Visual attention also modulates the LPP. In the present study, we sought to determine whether working memory load might exert its influence on ERPs by reducing fixations to arousing picture regions. We simultaneously recorded eye-tracking and EEG while participants performed a working memory task interspersed with the presentation of task-irrelevant fearful and neutral faces. As expected, fearful compared to neutral faces elicited larger N170 and LPP amplitudes; in addition, working memory load reduced the N170 and the LPP. Participants made more fixations to arousing regions of neutral faces and faces presented under high working memory load. Therefore, working memory load did not induce avoidance of arousing picture regions and visual attention cannot explain load effects on the N170 and LPP.     

Electrocortical and ocular indices of attention to fearful and neutral faces presented under high and low working memory load

Working memory load reduces the late positive potential (LPP), consistent with the notion that functional activation of the DLPFC attenuates neural indices of sustained attention. Visual attention also modulates the LPP. In the present study, we sought to determine whether working memory load might exert its influence on ERPs by reducing fixations to arousing picture regions. We simultaneously recorded eye-tracking and EEG while participants performed a working memory task interspersed with the presentation of task-irrelevant fearful and neutral faces. As expected, fearful compared to neutral faces elicited larger N170 and LPP amplitudes; in addition, working memory load reduced the N170 and the LPP. Participants made more fixations to arousing regions of neutral faces and faces presented under high working memory load. Therefore, working memory load did not induce avoidance of arousing picture regions and visual attention cannot explain load effects on the N170 and LPP.     

Does focused endogenous attention prevent attentional capture in pop‐out visual search?

To investigate whether salient visual singletons capture attention when they appear outside the current endogenous attentional focus, we measured the N2pc component as a marker of attentional capture in a visual search task where target or nontarget singletons were presented at locations previously cued as task-relevant, or in the uncued irrelevant hemifield. In two experiments, targets were either defined by color or by a combination of color and shape. The N2pc was elicited both for attended singletons and for singletons on the uncued side, demonstrating that focused endogenous attention cannot prevent attentional capture by salient unattended visual events. However, N2pc amplitudes were larger for attended and unattended singletons that shared features with the current target, suggesting that top-down task sets modulate the capacity of visual singletons to capture attention both within and outside the current attentional focus.     

Effects of display and memory load on the event-related potentials during visual letter search combined with selection by color]

This study investigated the effects of display and memory load on the event-related potentials (ERPs) in a visual letter search task combined with selection by color. ERPs were recorded from 10 adult subjects, when they were required to judge if test stimuli, which consisted of a horizontal array of five colored alphabets, contained target letters defined by shape and color. The results showed that the increase of display or memory load caused the enlargement of three different negativities. The first and second display load effects were observed in the ERPs at midline central electrode (Cz) between 240-280 ms latency range, and in a posterior-distributed negativity that was the maximum at about 650 ms after the stimulus onset. The third was the anterior-central negativity that was the largest at Cz about 550 ms after the stimulus onset. In contrast to the first two effects, this negativity was enhanced by display or memory load. In discussion, a tentative linkage between these ERP changes and two slave systems of working memory (phonological loop and visuo-spatial sketchpad) was proposed.     

The effect of category learning on visual attention and visual representation

Subordinate‐level category learning recruits neural resources associated with perceptual expertise, including the N250 component of the ERP, a posterolateral negative wave maximal between 230 and 330 ms. The N250 is a relatively late visual ERP and could plausibly be driven by attention to the features of categorized objects. Indeed, it has a latency and scalp distribution similar to the selection negativity (SN), an ERP component long known to be sensitive to attentional selection of target features. To clarify sensitivity of the N250 to attention and to more generally investigate the effect of category learning on attentional modulation of learned features, we independently manipulated subordinate‐level category learning and target detection in a speeded paradigm designed to optimally elicit the SN and accompanying frontal selection positivity (FSP). Participants first practiced categorizing a set of artificial animal stimuli and then performed a speeded target detection task on trained and untrained stimuli while ERPs were recorded. SN and FSP were roughly linearly related to the number of target features in the stimulus. Trained stimuli elicited a significantly larger N250 than untrained stimuli. The SN and N250 effects were additive, with all levels of target similarity equally affected by training, and had different time courses. Training had little effect on the FSP. The results suggest that (a) the N250 and SN have different sources, and (b) at the very least, the learning‐induced N250 indexes a different attentional subprocess from the target‐induced SN and could be driven by a different cognitive process altogether.     

The role of stimulus type in age-related changes of visual working memory

Aging is accompanied by increasing difficulty in working memory associated with the temporary storage and processing of goal-relevant information. Face recognition plays a preponderant role in human behavior, and one might therefore suggest that working memory for faces is spared from age-related decline compared to socially less important visual stimulus material. To test this hypothesis, we performed working memory (n-back) tasks with two different visual stimulus types, namely faces and doors, and compared them to tasks with primarily verbal material, namely letters. Age-related reaction time slowing was comparable for all three stimulus types, supporting hypotheses on general cognitive and motor slowing. In contrast, performance substantially declined with age for faces and doors, but little for letters. Working memory for faces resulted in significantly better performance than that for doors and was more sensitive to on-line manipulation errors such as the temporal order. All together, our results show that even though face perception might play a specific role in visual processing, visual working memory for faces undergoes the same age-related decline as it does for socially less relevant visual material. Moreover, these results suggest that working memory decline cannot be solely explained by increasing vulnerability in prefrontal cortex related to executive functioning, but indicate an age-related decrease in a visual short-term buffer, possibly located in the temporal cortex.     

The effects of visual material and temporal synchrony on the processing of letters and speech sounds

Associating letters with speech sounds is essential for reading skill acquisition. In the current study, we aimed at determining the effects of different types of visual material and temporal synchrony on the integration of letters and speech sounds. To this end, we recorded the mismatch negativity (MMN), an index of automatic change detection in the brain, from literate adults. Subjects were presented with auditory consonant–vowel syllable stimuli together with visual stimuli, which were either written syllables or scrambled pictures of the written syllables. The visual stimuli were presented in half of the blocks synchronously with the auditory stimuli and in the other half 200 ms before the auditory stimuli. The auditory stimuli were consonant, vowel or vowel length changes, or changes in syllable frequency or intensity presented by using the multi-feature paradigm. Changes in the auditory stimuli elicited MMNs in all conditions. MMN amplitudes for the consonant and frequency changes were generally larger for the sounds presented with written syllables than with scrambled syllables. Time delay diminished the MMN amplitude for all deviants. The results suggest that speech sound processing is modulated when the sounds are presented with letters versus non-linguistic visual stimuli, and further, that the integration of letters and speech sounds seems to be dependent on precise temporal alignment. Moreover, the results indicate that with our paradigm, a variety of parameters relevant and irrelevant for reading can be tested within one experiment.     

Selective attention to spatial and non-spatial visual stimuli is affected differentially by age: effects on event-related brain potentials and performance data.

To assess selective attention processes in young and old adults, behavioral and event-related potential (ERP) measures were recorded. Streams of visual stimuli were presented from left or right locations (Experiment 1) or from a central location and comprising two different spatial frequencies (Experiment 2). In both experiments, results were compared in visual-only and visual+auditory stimulus context conditions. Participants were forced to respond fast in both experiments, while maintaining high accuracy. In Experiment 1, no behavioral effects of aging were found; however, an enlargement of the N1 component in the older age group suggested that older adults initial selection process was larger than that of young adults. A late frontal effect following the P300 elicited by attended non-targets was larger in the visual+auditory condition than in the visual-only condition in the old age group. This effect was interpreted as reflecting a memory update of the relevant target location. In Experiment 2, older adults made relatively more errors in the visual+auditory condition than in visual-only condition, more so than the young adults. Older adults' ERP data were also characterized by an enlargement of the occipital selection negativity, compared to the young age group. In contrast to experiment 1, no late frontal post-P3 effect could be found, suggesting that the memory trace of the relevant stimulus feature was updated less frequently, explaining the reduction in response accuracy in the visual+auditory stimulus context conditions.     

The uniqueness of social attention revisited: working memory load interferes with endogenous but not social orienting

It is well known that perceived eye gaze direction influences attentional orienting. However, it still remains unclear whether social orienting involves exogenous or endogenous attentional control. To address this issue, we examined if social orienting and endogenous orienting were differentially modulated by working memory load, which is known to interfere with endogenous but not exogenous attention. To do so, we manipulated eye direction as either spatially counterpredictive in Experiment 1 or spatially predictive in Experiment 2 while participants performed a cueing task either in isolation or under working memory load. We found that when social attention and endogenous attention diverged spatially in Experiment 1, social orienting elicited by gaze direction remained intact while endogenous orienting elicited by the cue’s predictive meaning was suppressed under working memory load, suggesting independence between social orienting and endogenous orienting. Indeed, a comparison between the sum of isolated social orienting and endogenous orienting magnitudes from Experiment 1 relative to their combined measure from Experiment 2 confirmed that social attention and endogenous attention operated in parallel. Together, our data show that social orienting is independent from endogenous orienting and further suggest that paying attention to social information might involve either exogenous or unique attentional mechanisms.     

Dynamics of target and distractor processing in visual search: evidence from event-related brain potentials

When multiple objects are present in a visual scene, salient and behaviorally relevant objects are attentionally selected and receive enhanced processing at the expense of less salient or less relevant objects. Here we examined three lateralized components of the event-related potential (ERP) - the N2pc, Ptc, and SPCN - as indices of target and distractor processing in a visual search paradigm. Participants responded to the orientation of a target while ignoring an attentionally salient distractor and ERPs elicited by the target and the distractor were obtained. Results indicate that both the target and the distractor elicit an N2pc component which may index the initial attentional selection of both objects. In contrast, only the distractor elicited a significant Ptc, which may reflect the subsequent suppression of distracting or irrelevant information. Thus, the Ptc component appears to be similar to another ERP component - the Pd - which is also thought to reflect distractor suppression. Furthermore, only the target elicited an SPCN component which likely reflects the representation of the target in visual short term memory.     

The active inhibition for the processing of visual irrelevant conflict information.

We applied event-related potentials (ERPs) to assess brain activities with inhibition of irrelevant conflict information processing during visual selective attention. Fifteen healthy subjects matched two sequentially presented color spots (S1 and S2) in a visual selective attention task. The two spots might be presented in a fixed position of either the same color (match), or different colors (color conflict). They might be in different positions of the same color (position conflict), or different colors (color and position conjunction conflicts). Subjects matched the stimuli in two different sessions according to different attention tasks: attending to color (Ac) or attending to position (Ap). In the time window of 201-350 ms after the onset of S2, an ERP component the N270, was elicited in all the conflict conditions with the amplitude enhanced in the task-relevant conflict. The amplitude of the N270 in the conjunction conflict condition was lower than that of the task-relevant conflict condition in either Ac or Ap session. The results suggest that the conflict processing is modulated by attention. All kinds of conflicts might be processed in a common system in the human brain, the conflict processing system. Due to the limited capacity of the conflict processing system, the decrement of the N270 in the conjunction conflict condition is deduced to reflect an active inhibition for the processing of the irrelevant conflict.     

Aging effects on selective attention-related electroencephalographic patterns during face encoding

Previous electrophysiological studies revealed that human faces elicit an early visual event-related potential (ERP) within the occipito–temporal cortex, the N170 component. Although face perception has been proposed to rely on automatic processing, the impact of selective attention on N170 remains controversial both in young and elderly individuals. Using early visual ERP and alpha power analysis, we assessed the influence of aging on selective attention to faces during delayed-recognition tasks for face and letter stimuli, examining 36 elderly and 20 young adults with preserved cognition. Face recognition performance worsened with age. Aging induced a latency delay of the N1 component for faces and letters, as well as of the face N170 component. Contrasting with letters, ignored faces elicited larger N1 and N170 components than attended faces in both age groups. This counterintuitive attention effect on face processing persisted when scenes replaced letters. In contrast with young, elderly subjects failed to suppress irrelevant letters when attending faces. Whereas attended stimuli induced a parietal alpha band desynchronization within 300–1000 ms post-stimulus with bilateral-to-right distribution for faces and left lateralization for letters, ignored and passively viewed stimuli elicited a central alpha synchronization larger on the right hemisphere. Aging delayed the latency of this alpha synchronization for both face and letter stimuli, and reduced its amplitude for ignored letters. These results suggest that due to their social relevance, human faces may cause paradoxical attention effects on early visual ERP components, but they still undergo classical top–down control as a function of endogenous selective attention. Aging does not affect the face bottom–up alerting mechanism but reduces the top–down suppression of distracting letters, possibly impinging upon face recognition, and more generally delays the top–down suppression of task-irrelevant information.     

The allocation of attention in displays with simultaneously presented singletons

In an ERP experiment, we investigated whether a ‘permanent’ salient distractor changes the deployment of attention to target and nontarget singletons. Observers searched for a color target in a search array that mainly consisted of black vertical lines, but also always contained a line in a task-irrelevant color. Together with this distractor, a target or nontarget singleton was presented. Nontargets could be salient on the task-relevant dimension (color), or on a neutral dimension (line orientation). N2pc amplitude was maximal for targets, no N2pc was elicited by color nontargets, and orientation nontargets elicited an inverse N2pc. Targets and color nontargets elicited larger N2 amplitude than orientation singletons. P3 amplitude was high for relevant and low for irrelevant singletons. Targets also elicited higher reaction times and more errors. Attention seemed thus driven by the target feature, and by its feature dimension, even when constant distraction on that dimension had to be suppressed.     

Age-related deficits in component processes of working memory

Working memory deficits in normal aging have been well documented, and studies suggest that high memory load plus the presence of distraction negatively impacts successful memory performance to a greater degree in older individuals. However, characterization of the component processes that are impaired by these task manipulations is not clear. In this behavioral study, younger and older subjects were tested with a delayed-recognition and recall task in which the encoding and delay period were both manipulated. During the encoding period, the subjects were presented with either a single letter or multiple letters at their predetermined forward letter span, and the delay period was either uninterrupted or interrupted with a visual distraction. There was an age-related impairment of working memory recognition accuracy only in the combination of high memory load and distraction. These results suggest that when working memory maintenance systems are taxed, faulty recognition processes may underlie cognitive aging deficits in healthy older individuals.     

Novelty detection is enhanced when attention is otherwise engaged: an event-related potential study

Novel stimuli are detected and evaluated quickly, suggesting that processing them is a priority for the brain. In the present study, the effects of attention on this early visual novelty processing were investigated in two experiments using the event-related potential (ERP) technique. In the first experiment, participants performed two tasks that varied in the amount of attention available for novel stimuli. In the Visual Oddball task, participants responded to an infrequent target presented among standard and novel stimuli. In the Working Memory task, participants saw the same stimuli, but they could ignore them. Instead, participants had to keep six letters in working memory and report one of these letters at the end of the trial; attention was thus maximally allocated away from the visual oddball stimuli. In line with attention being fully occupied in the Working Memory task, the P3a to the visual oddball stimuli was smaller in the Working Memory than in the Visual Oddball task. In contrast, the anterior N2 component to task-irrelevant stimuli was enhanced in the Working Memory task. These findings suggest that the initial detection of novel stimuli is enhanced (large anterior N2) when few attentional resources are available, which is inconsistent with earlier findings that if anything, the N2 is enhanced by attention. In a second experiment, a condition was added in which working memory load was low, but visual oddball stimuli were task-irrelevant. Results from this experiment showed that while the reduction in P3a amplitude was due to task irrelevance, the enhanced anterior N2 was linked to a high working memory load. This suggests that novelty detection is enhanced when attention is otherwise engaged.     

Reward‐related distracters and working memory filtering

Reward‐related stimuli capture attention, even when they are task irrelevant. A consequence of attentional prioritization of reward‐related stimuli is that they may also have preferential access to working memory like other forms of emotional information. However, whether reward‐related distracters leak into working memory remains unknown. Here, using a well‐validated change detection task of visual working memory capacity and filtering, we conducted two studies to directly assess the impact of reward‐related distracters on working memory. In both studies, the distracters consisted of colored bars or circles that were previously associated with monetary reward. In Experiment 1, results indicated that previously rewarded distracters did not impact behavioral measures of working memory filtering efficiency compared to neutral distracters. In Experiment 2, using ERPs, we measured the contralateral delay activity (CDA), a psychophysiological index of the number of items retained in working memory, to further assess filtering efficiency. We observed that the CDA for high reward distracters was similar to low reward and neutral distracters. However, in early trials, behavioral measures revealed that previously rewarded stimuli negatively impacted working memory capacity, an effect not observed with neutral distracters. This effect, though, was not found for the CDA in early trials. In summary, our findings across two studies suggest that attentional capture by task‐irrelevant reward may have minimal impact on visual working memory—findings that have important implications for delineating the boundaries of reward‐cognition interactions.