Looks are (almost) everything: where drivers look to get information

OBJECTIVE: To describe the impact of Rockwell's early eye movements research. BACKGROUND: The advent of a new technology enabling measurements of eye movements in natural environments launched the seminal research of a Human Factors pioneer, Tom Rockwell, into how drivers process visual information. METHOD: In two seminal Human Factors articles -"Mapping Eye-Movement Pattern to the Visual Scene in Driving: An Exploratory Study" (Mourant & Rockwell, 1970) and "Strategies of Visual Search by Novice and Experienced Drivers" (Mourant & Rockwell, 1972)- Rockwell and his student, Ron Mourant, examined drivers' eye movements in naturalistic driving environments. RESULTS: The analyses of the visual fixations revealed systematic relationships between the sources of information the drivers needed to drive safely and the spatial distributions of their visual fixations. In addition, they showed that as drivers gain skill and experience, their pattern of fixations changes in a systematic manner. CONCLUSIONS: The research demonstrated that fixations and saccadic eye movements provide important insights into drivers' visual search behavior, information needs, and information acquisition processes. APPLICATION: This research has been a cornerstone for a myriad of driving-related studies, by Rockwell and other researchers. Building on Rockwell's pioneering work, these studies used eye-tracking systems to describe cognitive aspects of skill acquisition, and the effects of fatigue and other impairments on the process of attention and information gathering. A novel and potentially revolutionary application of this research is to use eye movement recordings for vehicle control and activation of in-vehicle safety systems.     

The influence of combined visual and tactile information on finger and eye movements during shape tracing

Abstract

Adaptation experiments in shape tracing were conducted to investigate finger and eye movements in various conditions of visual and tactile information. Maximum velocity, mean velocity, maximum acceleration and reacceleration point were calculated from finger movements. Number of eye fixations and lead time of eye fixation to finger position were calculated from eye movements. The results showed that for the finger movement the values of the indices studied were higher in the combined visual and tactile condition than in the visual only condition. The number of eye fixations decreased when subjects repeated the tracing and was more marked in the combined visual and tactile condition than in the visual only condition. The results suggest that finger movements become faster and use of vision is reduced when both visual and tactile information are given.     

Affects on visual search performance of individual differences in fixation time and number of fixations

Abstract

The purpose of this study was to provide a basis for improving individual visual performance of inspectors. The relationship between the correct count rate and eye movements of subjects when they counted dots arranged on samples presented for different lengths of time were analysed mainly to determine individual differences. Subjects' eye movements were measured with a corneal reflectance eye camera and analysed frame by frame with a video motion analyser. It was found that accuracy of visual inspection does not depend on length of search time and that a fast search time is not incompatible with a slow search speed. Furthermore, fixation time and number of fixations were considered the main factors governing accuracy of visual inspection. When limited time is allowed for search, a search strategy of prolonging the fixation time leads to high performance and consequently shorter inspection time. Several other findings were obtained which appear important in obtaining accurate information rapidly.     

Social facilitation effect in search and decision‐making components of visual inspection

This study aimed to examine the effect of social presence on the performance of the search and decision‐making components of visual inspection. A within‐subject design experiment was conducted. Participants performed easy and difficult search and decision‐making tasks in alone and audience‐present conditions with search time, decision time, decision accuracy rates, and subjective arousal level in both components measured. Results indicated that the presence of an audience shortened (lengthened) the response time in easy (difficult) search and decision‐making components but did not influence decision accuracy rates. The social facilitation intensity in the search component was stronger than that in the decision‐making component. For both components, the performance impairment in difficult tasks was stronger than the performance improvement in easy tasks, and arousal was greater in the audience‐present condition than when working alone. The findings helped clarify the influential mechanisms of the social facilitation effect in visual inspection tasks.     

Eye movements of younger and older drivers.

This 2-part study focuses on eye movements to explain driving-related visual performance in younger and older persons. In the first task, participants' eye movements were monitored as they viewed a traffic scene image with a numeric overlay and visually located the numbers in their sequential order. The results showed that older participants had significantly longer search episodes than younger participants, and that the visual search of older adults was characterized by more fixations and shorter saccades, although the average fixation durations remained the same. In the second task, participants viewed pictures of traffic scenes photographed from the driver's perspective. Their task was to assume the role of the driver and regard the image accordingly. Results in the second task showed that older participants allocated a larger percentage of their visual scan time to a small subset of areas in the image, whereas younger participants scanned the images more evenly. Also, older participants revisited the same areas and younger participants did not. The results suggest how aging might affect the efficacy of visual information processing. Potential applications of this research include training older drivers for a more effective visual search, and providing older drivers with redundant information in case some information is missed.     

Perceptual training for visual search

People are better at visual search than the best fully automated methods. Despite this, visual search remains a difficult perceptual task. The goal of this investigation was to experimentally test the ways in which visual search performance could be improved through two categories of training interventions: perceptual training and conceptual training. To determine the effects of each training on a later performance task, the two types of trainings were manipulated using a between-subjects design (conceptual vs. perceptual × training present vs. training absent). Perceptual training led to speed and accuracy improvements in visual search. Issues with the design and administration of the conceptual training limited conclusions on its effectiveness but provided useful lessons for conceptual training design. The results suggest that when the visual search task involves detecting heterogeneous or otherwise unpredictable stimuli, perceptual training can improve visual search performance. Similarly, careful consideration of the performance task and training design is required to evaluate the effectiveness of conceptual training.

Practitioner Summary: Visual search is a difficult, yet critical, task in industries such as baggage screening and radiology. This study investigated the effectiveness of perceptual training for visual search. The results suggest that when visual search involves detecting heterogeneous or otherwise unpredictable stimuli, perceptual training may improve the speed and accuracy of visual search.     

Effects of speed–accuracy instructions on oculomotor scanning and target recognition in a simulated baggage X-ray screening task

Visual search tasks are often carried out under high levels of time stress. Transportation security screeners, for example, face demands to achieve high levels of accuracy while maintaining rapid passenger throughput. An experiment examined the strategies by which operators regulate visual search performance under such conditions. Observers performed a simulated baggage-screening task under instructions to emphasise either response speed or accuracy. Behavioural measures and eye movements were recorded. Observers made fewer and briefer fixations under emphasise-speed than under emphasise-accuracy instructions. Losses in accuracy were produced by more frequent failures to fixate on targets and a decrease in the detection rate of non-fixated targets. The likelihood with which observers detected a fixated target was similar across speed–accuracy instructions. Results will inform efforts to model visual search in naturalistic tasks, allowing more accurate prediction of response times and error rate and may aid the design of training programmes and other interventions to improve search performance under stress.     

Combining experiences of race gaming and natural driving affects gaze location strategy in simulated context

Abstract

The aims of the study were to investigate the effects of race gaming experience in playing racing video games on gaze behaviour and performance of drivers and the effects of natural driving experience on gaze behaviour and performance of gamers. Thirty participants, divided into drivers-gamers, drivers-non-gamers and non-drivers-gamers, were asked to drive in a race circuit as fast as possible while their eye movements were recorded. Drivers-gamers spent more time looking at the lane than non-drivers-gamers. Furthermore, drivers-gamers performed greater number of fixations towards the speedometer and showed faster performance in the racing task than the drivers-non-gamers. Combining natural driving and race gaming experiences changed the gaze location strategy of drivers.

Practitioner summary: Racing video games practitioners have high propensity to exhibit attitudes and intentions of risky driving behaviour. Combining natural driving and race gaming experiences affects gaze behaviour strategy of drivers.

Abbreviations: DG: Drivers-gamers; DNG: Drivers-non-gamers; NDG: Non-drivers-gamers; AOIs: Areas of Interest; r-NUMFIX: Relative number of fixations; r-DURFIX: Relative fixations duration     

Visual search in modern human-computer interfaces

Abstract

This article reviews aspects of visual search in relation to computer visual display units. Theoretical issues such as eye movements in visual search are discussed as well as practical examples such as the role of array shape, the benefits of cursor-presented status (insert vs overtype) information, conspicuity of peripherally presented information, and possible benefits of anti-aliased fonts. In association with the four experiments relating to these aspects, the respective follow-up eye movement monitoring studies are also described as these allowed quantification of what otherwise might only have been inferred. The review concludes with four major recommendations. First, much scope remains for exploring the optimization of status (e.g., cursor-presented) information. Second, it would be worth exploring whether icons, like verbal labels, are susceptible to an array shape effect. Third, further work is required (e.g. in the possible role of articulatory differences) in accounting for the ease of locatability of icons over words. Fourth, spatial frequency analysis seems likely to be a major research field over the next decade.     

Determining effects of virtually and physically present co‐actor in evoking social facilitation

This study investigated whether a virtually present co‐actor (VPC) can evoke social facilitation effect in visual search tasks. The conducted experiment employed a 2 (target presence: present, absent) × 2 (task difficulty: easy, difficult) × 3 (social presence: alone, VPC, physically present co‐actor, PPC) within‐subject design. Twelve male university students participated in the experiment. Results showed that a VPC evoked social facilitation effect in visual search tasks, that is, shortened (lengthened) response time when easy (difficult) tasks were completed. Intensities of social facilitation effect induced by VPC and PPC showed similar strengths. However, both VPC and PPC presented no significant influence on response accuracy. These findings provide guidance in designing work organization modes for a team or a group performing tasks of different difficulty levels.     

Automatic eye fixations identification based on analysis of variance and covariance

Eye movement is the simplest and repetitive movement that enables humans to interact with the environment. The common daily activities, such as reading a book or watching television, involve this natural activity, which consists of rapidly shifting our gaze from one region to another. In clinical application, the identification of the main components of eye movement during visual exploration, such as fixations and saccades, is the objective of the analysis of eye movements: however, in patients affected by motor control disorder the identification of fixation is not banal. This work presents a new fixation identification algorithm based on the analysis of variance and covariance: the main idea was to use bivariate statistical analysis to compare variance over x and y to identify fixation. We describe the new algorithm, and we compare it with the common fixations algorithm based on dispersion. To demonstrate the performance of our approach, we tested the algorithm in a group of healthy subjects and patients affected by motor control disorder.     

Prospective memory in the ICU: the effect of visual cues on task execution in a representative simulation

Despite the potential dangers of clinical tasks being forgotten, few researchers have investigated prospective memory (PM) – the ability to remember to execute future tasks – in health-care contexts. Visual cues help people remember to execute intentions at the appropriate moment. Using an intensive care unit simulator, we investigated whether nurses' memory for future tasks improves when visual cues are present, and how nurses manage PM demands. Twenty-four nurses participated in a 40-minute scenario simulating the start of a morning shift. The scenario included eight PM tasks. The presence or absence of a visually conspicuous cue for each task was manipulated. The presence of a visual cue improved recall compared to no cue (64% vs. 50%, p = 0.03 one-tailed, η _p ² = 0.15). Nurses used deliberate reminders to manage their PM demands. PM in critical care might be supported by increasing the visibility of cues related to tasks.

Practitioner summary: Nurses must remember to execute multiple future tasks to ensure patient safety. We investigated the effect of visual cues on nurses' ability to remember future tasks. Experimental manipulation of cues in a representative intensive care unit simulation indicated that visual cues increase the likelihood that future tasks are executed.     

Insights from surgeons’ eye-movement data in a virtual simulation surgical training environment: effect of experience level and hand conditions

Today, with the advancements in the eye-tracking technology, it has become possible to follow surgeons’ eye movements while performing surgical tasks. Despite the availability of studies providing a better understanding of surgeons’ eye movements, research in the particular field of endoneurosurgery is very limited. Analysing surgeons’ eye-movement data can provide general insights into how to improve surgical education programmes. In this study, four simulation-based task-oriented endoscopic surgery training scenarios were developed and implemented by 23 surgical residents using three different hand conditions; dominant, non-dominant, and both. The participants’ recorded eye data comprised fixation number, fixation duration, saccade number, saccade duration, pursuit number, pursuit duration, and pupil size. This study has two main contributions: First, it reports on the eye-movement behaviours of surgical residents, demonstrating that novice residents tended to make more fixations and saccades than intermediate residents. They also had a higher fixation duration and followed the objects more frequently compared to the intermediates. Furthermore, hand conditions significantly affected the eye movements of the participants. Based on these results, it can be concluded that eye-movement data can be used to assess the skill levels of surgical residents and would be an important measure to better guide trainees in surgical education programmes. The second contribution of this study is the eye-movement event classifications of 10 different algorithms. Although the algorithms mostly provided similar results, there were a few conflicted values for some classifications, which offers a clue as to how researchers can utilise these algorithms with low sampling frequency eye trackers.     

Evaluating the human ongoing visual search performance by eye tracking application and sequencing tests

Human visual search is an everyday activity that enables humans to explore the real world. Given the visual input, during a visual search, it is necessary to select some aspects of input to shift the gaze to next target. The aim of the study is to develop a mathematical method able to evaluate the visual selection process during the execution of a high cognitively demanding task such as the trial making test part B (TMT). The TMT is a neuro-psychological instrument where numbers and letters should be connected to each other in numeric and alphabetic order. We adapted the TMT to an eye-tracking version, and we used a vector model, the "eight pointed star" (8PS), to discover how selection (fixations) guides next exploration (saccades) and how human top-down factors interact with bottom-up saliency. The results reported a trend to move away from the last fixations correlated to the number of distracters and the execution performance.     

A Human-Cognitive Perspective of Users’ Password Choices in Recognition-Based Graphical Authentication

ABSTRACT

Graphical password composition is an important part of graphical user authentication which affects the strength of the chosen password. Considering that graphical authentication is associated with visual search, perception, and information retrieval, in this paper we report on an eye-tracking study (N = 109) that aimed to investigate the effects of users’ cognitive styles toward the strength of the created passwords and shed light into whether and how the visual strategy of the users during graphical password composition is associated with the passwords’ strength. For doing so, we adopted Witkin’s Field Dependence-Independence theory, which underpins individual differences in visual information and cognitive processing, as graphical password composition tasks are associated with visual search. The analysis revealed that users with different cognitive processing characteristics followed different patterns of visual behavior during password composition which affected the strength of the created passwords. The findings underpin the need of considering human-cognitive characteristics as a design factor in graphical password schemes. The paper concludes by discussing implications for improving recognition-based graphical passwords through adaptation and personalization techniques based on individual cognitive characteristics.     

Evaluating camouflage design using eye movement data

This study investigates the characteristics of eye movements during a camouflaged target search task. Camouflaged targets were randomly presented on two natural landscapes. The performance of each camouflage design was assessed by target detection hit rate, detection time, number of fixations on display, first saccade amplitude to target, number of fixations on target, fixation duration on target, and subjective ratings of search task difficulty. The results showed that the camouflage patterns could significantly affect the eye-movement behavior, especially first saccade amplitude and fixation duration, and the findings could be used to increase the sensitivity of the camouflage assessment. We hypothesized that the assessment could be made with regard to the differences in detectability and discriminability of the camouflage patterns. These could explain less efficient search behavior in eye movements. Overall, data obtained from eye movements can be used to significantly enhance the interpretation of the effects of different camouflage design.     

Is visual content in textual search interfaces beneficial to dyslexic users?

Dyslexia is a learning disability characterised by problems with accurate or fluent word recognition, poor decoding, and poor spelling abilities. Although several studies have addressed dyslexia and Web accessibility, less is known about how dyslexia affects information search. This study investigated whether the inclusion of icons in search user interfaces enhances performance among dyslexics. A total of 21 dyslexics and 21 controls completed 52 search tasks in 4 conditions: icons only, words only, and both icons and words in a grid layout and a list layout, while eye movements were recorded. Dyslexics took significantly longer than controls to locate targets in tasks containing text, but not in the icon-only condition. Dyslexics had longer fixation durations than controls in both icon and text based search arrays, suggesting higher mental load associated with search tasks generally. The addition of words to icon arrays led to faster search times within controls, but not dyslexics. Dyslexics also exhibited more fixations on dual-modality tasks, and longer scanpaths than controls in list layout. Both groups were fastest searching the list layout, with icons and words listed in columns. Results are discussed in terms of the design of accessible search interfaces for dyslexic users, taking into account mental load of dual-modality information display, and the arrangement of search items. Empirical data is provided for the design of accessible search results interfaces for dyslexics.     

The impact of alerting designs on air traffic controller’s eye movement patterns and situation awareness

Abstract

This research investigated controller’ situation awareness by comparing COOPANS’s acoustic alerts with newly designed semantic alerts. The results demonstrate that ATCOs’ visual scan patterns had significant differences between acoustic and semantic designs. ATCOs established different eye movement patterns on fixations number, fixation duration and saccade velocity. Effective decision support systems require human-centered design with effective stimuli to direct ATCO’s attention to critical events. It is necessary to provide ATCOs with specific alerting information to reflect the nature of the critical situation in order to minimise the side effects of startle and inattentional deafness. Consequently, the design of a semantic alert can significantly reduce ATCOs’ response time, therefore providing valuable extra time in a time-limited situation to formulate and execute resolution strategies in critical air safety events. The findings of this research indicate that the context-specified design of semantic alerts could improve ATCO’s situational awareness and significantly reduce response time in the event of Short Term Conflict Alert (STCA) activation which alerts to two aircraft having less than the required lateral or vertical separation.

Practitioner Summary: Eye movements are closely linked with visual attention and can be analysed to explore shifting attention whilst performing monitoring tasks. This research has found that context-specific designed semantic alerts facilitated improved ATCO cognitive processing by integrating visual and auditory resources. Semantic designs have been demonstrated to be superior to acoustic design by directing the operator’s attention more quickly to critical situations.Abbreviations: APW: area proximity warning; ASRS: aviation safety reporting system; ATC: air traffic control; ATCO: air traffic controller; ATM: air traffic management; COOPANS: cooperation between air navigation service providers; HCI: human-computer interaction; IAA: irish aviation authority; MSAW: minimum safe altitude warning; MTCD: medium-term conflict detection; SA: situation awareness; STCA: short term conflict alert; TP: trajectory prediction     

Visual search tasks: measurement of dynamic visual lobe and relationship with display movement velocity

Visual lobe is a useful tool for predicting visual search performance. Up till now, no study has focused on dynamic visual lobe. This study developed a dynamic visual lobe measurement system (DVLMS) that could effectively map dynamic visual lobe and calculate visual lobe shape indices. The effects of display movement velocity on lobe shape indices were examined under four velocity conditions: 0, 4, 8 and 16 deg/s. In general, with the increase of display movement velocity, visual lobe area and perimeter became smaller, whereas lobe shape roundness, boundary smoothness, symmetry and regularity deteriorated. The elongation index was not affected by velocity. Regression analyses indicated that display movement velocity was important in determining dynamic visual lobe shape indices. Dynamic visual lobe provides another option for better understanding dynamic vision, in addition to dynamic visual acuity. Findings of this study can provide guidelines for analysing and designing dynamic visual tasks.

Practitioner Summary: Dynamic visual lobe is important in reflecting the visual ability of searching for a moving target. We developed a dynamic visual lobe measurement system (DVLMS) and examined display movement velocity’s effects on lobe shape. Findings revealed that velocity was a key factor affecting dynamic visual lobe shape indices.     

A model of head movement contribution for gaze transitions

Head posture has been associated with work-related neck symptoms and discomfort, but its relationship with visual tasks has received much less attention. Head movement amplitude is normally a fraction of the angular distance to a visual target, as gaze transition is usually achieved through the combination of both head and eye movement. In this study, the proportion of head orientation vs. target orientation, named head movement contribution ratio (HMCR), was quantified and modelled as a function of target location. Head movements were measured on subjects orienting and maintaining gaze for 2 s at randomly presented visual targets distributed along an arc placed horizontally or vertically. Bootstrap regression models showed that the horizontal HMCR was approximately 69% of target azimuth. The vertical HMCR was bilinear and corresponded to 52% and 8% and of target elevation for targets above and below the horizontal plane, respectively. The data also demonstrated that head orientation is affected by the kinematic coupling between horizontal and vertical components of head movement.

Statement of Relevance: Awkward head and neck posture is a risk factor for work-related musculoskeletal disorders. This study investigated the influence of visual target location on head orientation over a large range of target eccentricity, as an attempt to predict the head and neck posture required for visual target detection and identification.