Influence mechanism of icon semantics on visual search performance: Evidence from an eye-tracking study

Explicit icon semantics can reduce the difficulty of understanding complex visual information. Optimizing the icon semantics and text semantics of icons can effectively improve the cognitive performance of digital interfaces. This paper adopts visual search tasks to study the effects of different combinations of icon semantic familiarity and the presence or absence of text on icon search performance under horizontal and vertical layouts. The behavioral experiment results show that under two layouts: 1. The main effect of icon semantics is significant, and the search performance increases with the increase of semantic familiarity. 2. The main effect of text is significant, and the search performance is negatively correlated with the addition of text. The eye movement experiment found that the semantic familiarity of icons had a significant impact on average fixation time. Furthermore, the number of fixation points changed significantly after the text variable was added. Therefore, there was no significant difference in the number of fixation points in the horizontal layout, and icon semantics was the main influencing factor in visual search. In the vertical layout, there was no significant difference in average fixation time, and text was the main influencing factor of visual search. The results show that the semantic familiarity of icons and different combinations with or without text significantly affect visual search performance in horizontal and vertical layouts. This paper provides a theoretical reference for the combination of icons and text in interface design.     

The effects of icon internal characteristics on complex cognition

The interaction between humans and digital interfaces is a complex cognitive process which has a high demand on working memory resources. However, most previous research on icon perception seems to focus only on the influences of an icon's internal characteristics (e.g. visual complexity, concreteness and familiarity) on simple tasks such as visual search and recall tasks. The study reported here applied a math task to simulate the complicated process of human-computer interaction and investigated the effects of icon internal characteristics on the performance of complex cognition. First, we created and filtered 80 novel icons of 4 different types (simple & concrete, simple & abstract, complex & concrete, complex & abstract). Subsequently, participants were required to perform an icon-meaning recall task, during which icon familiarity was manipulated by different exposure frequencies. Finally, after viewing random icon-digit associations, participants' abilities to solve algebraic equations involving icons were tested. The results indicated that participants performed significantly better when the icons were simple, concrete or familiar and these beneficial effects increased as the task became more complex. Additionally, strong evidence was found to show that it was easier to encode and maintain familiar icons and associated information in working memory. The findings of this study provide useful information for the icon design and interface design of human-computer interaction.     

EvIcon: Designing High-Usability Icon with Human-in-the-loop Exploration and IconCLIP

Interface icons are prevalent in various digital applications. Due to limited time and budgets, many designers rely on informal evaluation, which often results in poor usability icons. In this paper, we propose a unique human-in-the-loop framework that allows our target users, that is novice and professional user interface (UI) designers, to improve the usability of interface icons efficiently. We formulate several usability criteria into a perceptual usability function and enable users to iteratively revise an icon set with an interactive design tool, EvIcon. We take a large-scale pre-trained joint image-text embedding (CLIP) and fine-tune it to embed icon visuals with icon tags in the same embedding space (IconCLIP). During the revision process, our design tool provides two types of instant perceptual usability feedback. First, we provide perceptual usability feedback modelled by deep learning models trained on IconCLIP embeddings and crowdsourced perceptual ratings. Second, we use the embedding space of IconCLIP to assist users in improving icons' visual distinguishability among icons within the user-prepared icon set. To provide the perceptual prediction, we compiled IconCEPT10K, the first large-scale dataset of perceptual usability ratings over 10,000 interface icons, by conducting a crowdsourcing study. We demonstrated that our framework could benefit UI designers' interface icon revision process with a wide range of professional experience. Moreover, the interface icons designed using our framework achieved better semantic distance and familiarity, verified by an additional online user study.     

From action icon to knowledge icon: Objective-oriented icon taxonomy in computer science

Icon plays a critical role in computer interface design. Studies on icon taxonomy explain the way in which various types of icon represent the objects and provide designers creation rules by which icons are more in line with users’ cognitive psychology. However, along with larger and larger use of icons, the previous classification criterion causes the boundary between categories blur. What’s more, Single classification standard is not able to well illustrate the icons applied in today’s computer applications. The purpose of this paper is to present an objective-oriented icon taxonomy which proposes to categorize icons into action icon and knowledge icon. To assess this proposition, we analyzed a sample of icons that applied in computer interface and suggest precise application domains to both action icon and knowledge icon categories. The results of this practice manifested that action icon and knowledge icon implied a high relation with applied environment and explicated the development trace of computer icons. This work is one of the first to point out the notion of knowledge icon and to highlight the importance of objective of icon application. Findings in this paper could enrich icon use in computer interface design, especially provides possible way to improve online knowledge sharing by visual tool like icon.     

How do app icon color and border shape influence visual search efficiency and user experience? Evidence from an eye-tracking study

Icon color and icon border shape are two key factors that affect search efficiency and user experience but have previously been studied separately. This study aimed to ascertain their separate and combined effects on smartphone interfaces. We conducted an experiment using eye tracking in addition to performance and experience measures to understand the effects of app icon color and border shape on visual efficiency and user experience. The results identified both features as essential attributes with interactive effects in the process of searching app icons on a smartphone interface. The study confirmed that varied colors across icons and a rounded square border shape helped to improve search efficiency, decrease cognitive effort, and lead to a more positive user experience.Relevance to industryUsers of smartphones are often confronted with the problem of selecting a single app from a great number of apps. Visual design of app icons plays a key role in influencing visual search efficiency and user experience. The results of this study have implications for designing app icons on the interface of smartphones to improve search efficiency and elicit positive user experience.     

Effects of smartphone icon background shapes and figure/background area ratios on visual search performance and user preferences

Smartphones are becoming increasingly popular, users are provided with various interface styles with different designed icons. Icon, as an important competent of user interface, is regarded to be more efficient and pleasurable. However, compared with desktop computers, fewer design principles on smartphone icon were proposed. This paper investigated the effects of icon background shape and the figure/background area ratio on visual search performance and user preference. Icon figures combined with six different geometric background shapes and five different figure/background area ratios were studied on three different screens in experiments with 40 subjects. The results of an analysis of variance (ANOVA) showed that these two independent variables (background shape and figure/background area ratio) significantly affected the visual search performance and user preference. On 3.5-in (1 in=0.025 4 m) and 4.0-in displays, unified backgroundwould be optimal, shapes such as square, circle and transitions between them (e.g., rounded square, squircle, etc.) are recommended because backgrounds in these shapes yield a better search time performance and subjective satisfaction for ease of use, search and visual preference. A 60% figure/background area ratio is the most appropriate for smartphone icon design on the 3.5-in screen, while a 50% area ratio could be a suggestion for both relatively optimized search performance and user preference on 4.0-in. In terms of the 4.7-in, icon figure is used directly for its better performance and preference compared with icons with background.     

The effects of color combinations,luminance contrast,and area ratio on icon visual search performance

The visual search of icons in a digital interface plays an important role in human-computer interaction. In this behavior study about the visual search of graphic symbols, the effects of the color combination, luminance contrast, and icon area ratio were investigated. A total of 28 participants (12 men and 16 women, ages ranging from 20 to 28 years, Mean = 22.4, SD = 1.7) were asked to perform an icon search task. Participants’ icon search accuracy and response times were measured as a function of the following independent variables: three levels of area ratios, four background colors, four foreground colors, and three levels of luminance contrast. The results showed that although there was no significant main effect of luminance contrast on the icon search accuracy, participants responded more quickly to medium luminance contrast than low or high luminance contrast. Similarly, the medium or low area ratio was more conducive to the participants identifying icons. Moreover, a strong interaction was observed between the effects of luminance contrast and icon area ratio. The icon search performance of the high luminance contrast improved significantly as the figure/background area ratio decreased. Icon color combinations also significantly affected visual search performance. White on black, yellow on black, turquoise on black, white on red, yellow on red, and white on purple resulted in better performance than other color combinations. The findings of this study provide useful guidelines for designing more efficient and user-friendly icons and interfaces in the future.     

Impact of spatial auditory feedback on the efficiency of iconic human–computer interfaces under conditions of visual impairment

This paper investigates the addition of spatial auditory feedback as a tool to assist people with visual impairments in the use of computers, specifically in tasks involving iconic visual search. In this augmented interface, unique sounds were mapped to visual icons on the screen. As the screen cursor traversed the screen, the user heard sounds of nearby icons, spatially, according to the relative position of each icon with respect to the screen cursor. A software prototype of the design was developed to evaluate the performance of users in the search of icons within the proposed interface. Experiments were conducted with simulated visual impairments on volunteer participants to evaluate if the addition of spatial auditory feedback makes the interface more accessible to users with impaired vision. Results demonstrated that spatialization of icon sounds provides additional remote navigational information to users, enabling new strategies for task completion. Directions for future research are discussed and prioritized.     

Semantic congruency of auditory warnings

Abstract

The aim of this study was to explore operator experience and performance for semantically congruent and incongruent auditory icons and abstract alarm sounds. It was expected that performance advantages for congruent sounds would be present initially but would reduce over time for both alarm types. Twenty-four participants (12M/12F) were placed into auditory icon or abstract alarm groupings. For each group both congruent and incongruent alarms were used to represent different driving task scenarios. Once sounded, participants were required to respond to each alarm by selecting a corresponding driving scenario. User performance for all sound types improved over time, however even with experience a decrement in speed of response remained for the incongruent iconic sounds and in accuracy of performance for the abstract warning sounds when compared to the congruent auditory icons. Semantic congruency was found to be of more importance for auditory icons than for abstract sounds.

Practitioner Summary: Alarms are used in many operating systems as emergency, alerting, or continuous monitoring signals for instance. This study found that the type and representativeness of an auditory warning will influence operator performance over time. Semantically congruent iconic sounds produced performance advantages over both incongruent iconic sounds and abstract warnings.     

How age affects the speed of perception of computer icons

The use of computers is constantly increasing. At the same time the population of the industrialised world is aging. In this study we investigated the speed with which users of different ages can find a specific computer icon from a group of others. Our results show that search performance slows with age when calculated across all three levels of inter-icon spacing (χ² (4) = 14.904, p < .05) and icon size (χ² (4) = 15.674, p < .05) used in this study. However, individual variability in search performance was very high within all age groups. Our study suggests that icons used in graphical user interfaces should be at least about 1 degree in size (about 0.7 cm at a viewing distance of 40 cm) for the majority of users to be able to perform their computerised tasks with relative ease. Also, the inter-icon spacing should be moderate, preferably about the same as the icon size. Ideally user interfaces should be adaptable to individual user needs and preferences.     

A Taxonomy of Representation Strategies in Iconic Communication

Predicting whether the intended audience will be able to recognize the meaning of an icon or pictograph is not an easy task. Many icon recognition studies have been conducted in the past. However, their findings cannot be generalized to other icons that were not included in the study, which, we argue, is their main limitation. In this paper, we propose a comprehensive taxonomy of icons that is intended to enable the generalization of the findings of recognition studies. To accomplish this, we analyzed a sample of more than eight hundred icons according to three axes: lexical category, semantic category, and representation strategy. Three basic representation strategies were identified: visual similarity; semantic association; and arbitrary convention. These representation strategies are in agreement with the strategies identified in previous taxonomies. However, a greater number of subcategories of these strategies were identified. Our results also indicate that the lexical and semantic attributes of a concept influence the choice of representation strategy.     

Using icons to convey medication schedule information

We examined older adult comprehension and memory for medication schedules conveyed by different types of visual icons as well as text. Three icons were compared: a timeline, a pair of 12 h clocks (one for AM and one for PM hours) and a 24 h clock. In Experiment 1, older and younger participants paraphrased and then recalled schedules that were conveyed by the three icons or by text. Text and timeline schedules were paraphrased more accurately than either clock icon. Paraphrase errors suggested that subjects had trouble integrating schedule information across the two 12 h clocks. Analysis of paraphrase times showed that the text schedule was paraphrased most quickly, followed by the timeline, the 24 h clock and the 12 h clock. No differences were found for the accuracy of recalling the icon and text schedules. Experiment 2 examined free and cued recall after limited study time by older adults. Because text superiority in Experiment 1 may have reflected greater familiarity with text than with icons, recall was examined across four trials in Experiment 2. Text was recalled most accurately, followed by the timeline, the 24 h clock and the 12 h clock. Study-test trials did not influence recall. Text schedules may have been generally more effective than the icons because of their greater familiarity. The timeline may have been the most effective icon because it was more compact and familiar than the clock icons used in the study. The findings suggest that including such icons in medication instructions requires training.     

Matching performance of vehicle icons in graphical and textual formats

The current research classified 82 vehicle icons into seven categories (image-related, concept-related, semi-abstract, arbitrary, abbreviation, word, and combined) for their matching accuracy, matching sequence, and matching time. These data can be compared and used as a framework for future icon development. Forty participants, all with a university degree, took part in this experiment. Half of the participants had intensive driving experience, while the other half never driven a car. The results indicated that on average, word icons had a significantly greater matching accuracy than the other icon formats; ranging from 4.7 to 20.8% difference. Regarding the matching sequence, participants matched image-related icons before other icon formats. Arbitrary and combined icons took significantly longer to match than other icon formats by 1.4–6.2 s. Based on the high matching accuracy (86.3%) and high ratings on subjective design features, word format can be used for functions describable using simple English for users with English reading ability. Confusion matrices showed that 63.2% of the misunderstandings were caused by similarity in format or function.     

基于因子分析法的图标设计评价方法

现有的图标设计评价方法大多基于审美评价或可用性而展开,缺少对图标间的关 系及设计因素的研究。为此,提出了一种基于因子分析法的界面图标设计评价方法,通过选取 图标设计中的6 个设计元变量,对其展开意象评价统计并建立均值矩阵,使用因子分析法对其 进行计算分析及评价。最终将图标设计的影响因子提取为图标外形和图标内部两个主要因子, 从而方便研究人员评价系列图标设计中各图标之间的关系和各构成要素的相互作用。该方法具 有可靠性,评价结果与现有系列图标设计的视觉表现及其市场表现相符,并得出图标外形因子 对于图标设计效果的影响作用大于图标内部因子这一主要观点。     

Development and validation of icons varying in their abstractness

Icons are used widely in human-computer interfaces. The level of abstractness-concreteness of an icon and its effect upon performance is of widespread interest. The authors have devised a quantitative measure for abstractness based on the complexity of the icon. They test their metric against subjective judgments of abstractness as identified by two different groups of subjects. After ranking two sets of ‘abstract’ and ‘concrete’ icons, the authors examined how well the icons were matched to the Pascal constructs that they represented. Further experiments were conducted using different groups of subjects to check whether correct matching of the icons with constructs was influenced by context. In summary the authors found that their metric was a good match for subjective measures of abstractness-concreteness. They also found that there is a better identification of concrete icons than abstract icons. Finally, it was shown that context does affect the correct identification of icons.     

Study on the correlation and inhibition of visual marking and industrial icons

As a carrier of complex information, industrial icons play an important role in intelligent manufacturing systems, and the superiority of its visual presentation affects the visual search efficiency of operators. This paper applied the theoretical knowledge of visual marking to the study of industrial icons, and the relationship between the inhibitory mechanism of visual marking and the information characteristic, semantic categories and spatial layout of industrial icons is explored from the inhibition of location, feature, category and spatial structure in visual marking. The experimental results show that: in the aspect of feature inhibition, the 2.5D style inhibits the flat style and the complex icon inhibits the simple icon. In the aspect of category inhibition, the inhibitory degree of four different categories are warning, monitoring, equipment and production in order from high to low. In the aspect of spatial layout inhibition, the regular layout inhibits the chaotic layout and the orderly arrangement inhibits the disorderly arrangement. It is concluded that there is a certain correlation among the industrial icons’ inhibition of feature, category and spatial layout, and the priority of category is higher than that of feature, the spatial layout inhibition is jointly influenced by the inhibition of feature and category, among which category inhibition takes the lead role.     

Iterative process of design and evaluation of icons for interactive TV menu

This paper shows an iterative process of design and evaluation of icons for future interactive TV services. In doing the RNRT (French National Network of Research in Telecommunications) iTV project, we tried to generate icons easy to identify, associate and memorise for 32 categories and services of our iTV system.

Through an iterative process, the Multiple Index Approach was applied until an acceptable icon set was achieved. In addition to existing evaluation criteria such as the intuitiveness, associativeness, preference and suitability with subjective certainty of users, we emphasised the importance of the learnability measured by recall tests.

As a conclusion, we propose a methodology of icon design and evaluation for information appliances that integrate unfamiliar features with common users.     

Learning and retention of associations between auditory icons and denotative referents: implications for the design of auditory warnings

OBJECTIVE: This study examined the way in which the type and preexisting strength of association between an auditory icon and a warning event affects the ease with which the icon/event pairing can be learned and retained. BACKGROUND: To be effective, an auditory warning must be audible, identifiable, interpretable, and heeded. Warnings consisting of familiar environmental sounds, or auditory icons, have potential to facilitate identification and interpretation. The ease with which pairings between auditory icons and warning events can be learned and retained is likely to depend on the type and strength of the preexisting icon/event association. METHOD: Sixty-three participants each learned eight auditory-icon/denotative-referent pairings and attempted to recall them 4 weeks later. Three icon/denotative-referent association types (direct, related, and unrelated) were employed. Participants rated the strength of the association for each pairing on a 7-point scale. RESULTS: The number of errors made while learning pairings was greater for unrelated than for either related or direct associations, whereas the number of errors made while attempting to recall pairings 4 weeks later was greater for unrelated than for related associations and for related than for direct associations. Irrespective of association type, both learning and retention performance remained at very high levels, provided the strength of the association was rated greater than 5. CONCLUSION: This suggests that strong preexisting associations are used to facilitate learning and retention of icon/denotative-referent pairings. APPLICATION: The practical implication of this study is that auditory icons having either direct or strong, indirect associations with warning events should be preferred.     

Iterative process of design and evaluation of icons for interactive TV menu

This paper shows an iterative process of design and evaluation of icons for future interactive TV services. In doing the RNRT (French National Network of Research in Telecommunications) iTV project, we tried to generate icons easy to identify, associate and memorise for 32 categories and services of our iTV system.

Through an iterative process, the Multiple Index Approach was applied until an acceptable icon set was achieved. In addition to existing evaluation criteria such as the intuitiveness, associativeness, preference and suitability with subjective certainty of users, we emphasised the importance of the learnability measured by recall tests.

As a conclusion, we propose a methodology of icon design and evaluation for information appliances that integrate unfamiliar features with common users.     

Interacting with VDL-based structured icons on crisis map for emergency coordination: Interactive design and experimental demonstration

Accurate and timely information sharing among multiple participants is one of the crucial factors for collaboration in crisis management. The icon-based map was frequently applied as an effective means for crisis interaction and collaboration. However, former studies only regarded the icons as supplementary visualization components during the whole crisis collaboration process. In this paper, the concept of a structured-icon-relied interactive method was proposed to directly integrate all kinds of crisis and coordination information through icons on the crisis map. Moreover, structured icons are supposed to explicitly show the correlation among crisis information and to improve the interactive experience of information search. The effectiveness of this interactive method was verified by a controlled experiment with three sub-tasks for simulated crisis rescue. The results of the experiment showed that the design of the crisis map based on structured icons had a positive impact on collaborative decision-making and crisis interaction. The dynamic characteristics of the structured icon could represent the temporal and spatial attributes of crisis information and enhance users’ crisis coordination ability. The study aims to enrich the theory of crisis information visualization and interaction based on structured iconic representation. In practice, this research could optimize the design of a crisis collaboration system based on the icon as well as the interaction between people, crisis information, and collaborative system, which in turn provides accurate and timely rescue decisions.