VA + Embeddings STAR: A State-of-the-Art Report on the Use of Embeddings in Visual Analytics

Over the past years, an increasing number of publications in information visualization, especially within the field of visual analytics, have mentioned the term “embedding” when describing the computational approach. Within this context, embeddings are usually (relatively) low-dimensional, distributed representations of various data types (such as texts or graphs), and since they have proven to be extremely useful for a variety of data analysis tasks across various disciplines and fields, they have become widely used. Existing visualization approaches aim to either support exploration and interpretation of the embedding space through visual representation and interaction, or aim to use embeddings as part of the computational pipeline for addressing downstream analytical tasks. To the best of our knowledge, this is the first survey that takes a detailed look at embedding methods through the lens of visual analytics, and the purpose of our survey article is to provide a systematic overview of the state of the art within the emerging field of embedding visualization. We design a categorization scheme for our approach, analyze the current research frontier based on peer-reviewed publications, and discuss existing trends, challenges, and potential research directions for using embeddings in the context of visual analytics. Furthermore, we provide an interactive survey browser for the collected and categorized survey data, which currently includes 122 entries that appeared between 2007 and 2023.     

Comparing alternatives for analyzing requirements trade-offs – In the absence of numerical data

ContextChoosing a design solution most often involves dealing with trade-offs and conflicts among requirements and design objectives. Making such trade-offs during early stages of requirements and design is challenging because costs and benefits of alternatives are often hard to quantify.ObjectiveThe objective of this work is to develop a decision analysis method that assists in making trade-offs in the absence of quantitative data.MethodIn this method, stakeholders qualitatively compare consequences of alternatives on decision criteria. We propose an algorithm that generates all possible consequences of alternatives on requirements, according to the rough qualitative comparisons that stakeholders made. The possible consequences generated by the algorithm are then analyzed by the Even Swaps Multi-Criteria Decision Analysis method to determine the best solution. The Even Swaps method is a technique developed in management science to assist in multi-criteria decision making when explicit value trade-offs are not available.Results and conclusionsOur algorithm teases out the need to accurately measure or estimate costs and benefits of alternative design solutions. The algorithm automates the Even Swap process, and reuses stakeholders’ value trade-offs throughout the Even Swaps process. We applied the prototype tool in several case studies to evaluate the utility of the method. The results of case studies provide evidence that our decision aid method selects the optimum solution correctly compared to results of other similar quantitative methods, while our method does not rely on detailed numerical assessment of alternatives and importance weights of criteria.     

Data Comets: Designing a Visualization Tool for Analyzing Autonomous Aerial Vehicle Logs with Grounded Evaluation

Autonomous unmanned aerial vehicles are complex systems of hardware, software, and human input. Understanding this complexity is key to their development and operation. Information visualizations already exist for exploring flight logs but comprehensive analyses currently require several disparate and custom tools. This design study helps address the pain points faced by autonomous unmanned aerial vehicle developers and operators. We contribute: a spiral development process model for grounded evaluation visualization development focused on progressively broadening target user involvement and refining user goals; a demonstration of the model as part of developing a deployed and adopted visualization system; a data and task abstraction for developers and operators performing post-flight analysis of autonomous unmanned aerial vehicle logs; the design and implementation of Data Comets , an open-source and web-based interactive visualization tool for post-flight log analysis incorporating temporal, geospatial, and multivariate data; and the results of a summative evaluation of the visualization system and our abstractions based on in-the-wild usage. A free copy of this paper and source code are available at osf.io/h4p7g     

Process and Pitfalls of Online Teaching and Learning with Design Study “Lite” Methodology: A Retrospective Analysis

Design studies are an integral method of visualization research with hundreds of instances in the literature. Although taught as a theory, the practical implementation of design studies is often excluded from visualization pedagogy due to the lengthy time commitments associated with such studies. Recent research has addressed this challenge and developed an expedited design study framework, the Design Study “Lite” Methodology (DSLM), which can implement design studies with novice students within just 14 weeks. The framework was developed and evaluated based on five semesters of in-person data visualization courses with 30 students or less and was implemented in conjunction with Service-Learning (S-L). With the growth and popularity of the data visualization field—and the teaching environment created by the COVID-19 pandemic—more academic institutions are offering visualization courses online. Therefore, in this paper, we strengthen and validate the epistemological foundations of the DSLM framework by testing its (1) adaptability to online learning environments and conditions and (2) scalability to larger classes with up to 57 students. We present two online implementations of the DSLM framework, with and without Service-Learning (S-L), to test the adaptability and scalability of the framework. We further demonstrate that the framework can be applied effectively without the S-L component. We reflect on our experience with the online DSLM implementations and contribute a detailed retrospective analysis using thematic analysis and grounded theory methods to draw valuable recommendations and guidelines for future applications of the framework. This work verifies that DSLM can be used successfully in online classes to teach design study methodology. Finally, we contribute novel additions to the DSLM framework to further enhance it for teaching and learning design studies in the classroom. The preprint and supplementary materials for this paper can be found at https://osf.io/6bjx5/.     

QUESTO: Interactive Construction of Objective Functions for Classification Tasks

Building effective classifiers requires providing the modeling algorithms with information about the training data and modeling goals in order to create a model that makes proper tradeoffs. Machine learning algorithms allow for flexible specification of such meta-information through the design of the objective functions that they solve. However, such objective functions are hard for users to specify as they are a specific mathematical formulation of their intents. In this paper, we present an approach that allows users to generate objective functions for classification problems through an interactive visual interface. Our approach adopts a semantic interaction design in that user interactions over data elements in the visualization are translated into objective function terms. The generated objective functions are solved by a machine learning solver that provides candidate models, which can be inspected by the user, and used to suggest refinements to the specifications. We demonstrate a visual analytics system QUESTO for users to manipulate objective functions to define domain-specific constraints. Through a user study we show that QUESTO helps users create various objective functions that satisfy their goals.     

The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

Machine learning (ML) models are nowadays used in complex applications in various domains, such as medicine, bioinformatics, and other sciences. Due to their black box nature, however, it may sometimes be hard to understand and trust the results they provide. This has increased the demand for reliable visualization tools related to enhancing trust in ML models, which has become a prominent topic of research in the visualization community over the past decades. To provide an overview and present the frontiers of current research on the topic, we present a State-of-the-Art Report (STAR) on enhancing trust in ML models with the use of interactive visualization. We define and describe the background of the topic, introduce a categorization for visualization techniques that aim to accomplish this goal, and discuss insights and opportunities for future research directions. Among our contributions is a categorization of trust against different facets of interactive ML, expanded and improved from previous research. Our results are investigated from different analytical perspectives: (a) providing a statistical overview, (b) summarizing key findings, (c) performing topic analyses, and (d) exploring the data sets used in the individual papers, all with the support of an interactive web-based survey browser. We intend this survey to be beneficial for visualization researchers whose interests involve making ML models more trustworthy, as well as researchers and practitioners from other disciplines in their search for effective visualization techniques suitable for solving their tasks with confidence and conveying meaning to their data.     

WYTIWYR: A User Intent-Aware Framework with Multi-modal Inputs for Visualization Retrieval

Retrieving charts from a large corpus is a fundamental task that can benefit numerous applications such as visualization recommendations. The retrieved results are expected to conform to both explicit visual attributes (e.g., chart type, colormap) and implicit user intents (e.g., design style, context information) that vary upon application scenarios. However, existing example-based chart retrieval methods are built upon non-decoupled and low-level visual features that are hard to interpret, while definition-based ones are constrained to pre-defined attributes that are hard to extend. In this work, we propose a new framework, namely WYTIWYR (What-You-Think-Is-What-You-Retrieve), that integrates user intents into the chart retrieval process. The framework consists of two stages: first, the Annotation stage disentangles the visual attributes within the query chart; and second, the Retrieval stage embeds the user's intent with customized text prompt as well as bitmap query chart, to recall targeted retrieval result. We develop aprototype WYTIWYR system leveraging a contrastive language-image pre-training (CLIP) model to achieve zero-shot classification as well as multi-modal input encoding, and test the prototype on a large corpus with charts crawled from the Internet. Quantitative experiments, case studies, and qualitative interviews are conducted. The results demonstrate the usability and effectiveness of our proposed framework.     

Phase Space Projection of Dynamical Systems

Dynamical systems are commonly used to describe the state of time-dependent systems. In many engineering and control problems, the state space is high-dimensional making it difficult to analyze and visualize the behavior of the system for varying input conditions. We present a novel dimensionality reduction technique that is tailored to high-dimensional dynamical systems. In contrast to standard general purpose dimensionality reduction algorithms, we use energy minimization to preserve properties of the flow in the high-dimensional space. Once the projection operator is optimized, further high-dimensional trajectories are projected easily. Our 3D projection maintains a number of useful flow properties, such as critical points and flow maps, and is optimized to match geometric characteristics of the high-dimensional input, as well as optional user constraints. We apply our method to trajectories traced in the phase spaces of second-order dynamical systems, including finite-sized objects in fluids, the circular restricted three-body problem and a damped double pendulum. We compare the projections with standard visualization techniques, such as PCA, t-SNE and UMAP, and visualize the dynamical systems with multiple coordinated views interactively, featuring a spatial embedding, projection to subspaces, our dimensionality reduction and a seed point exploration tool.     

Been There,Seen That: Visualization of Movement and 3D Eye Tracking Data from Real-World Environments

The distribution of visual attention can be evaluated using eye tracking, providing valuable insights into usability issues and interaction patterns. However, when used in real, augmented, and collaborative environments, new challenges arise that go beyond desktop scenarios and purely virtual environments. Toward addressing these challenges, we present a visualization technique that provides complementary views on the movement and eye tracking data recorded from multiple people in real-world environments. Our method is based on a space-time cube visualization and a linked 3D replay of recorded data. We showcase our approach with an experiment that examines how people investigate an artwork collection. The visualization provides insights into how people moved and inspected individual pictures in their spatial context over time. In contrast to existing methods, this analysis is possible for multiple participants without extensive annotation of areas of interest. Our technique was evaluated with a think-aloud experiment to investigate analysis strategies and an interview with domain experts to examine the applicability in other research fields.     

Don't Peek at My Chart: Privacy-preserving Visualization for Mobile Devices

Data visualizations have been widely used on mobile devices like smartphones for various tasks (e.g., visualizing personal health and financial data), making it convenient for people to view such data anytime and anywhere. However, others nearby can also easily peek at the visualizations, resulting in personal data disclosure. In this paper, we propose a perception-driven approach to transform mobile data visualizations into privacy-preserving ones. Specifically, based on human visual perception, we develop a masking scheme to adjust the spatial frequency and luminance contrast of colored visualizations. The resulting visualization retains its original information in close proximity but reduces visibility when viewed from a certain distance or farther away. We conducted two user studies to inform the design of our approach (N=16) and systematically evaluate its performance (N=18), respectively. The results demonstrate the effectiveness of our approach in terms of privacy preservation for mobile data visualizations.     

Structure and Empathy in Visual Data Storytelling: Evaluating their Influence on Attitude

In the visualization community, it is often assumed that visual data storytelling increases memorability and engagement, making it more effective at communicating information. However, many assumptions about the efficacy of storytelling in visualization lack empirical evaluation. Contributing to an emerging body of work, we study whether selected techniques commonly used in visual data storytelling influence people's attitudes towards immigration. We compare (a) personal visual narratives designed to generate empathy; (b) structured visual narratives of aggregates of people; and (c) an exploratory visualization without narrative acting as a control condition. We conducted two crowdsourced between-subject studies comparing the three conditions, each with 300 participants. To assess the differences in attitudes between conditions, we adopted established scales from the social sciences used in the European Social Survey (ESS). Although we found some differences between conditions, the effects on people's attitudes are smaller than we expected. Our findings suggest that we need to be more careful when it comes to our expectations about the effects visual data storytelling can have on attitudes. Additional material: https://flowstory.github.io/attitudes/.     

SEEVis: A Smart Emergency Evacuation Plan Visualization System with Data-Driven Shot Designs

Despite the significance of tracking human mobility dynamics in a large-scale earthquake evacuation for an effective first response and disaster relief, the general understanding of evacuation behaviors remains limited. Numerous individual movement trajectories, disaster damages of civil engineering, associated heterogeneous data attributes, as well as complex urban environment all obscure disaster evacuation analysis. Although visualization methods have demonstrated promising performance in emergency evacuation analysis, they cannot effectively identify and deliver the major features like speed or density, as well as the resulting evacuation events like congestion or turn-back. In this study, we propose a shot design approach to generate customized and narrative animations to track different evacuation features with different exploration purposes of users. Particularly, an intuitive scene feature graph that identifies the most dominating evacuation events is first constructed based on user-specific regions or their tracking purposes on a certain feature. An optimal camera route, i.e., a storyboard is then calculated based on the previous user-specific regions or features. For different evacuation events along this route, we employ the corresponding shot design to reveal the underlying feature evolution and its correlation with the environment. Several case studies confirm the efficacy of our system. The feedback from experts and users with different backgrounds suggests that our approach indeed helps them better embrace a comprehensive understanding of the earthquake evacuation.     

Visual Gaze Labeling for Augmented Reality Studies

Augmented Reality (AR) provides new ways for situated visualization and human-computer interaction in physical environments. Current evaluation procedures for AR applications rely primarily on questionnaires and interviews, providing qualitative means to assess usability and task solution strategies. Eye tracking extends these existing evaluation methodologies by providing indicators for visual attention to virtual and real elements in the environment. However, the analysis of viewing behavior, especially the comparison of multiple participants, is difficult to achieve in AR. Specifically, the definition of areas of interest (AOIs), which is often a prerequisite for such analysis, is cumbersome and tedious with existing approaches. To address this issue, we present a new visualization approach to define AOIs, label fixations, and investigate the resulting annotated scanpaths. Our approach utilizes automatic annotation of gaze on virtual objects and an image-based approach that also considers spatial context for the manual annotation of objects in the real world. Our results show, that with our approach, eye tracking data from AR scenes can be annotated and analyzed flexibly with respect to data aspects and annotation strategies.     

Feature Driven Combination of Animated Vector Field Visualizations

Animated visualizations are one of the methods for finding and understanding complex structures of time-dependent vector fields. Many visualization designs can be used to this end, such as streamlines, vector glyphs, and image-based techniques. While all such designs can depict any vector field, their effectiveness in highlighting particular field aspects has not been fully explored. To fill this gap, we compare three animated vector field visualization techniques, OLIC, IBFV, and particles, for a critical point detection-and-classification task through a user study. Our results show that the effectiveness of the studied techniques depends on the nature of the critical points. We use these results to design a new flow visualization technique that combines all studied techniques in a single view by locally using the most effective technique for the patterns present in the flow data at that location. A second user study shows that our technique is more efficient and less error prone than the three other techniques used individually for the critical point detection task.     

VisuaLint: Sketchy In Situ Annotations of Chart Construction Errors

Chart construction errors, such as truncated axes or inexpressive visual encodings, can hinder reading a visualization, or worse, imply misleading facts about the underlying data. These errors can be caught by critical readings of visualizations, but readers must have a high level of data and design literacy and must be paying close attention. To address this issue, we introduce VisuaLint: a technique for surfacing chart construction errors in situ. Inspired by the ubiquitous red wavy underline that indicates spelling mistakes, visualization elements that contain errors (e.g., axes and legends) are sketchily rendered and accompanied by a concise annotation. VisuaLint is unobtrusive — it does not interfere with reading a visualization — and its direct display establishes a close mapping between erroneous elements and the expression of error. We demonstrate five examples of VisualLint and present the results of a crowdsourced evaluation (N = 62) of its efficacy. These results contribute an empirical baseline proficiency for recognizing chart construction errors, and indicate near-universal difficulty in error identification. We find that people more reliably identify chart construction errors after being shown examples of VisuaLint, and prefer more verbose explanations for unfamiliar or less obvious flaws.     

Infomages: Embedding Data into Thematic Images

Recent studies have indicated that visually embellished charts such as infographics have the ability to engage viewers and positively affect memorability. Fueled by these findings, researchers have proposed a variety of infographic design tools. However, these tools do not cover the entire design space. In this work, we identify a subset of infographics that we call infomages. Infomages are casual visuals of data in which a data chart is embedded into a thematic image such that the content of the image reflects the subject and the designer's interpretation of the data. Creating an effective infomage, however, can require a fair amount of design expertise and is thus out of reach for most people. In order to also afford non-artists with the means to design convincing infomages, we first study the principled design of existing infomages and identify a set of key chart embedding techniques. Informed by these findings we build a design tool that links web-scale image search with a set of interactive image processing tools to empower novice users with the ability to design a wide variety of infomages. As the embedding process might introduce some amount of visual distortion of the data our tool also aids users to gauge the amount of this distortion, if any. We experimentally demonstrate the usability of our tool and conclude with a discussion of infomages and our design tool.     

Sunspot Plots: Model-based Structure Enhancement for Dense Scatter Plots

Scatter plots are a powerful and well-established technique for visualizing the relationships between two variables as a collection of discrete points. However, especially when dealing with large and dense data, scatter plots often exhibit problems such as overplotting, making the data interpretation arduous. Density plots are able to overcome these limitations in highly populated regions, but fail to provide accurate information of individual data points. This is particularly problematic in sparse regions where the density estimate may not provide a good representation of the underlying data. In this paper, we present sunspot plots, a visualization technique that communicates dense data as a continuous data distribution, while preserving the discrete nature of data samples in sparsely populated areas. We furthermore demonstrate the advantages of our approach on typical failure cases of scatter plots within synthetic and real-world data sets and validate its effectiveness in a user study.     

Warehouse Vis: A Visual Analytics Approach to Facilitating Warehouse Location Selection for Business Districts

Selecting a proper warehouse location serving to satisfy the demands of the goods from a certain business area is important to a successful retail business. However, the large solution space, uncertain traffic conditions, and varying business preferences impose great challenges on warehouse location selection. Conventional approaches mainly summarize relevant evaluation criteria and compile them into an analysis report to facilitate rapid data absorption but fail to support a comprehensive and joint decision-making process in warehouse location selection. In this paper, we propose a visual analytics approach to facilitating warehouse location selection. We first visually centralize relevant information of warehouses and adapts a widely-used methodology to efficiently rank warehouse candidates. We then design a delivering estimation model based on massive logistics trajectories to resolve the uncertainty issue of traffic conditions of warehouses. Based on these techniques, an interactive framework is proposed to generate and explore the candidate warehouses. We conduct a case study and a within-subject study with baseline systems to assess the efficacy of our system. Experts ‘feedback also suggests that our approach indeed helps them better tackle the problem of finding an ideal warehouse in the field of retail logistics management.     

Augmenting Node-Link Diagrams with Topographic Attribute Maps

We propose a novel visualization technique for graphs that are attributed with scalar data. In many scenarios, these attributes (e.g., birth date in a family network) provide ambient context information for the graph structure, whose consideration is important for different visual graph analysis tasks. Graph attributes are usually conveyed using different visual representations (e.g., color, size, shape) or by reordering the graph structure according to the attribute domain (e.g., timelines). While visual encodings allow graphs to be arranged in a readable layout, assessing contextual information such as the relative similarities of attributes across the graph is often cumbersome. In contrast, attribute-based graph reordering serves the comparison task of attributes, but typically strongly impairs the readability of the structural information given by the graph's topology. In this work, we augment force-directed node-link diagrams with a continuous ambient representation of the attribute context. This way, we provide a consistent overview of the graph's topological structure as well as its attributes, supporting a wide range of graph-related analysis tasks. We resort to an intuitive height field metaphor, illustrated by a topographic map rendering using contour lines and suitable color maps. Contour lines visually connect nodes of similar attribute values, and depict their relative arrangement within the global context. Moreover, our contextual representation supports visualizing attribute value ranges associated with graph nodes (e.g., lifespans in a family network) as trajectories routed through this height field. We discuss how user interaction with both the structural and the contextual information fosters exploratory graph analysis tasks. The effectiveness and versatility of our technique is confirmed in a user study and case studies from various application domains.