Prototype evaluation and redesign: structuring the design space through contextual techniques

This paper addresses the problems involved in the progress through evaluation and redesign from one interface prototype to the next stage of development. An approach is proposed based on situated action techniques for the early identification of user interface issues and their translation into design factors that lead to design improvements. The approach can be used within parallel prototyping or iterative development. Situated action is increasingly popular for the participative identification of user requirements for information systems and is usually implemented at an early stage in systems development. In contrast the most frequently used method for user participation within detailed interface design is within iterative user-based evaluation which is often performed relatively late in the development. The method described involves linking developer–user contextual evaluation sessions, where developers observe user's responses to an interactive prototype, and team evidence analysis sessions, where a group of designers work together to derive design decisions with the evidence collected from the users. The proposed method is tested in an experimental design. The proposed techniques provide a rich source of user evidence that can be brought to bear on the enhancement of prototype user interfaces. The study also showed how team working within a group of developers is important to effective design.     

SUMLOW: early design‐stage sketching of UML diagrams on an E‐whiteboard

Most visual diagramming tools provide point‐and‐click construction of computer‐drawn diagram elements using a conventional desktop computer and mouse. SUMLOW is a unified modelling language (UML) diagramming tool that uses an electronic whiteboard (E‐whiteboard) and sketching‐based user interface to support collaborative software design. SUMLOW allows designers to sketch UML constructs, mixing different UML diagram elements, diagram annotations, and hand‐drawn text. A key novelty of the tool is the preservation of hand‐drawn diagrams and support for manipulation of these sketches using pen‐based actions. Sketched diagrams can be automatically ‘formalized’ into computer‐recognized and ‐drawn UML diagrams and then exported to a third party CASE tool for further extension and use. We describe the motivation for SUMLOW, illustrate the use of the tool to sketch various UML diagram types, describe its key architecture abstractions and implementation approaches, and report on two evaluations of the toolset. We hope that our experiences will be useful for others developing sketching‐based design tools or those looking to leverage pen‐based interfaces in software applications. Copyright © 2007 John Wiley & Sons, Ltd.     

The Impact of Tangible User Interfaces on Designers' Spatial Cognition

ABSTRACT

Most studies on tangible user interfaces for the tabletop design systems are being undertaken from a technology viewpoint. Although there have been studies that focus on the development of new interactive environments employing tangible user interfaces for designers, there is a lack of evaluation with respect to designers' spatial cognition. In this research we study the effects of tangible user interfaces on designers' spatial cognition to provide empirical evidence for the anecdotal views of the effect of tangible user interfaces. To highlight the expected changes in spatial cognition while using tangible user interfaces, we compared designers using a tangible user interface on a tabletop system with 3D blocks to designers using a graphical user interface on a desktop computer with a mouse and keyboard. The ways in which designers use the two different interfaces for 3D design were examined using a protocol analysis method. The result reveals that designers using 3D blocks perceived more spatial relationships among multiple objects and spaces and discovered new visuo-spatial features when revisiting their design configurations. The designers using the tangible interfaces spent more time in relocating objects to different locations to test the moves, and interacted with the external representation through large body movements implying an immersion in the design model. These two physical actions assist in designers' spatial cognition by reducing cognitive load in mental visual reasoning. Further, designers using the tangible interfaces spent more time in restructuring the design problem by introducing new functional issues as design requirements and produced more discontinuities to the design processes, which provides opportunity for reflection and modification of the design. Therefore this research shows that tangible user interfaces changes designers' spatial cognition, and the changes of the spatial cognition are associated with creative design processes.     

Embarking on Spoken-Language NL Interface Design

Natural language (NL) user interfaces are growing in popularity. Unfortunately, the complexity of NL interaction makes these interfaces difficult to design. For NL interfaces to become successful, universal tools are needed to help support the NL design process. What work practice should these tools explicitly support? Interviews with NL designers and our own experiments have identified a specific work practice that designers should consider as they begin to incorporate NL into user interface designs. The work practice study highlights the value of using Wizard of Oz prototyping in NL design. We describe a tool that we have built, called SUEDE, to explicitly support the first stage of NL design for spoken-language user interfaces. Our tools and tools like it will help make NL in human-computer interaction (HCI) more commonplace.     

Wizard of Oz support throughout an iterative design process

The Wizard of Oz prototyping approach, widely used in human-computer interaction research, is particularly useful in exploring user interfaces for pervasive, ubiquitous, or mixed-reality systems that combine complex sensing and intelligent control logic. The vast design space for such nontraditional interfaces provides many possibilities for user interaction through one or more modalities and often requires challenging hardware and software implementations. The WOz method helps designers avoid getting locked into a particular design or working under an incorrect set of assumptions about user preferences, because it lets them explore and evaluate designs before investing the considerable development time needed to build a complete prototype.     

Developing Multimodal Adaptation Algorithm for Mobility Impaired Users by Evaluating Their Hand Strength

Recent research on interactive electronic systems like computer, digital TV, smartphones can improve the quality of life of many disabled and elderly people by helping them to engage more fully to the world. Previously, a simulator was developed that reflects the effect of impairment on interaction with electronic devices and thus helps designers in developing accessible systems. In this article, the scope of the simulator has been extended to multiple pointing devices. The way that hand strength affects pointing performance of people with and without mobility impairment in graphical user interfaces was investigated for four different input modalities, and a set of linear equations to predict pointing time and average number of submovements for different devices was developed. These models were used to develop an adaptation algorithm to facilitate pointing in electronic interfaces by users with motor impairment using different pointing devices. The algorithm attracts a pointer when it is near a target and thus helps to reduce random movement during homing and clicking. The algorithm was optimized using the simulator and then tested on a real-life application with multiple distractors involving three different pointing devices. The algorithm significantly reduces pointing time for different input modalities.     

Sketching interfaces: toward more human interface design

Researchers at University of California, Berkeley and Carnegie Mellon University have designed, implemented, and evaluated SILK (Sketching Interfaces Like Krazy), an informal sketching tool that combines many of the benefits of paper-based sketching with the merits of current electronic tools. With SILK, designers can quickly sketch an interface using an electronic pad and stylus, and SILK recognizes widgets and other interface elements as the designer draws them. Unlike paper-based sketching, however, designers can exercise these elements in their sketchy state. For example, a sketched scroll-bar is likely to contain an elevator or thumbnail, the small rectangle a user drags with a mouse. In a paper sketch, the elevator would just sit there, but in a SILK sketch, designers can drag it up and down, which lets them test component or widget behavior. SILK also supports the creation of storyboards-the arrangement of sketches to show how design elements behave, such as how a dialog box appears when the user activates a button. Storyboards are important because they give designers a way to show colleagues, customers, or end users early on how an interface will behave     

Creating Interaction Techniques by Demonstration

When creating highly interactive, direct-manipulation interfaces, one of the most difficult design and implementation tasks is handling the mouse and other input devices. Peridot, a new user interface management system, addresses this problem by allowing the designer of the user interface to demonstrate how the input device should be handled by giving an example of the interface in action. The designer uses sample values for parameters, and the system automatically infers the general operation and creates the code. After an interaction is specified, it can be executed rapid prototyping, since it is very easy to design, implement, and modify mouse-based interfaces. Perudit also supports such additional input devices as touch tablets, as well as multiple input devices operating in parallel (for example, one in each hand) in a natural, easy-to-specify manner. All interaction techniques are implemented using active values, which are like variables except that the objects that depend on active values are updated immediately whenever they change. Active values are a straightforward and efficient mechanism for implementing dynamic interactions.     

A Dynamic Gesture Language and Graphical Feedback for Interaction in a 3D User Interface

In user interfaces of modern systems, users get the impression of directly interacting with application objects. In 3D based user interfaces, novel input devices, like hand and force input devices, are being introduced. They aim at providing natural ways of interaction. The use of a hand input device allows the recognition of static poses and dynamic gestures performed by a user's hand. This paper describes the use of a hand input device for interacting with a 3D graphical application. A dynamic gesture language, which allows users to teach some hand gestures, is presented. Furthermore, a user interface integrating the recognition of these gestures and providing feedback for them, is introduced. Particular attention has been spent on implementing a tool for easy specification of dynamic gestures, and on strategies for providing graphical feedback to users' interactions. To demonstrate that the introduced 3D user interface features, and the way the system presents graphical feedback, are not restricted to a hand input device, a force input device has also been integrated into the user interface.     

Investigating three-dimensional sketching for early conceptual design—Results from expert discussions and user studies

As immersive 3D user interfaces reach broader acceptance, their use as sketching media is attracting both commercial and academic interests. So far, little is known about user requirements and cognitive aspects of immersive 3D sketching. Also the latter's integration into the workflow of virtual product development is far from being solved.We present results from two focus group expert discussions, a comparative user study on immersive 3D sketching conducted among professional furniture designers and a qualitative content analysis of user statements. The results of the focus group discussions show a strong interest in using the three-dimensional (3D) space as a medium for conceptual design. Users expect it to provide new means for the sketching process, namely spatiality, one-to-one proportions, associations, and formability. Eight groups of functions required for 3D sketching were outlined during the discussions.The comparative study was intended to find and investigate advantages of immersive three-dimensional space and its additional degrees-of-freedom for creative/reflective externalization processes. We compared a 3D and a 2D baseline condition in the same technical environment, a VR-Cave system. In both conditions, no haptic feedback was provided and the 2D condition was not intended to simulate traditional 2D sketching (on paper). The results from our user study show that both the sketching process and the resulting sketches differ in the 2D and 3D condition, namely in terms of the perceived fluency of sketch creation, in terms of the perceived appropriateness for the task, and in terms of the perceived stimulation by the medium, the movement speed, the sketch sizes, the degree of detail, the functional aspects, and the usage time. In order to validate the results of the focus group discussions, we produced a questionnaire to check for the subjectively perceived advantages and disadvantages in both the 2D and 3D conditions. A qualitative content analysis of the user statements revealed that the biggest advantage of 3D sketching lies in the sketching process itself. In particular, the participants emphasized the system's ability to foster inspiration and to improve the recognition of spatiality and spatial thinking.We argue that both 2D and 3D sketching are relevant for early conceptual design. As we progress towards 3D sketching, new tangible interactive tools are needed, which account for the user's perceptual and cognitive abilities.     

Personalized teleoperation via intention recognition

ABSTRACT

One of the challenges of teleoperation is the recognition of a user’s intended commands, particularly in the manning of highly dynamic systems such as drones. In this paper, we present a solution to this problem by developing a generalized scheme relying on a Convolutional Neural Network (CNN) that is trained to recognize a user’s intended commands, directed through a haptic device. Our proposed method allows the interface to be personalized for each user, by pre-training the CNN differently according to the input data that is specific to the intended end user. Experiments were conducted using two haptic devices and classification results demonstrate that the proposed system outperforms geometric-based approaches by nearly 12%. Furthermore, our system also lends itself to other human–machine interfaces where intention recognition is required.     

Adapting paper prototyping for designing user interfaces for multiple display environments

A multiple display environment (MDE) networks personal and shared devices to form a virtual workspace, and designers are just beginning to grapple with the challenges of developing interfaces tailored for these environments. To develop effective interfaces for MDEs, designers must employ methods that allow them to rapidly generate and test alternative designs early in the design process. Paper prototyping offers one promising method, but needs to be adapted to effectively simulate the use of multiple displays and allow testing with groups of users. In this paper, we share experiences from two projects in which paper prototyping was utilized to explore interfaces for MDEs. We identify problems encountered when applying the traditional method, describe how these problems were overcome, and distill our experiences into recommendations that others can draw upon. By following our recommendations, designers need only make minor modifications to the existing method to better realize benefits of paper prototyping for MDEs.     

Tele-Graffiti: A Camera-Projector Based Remote Sketching System with Hand-Based User Interface and Automatic Session Summarization

One way to build a remote sketching system is to use a video camera to image what each user draws at their site, transmit the video to the other sites, and display it there using an LCD projector. Such camera-projector based remote sketching systems date back to Paul Wellner's (largely unimplemented) Xerox Double DigitalDesk. To make such a system usable, however, the users have to be able to move the paper on which they are drawing, they have to be able to interact with the system using a convenient interface, and sketching sessions must be stored in a compact format so that they can be replayed later. We have recently developed Tele-Graffiti, a remote sketching system with the following three features: (1) real-time paper tracking to allow the users to move their paper during system operation, (2) a hand based user interface, and (3) automatic session summarization and playback. In this paper, we describe the design, implementation, and performance of Tele-Graffiti.Supplementary material to this paper is available in electronic form at http://dx.doi.org/10.1023/A:1023084706295     

Rapid Prototyping of Adaptable User Interfaces

User interface adaptation has been proposed in recent years as a means to achieve personalized accessibility and usability of user interfaces. Related user interface architectures, as well as a variety of related development method and tools, have also been elaborated. Admittedly, however, despite the recognized validity of the approach, which has been proved in practice in a series of prototype systems, the wider adoption and uptake of user interface adaptation approaches are still limited. One of the identified obstacles is the complexity intrinsic in designing such interfaces and the need of radically revising the current user interface design practice to account for (a) the alternative designs required for adaptation, (b) the parameters involved in driving adaptations (i.e., selecting among alternatives at a given point during interaction), and (c) the logic of adaptation at runtime. This article proposes a twofold tool-based support strategy for user interface adaptation development, based on (a) an adaptation development toolkit and related widget library, which directly embeds lexical level adaptations into common interactive widgets, and (b) embedding such a library in a common integrated development environment, thus allowing designers to define and view alternative adaptations at design time and create adaptable user interfaces through traditional prototyping. The aforementioned approach has been implemented in the domain of adaptable applications for older users, producing tools that are currently in use in the development of a large suite of interactive applications in various domains. The approach presented in this article is claimed to be the first and so far unique supporting rapid prototyping of adaptable user interfaces, thus minimizing the divergence between typical development practices and user interface adaptation development.     

Touchable area: An empirical study on design approach considering perception size and touch input behavior

As the use of mobile touch devices continues to increase, distinctive user experiences can be provided through a direct manipulation. Therefore, the characteristics of touch interfaces should be considered regarding their controllability. This study aims to provide a design approach for touch-based user interfaces. A derivation procedure for the touchable area is proposed as a design guideline based on input behavior. To these ends, two empirical tests were conducted through a smart phone interface. Fifty-five participants were asked to perform a series of input tasks on a screen. As results, touchable area with a desirable hit rate of 90% could be yielded depending on the icon design. To improve the applicability of the touchable area, user error was analyzed based on omission-commission classification. The most suitable design had a hit rate of 95% compared to 90 and 99%. This study contributes practical implications for user interaction design with finger-based controls.Relevance to industryThis research describes a distinctive design approach that guarantees the desired touch accuracy for effective use of mobile touch devices. Therefore, the results will encourage interface designers to take into account the input behavior of fingers from a user-centered perspective.     

Interactive multi-user video retrieval systems

In this paper we present two interactive multi-user systems for video search and browsing. The first is composed by web applications which allows multiuser interaction in a distributed environment; such applications are based on the Rich Internet Application paradigm, designed to obtain the levels of responsiveness and interactivity typical of a desktop application. The second system implements a multi-user collaborative application within a single location, exploiting multi-touch devices. Both systems use the same backend, based on a service oriented architecture (SOA) that provides services for automatic and manual annotation, and an ontology-based video search and browsing engine. Ontology-based browsing let users to inspect the content of video collections; user queries are expanded through ontology reasoning. User-centered field trials of the systems, conducted to assess the user experience and satisfaction, have shown that the approach followed to design these interfaces is extremely appreciated by professional archivists and people working on multimedia.     

Automatically generating personalized user interfaces with Supple

Today's computer–human interfaces are typically designed with the assumption that they are going to be used by an able-bodied person, who is using a typical set of input and output devices, who has typical perceptual and cognitive abilities, and who is sitting in a stable, warm environment. Any deviation from these assumptions may drastically hamper the person's effectiveness—not because of any inherent barrier to interaction, but because of a mismatch between the person's effective abilities and the assumptions underlying the interface design.We argue that automatic personalized interface generation is a feasible and scalable solution to this challenge. We present our Supple system, which can automatically generate interfaces adapted to a person's devices, tasks, preferences, and abilities. In this paper we formally define interface generation as an optimization problem and demonstrate that, despite a large solution space (of up to 10¹⁷ possible interfaces), the problem is computationally feasible. In fact, for a particular class of cost functions, Supple produces exact solutions in under a second for most cases, and in a little over a minute in the worst case encountered, thus enabling run-time generation of user interfaces. We further show how several different design criteria can be expressed in the cost function, enabling different kinds of personalization. We also demonstrate how this approach enables extensive user- and system-initiated run-time adaptations to the interfaces after they have been generated.Supple is not intended to replace human user interface designers—instead, it offers alternative user interfaces for those people whose devices, tasks, preferences, and abilities are not sufficiently addressed by the hand-crafted designs. Indeed, the results of our study show that, compared to manufacturers' defaults, interfaces automatically generated by Supple significantly improve speed, accuracy and satisfaction of people with motor impairments.     

Mediating intimacy in long-distance relationships using kiss messaging

Intimate interactions between remotely located individuals are not well supported by conventional communication tools, mainly due to the lack of physical contact. Also, haptic research has not focused on the use of a kiss as a mode of interaction that maintains intimacy in long distance relationships. In this study, we designed and developed a haptic device called Kissenger (Kiss-Messenger) for this issue. Kissenger is an interactive device that provides a physical interface for transmitting a kiss between two remotely connected people. Each device is paired with another and the amount of force and shape of the kiss by the user is sensed and transmitted to another device that is replicated using actuators. Kissenger is designed to augment already existing remote communication technologies. Challenges in the design and development of the system are addressed through an iterative design process involving constant evaluation by users after each stage. The devices are evaluated through a short- and a long-term user study with remotely located couples. The results point to an initial acceptance of the device with feedback from the users on directions to improve the overall experience. This study discusses potential issues that designers should be aware of when prototyping for remote intimate interactions.     

A language for representing and extracting 3D geometry semantics from paper-based sketches

Every product that exists, ranging from a toothbrush to a car, has first been conceived as a mental concept. Due to its efficacy in rapidly externalizing concepts, paper-based sketching is still extensively used by practising designers to gradually develop the three-dimensional (3D) geometric form of a concept. It is a common practice that form concepts are sketched on paper prior to generating 3D virtual models in commercial Computer-Aided Design (CAD) systems. However, the user-interface of such systems does not support automatic generation of 3D models from sketches. Furthermore, the inherent characteristics of form sketching (e.g. idiosyncrasy) pose a challenge to computer-based understanding of the form concept semantics expressed on paper. To address these issues, this paper is therefore concerned with the development of a visual language that is prescribed and to be used by product designers to annotate paper-based sketches such that the form geometry semantics can be formally represented; parsing the annotated sketch allows for the automatic generation of 3D virtual models in CAD. Inspired by re-usable 3D CAD modelling functions and the related environmental constraints and requirements, a prescribed sketching language, PSL, has been developed to annotate paper-based form sketches. The framework architecture which parses the annotated sketch and subsequently extracts the form concept semantics is described. Based on this framework, a prototype computer tool has been implemented and evaluated. Evaluation results provide a degree of evidence, first on the suitability of PSL in representing the semantics of a range of forms, and secondly on the designers׳ acceptance of taking up this annotated sketching approach in practice.     

Adapting to changing user requirements

Systems which have the capability of adapting to changing user requirements must be founded on accurate and perceptive models of the organisation in which they have to function. Design methods based on active user participation and the use of experimental and prototyping methods help to ensure accurate models. But because systems are expected to survive even when circumstances change, the designers have to have a view of what the future world will look like. A technique which helps to provide such a view is called “future analysis”. However, even designs based on the best prediction technique cannot guarantee a fit between the currently designed system and future needs. Hence it is important to design systems with built-in flexibility. A number of methods have been developed which reduce the disruption caused by the amendment or even replacement of a system or system's component.