Exertion interfaces for computer videogames using smartphones as input controllers

As mobile phones become smarter and include a wider and more powerful array of sensory components, the opportunity to leverage those capabilities in contexts other than telephony grows. We have in particular identified those sensory capabilities as key components for modern user interfaces that can detect movement, actions and intentions to enrich human-computer interaction in a natural way. In this work, we present research around using smartphones as input controllers in the context of exertion videogames. We propose a conceptual framework that identifies the core elements of such interfaces, regardless of the underlying technological platforms, and provides a design pattern for their integration into existing videogames without having to change the game’s source code. We present a proof of concept implementation for the framework, with two smartphone input controllers, which using a soft button and accelerometer data, interface to a target-shooting exertion game played while exercising on a stationary bicycle. We present findings from a user experience evaluation.     

Modeling input modality choice in mobile graphical and speech interfaces

In this paper, we review three experiments with a mobile application that integrates graphical input with a touch-screen and a speech interface and develop a model for input modality choice in multimodal interaction. The model aims to enable simulation of multimodal human–computer interaction for automatic usability evaluation. The experimental results indicate that modality efficiency and input performance are important moderators of modality choice. Accordingly, we establish a utility-driven model that provides probability estimations of modality usage, based on the parameters of modality efficiency and input performance. Four variants of the model that differ in training data are fitted by means of Sequential Least Squares Programming. The analysis reveals a considerable fit regarding averaged modality usage. When applied to individual modality usage profiles, the accuracy decreases significantly. In an application example it is shown how the modality choice mechanism can be deployed for simulating interaction in the field of automatic usability evaluation. Results and possible limitations are discussed.     

A real-time framework for eye detection and tracking

Epidemiological studies indicate that automobile drivers from varying demographics are confronted by difficult driving contexts such as negotiating intersections, yielding, merging and overtaking. We aim to detect and track the face and eyes of the driver during several driving scenarios, allowing for further understanding of a driver’s visual search pattern behavior. Traditionally, detection and tracking of objects in visual media has been performed using specific techniques. These techniques vary in terms of their robustness and computational cost. This research proposes a real-time framework that is built upon a foundation synonymous to boosting, which we extend from learners to trackers and demonstrate that the idea of an integrated framework employing multiple trackers is advantageous in forming a globally strong tracking methodology. In order to model the effectiveness of trackers, a confidence parameter is introduced to help minimize the errors produced by incorrect matches and allow more effective trackers with a higher confidence value to correct the perceived position of the target.     

Smooth Gaze: a framework for recovering tasks across devices using eye tracking

A user task is often distributed across devices, e.g., a student listening to a lecture in a classroom while watching slides on a projected screen and making notes on her laptop, and sometimes checking Twitter for comments on her smartphone. In scenarios like this, users move between heterogeneous devices and have to deal with task resumption overhead from both physical and mental perspectives. To address this problem, we created Smooth Gaze, a framework for recording the user’s work state and resuming it seamlessly across devices by leveraging implicit gaze input. In particular, we propose two novel and intuitive techniques, smart watching and smart posting, for detecting which display and target region the user is looking at, and transferring and integrating content across devices respectively. In addition, we designed and implemented a cross-device reading system SmoothReading that captures content from secondary devices and generates annotations based on eye tracking, to be displayed on the primary device. We conducted a study that showed that the system supported information seeking and task resumption, and improved users’ overall reading experience.     

A framework for search interfaces

Searching textual databases can be confusing for users. Popular search systems for the World Wide Web and stand alone systems typically provide a simple interface: users type in keywords and receive a relevance ranked list of 10 results. This is appealing in its simplicity, but users are often frustrated because search results are confusing or aspects of the search are out of their control. If we are to improve user performance, reduce mistaken assumptions, and increase successful searches, we need more predictable design. To coordinate design practice, we suggest a four-phase framework that would satisfy first time, intermittent, and frequent users accessing a variety of textual and multimedia libraries     

Robust tracking control for an air-breathing hypersonic vehicle with input constraints

The focus of this paper is on the design and simulation of robust tracking control for an air-breathing hypersonic vehicle (AHV), which is affected by high nonlinearity, uncertain parameters and input constraints. The linearisation method is employed for the longitudinal AHV model about a specific trim condition, and then considering the additive uncertainties of three parameters, the linearised model is just in the form of affine parameter dependence. From this point, the linear parameter-varying method is applied to design the desired controller. The poles for the closed-loop system of the linearised model are placed into a desired vertical strip, and the quadratic stability of the closed-loop system is guaranteed. Input constraints of the AHV are addressed by additional linear matrix inequalities. Finally, the designed controller is evaluated on the nonlinear AHV model and simulation results demonstrate excellent tracking performance with good robustness.     

Pursuit of methodology for data input related to taste in design: Using eye tracking technology

Being knowledgeable and using the knowledge accurately is the basis for all development. Knowledge sheds light for the designers on the topics of correct decision making and early intervention while assessing the success of current products. The aesthetic value of a product is the fundamental point of communication between the designer and the user. Therefore, with the aim of accessing the hidden aesthetic taste in the users via the user, eye tracking technology was used as a tool since the sight has a great influence on one’s internal world. Within this scope, it was decided to work on a static product group, and 40 armchair variations were created as a sample. The eye movements on armchairs with variable stylistic features and the taste evaluations of 60 participants were monitored via the three different stages of the experiments. In line with the main objective of the study, significant and guiding outputs were obtained. The results showed that the taste related to the product can be understood via the eye with viewing metrics such as time to first fixation, first fixation duration, fixation count and duration, visit count and duration, and that the knowledge of view could affect the design decisions. On the other hand, a road map and a procedure were created for the purposes of accessing the implicit taste information for the design practice through using the eye tracking technology.     

一个改进的用于流体界面追踪的等值面法

提出了一个改进的O(n)的多点匹配方法用于流体界面追踪。新的方法和快速匹配方法(FMM)一样,都是基于窄带的方法。与FMM不同的是,它不需要对窄带中的点进行排序,也不是逐点更新窄带上的网格点,而是应用一个多点调整的策略来同时更新一组网格点。二维、三维的数值实验表明了本方法的有效性。此方法已应用于动态虚拟环境中的流体界面追踪中。     

Robust camera pose tracking for augmented reality using particle filtering framework

In this paper, we present new solutions for the problem of estimating the camera pose using particle filtering framework. The proposed approach is suitable for real-time augmented reality (AR) applications in which the camera is held by the user. This work demonstrates that particle filtering improve the robustness of the tracking comparing to existing approaches, such as those based on the Kalman filter. We propose a tracking framework for both points and lines features, the particle filter is used to compute the posterior density for the camera 3D motion parameters. We also analyze the sensitivity of our technique when outliers are present in the match data. Outliers arise frequently due to incorrect correspondences which occur because of either image noise or occlusion. Results from real data in an augmented reality setup are then presented, demonstrating the efficiency and robustness of the proposed method.     

A strong tracking nonlinear robust filter for eye tracking

Non-intrusive methods for eye tracking are important for many applications of vision-based human computer interaction. However, due to the high nonlinearity of eye motion, how to ensure the robustness of external interference and accuracy of eye tracking pose the primary obstacle to the integration of eye movements into today’s interfaces. In this paper, we present a strong tracking unscented Kalman filter (ST-UKF) algorithm, aiming to overcome the difficulty in nonlinear eye tracking. In the proposed ST-UKF, the Suboptimal fading factor of strong tracking filtering is introduced to improve robustness and accuracy of eye tracking. Compared with the related Kalman filter for eye tracking, the proposed ST-UKF has potential advantages in robustness and tracking accuracy. The last experimental results show the validity of our method for eye tracking under realistic conditions.     

Automated eye tracking system calibration using artificial neural networks

The electro-oculogram (EOG) continues to be widely used to record eye movements especially in clinical settings. However, an efficient and accurate means of converting these recordings into eye position is lacking. An artificial neural network (ANN) that maps two-dimensional (2D) eye movement recordings into 2D eye positions can enhance the utility of such recordings. Multi-layer perceptrons (MLPs) with non-linear activation functions and trained with back propagation proved to be capable of calibrating simulated EOG data to a mean accuracy of 0.33 degrees . Linear perceptrons (LPs) were only nearly half as accurate. For five subjects, the mean accuracy provided by the MLPs was 1.09 degrees of visual angle ( degrees ) for EOG data, and 0.98 degrees for an infrared eye tracker. MLPs enabled calibration of 2D saccadic EOG to an accuracy not significantly different from that obtained with the infrared tracker. Using initial weights trained on another person reduced MLP training time, reaching convergence in as little as 20 iterations.     

Blink detection for real-time eye tracking

This work is motivated by our goal of providing non-contact head and eye based control of computer systems for people with motor difficulties. The system described here uses spatio-temporal filtering and variance maps to locate the head and find the eye-feature points, respectively. These feature points are accurately tracked in the succeeding frames by using a modified version of the Lucas–Kanade tracking algorithm with pyramidal implementation. Accurate head and eye tracking results are obtained at a processing rate of more than 30 frames per second (fps) in more than 90% cases with a low false positive blink detection rate of 0.01%. This is achieved under varying lighting conditions for people of different ethnicity, with and without wearing glasses.     

基于单固定摄像头的多目标跟踪框架

针对单固定摄像头的视频监控系统对合并遮挡目标跟踪效果不好导致跟踪失败的问题,提出了一个稀疏多目标跟踪框架.该框架对系统的目标的合并遮挡和跟踪滤波这两个部分作了改进.系统由运动目标检测、关联矩阵建立、目标交互处理和滤波四部分组成.首先提取前景区域并建立关联矩阵;然后用关联矩阵判断各目标运动状态并进行相应处理,当目标发生交互时,用TLD算法跟踪,为了提高TLD的跟踪效率和减少TLD的初始化异常情况,用双三次插值对目标和跟踪窗口进行同比例缩放;最后用分数阶卡尔曼滤波对跟踪结果进行滤波.实验结果证明,该框架能有效提高单固定摄像头对目标交互遮挡情况的处理能力.     

User satisfaction modeling framework for automotive audio interfaces

Currently the research on product satisfaction is evolving to integrate pleasure and delight alongside other, more traditional needs. Pleasure and delight are emotional and hedonic benefits experienced by the user when interacting with the product. Thus, user satisfaction models have to deal with high complexity of product properties that collectively contribute to satisfaction. The formulation of these models has to include not only a proper taxonomy, but also the organization of such properties in categories and their relation to well defined satisfaction dimensions. Successful models would help to find out the most effective technical and design specifications (engineering parameters) that give shape to specific attributes of design, sometimes called “brand feelings”, defined in accordance with the business strategy. The objective of this paper is to present a user satisfaction modeling framework that deals with the complexity of an in-car user interface, namely the audio one. The proposed framework defines three distinct types of product properties: engineering parameters (P), perceived product attributes (A) and satisfaction dimensions (D) and presents a methodology to link the perceived product attributes (A) to satisfaction dimension (D).     

面向服务Web用户界面的动态组合框架

针对当前Web服务的应用和研究只关注于对业务功能的封装和重用以及相应服务的组合,而对用户界面缺乏相应的支持,提出了一种动态的、面向服务的Web用户界面(web user interface,WebUI)组合框架.在此框架中,WebUI构件被封装为可重用的Web服务,因此可以被发现、选择和绑定,并根据WebUI建模时所生成的WebUI组合描述被动态地组合为相应的WebUI,显著地简化了WebUI的开发和维护工作.     

Designing for pen and speech input in an object-action framework: the case of email

This study presents a user interface that was intentionally designed to support multimodal interaction by compensating for the weaknesses of speech compared with pen input and vice versa. The test application was email using a web pad with pen and speech input. In the case of pen input, information was represented as visual objects, which were easily accessible. Graphical metaphors were used to enable faster and easier manipulation of data. Speech input was facilitated by displaying the system speech vocabulary to the user. All commands and accessible fields with text labels could be spoken in by name. Commands and objects that the user could access via speech input were shown on a dynamic basis in a window. Multimodal interaction was further enhanced by creating a flexible object-action order such that the user could utter or select a command with a pen followed by the object which was to be enacted upon, or the other way round (e.g., New Message or Message New). The flexible action-object interaction design combined with voice and pen input led to eight possible action-object-modality combinations. The complexity of the multimodal interface was further reduced by making generic commands such as New applicable across corresponding objects. Use of generic commands led to a simplification of menu structures by reducing the number of instances in which actions appeared. In this manner, more content information could be made visible and consistently accessible via pen and speech input. Results of a controlled experiment indicated that the shortest task completion times for the eight possible input conditions were when speech-only was used to refer to an object followed by the action to be performed. Speech-only input with action-object order was also relatively fast. In the case of pen input-only, the shortest task completion times were found when an object was selected first followed by the action to be performed. In multimodal trials in which both pen and speech were used, no significant effect was found for object-action order, suggesting benefits of providing users with a flexible action-object interaction style in multimodal or speech-only systems.     

Adaptive neural network asymptotical tracking control for an uncertain nonlinear system with input quantisation

In this paper, the problem of adaptive neural network asymptotical tracking is investigated for a class of nonlinear system with unknown function, external disturbances and input quantisation. Based on neural network technique, an adaptive asymptotical tracking controller is provided for an uncertain nonlinear system via backstepping method. In order to reduce complexity of the control algorithm in the backstepping design process, a sliding mode differentiator is employed to estimate the virtual control law and only two parameters need to be estimated via adaptive control technique. The stability of the closed-loop system is analysed by using Lyapunov function method and zero-tracking error performance is obtained in the presence of unknown nonlinear function, external disturbances and input quantisation. Finally, an application example is employed to demonstrate the effectiveness of the proposed scheme.     

In search of effective text input interfaces for off the desktop computing

It is generally recognized that today's frontier of HCI research lies beyond the traditional desktop computers whose GUI interfaces were built on the foundation of display—pointing device—full keyboard. Many interface challenges arise without such a physical UI foundation. Text writing—ranging from entering URLs and search queries, filling forms, typing commands, to taking notes and writing emails and chat messages—is one of the hard problems awaiting for solutions in off-desktop computing. This paper summarizes and synthesizes a research program on this topic at the IBM Almaden Research Center. It analyzes various dimensions that constitute a good text input interface; briefly reviews related literature; discusses the evaluation methodology issues of text input; presents the major ideas and results of two systems, ATOMIK and SHARK; and points out current and future directions in the area from our current vantage point.