User‐Guided Facial Animation through an Evolutionary Interface

We propose a design framework to assist with user‐generated content in facial animation — without requiring any animation experience or ground truth reference. Where conventional prototyping methods rely on handcrafting by experienced animators, our approach looks to encode the role of the animator as an Evolutionary Algorithm acting on animation controls, driven by visual feedback from a user. Presented as a simple interface, users sample control combinations and select favourable results to influence later sampling. Over multiple iterations of disregarding unfavourable control values, parameters converge towards the user's ideal. We demonstrate our framework through two non‐trivial applications: creating highly nuanced expressions by evolving control values of a face rig and non‐linear motion through evolving control point positions of animation curves.     

Comprehensive Facial Performance Capture

We present a system for recording a live dynamic facial performance, capturing highly detailed geometry and spatially varying diffuse and specular reflectance information for each frame of the performance. The result is a reproduction of the performance that can be rendered from novel viewpoints and novel lighting conditions, achieving photorealistic integration into any virtual environment. Dynamic performances are captured directly, without the need for any template geometry or static geometry scans, and processing is completely automatic, requiring no human input or guidance. Our key contributions are a heuristic for estimating facial reflectance information from gradient illumination photographs, and a geometry optimization framework that maximizes a principled likelihood function combining multi‐view stereo correspondence and photometric stereo, using multi‐resolution belief propagation. The output of our system is a sequence of geometries and reflectance maps, suitable for rendering in off‐the‐shelf software. We show results from our system rendered under novel viewpoints and lighting conditions, and validate our results by demonstrating a close match to ground truth photographs.     

Real‐Time Multi‐View Facial Capture with Synthetic Training

We present a real‐time multi‐view facial capture system facilitated by synthetic training imagery. Our method is able to achieve high‐quality markerless facial performance capture in real‐time from multi‐view helmet camera data, employing an actor specific regressor. The regressor training is tailored to specified actor appearance and we further condition it for the expected illumination conditions and the physical capture rig by generating the training data synthetically. In order to leverage the information present in live imagery, which is typically provided by multiple cameras, we propose a novel multi‐view regression algorithm that uses multi‐dimensional random ferns. We show that higher quality can be achieved by regressing on multiple video streams than previous approaches that were designed to operate on only a single view. Furthermore, we evaluate possible camera placements and propose a novel camera configuration that allows to mount cameras outside the field of view of the actor, which is very beneficial as the cameras are then less of a distraction for the actor and allow for an unobstructed line of sight to the director and other actors. Our new real‐time facial capture approach has immediate application in on‐set virtual production, in particular with the ever‐growing demand for motion‐captured facial animation in visual effects and video games.     

User‐Assisted Video Stabilization

We present a user‐assisted video stabilization algorithm that is able to stabilize challenging videos when state‐of‐the‐art automatic algorithms fail to generate a satisfactory result. Current methods do not give the user any control over the look of the final result. Users either have to accept the stabilized result as is, or discard it should the stabilization fail to generate a smooth output. Our system introduces two new modes of interaction that allow the user to improve the unsatisfactory stabilized video. First, we cluster tracks and visualize them on the warped video. The user ensures that appropriate tracks are selected by clicking on track clusters to include or exclude them. Second, the user can directly specify how regions in the output video should look by drawing quadrilaterals to select and deform parts of the frame. These user‐provided deformations reduce undesirable distortions in the video. Our algorithm then computes a stabilized video using the user‐selected tracks, while respecting the user‐modified regions. The process of interactively removing user‐identified artifacts can sometimes introduce new ones, though in most cases there is a net improvement. We demonstrate the effectiveness of our system with a variety of challenging hand held videos.     

User‐Controllable Color Transfer

This paper presents an image editing framework where users use reference images to indicate desired color edits. In our approach, users specify pairs of strokes to indicate corresponding regions in both the original and the reference image that should have the same color “style”. Within each stroke pair, a nonlinear constrained parametric transfer model is used to transfer the reference colors to the original. We estimate the model parameters by matching color distributions, under constraints that ensure no visual artifacts are present in the transfer result. To perform transfer on the whole image, we employ optimization methods to propagate the model parameters defined at each stroke location to spatially‐close regions of similar appearance. This stroke‐based formulation requires minimal user effort while retaining the high degree of user control necessary to allow artistic interpretations. We demonstrate our approach by performing color transfer on a number of image pairs varying in content and style, and show that our algorithm outperforms state‐of‐the‐art color transfer methods on both user‐controllability and visual qualities of the transfer results.     

Near‐Instant Capture of High‐Resolution Facial Geometry and Reflectance

We present a near‐instant method for acquiring facial geometry and reflectance using a set of commodity DSLR cameras and flashes. Our setup consists of twenty‐four cameras and six flashes which are fired in rapid succession with subsets of the cameras. Each camera records only a single photograph and the total capture time is less than the 67ms blink reflex. The cameras and flashes are specially arranged to produce an even distribution of specular highlights on the face. We employ this set of acquired images to estimate diffuse color, specular intensity, specular exponent, and surface orientation at each point on the face. We further refine the facial base geometry obtained from multi‐view stereo using estimated diffuse and specular photometric information. This allows final submillimeter surface mesostructure detail to be obtained via shape‐from‐specularity. The final system uses commodity components and produces models suitable for authoring high‐quality digital human characters.     

基于线索引导的面部运动捕捉与数据虚拟重用方法

多数无标记面部表情运动捕捉方法仅捕捉面部表情运动的平面位置变化,而无纵深变化描述。针对这一问题,提出了基于线索引导的面部运动捕捉与数据虚拟重用方法。首先,使用单目视觉系统对人脸主体区域进行定位;然后,采用级联回归模型对人脸地标特征逐步求精,通过利用人脸特征的主动地标线索和地标特征的深度线索得到特征点在三维空间上的位置变换关系;最后,借助面部骨骼节点的数据表示实现面部表情运动重构。通过在线的实时面部表情运动捕捉实验效果可以看出,该方法不仅能够实现不同视角下对应地标特征的精确匹配,而且能够较好地将真实面部运动赋予虚拟角色。     

User‐level operating system transactions

User‐level operating system transactions allow system administrators and ordinary users to perform a sequence of file operations and then commit them as a group, or abort them without leaving any trace behind. Such a facility can aid many system administration and software development tasks. The snapshot isolation concurrency control mechanism allows transactions to be implemented without locking individual system calls; conflicts are detected when the transaction is ready to commit. Along these lines we have implemented a user‐space transaction monitor that is based on ZFS snapshots and a file system event monitor. Transactions are committed through a robust and efficient algorithm that merges the operations performed on a file system's clone back to its parent. Both the performance impact and the implementation cost of the transaction monitor we describe are fairly small. Copyright © 2009 John Wiley & Sons, Ltd.     

Towards User‐Centered Active Learning Algorithms

The labeling of data sets is a time‐consuming task, which is, however, an important prerequisite for machine learning and visual analytics. Visual‐interactive labeling (VIAL) provides users an active role in the process of labeling, with the goal to combine the potentials of humans and machines to make labeling more efficient. Recent experiments showed that users apply different strategies when selecting instances for labeling with visual‐interactive interfaces. In this paper, we contribute a systematic quantitative analysis of such user strategies. We identify computational building blocks of user strategies, formalize them, and investigate their potentials for different machine learning tasks in systematic experiments. The core insights of our experiments are as follows. First, we identified that particular user strategies can be used to considerably mitigate the bootstrap (cold start) problem in early labeling phases. Second, we observed that they have the potential to outperform existing active learning strategies in later phases. Third, we analyzed the identified core building blocks, which can serve as the basis for novel selection strategies. Overall, we observed that data‐based user strategies (clusters, dense areas) work considerably well in early phases, while model‐based user strategies (e.g., class separation) perform better during later phases. The insights gained from this work can be applied to develop novel active learning approaches as well as to better guide users in visual interactive labeling.     

Deep Video‐Based Performance Synthesis from Sparse Multi‐View Capture

We present a deep learning based technique that enables novel‐view videos of human performances to be synthesized from sparse multi‐view captures. While performance capturing from a sparse set of videos has received significant attention, there has been relatively less progress which is about non‐rigid objects (e.g., human bodies). The rich articulation modes of human body make it rather challenging to synthesize and interpolate the model well. To address this problem, we propose a novel deep learning based framework that directly predicts novel‐view videos of human performances without explicit 3D reconstruction. Our method is a composition of two steps: novel‐view prediction and detail enhancement. We first learn a novel deep generative query network for view prediction. We synthesize novel‐view performances from a sparse set of just five or less camera videos. Then, we use a new generative adversarial network to enhance fine‐scale details of the first step results. This opens up the possibility of high‐quality low‐cost video‐based performance synthesis, which is gaining popularity for VA and AR applications. We demonstrate a variety of promising results, where our method is able to synthesis more robust and accurate performances than existing state‐of‐the‐art approaches when only sparse views are available.     

Woodification: User‐Controlled Cambial Growth Modeling

We present a botanical simulation of secondary (cambial) tree growth coupled to a physical cracking simulation of its bark. Whereas level set growth would use a fixed resolution voxel grid, our system extends the deformable simplicial complex (DSC), supporting new biological growth functions robustly on any surface polygonal mesh with adaptive subdivision, collision detection and topological control. We extend the DSC with temporally coherent texturing, and surface cracking with a user‐controllable biological model coupled to the stresses introduced by the cambial growth model.     

A User‐based Visual Analytics Workflow for Exploratory Model Analysis

Many visual analytics systems allow users to interact with machine learning models towards the goals of data exploration and insight generation on a given dataset. However, in some situations, insights may be less important than the production of an accurate predictive model for future use. In that case, users are more interested in generating of diverse and robust predictive models, verifying their performance on holdout data, and selecting the most suitable model for their usage scenario. In this paper, we consider the concept of Exploratory Model Analysis (EMA), which is defined as the process of discovering and selecting relevant models that can be used to make predictions on a data source. We delineate the differences between EMA and the well‐known term exploratory data analysis in terms of the desired outcome of the analytic process: insights into the data or a set of deployable models. The contributions of this work are a visual analytics system workflow for EMA, a user study, and two use cases validating the effectiveness of the workflow. We found that our system workflow enabled users to generate complex models, to assess them for various qualities, and to select the most relevant model for their task.     

User‐interface design heuristics for developing usable industrial hypermedia applications

Shop floor operators urgently need help to cope with the pressures of the present manufacturing context. One proposed solution is industrial hypermedia applications (IHA). A review of published papers on IHAs has shown that while the potential benefits of such applications, if accepted and used by operators, are well documented, user‐interface design aspects are not. In other words, the rationale used for the user‐interface design is not explained. This article fills the gap by drawing on human–computer interaction and hypermedia literature to present user‐interface design guidelines or heuristics. The latter were incorporated in 2 IHAs developed for this study. Shop floor operators empirically assessed the IHAs through usability trials held in the workplace and provided data on their perceptions of the IHA's ease of use. The findings show that the 7 design heuristics contributed to the usability of the IHAs. However, developers should not apply the design heuristics in isolation but should also elicit user characteristics and task domain. © 2002 Wiley Periodicals, Inc.     

Performance Capture of High‐Speed Motion Using Staggered Multi‐View Recording

We present a markerless performance capture system that can acquire the motion and the texture of human actors performing fast movements using only commodity hardware. To this end we introduce two novel concepts: First, a staggered surround multi‐view recording setup that enables us to perform model‐based motion capture on motion‐blurred images, and second, a model‐based deblurring algorithm which is able to handle disocclusion, self‐occlusion and complex object motions. We show that the model‐based approach is not only a powerful strategy for tracking but also for deblurring highly complex blur patterns.     

Orthogonal-Blendshape-Based Editing System for Facial Motion Capture Data

The authors present a novel data-driven 3D facial motion capture data editing system using automated construction of an orthogonal blendshape face model and constrained weight propagation, aiming to bridge the popular facial motion capture technique and blendshape approach. In this work, a 3D facial-motion-capture-editing problem is transformed to a blendshape-animation-editing problem. Given a collected facial motion capture data set, we construct a truncated PCA space spanned by the greatest retained eigenvectors and a corresponding blendshape face model for each anatomical region of the human face. As such, modifying blendshape weights (PCA coefficients) is equivalent to editing their corresponding motion capture sequence. In addition, a constrained weight propagation technique allows animators to balance automation and flexible controls.     

A User‐Centric Evaluation Procedure for the Design of Child‐Resistant Medicine Bottles

Recently, people have begun to realize the importance of child‐resistant (CR) medicine packaging. However, most manufacturers and designers have not been able to effectively provide prevention strategies or design criteria to protect consumers. This research proposes a systematic approach to analyze CR packaging design problems and experiments to evaluate the identified design parameters and to generate the most suitable CR medicine packaging design. The design of a CR packaging bottle is used as an example to help explain the development procedure. During the development procedure, user trials, questionnaires, and children's anthropometric data on bottle opening and hand operations are analyzed. Five design parameters, specifically the cap diameter, cap height, bottle height, bottle diameter, and torsion, are identified and used to perform a Taguchi orthogonal array experimental analysis. A computer‐aided design system is also built to help generate the most suitable design alternatives. The results should assist designers in determining the most important CR design parameters and their most suitable combinations for bottle and related CR medicine packaging design.     

COPERNICUS: Context‐Preserving Engine for Route Navigation with Interactive User‐modifiable Scaling

In this paper, we present an automated system for generating context‐preserving route maps that depict navigation routes as a path between nodes and edges inside a topographic network. Our application identifies relevant context information to support navigation and orientation, and generates customizable route maps according to design principles that communicate all relevant context information clearly visible on one single page. Interactive scaling allows seamless transition between the original undistorted map and our new map design, and supports user‐specified scaling of regions of interest to create personalized driving directions according to the drivers needs.     

User‐centric trust and reputation model for personal and trusted service selection

Improving quality of services (QoS) through applying trust and reputation management technology is increasingly popular in the literature and industry. Most existing trust and reputation systems calculate a general trust value or vector based on the gathered feedback without regard to trust's locality and subjectivity; therefore, they cannot effectively support a personal selection with consumer preferences. Our goal is to build a trust and reputation mechanism for facilitating a trustworthy and personal service selection in a service‐oriented Web, where service peers can act as a service provider and/or a service consumer. A user‐centric trust and reputation mechanism distinguishing the different trust context and content to enable a personal service selection with regard to trust preference in a service‐oriented Web is represented in detail. It is widely recognized that reputation‐based trust methods must face the challenge of malicious behaviors. To deal with the malicious feedback behaviors, we introduce a “bidirectional'' feedback mechanism based on QoS experience similarity in our trust and reputation framework. The test run demonstrates that our method can significantly increase the success rate of service transactions and is effective in resisting various malicious behaviors of service peers, when it is compared to other similar methods. © 2011 Wiley Periodicals, Inc.     

Self‐similarity Analysis for Motion Capture Cleaning

Motion capture sequences may contain erroneous data, especially when the motion is complex or performers are interacting closely and occlusions are frequent. Common practice is to have specialists visually detect the abnormalities and fix them manually. In this paper, we present a method to automatically analyze and fix motion capture sequences by using self‐similarity analysis. The premise of this work is that human motion data has a high‐degree of self‐similarity. Therefore, given enough motion data, erroneous motions are distinct when compared to other motions. We utilize motion‐words that consist of short sequences of transformations of groups of joints around a given motion frame. We search for the K‐nearest neighbors (KNN) set of each word using dynamic time warping and use it to detect and fix erroneous motions automatically. We demonstrate the effectiveness of our method in various examples, and evaluate by comparing to alternative methods and to manual cleaning.     

Simulation‐Ready Hair Capture

Physical simulation has long been the approach of choice for generating realistic hair animations in CG. A constant drawback of simulation, however, is the necessity to manually set the physical parameters of the simulation model in order to get the desired dynamic behavior. To alleviate this, researchers have begun to explore methods for reconstructing hair from the real world and even to estimate the corresponding simulation parameters through the process of inversion. So far, however, these methods have had limited applicability, because dynamic hair capture can only be played back without the ability to edit, and solving for simulation parameters can only be accomplished for static hairstyles, ignoring the dynamic behavior. We present the first method for capturing dynamic hair and automatically determining the physical properties for simulating the observed hairstyle in motion. Since our dynamic inversion is agnostic to the simulation model, the proposed method applies to virtually any hair simulation technique, which we demonstrate using two state‐of‐the‐art hair simulation models. The output of our method is a fully simulation‐ready hairstyle, consisting of both the static hair geometry as well as its physical properties. The hairstyle can be easily edited by adding additional external forces, changing the head motion, or re‐simulating in completely different environments, all while remaining faithful to the captured hairstyle.