Real‐time Animation of Sand‐Water Interaction

Recent advances in physically‐based simulations have made it possible to generate realistic animations. However, in the case of solid‐fluid coupling, wetting effects have rarely been noticed despite their visual importance especially in interactions between fluids and granular materials. This paper presents a simple particle‐based method to model the physical mechanism of wetness propagating through granular materials; Fluid particles are absorbed in the spaces between the granular particles and these wetted granular particles then stick together due to liquid bridges that are caused by surface tension and which will subsequently disappear when over‐wetting occurs. Our method can handle these phenomena by introducing a wetness value for each granular particle and by integrating those aspects of behavior that are dependent on wetness into the simulation framework. Using this method, a GPU‐based simulator can achieve highly dynamic animations that include wetting effects in real time.     

Adaptive Image‐Space Sampling for Gaze‐Contingent Real‐time Rendering

With ever‐increasing display resolution for wide field‐of‐view displays—such as head‐mounted displays or 8k projectors—shading has become the major computational cost in rasterization. To reduce computational effort, we propose an algorithm that only shades visible features of the image while cost‐effectively interpolating the remaining features without affecting perceived quality. In contrast to previous approaches we do not only simulate acuity falloff but also introduce a sampling scheme that incorporates multiple aspects of the human visual system: acuity, eye motion, contrast (stemming from geometry, material or lighting properties), and brightness adaptation. Our sampling scheme is incorporated into a deferred shading pipeline to shade the image's perceptually relevant fragments while a pull‐push algorithm interpolates the radiance for the rest of the image. Our approach does not impose any restrictions on the performed shading. We conduct a number of psycho‐visual experiments to validate scene‐ and task‐independence of our approach. The number of fragments that need to be shaded is reduced by 50 % to 80 %. Our algorithm scales favorably with increasing resolution and field‐of‐view, rendering it well‐suited for head‐mounted displays and wide‐field‐of‐view projection.     

Real‐time and user‐independent feature classification of forearm using EMG signals

Electromyography (EMG) signals contain various information about human motion. How to extract the EMG signals of the human body by appropriate methods for classification is a hot issue in current research. Unfortunately, the main problem with the classification of EMG signals is that only certain actions can be identified. Once the individual is changed, the recognition accuracy rate will be greatly reduced. This study introduces a method for classifying the forearm using back propagation (BP) neural networks. This mode extracted five features of the EMG signals. Participants were required to train their own actions during the test. Six participants selected four to six actions to identify them, and the average accuracy was more than 90%. The results suggest that the method can be used among different individuals and provides a good classification method.     

Real‐time generation of augmented video sequences by background tracking

We present a real‐time system that outputs a multi‐layer augmented video from a normal video stream. The input sequence is captured in a known environment: a panoramic image (the reference background) is provided before shooting. We use background tracking to register each input frame with this reference, and background subtraction to segment the foreground objects. Our background tracking method is a coarse‐to‐fine integration of three state‐of‐the‐art algorithms that we make resistant to occlusions. In particular, we introduce an algebraic technique to properly adapt the Jurie and Dhome (JD) tracker (the most robust of them). We report experimental results on real and synthetic data which validate our approach. We generate our augmented video sequences by compositing layers: a natural or synthetic background and several natural or synthetic foregrounds. Copyright © 2006 John Wiley & Sons, Ltd.     

Learnt Real‐time Meshless Simulation

We present a new real‐time approach to simulate deformable objects using a learnt statistical model to achieve a high degree of realism. Our approach improves upon state‐of‐the‐art interactive shape‐matching meshless simulation methods by not only capturing important nuances of an object's kinematics but also of its dynamic texture variation. We are able to achieve this in an automated pipeline from data capture to simulation. Our system allows for the capture of idiosyncratic characteristics of an object's dynamics which for many simulations (e.g. facial animation) is essential. We allow for the plausible simulation of mechanically complex objects without knowledge of their inner workings. The main idea of our approach is to use a flexible statistical model to achieve a geometrically‐driven simulation that allows for arbitrarily complex yet easily learned deformations while at the same time preserving the desirable properties (stability, speed and memory efficiency) of current shape‐matching simulation systems. The principal advantage of our approach is the ease with which a pseudo‐mechanical model can be learned from 3D scanner data to yield realistic animation. We present examples of non‐trivial biomechanical objects simulated on a desktop machine in real‐time, demonstrating superior realism over current geometrically motivated simulation techniques.     

Real‐time Shading with Filtered Importance Sampling

We propose an analysis of numerical integration based on sampling theory, whereby the integration error caused by aliasing is suppressed by pre‐filtering. We derive a pre‐filter for evaluating the illumination integral yielding filtered importance sampling, a simple GPU‐based rendering algorithm for image‐based lighting. Furthermore, we extend the algorithm with real‐time visibility computation. Free from any pre‐computation, the algorithm supports fully dynamic scenes and, above all, is simple to implement.     

A 2D–3D integrated tabletop environment for multi‐user collaboration

This paper proposes a novel tabletop display system for natural communication and flexible information sharing. The proposed system is specifically designed to integrate two‐dimensional (2D) and three‐dimensional (3D) user interfaces by using a multi‐user stereoscopic display, IllusionHole. The proposed system takes awareness into consideration and provides both 2D and 3D information and user interfaces. On the display, a number of standard Windows desktop environments are provided as personal workspaces, as well as a shared workspace with a dedicated graphical user interface. In the personal workspaces, users can simultaneously access existing applications and data, and exchange information between personal and shared workspaces. In this way, the proposed system can seamlessly integrate personal, shared, 2D, and 3D workspaces with conventional user interfaces and effectively support communication and information sharing. To demonstrate the capabilities of the proposed display system, a modeling application was implemented. A preliminary experiment confirmed the effectiveness of this system. Copyright © 2006 John Wiley & Sons, Ltd.     

Real‐time hybrid control of electrohydraulic active suspension

Electrohydraulic actuators are an attractive choice for active suspension, because these systems provide a high power‐to‐weight ratio. However, their dynamics are highly nonlinear. In addition, the use of one simple controller for both position and force is complicated, because there is a compromise between them in the case of active suspension. Most existing controllers do not efficiently fulfill the requirements, because only one state variable is considered. In this paper, we address these problems by developing a new hybrid controller for both position and force and implementing it in a real‐time test bench. Our goal is to control the vertical position of the passenger seat while tracking the force transmitted to passengers and keeping it within tolerable and comfortable limits. Therefore, the proposed controller is a combination of two controllers. Its flexible structure redirects the control signal to control the proper controlled state variable. The real‐time results of the newly designed hybrid controller are compared with those obtained using a classical proportional integral derivative controller, because this is the most widely used controller in the industry. As expected, the proposed controller demonstrates better performance in real‐time operation. Copyright © 2017 John Wiley & Sons, Ltd.     

Decoupled Shading for Real‐time Heterogeneous Volume Illumination

Existing real‐time volume rendering techniques which support global illumination are limited in modeling distinct realistic appearances for classified volume data, which is a desired capability in many fields of study for illustration and education. Directly extending the emission‐absorption volume integral with heterogeneous material shading becomes unaffordable for real‐time applications because the high‐frequency view‐dependent global lighting needs to be evaluated per sample along the volume integral. In this paper, we present a decoupled shading algorithm for multi‐material volume rendering that separates global incident lighting evaluation from per‐sample material shading under multiple light sources. We show how the incident lighting calculation can be optimized through a sparse volume integration method. The quality, performance and usefulness of our new multi‐material volume rendering method is demonstrated through several examples.     

3D interactive system based on vision computing of direct‐flective cameras

A bare‐finger 3D interactive technology for portable devices was developed. Using directive‐flective cameras to reform the field of viewing, a blind working range close to the camera is eliminated. Moreover, the algorithm of vision computing, different from skin color detection, is presented to determine the positions of fingertips. The interactive range is workable from 1.5 to 50 cm above the entire surface of the display. The mean position error of less than 1 cm is achieved. This accuracy realizes a camera‐based 3D interactive system allowing for near‐distance functionality. Therefore, floating 3D images can be touched and interacted with, potentially creating more application and intuitive user‐machine interface.     

L4RW: Laziness‐based Realistic Real‐time Responsive Rebalance in Walking

We present a novel L4RW (Laziness‐based Realistic Real‐time Responsive Rebalance in Walking) technique to synthesize 4RW animations under unexpected external perturbations with minimal locomotion effort. We first devise a lazy dynamic rebalance model, which specifies the dynamic balance conditions, defines the rebalance effort, and selects the suitable rebalance strategy automatically using the laziness law after an unexpected perturbation. Based on the model, L4RW searches over a motion capture (mocap) database for an appropriate motion segment to follow, and the transition‐to motions is generated by interpolating the active response dynamic motion. A support vector machine (SVM) based training, classification, and predication algorithm is applied to reduce the search space, and it is trained offline only once. Our algorithm classifies the mocap database into many rebalance strategy‐specified subsets and then online predicts responsive motions in the subset according to the selected strategy. The rebalance effort, the ‘extrapolated center of mass’ (XCoM) and environment constraints are selected as feature attributes for the SVM feature vector. Furthermore, the subset's segments are sorted through the rebalance effort, then our algorithm searches for an acceptable segment starting from the least‐effort segment. Compared with previous methods, our search increases speed by over two orders of magnitude, and our algorithm creates more realistic and smooth 4RW animation.     

Geometric Stiffness for Real‐time Constrained Multibody Dynamics

This paper focuses on the stable and efficient simulation of articulated rigid body systems for real‐time applications. Specifically, we focus on the use of geometric stiffness which can dramatically increase simulation stability. We examine several numerical problems with the inclusion of geometric stiffness in the equations of motion, as proposed by previous work, and address these issues by introducing a novel method for efficiently building the linear system. This offers improved tractability and numerical efficiency. Furthermore, geometric stiffness tends to significantly dissipate kinetic energy. We propose an adaptive damping scheme, inspired by the geometric stiffness, that uses a stability criterion based on the numerical integrator to determine the amount of non‐constitutive damping required to stabilize the simulation. With this approach, not only is the dynamical behavior better preserved, but the simulation remains stable for mass ratios of 1,000,000‐to‐1 at time steps up to 0.1 s. We present a number of challenging scenarios to demonstrate that our method improves efficiency, and that it increases stability by orders of magnitude compared to previous work.     

Real‐time Image‐based Lighting of Microfacet BRDFs with Varying Iridescence

Iridescence is a natural phenomenon that is perceived as gradual color changes, depending on the view and illumination direction. Prominent examples are the colors seen in oil films and soap bubbles. Unfortunately, iridescent effects are particularly difficult to recreate in real‐time computer graphics. We present a high‐quality real‐time method for rendering iridescent effects under image‐based lighting. Previous methods model dielectric thin‐films of varying thickness on top of an arbitrary micro‐facet model with a conducting or dielectric base material, and evaluate the resulting reflectance term, responsible for the iridescent effects, only for a single direction when using real‐time image‐based lighting. This leads to bright halos at grazing angles and over‐saturated colors on rough surfaces, which causes an unnatural appearance that is not observed in ground truth data. We address this problem by taking the distribution of light directions, given by the environment map and surface roughness, into account when evaluating the reflectance term. In particular, our approach prefilters the first and second moments of the light direction, which are used to evaluate a filtered version of the reflectance term. We show that the visual quality of our approach is superior to the ones previously achieved, while having only a small negative impact on performance.     

Improving Memory Space Efficiency of Kd‐tree for Real‐time Ray Tracing

Compared with its competitors such as the bounding volume hierarchy, a drawback of the kd‐tree structure is that a large number of triangles are repeatedly duplicated during its construction, which often leads to inefficient, large and tall binary trees with high triangle redundancy. In this paper, we propose a space‐efficient kd‐tree representation where, unlike commonly used methods, an inner node is allowed to optionally store a reference to a triangle, so highly redundant triangles in a kd‐tree can be culled from the leaf nodes and moved to the inner nodes. To avoid the construction of ineffective kd‐trees entailing computational inefficiencies due to early, possibly unnecessary, ray‐triangle intersection calculations that now have to be performed in the inner nodes during the kd‐tree traversal, we present heuristic measures for determining when and how to choose triangles for inner nodes during kd‐tree construction. Based on these metrics, we describe how the new form of kd‐tree is constructed and stored compactly using a carefully designed data layout. Our experiments with several example scenes showed that our kd‐tree representation technique significantly reduced the memory requirements for storing the kd‐tree structure, while effectively suppressing the unavoidable frame‐rate degradation observed during ray tracing.     

High‐resolution 360 Video Foveated Stitching for Real‐time VR

In virtual reality (VR) applications, the contents are usually generated by creating a 360° Video panorama of a real‐world scene. Although many capture devices are being released, getting high‐resolution panoramas and displaying a virtual world in real‐time remains challenging due to its computationally demanding nature. In this paper, we propose a real‐time 360° Video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high‐resolution panoramic Video in real‐time that can be streamed directly to the client. Our foveated stitching algorithm takes Videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.     

Gradient‐based Interpolation and Sampling for Real‐time Rendering of Inhomogeneous,Single‐scattering Media

We present a real‐time rendering algorithm for inhomogeneous, single scattering media, where all‐frequency shading effects such as glows, light shafts, and volumetric shadows can all be captured. The algorithm first computes source radiance at a small number of sample points in the medium, then interpolates these values at other points in the volume using a gradient‐based scheme that is efficiently applied by sample splatting. The sample points are dynamically determined based on a recursive sample splitting procedure that adapts the number and locations of sample points for accurate and efficient reproduction of shading variations in the medium. The entire pipeline can be easily implemented on the GPU to achieve real‐time performance for dynamic lighting and scenes. Rendering results of our method are shown to be comparable to those from ray tracing.     

Real‐time Realistic Ocean Lighting using Seamless Transitions from Geometry to BRDF

Realistic animation and rendering of the ocean is an important aspect for simulators, movies and video games. By nature, the ocean is a difficult problem for Computer Graphics: it is a dynamic system, it combines wave trains at all scales, ranging from kilometric to millimetric. Worse, the ocean is usually viewed at several distances, from very close to the viewpoint to the horizon, increasing the multi‐scale issue, and resulting in aliasing problems. The illumination comes from natural light sources (the Sun and the sky dome), is also dynamic, and often underlines the aliasing issues. In this paper, we present a new algorithm for modelling, animation, illumination and rendering of the ocean, in real‐time, at all scales and for all viewing distances. Our algorithm is based on a hierarchical representation, combining geometry, normals and BRDF. For each viewing distance, we compute a simplified version of the geometry, and encode the missing details into the normal and the BRDF, depending on the level of detail required. We then use this hierarchical representation for illumination and rendering. Our algorithm runs in real‐time, and produces highly realistic pictures and animations.     

Real‐time Bas‐Relief Generation from Depth‐and‐Normal Maps on GPU

To design a bas‐relief from a 3D scene is an inherently interactive task in many scenarios. The user normally needs to get instant feedback to select a proper viewpoint. However, current methods are too slow to facilitate this interaction. This paper proposes a two‐scale bas‐relief modeling method, which is computationally efficient and easy to produce different styles of bas‐reliefs. The input 3D scene is first rendered into two textures, one recording the depth information and the other recording the normal information. The depth map is then compressed to produce a base surface with level‐of‐depth, and the normal map is used to extract local details with two different schemes. One scheme provides certain freedom to design bas‐reliefs with different visual appearances, and the other provides a control over the level of detail. Finally, the local feature details are added into the base surface to produce the final result. Our approach allows for real‐time computation due to its implementation on graphics hardware. Experiments with a wide range of 3D models and scenes show that our approach can effectively generate digital bas‐reliefs in real time.     

Real‐time horse gait synthesis

Horse locomotion exhibits rich variations in gaits and styles. Although there have been many approaches proposed for animating quadrupeds, there is not much research on synthesizing horse locomotion. In this paper, we present a horse locomotion synthesis approach. A user can arbitrarily change a horse's moving speed and direction, and our system would automatically adjust the horse's motion to fulfill the user's commands. At preprocessing, we manually capture horse locomotion data from Eadweard Muybridge's famous photographs of animal locomotion and expand the captured motion database to various speeds for each gait. At runtime, our approach automatically changes gaits based on speed, synthesizes the horse's root trajectory, and adjusts its body orientation based on the horse's turning direction. We propose an asynchronous time warping approach to handle gait transition, which is critical for generating realistic and controllable horse locomotion. Our experiments demonstrate that our system can produce smooth, rich, and controllable horse locomotion in real time. Copyright © 2012 John Wiley & Sons, Ltd.     

Real‐Time Indirect Illumination and Soft Shadows in Dynamic Scenes Using Spherical Lights

We present a method for rendering approximate soft shadows and diffuse indirect illumination in dynamic scenes. The proposed method approximates the original scene geometry with a set of tightly fitting spheres. In previous work, such spheres have been used to dynamically evaluate the visibility function to render soft shadows. In this paper, each sphere also acts as a low‐frequency secondary light source, thereby providing diffuse one‐bounce indirect illumination. The method is completely dynamic and proceeds in two passes: In a first pass, the light intensity distribution on each sphere is updated based on sample points on the corresponding object surface and converted into the spherical harmonics basis. In a second pass, this radiance information and the visibility are accumulated to shade final image pixels. The sphere approximation allows us to compute visibility and diffuse reflections of an object at interactive frame rates of over 20 fps for moderately complex scenes.