基于混合遗传算法的人体运动捕获数据关键帧提取

为实现基于最佳关键帧集合的人体运动的紧致表示,提出一种遗传算法与单纯形法结合的人体运动捕获数据关键帧提取方法。以重构误差最小化和压缩率最优为目标,定义适应度函数,度量重构运动与原始运动之间的重构误差,通过关节位置和关节速率加权计算,并考虑数据的运动特性。利用背景知识对初始种群的个体进行优化,保证进化的良好基础和种群的多样性。将遗传算法和局部搜索技术结合,提高算法运行效率和求解质量。实验结果表明,该方法能够高效地从运动捕获数据中提取出最优的关键帧集合,较好地满足运动数据的紧致表示,且能高质量重构其它帧。     

基于拉普拉斯分值特征选择的运动捕获数据关键帧提取

针对已有的运动捕获数据关键帧提取方法常常忽略运动数据局部拓扑结构特性问题,提出了一种基于拉普拉斯分值LS特征选择的人体运动数据关键帧提取方法。该方法首先从原始运动数据集中提取两种代表性的特征向量并对其归一化,利用LS算法对组合后的特征向量进行打分和特征权重学习,以获取能够判别性揭示局部运动信息的特征子向量;其次,通过构建综合特征函数并基于极值判别原理,得到初始候选关键帧序列;最后,根据时间阈值约束和姿态相似判别策略,利用改进的k-means算法对候选帧进行聚类筛选,以达到去除冗余关键帧的目的,从而得到最终关键帧序列集合。仿真实验结果表明,该方法提取的关键帧序列具有典型性,能较好地对整体运动捕获数据进行视觉概括。     

从运动捕获数据中提取关键帧

在四元数表示下导出了两个旋转之间差异的一种简单形式，将人体各关节上总的旋转变化作为帧间距，设计了一种从运动捕获数据中提取关键帧的高效算法，通过向量的线性插值和四元数的球面线性插值重建了原始动画。实验表明，该算法具有很好的数据压缩效果，而且提取出的关键帧在视觉上对原始动画具有概括力。     

重建误差最优化的运动捕获数据关键帧提取

为了实现运动捕获数据的实时压缩,提出一种重建误差最优化的关键帧提取方法.定义重建误差作为关键帧有效性的度量标准;利用帧消减算法得到各消减帧的重建误差,并根据误差大小对消减帧进行排序;最后以消减帧排序为基础计算出重建误差曲线,并依据曲线确定最优压缩率来提取出相应数量的关键帧.实验结果表明,该方法能够高效地从运动捕获数据中提取出具有最优重建误差的关键帧,较好地满足了数据实时压缩的需要.     

Human motion capture data retrieval based on semantic thumbnail

We present a method for the efficient retrieval and browsing of immense amounts of realistic 3D human body motion capture data. The proposed method organizes motion capture data based on statistical K-means (SK–means), democratic decision making, unsupervised learning, and visual key frame extraction, thus achieving intuitive retrieval by browsing thumbnails of semantic key frames. We apply three steps for the efficient retrieval of motion capture data. The first is obtaining the basic type clusters by clustering motion capture data using the novel SK-means algorithm, and after which, immediately performing character matching. The second is learning the retrieval information of users during the retrieval process and updating the successful retrieval rate of each data; the search results are then ranked on the basis of successful retrieval rate by democratic decision making to improve accuracy. The last step is generating thumbnails with semantic generalization, which is conducted by using a novel key frame extraction algorithm based on visualized data analysis. The experiment demonstrates that this method can be utilised for the efficient organization and retrieval of enormous motion capture data.     

稀疏表达的运动数据压缩

随着运动数据越来越多地被应用于动画制作和科研领域,高效的运动数据压缩技术也逐渐成为一个热门的研究课题.基于稀疏表达提出一种新的运动数据有损压缩方法.首先对输入的运动数据进行分析生成稀疏表达字典;然后基于稀疏表达字典对运动数据中的每一帧进行稀疏线性表达;最后用K-SVD算法对字典和稀疏表示进行迭代优化.实验结果表明,本文方法可以达到较高的压缩比(50倍左右),同时保持原始运动数据的完整性,还原后可控制重建误差在肉眼不易分辨的范围内(平均RMS误差2.0以下),并且本文方法特别适用于对较短运动数据的压缩.     

Optimization-based posture reconstruction for digital human models

Digital human modeling provides a valuable tool for designers when implemented early in the design process. Motion capture experiments offer a means of validation of the digital human simulation models. However, there is a gap between the motion capture experiments and the simulation models, as the motion capture results are marker positions in Cartesian space and the simulation model is based on joint space. Therefore, it is necessary to map the motion capture data to simulation models by employing a posture reconstruction algorithm. Posture reconstruction is an inherently redundant problem where the collective distance error between experimental joint centers and simulation joint centers is minimized. This paper presents an optimization-based method for determining an accurate and efficient solution to the posture reconstruction problem. The procedure is used to recreate 120 experimental postures. For each posture, the algorithm minimizes the distance between the simulation model joint centers and the corresponding experimental subject joint centers which is called the mean measurement error.     

Video super-resolution reconstruction method based on deep Back projection and motion feature fusion

How to effectively utilize inter-frame redundancies is the key to improve the accuracy and speed of video super-resolution reconstruction methods. Previous methods usually process every frame in the whole video in the same way, and do not make full use of redundant information between frames, resulting in low accuracy or long reconstruction time. In this paper, we propose the idea of reconstructing key frames and non-key frames respectively, and give a video super-resolution reconstruction method based on deep back projection and motion feature fusion. Key-frame reconstruction subnet can obtain key frame features and reconstruction results with high accuracy. For non-key frames, key frame features can be reused by fusing them and motion features, so as to obtain accurate non-key frame features and reconstruction results quickly. Experiments on several public datasets show that the proposed method performs better than the state-of-the-art methods, and has good robustness.

     

Markerless human motion capture by Markov random field and dynamic graph cuts with color constraints

Currently, many vision-based motion capture systems require passive markers attached to key locations on the human body. However, such systems are intrusive with limited application. The algorithm that we use for human motion capture in this paper is based on Markov random field (MRF) and dynamic graph cuts. It takes full account of the impact of 3D reconstruction error and integrates human motion capture and 3D reconstruction into MRF-MAP framework. For more accurate and robust performance, we extend our algorithm by incorporating color constraints into the pose estimation process. The advantages of incorporating color constraints are demonstrated by experimental results on several video sequences. Supported by the National Basic Research Program of China (Grant No. 2006CB303105)     

A recognition-based motion capture baseline on the HumanEva II test data

The advent of the HumanEva standardized motion capture data sets has enabled quantitative evaluation of motion capture algorithms on comparable terms. This paper measures the performance of an existing monocular recognition-based pose recovery algorithm on select HumanEva data, including all the HumanEva II clips. The method uses a physically motivated Markov process to connect adjacent frames and achieve a 3D relative mean error of 8.9 cm per joint. It further investigates factors contributing to the error and finds that research into better pose retrieval methods offers promise for improvement of this technique and those related to it. Finally, it investigates the effects of local search optimization with the same recognition-based algorithm and finds no significant deterioration in the results, indicating that processing speed can be largely independent of the size of the recognition library for this approach.     

联合弹性碰撞与梯度追踪的WSNs压缩感知重构

为提高压缩感知(Compressed sensing,CS)大规模稀疏信号重构精度,提出了一种联合弹性碰撞优化与改进梯度追踪的WSNs(Wireless sensor networks)压缩感知重构算法.首先,创新地提出一种全新的智能优化算法|弹性碰撞优化算法(Elastic collision optimization algorithm,ECO),ECO模拟物理碰撞信息交互过程,利用自身历史最优解和种群最优解指导进化方向,并且个体以N(0,1)概率形式散落于种群最优解周围,在有效提升收敛速度的同时扩展了个体搜索空间,理论定性分析表明ECO依概率1收敛于全局最优解,而种群多样性指标分析证明了算法全局寻优能力.其次,针对贪婪重构算法高维稀疏信号重构效率低、稀疏度事先设定的缺陷,在设计重构有效性指数的基础上将ECO应用于压缩感知重构算法中,并引入拟牛顿梯度追踪策略,从而实现对大规模稀疏度未知数据的准确重构.最后,利用多维测试函数和WSNs数据采集环境进行仿真,仿真结果表明,ECO在收敛精度和成功率上具有一定优势,而且相比于其他重构算法,高维稀疏信号重构结果明显改善.     

基于少量关键序列帧的人体姿态识别方法

针对传统人体姿态识别数据采集易受环境干扰、难以解决人体运动姿态的相似性和 人体运动执行者的特征差异性等问题,提出一种基于少量关键序列帧的人体姿态识别方法。首先 对原有运动序列进行预选,通过运动轨迹取极值的方法构造初选关键帧序列,再利用帧消减算法 获取最终关键帧序列;然后对不同人体姿态分别建立隐马尔科夫模型,利用 Baum-Welch 算法计 算得到初始概率矩阵、混淆矩阵、状态转移矩阵,获得训练后模型;最后输入待测数据,应用前 向算法,得到对于每个模型的概率,比较并选取最大概率对应的姿态作为识别结果。实验结果表 明,该方法能够有效的选取原始运动序列的关键帧,提高人体姿态识别的准确性。     

基于时空域相关性的屏幕内容帧间快速编码算法

针对屏幕内容视频帧间编码的高复杂度问题,提出了一种基于时空域特性的帧间快速编码算法。首先,根据运动静止检测算法将待编码帧分为静止帧和运动帧;然后,对运动和静止帧分别采用不同的编码策略。对于静止帧,在统计分析时域对应编码单元（CU）分割深度和预测模式的基础上,确定CU最佳分割深度和最优预测模式。对于运动帧中的静止最大编码单元（LCU）,利用时域相关特性提前终止CU分割,模式选取则只针对大尺寸模式进行预测;对于运动帧中的运动LCU,根据其相邻LCU的运动静止特性确定CU分割深度以及预测模式。实验结果表明,所提算法相比原始编码平台,在BDBR平均上升3.65%的情况下,编码时间平均节省46.40%。所提算法在率失真性能损失可接受的前提下,有效地降低了屏幕内容视频帧间编码复杂度,有利于屏幕内容视频的实时应用。     

A system for the capture and synthesis of insect motion

We present an integrated system that enables the capture and synthesis of 3D motions of small scale dynamic creatures, typically insects and arachnids, in order to drive computer generated models. The system consists of a number of stages, initially, the acquisition of a multi-view calibration scene and synchronised video footage of a subject performing some action is carried out. A user guided labelling process, that can be semi-automated using tracking techniques and a 3D point generating algorithm, then enables a full metric calibration and captures the motions of specific points on the subject. The 3D motions extracted, which often come from a limited number of frames of the original footage, are then extended to generate potentially infinitely long, characteristic motion sequences for multiple similar subjects. Finally a novel path following algorithm is used to find optimal path along with coherent motion for synthetic subjects.     

Pixel-wise video stabilization

In this paper, we present a novel video stabilization method with a pixel-wise motion model. In order to avoid distortion introduced by traditional feature points based motion models, we focus on constructing a more accurate model to capture the motion in videos. By taking advantage of dense optical flow, we can obtain the dense motion field between adjacent frames and set up a pixel-wise motion model which is accurate enough. Our method first estimates dense motion field between adjacent frames. A PatchMatch based dense motion field estimation algorithm is proposed. This algorithm is specially designed for similar video frames rather than arbitrary images to reach higher speed and better performance. Then, a simple and fast smoothing algorithm is performed to make the jittered motion stabilized. After that, we warp input frames using a weighted average algorithm to construct the output frames. Some pixels in output frames may be still empty after the warping step, so in the last step, these empty pixels are filled using a patch based image completion algorithm. We test our method on many challenging videos and demonstrate the accuracy of our model and the effectiveness of our method.     

Structured Sparse Coding With the Group Log-regularizer for Key Frame Extraction

Key frame extraction based on sparse coding can reduce the redundancy of continuous frames and concisely express the entire video. However, how to develop a key frame extraction algorithm that can automatically extract a few frames with a low reconstruction error remains a challenge. In this paper, we propose a novel model of structured sparse-coding-based key frame extraction, wherein a nonconvex group log-regularizer is used with strong sparsity and a low reconstruction error. To automatically extract key frames, a decomposition scheme is designed to separate the sparse coefficient matrix by rows. The rows enforced by the nonconvex group log-regularizer become zero or nonzero, leading to the learning of the structured sparse coefficient matrix. To solve the nonconvex problems due to the log-regularizer, the difference of convex algorithm (DCA) is employed to decompose the log-regularizer into the difference of two convex functions related to the l₁ norm, which can be directly obtained through the proximal operator. Therefore, an efficient structured sparse coding algorithm with the group log-regularizer for key frame extraction is developed, which can automatically extract a few frames directly from the video to represent the entire video with a low reconstruction error. Experimental results demonstrate that the proposed algorithm can extract more accurate key frames from most SumMe videos compared to the state-of-the-art methods. Furthermore, the proposed algorithm can obtain a higher compression with a nearly 18% increase compared to sparse modeling representation selection (SMRS) and an 8% increase compared to SC-det on the VSUMM dataset.      

分布式视频编码中关键帧丢失错误保护

目的 分布式视频编码较其传统视频编码具有编码简单、误码鲁棒性高等特点,可以很好地满足如无人机航拍、无线监控等新型视频业务的需求。在分布式视频编码中,视频图像被交替分为关键帧和Wyner-Ziv帧,由于受到信道衰落和干扰等因素的影响,采用传统帧内编码方式的关键帧的误码鲁棒性远不如基于信道编码的Wyner-Ziv帧。关键帧能否正确传输和解码对于Wyner-Ziv帧能否正确解码起着决定性的作用,进而影响着整个系统的压缩效率和率失真性能。为此针对关键帧在异构网络中的鲁棒性传输问题,提出一种基于小波域的关键帧质量可分级保护传输方案。方法 在编码端对关键帧同时进行传统的帧内视频编码和基于小波域的Wyner-Ziv编码,解码端将经过错误隐藏后的误码关键帧作为基本层,Wyner-Ziv编码产生的校验信息码流作为增强层。为了提高系统的分层特性以便使系统的码率适应不同的网络条件,进一步将小波分解后图像的各个不同层的低频带和高频带组合成不同的增强层,根据不同信道环境,传输不同层的Wyner-Ziv校验数据。同时对误码情况下关键帧的虚拟噪声模型进行了改进,利用第1个增强层已解码重建的频带与其对应边信息来获得第2个和第3个增强层对应频带的更加符合实际的虚拟信道模型的估计。结果 针对不同的视频序列在关键帧误码率为1%20%时,相比较于传统的帧内错误隐藏算法,所提方案可以提高视频重建图像的主观质量和整体系统的率失真性能。例如在关键帧误码率为5%时,通过传输第1个增强层,不同的视频序列峰值信噪比（PSNR）提升可达25 dB左右;如果继续传输第2个增强层的校验信息,视频图像的PSNR也可以提升0.51.6 dB左右;如果3个增强层的校验信息都传输的话,基本上可以达到无误码情况下关键帧的PSNR。结论 本文所提方案可以很好地解决分布式视频编码系统中的关键帧在实际信道传输过程中可能出现的误码问题,同时采用的分层传输方案可以适应不同网络的信道情况。     

Hyperspectral Inverse Skinning

In example-based inverse linear blend skinning (LBS), a collection of poses (e.g. animation frames) are given, and the goal is finding skinning weights and transformation matrices that closely reproduce the input. These poses may come from physical simulation, direct mesh editing, motion capture or another deformation rig. We provide a re-formulation of inverse skinning as a problem in high-dimensional Euclidean space. The transformation matrices applied to a vertex across all poses can be thought of as a point in high dimensions. We cast the inverse LBS problem as one of finding a tight-fitting simplex around these points (a well-studied problem in hyperspectral imaging). Although we do not observe transformation matrices directly, the 3D position of a vertex across all of its poses defines an affine subspace, or flat. We solve a ‘closest flat’ optimization problem to find points on these flats, and then compute a minimum-volume enclosing simplex whose vertices are the transformation matrices and whose barycentric coordinates are the skinning weights. We are able to create LBS rigs with state-of-the-art reconstruction error and state-of-the-art compression ratios for mesh animation sequences. Our solution does not consider weight sparsity or the rigidity of recovered transformations. We include observations and insights into the closest flat problem. Its ideal solution and optimal LBS reconstruction error remain an open problem.     

RGBD融合明暗恢复形状的全视角三维重建技术研究

为了高效、高精度、低成本地实现对物体的全视角三维重建, 提出一种使用深度相机融合光照约束实现全视角三维重建的方法。该重建方法中,在进行单帧重建时采用RGBD深度图像融合明暗恢复形状(Shape from shading,SFS)的重建方法, 即在原有的深度数据上加上额外的光照约束来优化深度值; 在相邻两帧配准时, 采用快速点特征直方图（Fast point feature histograms, FPFH）特征进行匹配并通过随机采样一致性（Random sample consensus, RANSAC）滤除错误的匹配点对求解粗配准矩阵, 然后通过迭代最近点（Iterative closest point, ICP）算法进行精配准得出两帧间的配准矩阵; 在进行全视角的三维重建时, 采用光束平差法优化相机位姿, 从而消除累积误差使首尾帧完全重合, 最后融合生成一个完整的模型。该方法融入了物体表面的光照信息,因此生成的三维模型更为光顺,也包含了更多物体表面的细节信息,提高了重建精度;同时该方法仅通过单张照片就能在自然光环境下完成对多反射率三维物体的重建,适用范围更广。本文方法的整个实验过程通过手持深度相机就能完成,不需要借助转台,操作更加方便。