Semi-Supervised Learning on Riemannian Manifolds

We consider the general problem of utilizing both labeled and unlabeled data to improve classification accuracy. Under the assumption that the data lie on a submanifold in a high dimensional space, we develop an algorithmic framework to classify a partially labeled data set in a principled manner. The central idea of our approach is that classification functions are naturally defined only on the submanifold in question rather than the total ambient space. Using the Laplace-Beltrami operator one produces a basis (the Laplacian Eigenmaps) for a Hilbert space of square integrable functions on the submanifold. To recover such a basis, only unlabeled examples are required. Once such a basis is obtained, training can be performed using the labeled data set. Our algorithm models the manifold using the adjacency graph for the data and approximates the Laplace-Beltrami operator by the graph Laplacian. We provide details of the algorithm, its theoretical justification, and several practical applications for image, speech, and text classification.     

Intelligent Learning for Deformable Object Manipulation

The majority of manipulation systems are designed with the assumption that the objects being handled are rigid and do not deform when grasped. This paper addresses the problem of robotic grasping and manipulation of 3-D deformable objects, such as rubber balls or bags filled with sand. Specifically, we have developed a generalized learning algorithm for handling of 3-D deformable objects in which prior knowledge of object attributes is not required and thus it can be applied to a large class of object types. Our methodology relies on the implementation of two main tasks. Our first task is to calculate deformation characteristics for a non-rigid object represented by a physically-based model. Using nonlinear partial differential equations, we model the particle motion of the deformable object in order to calculate the deformation characteristics. For our second task, we must calculate the minimum force required to successfully lift the deformable object. This minimum lifting force can be learned using a technique called iterative lifting. Once the deformation characteristics and the associated lifting force term are determined, they are used to train a neural network for extracting the minimum force required for subsequent deformable object manipulation tasks. Our developed algorithm is validated with two sets of experiments. The first experimental results are derived from the implementation of the algorithm in a simulated environment. The second set involves a physical implementation of the technique whose outcome is compared with the simulation results to test the real world validity of the developed methodology.     

Robust Shape Control of Deformable Objects Using Model-Based Techniques

Shape control of a deformable object by a robotic system is a challenging problem because of the difficulty of imposing shape change by a finite number of actuation points to an essentially infinite-dimensional object. In this paper, a new approach to shape changing of deformable objects by a system of manipulators is presented. First, an integrated dynamic equation of motion for a system of multiple manipulators handling a deformable object is developed. A shape correspondence between the initial contact points of the multiple manipulators on a deformable object and a two-dimensional curve that represents the final desired shape is determined. A shape Jacobian that contains the local shape information of the desired shape of the object is formulated and is introduced into the control law. We develop a shape estimator with a second-order dynamics that is used to estimate the curve parameters corresponding to the end-effector position in each time step as the initial object is deformed to its desired final shape. Finally, we design a robust controller for the shape changing task that can work in the presence of modeling uncertainty. The simulation results demonstrate the efficacy of the proposed method.     

Autonomous Shape Control of a Deformable Object by Multiple Manipulators

Shape control of a deformable object by a robotic system is a challenging problem because of the difficulty of imposing shape change by a finite number actuation points to an essentially infinite dimensional object. In this paper, a new approach to shape changing of deformable objects by a system of manipulators is presented. First, an integrated dynamic equation of motion for a system of multiple manipulators handling a deformable object is developed. The initial and the final shapes of the deformable object are specified by curves that represent the boundary of the object. We design an optimization-based planner that minimizes an energy-like criterion to determine the locations of the contact points on the desired curve representing the final shape of the object. The motion of each manipulator is controlled independently without any communication between them. Finally we design a robust controller for shape changing that can work in the presence of modeling uncertainty. The simulation results demonstrate the efficacy of the proposed method.     

Deformable 3D Shape Registration Based on Local Similarity Transforms

In this paper, a new method for deformable 3D shape registration is proposed. The algorithm computes shape transitions based on local similarity transforms which allows to model not only as‐rigid‐as‐possible deformations but also local and global scale. We formulate an ordinary differential equation (ODE) which describes the transition of a source shape towards a target shape. We assume that both shapes are roughly pre‐aligned (e.g., frames of a motion sequence). The ODE consists of two terms. The first one causes the deformation by pulling the source shape points towards corresponding points on the target shape. Initial correspondences are estimated by closest‐point search and then refined by an efficient smoothing scheme. The second term regularizes the deformation by drawing the points towards locally defined rest positions. These are given by the optimal similarity transform which matches the initial (undeformed) neighborhood of a source point to its current (deformed) neighborhood. The proposed ODE allows for a very efficient explicit numerical integration. This avoids the repeated solution of large linear systems usually done when solving the registration problem within general‐purpose non‐linear optimization frameworks. We experimentally validate the proposed method on a variety of real data and perform a comparison with several state‐of‐the‐art approaches.     

Unsupervised Learning of Image Manifolds by Semidefinite Programming

Can we detect low dimensional structure in high dimensional data sets of images? In this paper, we propose an algorithm for unsupervised learning of image manifolds by semidefinite programming. Given a data set of images, our algorithm computes a low dimensional representation of each image with the property that distances between nearby images are preserved. More generally, it can be used to analyze high dimensional data that lies on or near a low dimensional manifold. We illustrate the algorithm on easily visualized examples of curves and surfaces, as well as on actual images of faces, handwritten digits, and solid objects.     

一种迁移学习和可变形卷积深度学习的蝴蝶检测算法

针对自然生态蝴蝶多种特征检测的实际需求,以及生态环境下蝴蝶检测效率低、精度差问题,本文提出了一种基于迁移学习和可变形卷积深度神经网络的蝴蝶检测算法（Transfer learning and deformable convolution deep learning network,TDDNET）.该算法首先使用可变形卷积模型重建ResNet-101卷积层,强化特征提取网络对蝴蝶特征的学习,并以此结合区域建议网络（Region proposal network,RPN）构建二分类蝴蝶检测网络,以下简称DNET-base;然后在DNET-base的模型上,构建RPN网络来指导可变形的敏感位置兴趣区域池化层,以便获得多尺度目标的评分特征图和更准确的位置,再由弱化非极大值抑制（Soft non-maximum suppression,Soft-NMS）精准分类形成TDDNET模型.随后通过模型迁移,将DNET-base训练参数迁移至TDDNET,有效降低数据分布不均造成的训练困难与检测性能差的影响,再由Fine-tuning方式快速训练TDDNET多分类网络,最终实现了对蝴蝶的精确检测.所提算法在854张蝴蝶测试集上对蝴蝶检测结果的mAP0.5为0.9414、mAP0.7为0.9235、检出率DR为0.9082以及分类准确率ACC为0.9370,均高于在同等硬件配置环境下的对比算法.对比实验表明,所提算法对生态照蝴蝶可实现较高精度的检测.     

Kernel-Based Subspace Learning on Riemannian Manifolds for Visual Recognition

Neural Processing Letters - Covariance matrices have attracted increasing attention for data representation in many computer vision tasks. The nonsingular covariance matrices are regarded as points...     

Variational Segmentation of Image Sequences Using Region-Based Active Contours and Deformable Shape Priors

In this paper we address the problem of segmentation in image sequences using region-based active contours and level set methods. We propose a novel method for variational segmentation of image sequences containing nonrigid, moving objects. The method is based on the classical Chan-Vese model augmented with a novel frame-to-frame interaction term, which allow us to update the segmentation result from one image frame to the next using the previous segmentation result as a shape prior. The interaction term is constructed to be pose-invariant and to allow moderate deformations in shape. It is expected to handle the appearance of occlusions which otherwise can make segmentation fail. The performance of the model is illustrated with experiments on synthetic and real image sequences.     

Geometric Flows of Curves in Shape Space for Processing Motion of Deformable Objects

We introduce techniques for the processing of motion and animations of non‐rigid shapes. The idea is to regard animations of deformable objects as curves in shape space. Then, we use the geometric structure on shape space to transfer concepts from curve processing in ?ⁿ to the processing of motion of non‐rigid shapes. Following this principle, we introduce a discrete geometric flow for curves in shape space. The flow iteratively replaces every shape with a weighted average shape of a local neighborhood and thereby globally decreases an energy whose minimizers are discrete geodesics in shape space. Based on the flow, we devise a novel smoothing filter for motions and animations of deformable shapes. By shortening the length in shape space of an animation, it systematically regularizes the deformations between consecutive frames of the animation. The scheme can be used for smoothing and noise removal, e.g., for reducing jittering artifacts in motion capture data. We introduce a reduced‐order method for the computation of the flow. In addition to being efficient for the smoothing of curves, it is a novel scheme for computing geodesics in shape space. We use the scheme to construct non‐linear “Bézier curves” by executing de Casteljau's algorithm in shape space.     

Deformable Pedal Curves and Surfaces: Hybrid Geometric Active Models for Shape Recovery

In this paper, we propose significant extensions to the snake pedal model, a powerful geometric shape modeling scheme introduced in (Vemuri and Guo, 1998). The extension allows the model to automatically cope with topological changes and for the first time, introduces the concept of a compact global shape into geometric active models. The ability to characterize global shape of an object using very few parameters facilitates shape learning and recognition. In this new modeling scheme, object shapes are represented using a parameterized function—called the generator—which accounts for the global shape of an object and the pedal curve (surface) of this global shape with respect to a geometric snake to represent any local detail. Traditionally, pedal curves (surfaces) are defined as the loci of the feet of perpendiculars to the tangents of the generator from a fixed point called the pedal point. Local shape control is achieved by introducing a set of pedal points—lying on a snake—for each point on the generator. The model dubbed as a snake pedal allows for interactive manipulation via forces applied to the snake. In this work, we replace the snake by a geometric snake and derive all the necessary mathematics for evolving the geometric snake when the snake pedal is assumed to evolve as a function of its curvature. Automatic topological changes of the model may be achieved by implementing the geometric snake in a level-set framework. We demonstrate the applicability of this modeling scheme via examples of shape recovery from a variety of 2D and 3D image data.     

End-to-End Learning of Latent Deformable Part-Based Representations for Object Detection

International Journal of Computer Vision - Object detection methods usually represent objects through rectangular bounding boxes from which they extract features, regardless of their actual shapes....     

基于形变模型由立体序列图象恢复物体的3D形状

结合立体视觉和形变模型提出了一种新的物体3D形状的恢复方法。采用立体视觉方法导出物体表面的3D坐标;利用光流模型估计物体的3D运动,根据此运动移动形变模型,使其对准物体的表面块;由形变模型将由各幅图象得到的离散的3D点融为一起,得到物体的表面形状。实验结果表明该方法能用于形状复杂的物体恢复。     

Simultaneous Learning of Nonlinear Manifolds Based on the Bottleneck Neural Network

Manifold learning methods are important techniques for nonlinear extraction of high-dimensional data structures. These methods usually extract a global manifold for data. However, in many real-world problems, there is not only one global manifold, but also additional information about the objects is shared by a large number of manifolds. These manifolds can share information for data reconstruction. To simultaneously extract these data manifolds, this paper proposes a nonlinear method based on the deep neural network (NN) named nonlinear manifold separator NN (NMSNN). Unlike unsupervised learning of bottleneck NN, data labels were used for simultaneous manifold learning. This paper makes use of NMSNN for extracting both expression and identity manifolds for facial images of the CK+ database. These manifolds have been evaluated by different metrics. The identity manifold is used for changing image identity. The result of identity recognition by K-nearest neighbor classifier shows that virtual identities are exactly sanitized. The virtual images for different expressions of test subjects are generated by expression manifold. The facial expression recognition rate of 92.86 % is achieved for virtual expressions of test persons. It is shown that NMSNN can be used to enrich datasets by sanitizing virtual images. As a result, 8 and 19 % improvements are gained in the face recognition task by a single image of each person on CK+ and Bosphorus databases, respectively.     

Inference and Learning with Hierarchical Shape Models

In this work we introduce a hierarchical representation for object detection. We represent an object in terms of parts composed of contours corresponding to object boundaries and symmetry axes; these are in turn related to edge and ridge features that are extracted from the image.     

Learning to Caricature via Semantic Shape Transform

International Journal of Computer Vision - Caricature is an artistic drawing created to abstract or exaggerate facial features of a person. Rendering visually pleasing caricatures is a difficult...     

基于期望首达时间的形状距离学习算法

由于逐对形状匹配不能很好地反映形状间相似度,因此需要引入后期处理步骤提升检索精度. 为了得到上下文敏感的形状相似度,本文提出了一种基于期望首达时间（Mean first-passage time,MFPT）的形状距离学习方法. 在利用标准形状匹配方法得到距离矩阵的基础上,建立离散时间马尔可夫链对形状流形结构进行分析.将形状样本视作状态,利用不同状态之间完成一次状态转移的平均时间步长,即期望首达时间,表示形状间的距离.期望首达时间能够结合测地距离发掘空间流形结构,并可以通过线性方程进行有效求解.分别对不同数据进行实验分析,本文所提出的方法在相同条件下能够达到更高的形状检索精度.