Study on eye gaze estimation

There are two components to the human visual line-of-sight: pose of human head and the orientation of the eye within their sockets. We have investigated these two aspects but will concentrate on eye gaze estimation. We present a novel approach called the "one-circle" algorithm for measuring the eye gaze using a monocular image that zooms in on only one eye of a person. Observing that the iris contour is a circle, we estimate the normal direction of this iris circle, considered as the eye gaze, from its elliptical image. From basic projective geometry, an ellipse can be back-projected into space onto two circles of different orientations. However, by using a geometric constraint, namely, that the distance between the eyeball's center and the two eye corners should be equal to each other, the correct solution can be disambiguated. This allows us to obtain a higher resolution image of the iris with a zoom-in camera, thereby achieving higher accuracies in the estimation. A general approach that combines head pose determination with eye gaze estimation is also proposed. The searching of the eye gaze is guided by the head pose information. The robustness of our gaze determination approach was verified statistically by the extensive experiments on synthetic and real image data. The two key contributions are that we show the possibility of finding the unique eye gaze direction from a single image of one eye and that one can obtain better accuracy as a consequence of this.     

An integrated head pose and eye gaze tracking approach to non-intrusive visual attention measurement for wide FOV simulators

Eye gaze tracking is very useful for quantitatively measuring visual attention in virtual environments. However, most eye trackers have a limited tracking range, e.g., ±35° in the horizontal direction. This paper proposed a method to combine head pose tracking and eye gaze tracking together to achieve a large range of tracking in virtual driving simulation environments. Multiple parallel multilayer perceptrons were used to reconstruct the relationship between head images and head poses. Head images were represented with the coefficients extracted from Principal Component Analysis. Eye gaze tracking provides precise results on the front view, while head pose tracking is more suitable for tracking areas of interest than for tracking points of interest on the side view.     

人体自由运动状态下的视线追踪算法研究*

针对人体在大空间范围内自由运动时视线方向难以追踪的问题,构建了一套基于光学跟踪设备的头戴式视线追踪系统。系统通过被动式光学追踪设备和头戴式眼部摄像机获取使用者的头部运动状态与眼部图像,然后依据初始标定结果来估计使用者自由运动状态下的视线方向;最后对系统进行简化,得到了适用于同类环境、与具体硬件设备无关的视线跟踪三点三面三变换几何模型。对系统进行应用实验和误差分析表明,使用者在3.0 * 3.2 * 2.0 m的大工作空间内自由运动时视线追踪误差为1.69度,频率为20赫兹。     

基于表观的归一化坐标系分类视线估计方法

视线估计能够反映人的关注焦点,对理解人类的情感、兴趣等主观意识有重要作用。但目前用于视线估计的单目眼睛图像容易因头部姿态的变化而失真,导致视线估计的准确性下降。提出一种新型分类视线估计方法,利用三维人脸模型与单目相机的内在参数,通过人脸的眼睛与嘴巴中心的三维坐标形成头部姿态坐标系,从而合成相机坐标系与头部姿态坐标系,并建立归一化坐标系,实现相机坐标系的校正。复原并放大归一化得到的灰度眼部图像,建立基于表观的卷积神经网络模型分类方法以估计视线方向,并利用黄金分割法优化搜索,进一步降低误差。在MPIIGaze数据集上的实验结果表明,相比已公开的同类算法,该方法能降低约7.4%的平均角度误差。     

What Are You Looking at?

In this paper we present a novel mechanism to obtain enhanced gaze estimation for subjects looking at a scene or an image. The system makes use of prior knowledge about the scene (e.g. an image on a computer screen), to define a probability map of the scene the subject is gazing at, in order to find the most probable location. The proposed system helps in correcting the fixations which are erroneously estimated by the gaze estimation device by employing a saliency framework to adjust the resulting gaze point vector. The system is tested on three scenarios: using eye tracking data, enhancing a low accuracy webcam based eye tracker, and using a head pose tracker. The correlation between the subjects in the commercial eye tracking data is improved by an average of 13.91%. The correlation on the low accuracy eye gaze tracker is improved by 59.85%, and for the head pose tracker we obtain an improvement of 10.23%. These results show the potential of the system as a way to enhance and self-calibrate different visual gaze estimation systems.     

Model free head pose estimation using stereovision

In this paper we present a stereovision based model free 3D head pose (orientation and position) estimation system suitable for human–machine interface applications. The system works by obtaining a ‘face plane’ from the 3D reconstructed face data, which is then used for head pose estimation. The key novelty in this work is the utilization of the face plane together with the eye locations on the reconstructed face data to obtain a robust head pose estimate. This approach leads to a model and initialization free head pose estimation system; therefore it is suitable for natural human–machine interfaces. In order to quantitatively asses the accuracy of the system for such applications, several evaluation experiments were conducted using a commercial motion capture system. The evaluation results indicate that this system can be used in human–computer and human–robot applications.     

A neural-based remote eye gaze tracker under natural head motion

A novel approach to view-based eye gaze tracking for human computer interface (HCI) is presented. The proposed method combines different techniques to address the problems of head motion, illumination and usability in the framework of low cost applications. Feature detection and tracking algorithms have been designed to obtain an automatic setup and strengthen the robustness to light conditions. An extensive analysis of neural solutions has been performed to deal with the non-linearity associated with gaze mapping under free-head conditions. No specific hardware, such as infrared illumination or high-resolution cameras, is needed, rather a simple commercial webcam working in visible light spectrum suffices. The system is able to classify the gaze direction of the user over a 15-zone graphical interface, with a success rate of 95% and a global accuracy of around 2 degrees , comparable with the vast majority of existing remote gaze trackers.     

Human head pose estimation based on HF method

This Paper addresses the problem of head pose estimation. Driving assistance technology utilizes head pose estimation as an indicator for visual focus and mental attention of the driver. Head pose estimation detects head orientation with respect to the camera. Model based and appearance-based methods are the two approaches in head pose estimation. The first approach uses the facial features to create a face geometrical models whereas the second method only takes into consideration the entire face image. The proposed appearance-based method work is performed using Hough transform and random forest to classify ninety-three classes of Hough values in order to find the exact head pose. The performance of the proposed work is evaluated based on accuracy and the time taken to detect the head pose. The paper outperforms many other previous works.     

Unseen head pose prediction using dense multivariate label distribution

Accurate head poses are useful for many face-related tasks such as face recognition, gaze estimation, and emotion analysis. Most existing methods estimate head poses that are included in the training data (i.e., previously seen head poses). To predict head poses that are not seen in the training data, some regression-based methods have been proposed. However, they focus on estimating continuous head pose angles, and thus do not systematically evaluate the performance on predicting unseen head poses. In this paper, we use a dense multivariate label distribution (MLD) to represent the pose angle of a face image. By incorporating both seen and unseen pose angles into MLD, the head pose predictor can estimate unseen head poses with an accuracy comparable to that of estimating seen head poses. On the Pointing’04 database, the mean absolute errors of results for yaw and pitch are 4.01° and 2.13°, respectively. In addition, experiments on the CAS-PEAL and CMU Multi-PIE databases show that the proposed dense MLD-based head pose estimation method can obtain the state-of-the-art performance when compared to some existing methods.     

基于改进头姿估计方法的机器人轮椅交互控制

针对现有迭代最近点（ICP）头姿估计算法存在迭代次数偏多且易陷于局部最优、而随机森林（RF）头姿估计算法准确性和稳定性不高的问题,提出一种新的头姿估计改进方法,并基于该改进方法构建机器人轮椅实时交互控制接口．首先,分析现有迭代最近点头姿算法与随机森林头姿算法在准确性、实时性及稳定性方面存在的问题,并提出一种新的基于随机森林与迭代最近点算法融合的头姿估计改进方法;其次,为实现头姿估计到机器人轮椅交互控制的无缝连接,建立基于传统机器人轮椅操纵杆的头部姿态运动空间映射;最后,在基于标准头姿数据库分析改进头姿估计方法性能的基础上,构建机器人轮椅实验平台并规划运动轨迹,以进一步验证基于改进头姿估计方法的人机交互接口在机器人轮椅实时控制方面的有效性．实验结果表明,改进后的头姿估计方法较传统迭代最近点算法减少了迭代次数且避免了陷于局部最优,在仅增加少量运算时间的基础上,其准确性和稳定性都优于传统随机森林算法;同时,基于改进头姿估计方法的人机交互接口亦能实时平稳地控制机器人轮椅沿既定的轨迹运动．     

使用人眼几何特征的视线追踪方法

目的 视线追踪是人机交互的辅助系统,针对传统的虹膜定位方法误判率高且耗时较长的问题,本文提出了一种基于人眼几何特征的视线追踪方法,以提高在2维环境下视线追踪的准确率。方法 首先通过人脸定位算法定位人脸位置,使用人脸特征点检测的特征点定位眼角点位置,通过眼角点计算出人眼的位置。直接使用虹膜中心定位算法的耗时较长,为了使虹膜中心定位的速度加快,先利用虹膜图片建立虹膜模板,然后利用虹膜模板检测出虹膜区域的位置,通过虹膜中心精定位算法定位虹膜中心的位置,最后提取出眼角点、虹膜中心点等信息,对点中包含的角度信息、距离信息进行提取,组合成眼动向量特征。使用神经网络模型进行分类,建立注视点映射关系,实现视线的追踪。通过图像的预处理对图像进行增强,之后提取到了相对的虹膜中心。提取到需要的特征点,建立相对稳定的几何特征代表眼动特征。结果 在普通的实验光照环境中,头部姿态固定的情况下,识别率最高达到98.9%,平均识别率达到95.74%。而当头部姿态在限制区域内发生变化时,仍能保持较高的识别率,平均识别率达到了90%以上。通过实验分析发现,在头部变化的限制区域内,本文方法具有良好的鲁棒性。结论 本文提出使用模板匹配与虹膜精定位相结合的方法来快速定位虹膜中心,利用神经网络来对视线落点进行映射,计算视线落点区域,实验证明本文方法具有较高的精度。     

Recognizing Visual Focus of Attention From Head Pose in Natural Meetings

We address the problem of recognizing the visual focus of attention (VFOA) of meeting participants based on their head pose. To this end, the head pose observations are modeled using a Gaussian mixture model (GMM) or a hidden Markov model (HMM) whose hidden states correspond to the VFOA. The novelties of this paper are threefold. First, contrary to previous studies on the topic, in our setup, the potential VFOA of a person is not restricted to other participants only. It includes environmental targets as well (a table and a projection screen), which increases the complexity of the task, with more VFOA targets spread in the pan as well as tilt gaze space. Second, we propose a geometric model to set the GMM or HMM parameters by exploiting results from cognitive science on saccadic eye motion, which allows the prediction of the head pose given a gaze target. Third, an unsupervised parameter adaptation step not using any labeled data is proposed, which accounts for the specific gazing behavior of each participant. Using a publicly available corpus of eight meetings featuring four persons, we analyze the above methods by evaluating, through objective performance measures, the recognition of the VFOA from head pose information obtained either using a magnetic sensor device or a vision-based tracking system. The results clearly show that in such complex but realistic situations, the VFOA recognition performance is highly dependent on how well the visual targets are separated for a given meeting participant. In addition, the results show that the use of a geometric model with unsupervised adaptation achieves better results than the use of training data to set the HMM parameters.     

A model of head movement contribution for gaze transitions

Head posture has been associated with work-related neck symptoms and discomfort, but its relationship with visual tasks has received much less attention. Head movement amplitude is normally a fraction of the angular distance to a visual target, as gaze transition is usually achieved through the combination of both head and eye movement. In this study, the proportion of head orientation vs. target orientation, named head movement contribution ratio (HMCR), was quantified and modelled as a function of target location. Head movements were measured on subjects orienting and maintaining gaze for 2 s at randomly presented visual targets distributed along an arc placed horizontally or vertically. Bootstrap regression models showed that the horizontal HMCR was approximately 69% of target azimuth. The vertical HMCR was bilinear and corresponded to 52% and 8% and of target elevation for targets above and below the horizontal plane, respectively. The data also demonstrated that head orientation is affected by the kinematic coupling between horizontal and vertical components of head movement.

Statement of Relevance: Awkward head and neck posture is a risk factor for work-related musculoskeletal disorders. This study investigated the influence of visual target location on head orientation over a large range of target eccentricity, as an attempt to predict the head and neck posture required for visual target detection and identification.     

A real-time gaze position estimation method based on a 3-D eye model.

This paper proposes a new gaze-detection method based on a 3-D eye position and the gaze vector of the human eyeball. Seven new developments compared to previous works are presented. First, a method of using three camera systems, i.e., one wide-view camera and two narrow-view cameras, is proposed. The narrow-view cameras use autozooming, focusing, panning, and tilting procedures (based on the detected 3-D eye feature position) for gaze detection. This allows for natural head and eye movement by users. Second, in previous conventional gaze-detection research, one or multiple illuminators were used. These studies did not consider specular reflection (SR) problems, which were caused by the illuminators when working with users who wore glasses. To solve this problem, a method based on dual illuminators is proposed in this paper. Third, the proposed method does not require user-dependent calibration, so all procedures for detecting gaze position operate automatically without human intervention. Fourth, the intrinsic characteristics of the human eye, such as the disparity between the pupillary and the visual axes in order to obtain accurate gaze positions, are considered. Fifth, all the coordinates obtained by the left and right narrow-view cameras, as well as the wide-view camera coordinates and the monitor coordinates, are unified. This simplifies the complex 3-D converting calculation and allows for calculation of the 3-D feature position and gaze position on the monitor. Sixth, to upgrade eye-detection performance when using a wide-view camera, the adaptive-selection method is used. This involves an IR-LED on/off scheme, an AdaBoost classifier, and a principle component analysis method based on the number of SR elements. Finally, the proposed method uses an eigenvector matrix (instead of simply averaging six gaze vectors) in order to obtain a more accurate final gaze vector that can compensate for noise. Experimental results show that the root mean square error of gaze detection was about 0.627 cm on a 19-in monitor. The processing speed of the proposed method (used to obtain the gaze position on the monitor) was 32 ms (using a Pentium IV 1.8-GHz PC). It was possible to detect the user's gaze position at real-time speed.     

A novel approach to 3-D gaze tracking using stereo cameras

A novel approach to three-dimensional (3-D) gaze tracking using 3-D computer vision techniques is proposed in this paper. This method employs multiple cameras and multiple point light sources to estimate the optical axis of user's eye without using any user-dependent parameters. Thus, it renders the inconvenient system calibration process which may produce possible calibration errors unnecessary. A real-time 3-D gaze tracking system has been developed which can provide 30 gaze measurements per second. Moreover, a simple and accurate calibration method is proposed to calibrate the gaze tracking system. Before using the system, each user only has to stare at a target point for a few (2-3) seconds so that the constant angle between the 3-D line of sight and the optical axis can be estimated. The test results of six subjects showed that the gaze tracking system is very promising achieving an average estimation error of under 1 degrees.     

基于瞳孔角膜反射技术的视线估计方法

作为信息获取与人机交互的一种新型方式,视线跟踪技术已经成为计算机视觉领域的热门研究方向。视线跟踪的核心技术是视线估计。针对现有视线估计方法标定复杂、限制头部运动等问题,提出了一种改进的基于二维瞳孔角膜反射技术的视线估计方法。在单相机单光源条件下,通过建立瞳孔角膜反射模型、补偿个体差异误差、补偿头部运动误差等步骤实现单点标定视线估计。实验结果表明,用该算法估计视线,在一定范围内,头部移动不会带来精度的明显下降。     

Accurate and fast 3D head pose estimation with noisy RGBD images

Head pose estimation plays an essential role in many high-level face analysis tasks. However, accurate and robust pose estimation with existing approaches remains challenging. In this paper, we propose a novel method for accurate three-dimensional (3D) head pose estimation with noisy depth maps and high-resolution color images that are typically produced by popular RGBD cameras such as the Microsoft Kinect. Our method combines the advantages of the high-resolution RGB image with the 3D information of the depth image. For better accuracy and robustness, features are first detected using only the color image, and then the 3D feature points used for matching are obtained by combining depth information. The outliers are then filtered with depth information using rules proposed for depth consistency, normal consistency, and re-projection consistency, which effectively eliminate the influence of depth noise. The pose parameters are then iteratively optimized using the Extended LM (Levenberg-Marquardt) method. Finally, a Kalman filter is used to smooth the parameters. To evaluate our method, we built a database of more than 10K RGBD images with ground-truth poses recorded using motion capture. Both qualitative and quantitative evaluations show that our method produces notably smaller errors than previous methods.     

Single image face orientation and gaze detection

We introduce a system to compute both head orientation and gaze detection from a single image. The system uses a camera with fixed parameters and requires no user calibration. Our approach to head orientation is based on a geometrical model of the human face, and is derived form morphological and physiological data. Eye gaze detection is based on a geometrical model of the human eye. Two new algorithms are introduced that require either two or three feature points to be extracted from each image. Our algorithms are robust and run in real-time on a typical PC, which makes our system useful for a large variety of needs, from driver attention monitoring to machine-human interaction.