足球场地标志线的自动提取

介绍一种从足球比赛录像中自动提取禁区线和球门线的方法．作为足球比赛场景三维重建和视频解说系统的一部分，提取的结果可用于摄像机定标和球场的重建．使用数学形态学、Hough变换、最小二乘法等多种方法，可以从复杂的足球比赛录像中自动提取出所有禁区线和球门线的直线参数．实验结果表明，文中方法对类似背景的直线提取非常有效，可满足摄像机定标和球场重建的需要．     

Automatic scene calibration for detecting and tracking people using a single camera

Segmenting and tracking of people is an important aim in the video analysis with multitudinous applications. The scene calibration enables the system to process the input video in a different way depending on the camera position and the scene characteristics for arising the successful results. In complex situations there exists extended occlusions, shadows and/or rejections, so that an appropriate calibration is required in order to achieve a highly developed people's segmentation as well as a tracking algorithm. In the majority of cases, once the system has been installed in a certain scene, it is difficult to obtain the calibration information of the scene. In this paper, an automatic method to calibrate the scene for detecting and tracking people systems is presented based on measurements of video sequences captured from a stationary camera.     

A Video-Based 3D-Reconstruction of Soccer Games

In this paper we present SoccerMan, a reconstruction system designed to generate animated, virtual 3D views from two synchronous video sequences of a short part of a given soccer game. After the reconstruction process, which needs also some manual interaction, the virtual 3D scene can be examined and 'replayed' from any viewpoint. Players are modeled as so-called animated texture objects, i.e. 2D player shapes are extracted from video and texture-mapped onto rectangles in 3D space. Animated texture objects have shown very appropriate as a 3D representation of soccer players in motion, as the visual nature of the original human motion is preserved. The trajectories of the players and the ball in 3D space are reconstructed accurately. In order to create a 3D reconstruction of a given soccer scene, the following steps have to be executed: 1) Camera parameters of all frames of both sequences are computed (camera calibration). 2) The playground texture is extracted from the video sequences. 3) Trajectories of the ball and the players' heads are computed after manually specifying their image positions in a few key frames. 4) Player textures are extracted automatically from video. 5) The shapes of colliding or occluding players are separated automatically. 6) For visualization, player shapes are texture-mapped onto appropriately placed rectangles in virtual space. SoccerMan is a novel experimental sports analysis system with fairly ambitious objectives. Its design decisions, in particular to start from two synchronous video sequences and to model players by texture objects, have already proven promising.     

一种基于球场模型的广播足球视频摄像机自动定标方法

随着足球视频语义内容分析和理解、自动视频索引、虚拟增强现实表示等研究的不断深入,摄像机定标技术扮演了越来越重要的角色.基于此,提出了一种基于球场模型的广播足球视频摄像机自动定标算法,该算法建立在所定义的球场模型基础之上,首先通过改进的球场线探测算法得到球场平面的特征点,并求解摄像机同形 (Homography)矩阵,然后,利用所求解的同形矩阵来定位球门,得到球门平面的两个特征点,最后根据基于K均值聚类和Hough-like搜索的方法来得到最优的摄像机内外参数.初步的实验结果表明,依据本文算法所求解的摄像机参数最优值相比初始值精确度更高,并且其在连续帧中也更加稳定.     

Camera calibration in sport event scenarios

The main goal of this paper is the design of a novel and robust methodology for calibrating cameras from a single image in sport scenarios, such as a soccer field, or a basketball or tennis court. In these sport scenarios, the only references we use to calibrate the camera are the lines and circles delimiting the different regions. The first problem we address is the extraction of image primitives, including the challenging problems of shaded regions and lens distortion. From these primitives, we automatically recognise the location of the sport court in the scene by estimating the homography which matches the actual court with its projection onto the image. This is achieved even when only a few primitives are available. Finally, from this homography, we recover the camera calibration parameters. In particular, we estimate the focal length as well as the position and orientation in the 3D space. We present some experiments on models and real courts which illustrate the accuracy of the proposed methodology.     

Continuous localization and mapping of a pan–tilt–zoom camera for wide area tracking

Pan–tilt–zoom (PTZ) cameras are well suited for object identification and recognition in far-field scenes. However, the effective use of PTZ cameras is complicated by the fact that a continuous online camera calibration is needed and the absolute pan, tilt and zoom values provided by the camera actuators cannot be used because they are not synchronized with the video stream. So, accurate calibration must be directly extracted from the visual content of the frames. Moreover, the large and abrupt scale changes, the scene background changes due to the camera operation and the need of camera motion compensation make target tracking with these cameras extremely challenging. In this paper, we present a solution that provides continuous online calibration of PTZ cameras which is robust to rapid camera motion, changes of the environment due to varying illumination or moving objects. The approach also scales beyond thousands of scene landmarks extracted with the SURF keypoint detector. The method directly derives the relationship between the position of a target in the ground plane and the corresponding scale and position in the image and allows real-time tracking of multiple targets with high and stable degree of accuracy even at far distances and any zoom level.     

基于HALCON的全景摄像机标定及应用研究

现有全景摄像机标定算法中,场景的先验知识在很多情况下难以准确获取。提出了一种基于HALCON的双曲面折反射全景摄像机标定算法。通过摄像机或者标定板的相互移动来获取不同位置的标定板图像;通过HALCON预处理后提取角点坐标;通过最小二乘法求解标定点投影方程的展开系数得到相机的内外参数。该算法中摄像机只需满足单视点要求,不需要场景的先验知识,也不需要特殊的装置和设备。实验结果表明能够获得较高的标定精度,并且在足球机器人边线距离的测量中得到了准确的结果。     

A hierarchy of cameras for 3D photography

The view-independent visualization of 3D scenes is most often based on rendering accurate 3D models or utilizes image-based rendering techniques. To compute the 3D structure of a scene from a moving vision sensor or to use image-based rendering approaches, we need to be able to estimate the motion of the sensor from the recorded image information with high accuracy, a problem that has been well-studied. In this work, we investigate the relationship between camera design and our ability to perform accurate 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for which the motion estimation problem can be solved independently of the depth of the scene which leads to fast and robust algorithms for 3D Photography. In between are multiple view cameras with a large field of view which we have built, as well as omni-directional sensors.     

基于视觉注意模型和HMM的足球视频语义分析

HMM模型具有良好的适应性,可以自动学习,对预测随机时序数据性能良好。场景是足球视频的基本特征,场景的转换体现了足球视频的摄制、编辑模式,表现了足球视频的语义。提出了一种基于场景分析和HMM的视频语义分析框架,用于识别足球视频中的一些语义事件。为了克服以往基于主颜色和其他底层特征的视频场景分析中存在的较大误差,又提出基于视觉注意模型对足球视频中的场景进行分析。实验结果表明,基于场景分析和HMM的事件识别方法对足球视频中的任意球事件有良好的识别效果     

Virtual Viewpoint Replay for a Soccer Match by View Interpolation From Multiple Cameras

This paper presents a novel method for virtual view synthesis that allows viewers to virtually fly through real soccer scenes, which are captured by multiple cameras in a stadium. The proposed method generates images of arbitrary viewpoints by view interpolation of real camera images near the chosen viewpoints. In this method, cameras do not need to be strongly calibrated since projective geometry between cameras is employed for the interpolation. For avoiding the complex and unreliable process of 3-D recovery, object scenes are segmented into several regions according to the geometric property of the scene. Dense correspondence between real views, which is necessary for intermediate view generation, is automatically obtained by applying projective geometry to each region. By superimposing intermediate images for all regions, virtual views for the entire soccer scene are generated. The efforts for camera calibration are reduced and correspondence matching requires no manual operation; hence, the proposed method can be easily applied to dynamic events in a large space. An application for fly-through observations of soccer match replays is introduced along with the algorithm of view synthesis and experimental results. This is a new approach for providing arbitrary views of an entire dynamic event.     

Camera calibration for a mobile robot prototype

Stereovision is an effective technique to use a CCD video camera to determine the 3D position of a target object from two or more simultaneous views of the scene. Camera calibration is a central issue in finding the position of objects in a stereovision system. This is usually carried out by calibrating each camera independently, and then applying a geometric transformation of the external parameters to find the geometry of the stereo setting. After calibration, the distance of various target objects in the scene can be calculated with CCD video cameras, and recovering the 3D structure from 2D images becomes simpler. However, the process of camera calibration is complicated. Based on the ideal pinhole model of a camera, we describe formulas to calculate intrinsic parameters that specify the correct camera characteristics, and extrinsic parameters that describe the spatial relationship between the camera and the world coordinate system. A simple camera calibration method for our CCD video cameras and corresponding experiment results are also given. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

Using pedestrians walking on uneven terrains for camera calibration

A calibrated camera is essential for computer vision systems: the prime reason being that such a camera acts as an angle measuring device. Once the camera is calibrated, applications like three-dimensional reconstruction or metrology or other applications requiring real world information from the video sequences can be envisioned. Motivated by this, we address the problem of calibrating multiple cameras, with an overlapping field of view observing pedestrians in a scene walking on an uneven terrain. This problem of calibration on an uneven terrain has so far not been addressed in the vision community. We automatically estimate vertical and horizontal vanishing points by observing pedestrians in each camera and use the corresponding vanishing points to estimate the infinite homography existing between the different cameras. This homography provides constraints on intrinsic (or interior) camera parameters while also enabling us to estimate the extrinsic (or exterior) camera parameters. We test the proposed method on real as well as synthetic data, in addition to motion capture dataset and compare our results with the state of the art.     

Common Visual Cues for Sports Highlights Modeling

Automatic annotation of semantic events allows effective retrieval of video content. In this work, we present solutions for highlights detection in sports videos. The proposed approach exploits the typical structure of a wide class of sports videos, namely those related to sports which are played in delimited venues with playfields of well known geometry, like soccer, basketball, swimming, track and field disciplines, and so on. For these sports, a modeling scheme based on a limited set of visual cues and on finite state machines that encode the temporal evolution of highlights is presented, that is of general applicability to this class of sports. Visual cues encode position and speed information coming from the camera and from the object/athletes that are present in the scene, and are estimated automatically from the video stream. Algorithms for model checking and for visual cues estimation are discussed, as well as applications of the representation to different sport domains.     

SecondSkin: An interactive method for appearance transfer

SecondSkin estimates an appearance model for an object visible in a video sequence, without the need for complex interaction or any calibration apparatus. This model can then be transferred to other objects, allowing a non‐expert user to insert a synthetic object into a real video sequence so that its appearance matches that of an existing object, and changes appropriately throughout the sequence. As the method does not require any prior knowledge about the scene, the lighting conditions, or the camera, it is applicable to video which was not captured with this purpose in mind. However, this lack of prior knowledge precludes the recovery of separate lighting and surface reflectance information. The SecondSkin appearance model therefore combines these factors. The appearance model does require a dominant light‐source direction, which we estimate via a novel process involving a small amount of user interaction. The resulting model estimate provides exactly the information required to transfer the appearance of the original object to new geometry composited into the same video sequence.     

城市视频实景地图构建初探

城市视频实景地图兼具地图立体空间、视频时间4维度层面信息统一表达能力，对于我国城市立体监控系统构建、互联网地图产品发展，以及未来实景3维中国建设战略实施具有重要意义和应用价值。为引起更多研究者进行探索，对城市视频实景地图构建方法、技术及其应用前景进行讨论。从增强虚拟环境技术（AVE）角度出发，对融合全景视频与地理3维模型构建城市视频实景地图涉及的全景摄像机标定、全景视频空间配准及视频纹理映射、实时渲染系列技术、方法进行了梳理。经过分析得出：1）适合传统“针孔”模型的摄像机标定、影像空间配准理论和方法，需根据全景摄像机球面投影模型进行拓展；2）适合静态纹理的大规模3D场景渲染LOD（levels of detail）技术和策略，需结合视频传输带宽限制、高帧率特点进行技术创新。城市视频实景地图构建是一项值得重视的崭新课题，将有力促进互联网、人工智能前沿技术发展，有望给相关行业带来万亿级市场机遇。     

基于C3D的足球视频场景分类算法

足球视频整场比赛持续时间较长,许多视频内容并非广大观众的兴趣所在,因此足球视频场景分类成为了近几十年来研究界的一项重要课题,许多机器学习方法也被应用于这个课题上.本文提出的基于C3D （三维卷积神经网络）的足球视频场景分类算法,将三维卷积运用于足球视频领域,并通过实验验证了本文算法的可行性.本文实验的流程如下：首先,基于帧间差分法和徽标检测法检测法对足球视频场景切换进行检测,实现镜头分割.在此基础上,提取分割镜头的语义特征并将其进行标记,然后通过C3D对足球事件进行分类.本文将足球视频分为7类,分别为远镜头、中镜头、特写镜头、回放镜头、观众镜头、开场镜头及VAR （视频助理裁判）镜头.实验结果表明,该模型在足球视频数据集上的分类准确率为96%.     

Camera–projector matching using unstructured video

This paper presents a novel approach for matching 2-D points between a video projector and a digital camera. Our method is motivated by camera–projector applications for which the projected image needs to be warped to prevent geometric distortion. Since the warping process often needs geometric information on the 3-D scene obtained from a triangulation, we propose a technique for matching points in the projector to points in the camera based on arbitrary video sequences. The novelty of our method lies in the fact that it does not require the use of pre-designed structured light patterns as is usually the case. The backbone of our application lies in a function that matches activity patterns instead of colors. This makes our method robust to pose, severe photometric and geometric distortions. It also does not require calibration of the color response curve of the camera–projector system. We present quantitative and qualitative results with synthetic and real-life examples, and compare the proposed method with the scale invariant feature transform (SIFT) method and with a state-of-the-art structured light technique. We show that our method performs almost as well as structured light methods and significantly outperforms SIFT when the contrast of the video captured by the camera is degraded.     

基于图像运动的三维重建及虚拟化应用

提出了一种稳定、快速地获取摄像机视频运动图像的三维重建方法,并对该运动图像做适当的虚拟化处理以展示重建效果。采用基于尺度不变特征点匹配的摄像机标定进行三维重建。尺度不变特征对于视频图像中的特征具有优秀敏锐的匹配能力,极大地放宽了摄像机标定对于设备上的限制,拓宽了实时三维重建的适用范围。通过对系统的一系列优化,不但提升了三维重建的精度,减少了错误匹配对摄像机标定的影响,而且进一步提升了处理速度。通过在三维重建的基础之上进行虚拟化处理,展示了本系统的三维重建效果。实验结果表明,该系统适用范围广,处理速度较快,重建精度高,实现了基于视频运动图像的三维重建。     

Designing educational experiences using ubiquitous technology

The aim of the study was to explore the design of situated educational and compelling experiences using SoundScape technology – which is a ubiquitous technology. Two experiences were designed: one that had a goal and the other that did not. Twenty six children then explored both experiences. The children found that the SoundScape with a goal was more interesting and were more likely to want to have another go than the non-goal version. They also made more positive comments concerning the goal version. Mentioning the presence of a goal as important factor in determining why they liked the game. More negative comments were made concerning the non-goal version and the children mentioned the lack of a goal as negative aspect of the non-goal version. One improvement they suggested for the non-goal version was to add a goal and to include more sensory stimulation. They overwhelmingly preferred the goal version compared to the non-goal version and when asked by far the most popular reason was the presence of a goal.     

Stratified self-calibration with the modulus constraint

In computer vision and especially for 3D reconstruction, one of the key issues is the retrieval of the calibration parameters of the camera. These are needed to obtain metric information about the scene from the camera. Often these parameters are obtained through cumbersome calibration procedures. There is a way to avoid explicit calibration of the camera. Self-calibration is based on finding the set of calibration parameters which satisfy some constraints (e.g., constant calibration parameters). Several techniques have been proposed but it often proved difficult to reach a metric calibration at once. Therefore, in the paper, a stratified approach is proposed, which goes from projective through affine to metric. The key concept to achieve this is the modulus constraint. It allows retrieval of the affine calibration for constant intrinsic parameters. It is also suited for use in conjunction with scene knowledge. In addition, if the affine calibration is known, it can also be used to cope with a changing focal length