Computer Vision for a 3-D Visualisation and Telepresence Collaborative Working Environment

An integrated media system is a computer-based environment that supports the creation, sharing, distribution and effective communication of multimodal information across the boundaries of space and time. The EU Information Societies Technology (IST) project — VIRTUE (Virtual Team User Environment) is working steadily towards the realisation of most aspects and properties of such a system, with particular emphasis on a three-way semi-immersive telepresence videoconferencing scenario. In contrast with the traditional videoconferencing system that we know now, the outcome of the project is expected to demonstrate distinctive presence features and experience for the conference participants. These include views of full-body-size realistically rendered images, eye-to-eye contact, gaze awareness, normal hand gesturing and direct body language. The purpose of this paper is to describe the current work in its related technical field, and the main objectives and scope of this project. One optional software system framework is outlined, and also illustrated are some component technologies in 3-D computer vision analysis that are being developed. The application of these component technologies, notably the dense-disparity estimation and the novel view synthesis, in 3-D interactive video manipulation and visualisation, are widely expected.     

新一代(第四代)人机交互的概念框架特征及关键技术

人机交互是研究人与计算机之间通过相互理解的交流与通信,在最大程度上为人们完成信息管理,服务和处理等功能的一门技术科学.本文给出新一代(第四代)人机交互的概念框架和其四个标志性特征,即:(1)具有多模感知(听觉,视觉,手势,笔势等)功能的人机交互方式;(2)可进行基于Agent的听、视觉对话,作为人机交互的界面;(3)具有Internet数据仓库和基于内容检索的知识处理能力,作为人机交互内容;(4)可以在二维或虚拟的三维环境中实现人机通信,作为人机交互的环境.文中给出了解决和实现该四大特征的关键技术,描述了实验系统平台的全面功能,可以为同类研究人员作参考.     

Detection of subtle nocturnal motor activity from 3-D accelerometry recordings in epilepsy patients

This paper presents a first step towards reliable detection of nocturnal epileptic seizures based on 3-D accelerometry (ACM) recordings. The main goal is to distinguish between data with and without subtle nocturnal motor activity, thus reducing the amount of data that needs further (more complex) analysis for seizure detection. From 15 ACM signals (measured on five positions on the body), two features are computed, the variance and the jerk. In the resulting 2-D feature space, a linear threshold function is used for classification. For training and testing, the algorithm ACM data along with video data is used from nocturnal registrations in seven mentally retarded patients with severe epilepsy. Per patient, the algorithm detected 100% of the periods of motor activity that are marked in video recordings and the ACM signals by experts. From all the detections, 43%-89% was correct (mean=65%). We were able to reduce the amount of data that need to be analyzed considerably. The results show that our approach can be used for detection of subtle nocturnal motor activity. Furthermore, our results indicate that our algorithm is robust for fluctuations across patients. Consequently, there is no need for training the algorithm for each new patient.     

金属3-D打印制造技术的发展

归纳了当前金属3-D打印技术的发展情况，指出了各类3-D打印技术优缺点，从发展历史、工作原理等方面讨论了典型3-D打印技术的技术特点; 在此基础上，对选区激光熔化技术的研究前景进行展望，即激光选区熔化技术作为金属3-D打印一个重要分支在各领域具有更广泛的应用; 提高材料性能、设备功能、结构设计及制造工艺的研发水平，可极大推动金属3-D打印技术的发展。随着金属打印技术的成熟，3-D打印的应用必将会覆盖更多金属制造产业，成为未来最重要、最具战略意义的制造技术。     

小视频内容分析技术发展探讨

小视频数量呈爆炸式增长态势,并引发了许多技术需求,包括小视频的编辑、搜索、推荐、溯源、审查和监管等。介绍了小视频数据的主要特点和小视频内容分析技术面临的挑战,并对目标检测、追踪、重识别等小视频内容分析技术的研究进展做了综合分析。认为只有构建一个整合多种不同算法的系统,才能够更准确、更鲁棒地解决分析问题,才能系统性地完成小视频内容分析任务。     

Invertible temporal subband/wavelet filter banks with half-pixel-accurate motion compensation

Three-dimensional (3-D) subband/wavelet coding with motion compensation has been demonstrated to be an efficient technique for video coding applications in some recent research works. When motion compensation is performed with half-pixel accuracy, images need to be interpolated in both temporal subband analysis and synthesis stages. The resulting subband filter banks developed in these former algorithms were not invertible due to image interpolation. In this paper, an invertible temporal analysis/synthesis system with half-pixel-accurate motion compensation is presented. We look at temporal decomposition of image sequences as a kind of down-conversion of the sampling lattices. The earlier motion-compensated (MC) interlaced/progressive scan conversion scheme is extended for temporal subband analysis/synthesis. The proposed subband/wavelet filter banks allow perfect reconstruction of the decomposed video signal while retaining high energy compaction of subband transforms. The invertible filter banks are then utilized in our 3-D subband video coder. This video coding system does not contain the temporal DPCM loop employed in the conventional hybrid coder and the earlier MC 3-D subband coders. The experimental results show a significant PSNR improvement by the proposed method. The generalization of our algorithm for MC temporal filtering at arbitrary subpixel accuracy is also discussed.     

A 3-D reconstruction system for the human jaw using a sequence of optical images

This paper presents a model-based vision system for dentistry that will assist in diagnosis, treatment planning, and surgical simulation. Dentistry requires an accurate three-dimensional (3-D) representation of the teeth and jaws for diagnostic and treatment purposes. The proposed integrated computer vision system constructs a 3-D model of the patient's dental occlusion using an intraoral video camera. A modified shape from shading (SFS) technique, using perspective projection and camera calibration, extracts the 3-D information from a sequence of two-dimensional (2-D) images of the jaw. Data fusion of range data and 3-D registration techniques develop the complete jaw model. Triangulation is then performed, and a solid 3-D model is reconstructed. The system performance is investigated using ground truth data, and the results show acceptable reconstruction accuracy.     

Advancements in noncontact, multiparameter physiological measurements using a webcam

We present a simple, low-cost method for measuring multiple physiological parameters using a basic webcam. By applying independent component analysis on the color channels in video recordings, we extracted the blood volume pulse from the facial regions. Heart rate (HR), respiratory rate, and HR variability (HRV, an index for cardiac autonomic activity) were subsequently quantified and compared to corresponding measurements using Food and Drug Administration-approved sensors. High degrees of agreement were achieved between the measurements across all physiological parameters. This technology has significant potential for advancing personal health care and telemedicine.     

Unconventional Image-Sensing and Light-Emitting Devices for Extended Reality

Extended reality (XR) refers to a space where physical and digital elements coexist and comprises three elements, namely, environment, human, and computer, which interact with each other. Image sensors and displays are the core elements of XR systems because visual information is important for recognizing and judging objects. Recently, new features of image sensors and displays that are useful for developing next-generation XR systems have been reported. For example, a miniaturized version of image sensors with the superb object detection and recognition capability offers new opportunities for machine vision technology. Furthermore, transparent and deformable displays are the key components of XR systems because they not only provide highly realistic virtual image information but also serve as efficient user interfaces. Herein, the recent progresses in such unconventional image sensors and display technologies are reviewed. First, image sensors with features of wavelength-selective photodetection for color discrimination, neuromorphic image acquisition for facile pattern recognition, and curved image sensor designs inspired by biological eyes for miniaturization and unconventional imaging performances are discussed. Then, light-emitting device technologies focusing on devices with transparency and deformable form factors are described. Finally, the review is concluded with a brief summary and a future outlook.     

激光三维成像技术及其主要应用

阐述了目前三维成像在其常见应用领域中的研究,主要致力于研究高分辨率三维成像系统。三维激光成像是一项可以应用于探测隐藏目标、地形测绘、构建虚拟环境、城市建模、目标识别等领域中的技术。在区域成像技术中,除了如立体视觉和结构化灯光等更常规的技术,实时三维传感也具有现实可操作性。当前三维激光成像技术已经发展到有能力提供厘米级波长的高分辨率三维成像,这将给许多领域提供方便,包括法律的实施和法医调查。与CCD和红外技术等传统的被动成像系统相比,激光成像技术不仅能提供强度和范围信息,还能穿透植被和窗户等特定情景元素。这意味着激光三维成像系统在目标识别与辨认等方面具备新的潜力。结果表明,激光三维成像系统可以在许多情况下得到应用。     

Three-dimensional modeling from two-dimensional video

This paper presents the surface-based factorization method to recover three-dimensional (3-D) structure, i.e., the 3-D shape and 3-D motion, of a rigid object from a two-dimensional (2-D) video sequence. The main ingredients of our approach are as follows: 1) we describe the unknown shape of the 3-D rigid object by polynomial patches; 2) projections of these patches in the image plane move according to parametric 2-D motion models; 3) we recover the parameters describing the 3-D shape and 3-D motion from the 2-D motion parameters by factorizing a matrix that is rank 1 in a noiseless situation. Our method is simultaneously an extension and a simplification of the original factorization method of Tomasi and Kanade (1992). We track regions where the 2-D motion in the image plane is described by a single set of parameters, avoiding the need to track a large number of pointwise features, in general, a difficult task. Then our method estimates the parameters describing the 3-D structure by factoring a rank 1 matrix, not rank 3 as in Tomasi and Kanade. This allows the use of fast iterative algorithms to compute the 3-D structure that best fits the data. Experimental results with real-life video sequences illustrate the good performance of our approach     

机器视觉技术、标准及安全产业应用

聚焦机器视觉技术及标准化,分析了机器视觉、计算机视觉与人类视觉的异同,列举了与机器视觉相关的技术,介绍了网络信息安全、智能监控和智能交通等机器视觉产业典型应用,以及国际及国内主流标准化组织针对机器视觉技术、系统和应用的标准化工作进展、现状和布局。重点介绍了VCM和DCM等涉及机器视觉核心技术——机器视觉编码标准的工作,以及机器视觉系统和应用标准化组织。在此基础上,分析了机器视觉技术研究和标准化的未来发展方向。     

An Introduction to Distributed Smart Cameras

Distributed smart cameras (DSCs) are real-time distributed embedded systems that perform computer vision using multiple cameras. This new approach has emerged thanks to a confluence of simultaneous advances in four key disciplines: computer vision, image sensors, embedded computing, and sensor networks. Processing images in a network of distributed smart cameras introduces several complications. However, we believe that the problems DSCs solve are much more important than the challenges of designing and building a distributed video system. We argue that distributed smart cameras represent key components for future embedded computer vision systems and that smart cameras will become an enabling technology for many new applications. We summarize smart camera technology and applications, discuss current trends, and identify important research challenges.      

双传感器激光视觉测量系统的校准

构建了由2个结构光传感器组成的激光视觉三维测量系统,建立了测量系统的数学模型.提出了一种基于未知运动的平面靶标获得测量系统全部参数的现场校准方法.实验表明,传感器校准后的三维坐标精度为0.011、0.008和0.115 mm,所提出的校准方法无需依赖专门的三维测量设备,操作简单,促进了双传感器激光视觉测量系统的工程化三维测量应用.     

共轴立体视觉深度测量

共轴立体视觉测距是一种深度恢复的新方法,利用光轴处于同一直线的两架相机,从单一角度获取图像信息,将三维空间的深度信息转换成二维空间的缩放视差,从而通过图像旋转与缩放、特征点提取、图像匹配、中心点估计等技术恢复深度信息。该技术具有测量系统体积小、实时性强等特点。介绍了该技术的深度恢复公式和算法,并设计了共轴立体摄影测距传感器,用实景实验和3DSMAX模拟实验对该技术进行了验证。试验结果表明,中长距离深度测量精度优于0．1％。     

New 3-D Chip Stacking Architectures by Wire-On-Bump and Bump-On-Flex

Two new 3-D chip stacking technologies, wire-on-bump (WOB) and bump-on-flex (BOF), are proposed and demonstrated with their prototypes. The WOB and BOF technologies are for low cost 3-D stacking of memory chips by vertical side interconnections with metal wires and flex-circuits, respectively. These new 3-D chip stacking technologies have benefits such as a shorter signal path and 3-D stackability of an unlimited number of chips compared to wire-bonded chip stacking. In the case of the BOF technology, additional active and passive components can be either surface-mounted onto or embedded into the flex-circuit, which is an added value that other chip stacking technologies have not demonstrated so far. More importantly, the WOB and BOF technologies enable lower cost processes than Si through-via technology, which is thus more suitable for memory chip stacking. This paper describes the detailed processes for our unique chip stacking structures with vertical interconnection methods of the WOB and BOF. Finite-element modeling and thermal cycle (TC) tests are also performed to address their thermo-mechanical reliability.     

基于多特征和多分类融合的视频概念检测

The rapid growth of multimedia content necessitates powerful technologies to filter, classify, index and retrieve video documents more efficiently. However, the essential bottleneck of image and video analysis is the problem of semantic gap that low level features extracted by computers always fail to coincide with high-level concepts interpreted by humans. In this paper, we present a generic scheme for the detection video semantic concepts based on multiple visual features machine learning. Various global and local low-level visual features are systematically investigated, and kernel-based learning method equips the concept detection system to explore the potential of these features. Then we combine the different features and sub-systems on both classifier-level and kernel-level fusion that contribute to a more robust system. Our proposed system is tested on the TRECVID dataset. The resulted Mean Average Precision (MAP) score is much better than the benchmark performance, which proves that our concepts detection engine develops a generic model and performs well on both object and scene type concepts.     

星上DVB-S2载波快速搜索技术

分析数字视频广播(DVB)-S2相对DVB-S的新特性和发展现状,提出一种基于TDQR-C005F的DVB-S/S2载波信号接收参数的搜索方案。重点针对宽带信号载波的中心频率和带宽参数的搜索技术,以及信号频道的符码率、编码码率和导频参数的三级搜索技术进行分析研究。与传统算法相比,算法和系统快速有效地解决了对DVB-S/S2突发信号的检测识别。实验表明,除对极少带宽小于2 MHz的DVB信号难以锁定外,系统对其他常规DVB-S/S2信号实现了准确锁定,识别搜索率达到97.5%。     

High-Speed and Broad-Band Communication Systems in Japan

This paper presents the current state of each high-speed and broad-band system and the development of the high-speed and broad-band network (HBN) in Japan. System configuration of a video conference system, a video transmission system, a high-speed digital circuit system, and so on, are introduced. Fundamental technologies to realize HBN, 1) switching technologies to handle broad bit-rate class transmission, 2) high-capacity transmission and high-speed digital terminal technology, 3) economical fiber optic subscriber loop technology, and 4) high-performance video codec technology, are discussed. Based on the above discussion, configuration of HBN is proposed.