3G wireless communications for mobile robotic tele-ultrasonography systems

Mobile healthcare (m-health) is a new paradigm that brings together the evolution of emerging wireless communications and network technologies with the concept of 'connected healthcare' anytime and anywhere. In this article, we present the performance analysis of an end-to-end mobile tele-echography using an ultra-light robot (OTELO), over the third-generation (3G) mobile communications network. The experimental setup of the OTELO system over a 3G connectivity link used to measure the system performance is described. The performance of the relevant medical data and the relevant quality of service (QoS) issues defined in terms of the average throughput, delta-time packet delay, and jitter delay are investigated. The real-time 3G performance results show the successful operation of this bandwidth demanding robotic m-health system.     

3D augmented reality teleoperated robot system based on dual vision

A 3D augmented reality navigation system using stereoscopic images is developed for teleoperated robot systems.The accurate matching between the simulated model and the video image of the actual robot can be realized,which helps the operator to accomplish the remote control task correctly and reliably.The system introduces the disparity map translation transformation method to take parallax images for stereoscopic displays,providing the operator an immersive 3D experience.Meanwhile,a fast and accurate registration method of dynamic stereo video is proposed,and effective integration of a virtual robot and the real stereo scene can be achieved.Preliminary experiments show that operation error of the system is maintained at less than 2.2 mm and the average error is 0.854 7,0.909 3 and 0.697 2 mm at x,y,z direction respectively.Lots of experiments such as pressing the button,pulling the drawer and so on are also conducted to evaluate the performance of the system.The feasibility studies show that the depth information of structure can be rapidly and recognized in remote environment site.The augmented reality of the image overlay system could increase the operating accuracy and reduce the procedure time as a result of intuitive 3D viewing.     

Exploratory design and evaluation of a homecare teleassistive mobile robotic system

Telehealth assistive technologies for homes constitute a very promising avenue to decrease load on the health care system, to reduce hospitalization period and to improve quality of life. Teleoperated from a distant location, a mobile robot can become a beneficial tool in health applications. However, design issues related to such systems are broad and mostly unexplored (e.g., locomotion and navigation in-home settings, remote interaction and patient acceptability, evaluation of clinical needs and their integration into health care information systems). Designing a safe and effective robotic system for in-home teleassistance requires taking into consideration the complexities of having novice users remotely navigate a mobile robot in a home environment while they interact with patients. This paper presents the progress made by adopting an interdisciplinary and exploratory design methodology to develop a telepresence assistive mobile robot for homecare assistance of elderly people. Preliminary studies using robots, focus groups and interviews allowed us to derive preliminary specifications to design a new mobile robotic system named Telerobot. Telerobot’s locomotion mechanism provides improved mobility when moving on uneven surfaces, helping to provide stable video feed to the user. Its control system is implemented for safe teleoperation. A study involving 10 rehabilitation professionals confirms that the system is usable in-home environments. Analysis of teleoperation strategies used by novice teleoperators suggest that it is essential in a home environment that the teleoperation interface provides the user with a visual feedback of the objects surrounding the robot, their distances relative to the robot and the size of the robot in the environment. Enhanced user interfaces to augment the operator’s perception of the environment were elaborated and tested in controlled conditions. Based on the progress made so far, the paper outlines issues that will be addressed in future work with the objective of coming up with a complete, efficient and usable in-home teleassistance mobile robotic system.     

Automatic extraction and tracking of the tongue contours

Computerized analysis of the tongue surface movement can provide valuable information to speech and swallowing research. Ultrasound technology is currently the most attractive modality for the tongue imaging mainly because of its high video frame rate. However, problems with ultrasound imaging, such as noise and echo artifacts, refractions, and unrelated reflections pose significant challenges for computer analysis of the tongue images and hence specific methods must be developed. This paper presents a system that is developed for automatic extraction and tracking of the tongue surface movements from ultrasound image sequences. The ultrasound images are supplied by the head and transducer support system (HATS), which was developed in order to fix the head and support the transducer under the chin in a known position without disturbing speech. In this work, we propose a novel scheme for the analysis of the tongue images using deformable contours. We incorporate novel mechanisms to 1) impose speech related constraints on the deformations; 2) perform spatiotemporal smoothing using a contour postprocessing stage; 3) utilize optical flow techniques to speed up the search process; and 4) propagate user supplied information to the analysis of all image frames. We tested the system's performance qualitatively and quantitatively in consultation with speech scientists. Our system produced contours that are within the range of manual measurement variations. The results of our system are extremely encouraging and the system can be used in practical speech and swallowing research in the field of otolaryngology.     

A system for real-time measurement of the brachial artery diameter in B-mode ultrasound images.

The measurement of the brachial artery diameter is frequently used in clinical studies for evaluating the flow-mediated dilation and, in conjunction with the blood pressure value, for assessing arterial stiffness. This paper presents a system for computing the brachial artery diameter in real-time by analyzing B-mode ultrasound images. The method is based on a robust edge detection algorithm which is used to automatically locate the two walls of the vessel. The measure of the diameter is obtained with subpixel precision and with a temporal resolution of 25 samples/s, so that the small dilations induced by the cardiac cycle can also be retrieved. The algorithm is implemented on a standalone video processing board which acquires the analog video signal from the ultrasound equipment. Results are shown in real-time on a graphical user interface. The system was tested both on synthetic ultrasound images and in clinical studies of flow-mediated dilation. Accuracy, robustness, and intra/inter observer variability of the method were evaluated.     

A System for Real-Time Measurement of the Brachial Artery Diameter in B-Mode Ultrasound Images

The measurement of the brachial artery diameter is frequently used in clinical studies for evaluating the flow-mediated dilation and, in conjunction with the blood pressure value, for assessing arterial stiffness. This paper presents a system for computing the brachial artery diameter in real-time by analyzing B-mode ultrasound images. The method is based on a robust edge detection algorithm which is used to automatically locate the two walls of the vessel. The measure of the diameter is obtained with subpixel precision and with a temporal resolution of 25 samples/s, so that the small dilations induced by the cardiac cycle can also be retrieved. The algorithm is implemented on a standalone video processing board which acquires the analog video signal from the ultrasound equipment. Results are shown in real-time on a graphical user interface. The system was tested both on synthetic ultrasound images and in clinical studies of flow-mediated dilation. Accuracy, robustness, and intra/inter observer variability of the method were evaluated     

Depth-enhanced mobile robot teleguide based on laser images

3D stereoscopic visualization may provide a user with higher comprehension of remote environment in teleoperation when compared to 2D viewing. Works in the literature have addressed the contribution of stereo vision to improve perception of some depth cues often for abstract tasks, and it is hard to find contributions specifically addressing mobile robot teleguide. The authors of this paper have investigated stereoscopic viewing in mobile robot teleguide based on video images in a previous work and pointed out advantages of stereo viewing in this type of application as well as shortcomings inherent to the use of visual sensor, e.g. image transmission delay. The proposed investigation aims at testing mobile robot teleguide based on a different sensor: the laser sensor. The use of laser is expected to solve some problems related to visual sensor while maintaining the advantage of having stereoscopic visualization of a remote environment. A usability evaluation is proposed to assess system performance. The evaluation runs under the same setup of the previous study so to have an experimental outcome comparable to the previous one. The evaluation involves several users and two different 3D visualization technologies. The results show a strong improvement in users’ performance when mobile robot teleguide based on laser sensor is (depth-) enhanced by stereo viewing. Some differences are detected between the use of laser and visual sensor which are discussed.     

Web-based video for real-time monitoring of radiological procedures

Web-based video transmission of images from CT and MRI consoles is implemented in an intranet environment for real-time monitoring of ongoing procedures. Images captured from the consoles are compressed to video resolution and then broadcast through a Web server. When called upon, the attending radiologists can view these live images on any computer within the secured intranet network. With adequate compression, these images can be displayed simultaneously in different locations at a rate of 2 to 5 images/s through a standard local area network. While the quality of the images was insufficient for diagnostic purposes, our user survey showed that they were suitable for supervising a procedure, for positioning the imaging slices, and for routine quality checking before completion of a study. The system was implemented at UCLA to monitor nine CTs and six MRIs distributed in four different buildings. This system significantly improved the radiologists' productivity by saving valuable time spent in trips between reading rooms and examination rooms. It also improved patient care and throughput by reducing the time spent waiting for the radiologists to check a study before removing the patient from the scanner     

Perceived Quality Measurement Model Supporting Full Session Mobility in Multimedia Service Delivery

Context-awareness is associated with the concept of machines being able to both recognize and react to the surrounding environment, which has accelerated multimedia computing technology to provide user-oriented intelligent services. The goal of this research focuses on the modeling of video quality measurement in seamless video service delivery process considering the mobility patterns of users. Mobility supporting schemes providing seamless multimedia services are classified into host mobility and user mobility. The former refers to a host-level handoff while the latter indicates a user-level handoff. In host-level handoff, the decisive factors affecting the quality of video contents consumption are the total distance between hosts, the distance to resume streaming while the user is in mobility mode as well as the screen size of the end host. The influential parameters affecting the degree of video quality and seamlessness are evaluated by a subjective quality assessment and then a perceived video quality model was developed, accordingly. In addition, the optimal video delivery switching point to enable user mobility based video service is studied through the quality analysis of host mobility services at varying distances among the hosts. Experimental results show that the proposed quality model has high correlation with assessed quality and is useful to enable an adequate seamless mobility for multimedia service delivery.     

Mosaic-based 3D scene representation and rendering

In this paper we address the problem of fusing images from many video cameras or a moving video camera. The captured images have obvious motion parallax, but they will be aligned and integrated into a few mosaics with a large field-of-view (FOV) that preserve 3D information. We have developed a compact geometric representation that can re-organize the original perspective images into a set of parallel projections with different oblique viewing angles. In addition to providing a wide field of view, mosaics with various oblique views well represent occlusion regions that cannot be seen in a usual nadir view. Stereo mosaic pairs can be formed from mosaics with different oblique viewing angles. This representation can be used as both an advanced interface for interactive 3D video and a pre-processing step for 3D reconstruction. A ray interpolation approach for generating the parallel-projection mosaics is presented, and efficient 3D scene/object rendering based on multiple parallel-projection mosaics is discussed. Several real-world examples are provided, with applications ranging from aerial video surveillance/environmental monitoring, ground mobile robot navigation, to under-vehicle inspection.     

Gesture-based interaction with voice feedback for a tour-guide robot

Any service robot should possess some kind of human–robot interaction to communicate with users. In this paper we present a tour-guide robot able to recognize hand gestures and provide voice feedback. A user can communicate with the robot using hand movements, but also through virtual buttons which are shown in a augmented reality environment in the robot’s screen. Moreover, the user continuously receives feedback from the robot’s screen or by means of voice messages. The result is a robot that understands the users’ needs and which keeps them informed about what it senses, which increases the usefulness of our robot. Finally, to measure the quality in use, we have conducted two user studies, which we have used to reveal and correct the weakness of the robot. The guide robot was successfully tested in several real world environments.     

User-Agent Cooperation in Multiagent IVUS Image Segmentation

Automated interpretation of complex images requires elaborate knowledge and model-based image analysis, but often needs interaction with an expert as well. This research describes expert interaction with a multiagent image interpretation system using only a restricted vocabulary of high-level user interactions. The aim is to minimize inter- and intra-observer variability by keeping the total number of interactions as low and simple as possible. The multiagent image interpretation system has elaborate high-level knowledge-based control over low-level image segmentation algorithms. Agents use contextual knowledge to keep the number of interactions low but, when in doubt, present the user with the most likely interpretation of the situation. The user, in turn, can correct, supplement, and/or confirm the results of image-processing agents. This is done at a very high level of abstraction such that no knowledge of the underlying segmentation methods, parameters or agent functioning is needed. High-level interaction thereby replaces more traditional contour correction methods like inserting points and/or (re)drawing contours. This makes it easier for the user to obtain good results, while inter- and intra-observer variability are kept minimal, since the image segmentation itself remains under control of image-processing agents. The system has been applied to intravascular ultrasound (IVUS) images. Experiments show that with an average of 2-3 high-level user interactions per correction, segmentation results substantially improve while the variation is greatly reduced. The achieved level of accuracy and repeatability is equivalent to that of manual drawing by an expert.      

Real-time 16K video coding on a GPU with complexity scalable BPC-PaCo

The advent of new technologies such as high dynamic range or 8K screens has enhanced the quality of digital images but it has also increased the codecs’ computational demands to process such data. This paper presents a video codec that, while providing the same coding features and performance as those of JPEG2000, can process 16K video in real time using a consumer-grade GPU. This high throughput is achieved with a technique that introduces complexity scalability to a bitplane coding engine, which is the most computationally complex stage of the coding pipeline. The resulting codec can trade throughput for coding performance depending on the user’s needs. Experimental results suggest that our method can double the throughput achieved by CPU implementations of the recently approved High-Throughput JPEG2000 and by hardwired implementations of HEVC in a GPU.     

A tele-operated mobile ultrasound scanner using a light-weight robot.

This paper presents a new tele-operated robotic chain for real-time ultrasound image acquisition and medical diagnosis. This system has been developed in the frame of the Mobile Tele-Echography Using an Ultralight Robot European Project. A light-weight six degrees-of-freedom serial robot, with a remote center of motion, has been specially designed for this application. It holds and moves a real probe on a distant patient according to the expert gesture and permits an image acquisition using a standard ultrasound device. The combination of mechanical structure choice for the robot and dedicated control law, particularly nearby the singular configuration allows a good path following and a robotized gesture accuracy. The choice of compression techniques for image transmission enables a compromise between flow and quality. These combined approaches, for robotics and image processing, enable the medical specialist to better control the remote ultrasound probe holder system and to receive stable and good quality ultrasound images to make a diagnosis via any type of communication link from terrestrial to satellite. Clinical tests have been performed since April 2003. They used both satellite or Integrated Services Digital Network lines with a theoretical bandwidth of 384 Kb/s. They showed the tele-echography system helped to identify 66% of lesions and 83% of symptomatic pathologies.     

Including efficient object recognition capabilities in online robots: from a statistical to a Neural-network classifier

For those situations in which the user wants to interact with the system by using, for example, voice commands, it would be convenient to refer to the objects by their names (e.g., "cube") instead of other types of interactions (e.g., "grasp object 1"). Thus, automatic object recognition is the first step in order to acquire a higher level of interaction between the user and the robot. Nevertheless, applying object recognition techniques when the camera images are being transmitted through the web is not an easy task. In this situation, images cannot have a very high resolution, which affects enormously the recognition process due to the inclusion of more errors while digitalizing the real image. Some experiments with the Universitat Jaume I Online Robot evaluate the performance of different neural-network implementations, comparing it to that of some distance-based object recognition algorithms. Results will show which combination of object features, and algorithms (both statistical and neural networks) is more appropriate to our purpose in terms of both effectiveness and computing time.     

Controlled manipulation of molecular samples with thenanoManipulator

The nanoManipulator system adds a virtual-reality interface to an atomic-force microscope (AFM), thus providing a tool that can be used by scientists to image and manipulate nanometer-sized molecular structures in a controlled manner. As the AFM tip scans the sample, the tip-sample interaction forces are monitored, which, in turn, can yield information about the frictional, mechanical, material, and topological properties of the sample. Computer graphics are used to reconstruct the surface for the user, with color or contours overlaid to indicate additional data sets. Moreover, a force feedback stylus, which is connected to the tip via software, allows the user to directly interact with the macromolecules. This system is being used to investigate carbon nanotubes, DNA, fibrin, adeno- and tobacco mosaic virus. It is now also possible to insert this system into a scanning electron microscope which provides the user with continuous images of the sample, even while the AFM tip is being used for manipulations     

Semiautomatic 3-D prostate segmentation from TRUS images using spherical harmonics

Prostate brachytherapy quality assessment procedure should be performed while the patient is still on the operating table since this would enable physicians to implant additional seeds immediately into the prostate if necessary thus reducing the costs and increasing patient outcome. Seed placement procedure is readily performed under fluoroscopy and ultrasound guidance. Therefore, it has been proposed that seed locations be reconstructed from fluoroscopic images and prostate boundaries be identified in ultrasound images to perform dosimetry in the operating room. However, there is a key hurdle that needs to be overcome to perform the ultrasound and fluoroscopy-based dosimetry: it is highly time-consuming for physicians to outline prostate boundaries in ultrasound images manually, and there is no method that enables physicians to identify three-dimensional (3-D) prostate boundaries in postimplant ultrasound images in a fast and robust fashion. In this paper, we propose a new method where the segmentation is defined in an optimization framework as fitting the best surface to the underlying images under shape constraints. To derive these constraints, we modeled the shape of the prostate using spherical harmonics of degree eight and performed statistical analysis on the shape parameters. After user initialization, our algorithm identifies the prostate boundaries on the average in 2 min. For algorithm validation, we collected 30 postimplant prostate volume sets, each consisting of axial transrectal ultrasound images acquired at 1-mm increments. For each volume set, three experts outlined the prostate boundaries first manually and then using our algorithm. By treating the average of manual boundaries as the ground truth, we computed the segmentation error. The overall mean absolute distance error was 1.26 +/- 0.41 mm while the percent volume overlap was 83.5 +/- 4.2. We found the segmentation error to be slightly less than the clinically-observed interobserver variability.     

Transcoding of single-layer MPEG video into lower rates

Forthcoming interactive video services such as video on demand will use pre-encoded bit streams for transmission. A great lack of flexibility arises when heterogeneous networks are used or when the user is allowed to use a bandwidth of his/her own choice. In the paper, a mechanism capable of decoupling video encoders from transmission-network constraints and user demands is proposed. The authors devise a low-cost, low-delay video transcoder capable of providing transmission flexibility to pre-encoded bit streams, by reducing their bit rates according to either channel capacity or user demand. Simple techniques such as open-loop coarse requantisation or drop of high-frequency transform coefficients are shown to be inefficient, because of the drift introduced in the transcoded pictures. It is shown that transcoded pictures are drift-free and their quality, on average, is only about 1 dB worse than those directly encoded at the same bit rate. The proposed transcoder is far less complex than a cascade of decoder-encoder, while the picture quality is shown to be better for almost all frames