Image-to-MIDI mapping based on dynamic fuzzy color segmentation for visually impaired people

In this paper, the RGB ratio is defined according to a reference color so that an image can be transformed from a conventional color space to the RGB ratio space. Different to traditional distance measurement, a road color model is determined by an ellipse area in the RGB ratio space enclosed by the estimated boundaries. The proposed dynamic fuzzy logic, where fuzzy membership functions are defined according to estimated boundaries, is introduced to implement clustering rules, such that each pixel will have its own fuzzy membership function corresponding to its intensity. A basic neural network is trained and used to achieve parameter optimization. Experimental results for road detection demonstrate the robustness of the proposed approach to variations in intensity. To provide obstacle information, especially for visually impaired people, Musical Instrument Digital Interface (MIDI) is introduced as the sound generator, and image-to-MIDI mapping algorithm is proposed. Experimental results show that the proposed method can adapt to various road types, and the resulting audio information successfully indicates the position and size of obstacles.     

Supporting visually impaired children with software agents in a multimodal learning environment

Visually impaired children have a great disadvantage in the modern society since their ability to use modern computer technology is limited due to inappropriate user interfaces. The aim of the work presented in this paper was to develop a multimodal software architecture and applications to support learning of visually impaired children. The software architecture is based on software agents, and has specific support for visual, auditory and haptic interaction. It has been used successfully with different groups of 7-8 year-old and 12 year-old visually impaired children. In this paper we discuss the enabling software technology and interaction techniques aimed to realize our goal and our experiences in the actual use of the system.     

Uncalibrated obstacle detection using normal flow

This paper addresses the problem of obstacle detection for mobile robots. The visual information provided by a single on-board camera is used as input. We assume that the robot is moving on a planar pavement, and any point lying outside this plane is treated as an obstacle. We address the problem of obstacle detection by exploiting the geometric arrangement between the robot, the camera, and the scene. During an initialization stage, we estimate an inverse perspective transformation that maps the image plane onto the horizontal plane. During normal operation, the normal flow is computed and inversely projected onto the horizontal plane. This simplifies the resultant flow pattern, and fast tests can be used to detect obstacles. A salient feature of our method is that only the normal flow information, or first order time-and-space image derivatives, is used, and thus we cope with the aperture problem. Another important issue is that, contrasting with other methods, the vehicle motion and intrinsic and extrinsic parameters of the camera need not be known or calibrated. Both translational and rotational motion can be dealt with. We present motion estimation results on synthetic and real-image data. A real-time version implemented on a mobile robot, is described.     

Accurate 3D-vision-based obstacle detection for an autonomous train

In this paper we present a 3D-vision based obstacle detection system for an autonomously operating train in open terrain environments. The system produces dense depth data in real-time from a stereo camera system with a baseline of 1.4 m to fulfill accuracy requirements for reliable obstacle detection 80 m ahead. On an existing high speed stereo engine, several modifications have been applied to significantly improve the overall performance of the system. Hierarchical stereo matching and slanted correlation masks increased the quality of the depth data in a way that the obstacle detection rate increased from 89.4% to 97.75% while the false positive detection rate could be kept as low as 0.25%. The evaluation results have been obtained from extensive real-world test data. An additional stereo matching speed-up of factor 2.15 was achieved and the overall latency of obstacle detection is considerably faster than 300 ms.     

AudioBrowser: a mobile browsable information access for the visually impaired

Although a large amount of research has been conducted on building interfaces for the visually impaired that allows users to read web pages and generate and access information on computers, little development addresses two problems faced by the blind users. First, sighted users can rapidly browse and select information they find useful, and second, sighted users can make much useful information portable through the recent proliferation of personal digital assistants (PDAs). These possibilities are not currently available for blind users. This paper describes an interface that has been built on a standard PDA and allows its user to browse the information stored on it through a combination of screen touches coupled with auditory feedback. The system also supports the storage and management of personal information so that addresses, music, directions, and other supportive information can be readily created and then accessed anytime and anywhere by the PDA user. The paper describes the system along with the related design choices and design rationale. A user study is also reported.     

A sonar approach to obstacle detection for a vision-based autonomous wheelchair

An advanced prototype Computer Controlled Power Wheelchair Navigation System or CCPWNS has been developed to provide autonomy for highly disabled users, whose mix of disabilities makes it difficult or impossible to control their own power chairs in their homes. The working paradigm is “teach and repeat” a mode of control for typical industrial holonomic robots. Ultrasound sensors, which during subsequent autonomous tracking will be used to detect obstacles, also are active during teaching. Based upon post-processed data collected during this teaching event, elaborate trajectories–which may involve multiple direction changes, pivoting and so on, depending upon the requirements of the typically restricted spaces within which the chair must operate–will later be called upon by the disabled rider. An off-line postprocessor assigns an ultrasound profile to the sequence of poses of any taught trajectory. Use of this profile during tracking obviates most of the inherent problems of using ultrasound to avoid obstacles while retaining the ability to near solid objects, such as when passing through a narrow doorway, where required by the environment and trajectory objectives. The work in this article describes a procedure to obtain consistent maps of sonar boundaries during the teaching process, and a preliminary approach to use this information during the tracking phase. The approach is illustrated by results obtained by using the CCPWNS prototype.     

Real-time pedestrian crossing lights detection algorithm for the visually impaired

In defect of intelligent assistant approaches, the visually impaired feel hard to cross the roads in urban environments. Aiming to tackle the problem, a real-time Pedestrian Crossing Lights (PCL) detection algorithm for the visually impaired is proposed in this paper. Different from previous works which utilize analytic image processing to detect the PCL in ideal scenarios, the proposed algorithm detects PCL using machine learning scheme in the challenging scenarios, where PCL have arbitrary sizes and locations in acquired image and suffer from the shake and movement of camera. In order to achieve the robustness and efficiency in those scenarios, the detection algorithm is designed to include three procedures: candidate extraction, candidate recognition and temporal-spatial analysis. A public dataset of PCL, which includes manually labeled ground truth data, is established for tuning parameters, training samples and evaluating the performance. The algorithm is implemented on a portable PC with color camera. The experiments carried out in various practical scenarios prove that the precision and recall of detection are both close to 100%, meanwhile the frame rate is up to 21 frames per second (FPS).     

一种障碍物模型检测算法

针对观测场景结构复杂、建模困难的问题,提出了一种基于运动目标跟踪的场景障碍物模型检测方法.结合对目标的检测与跟踪过程,利用空间投影关系确定场景中障碍物的位置,并将其映射到场景模型上,即可得到场景观测模型.该方法只需要对目标运动进行简单地跟踪和统计,不需要进行三维结构恢复以及图像拼接处理,能够有效解决场景的在线自适应建模问题.     

Understanding the requirements of geographical data for blind and partially sighted people to make journeys more independently

Previous research has highlighted that blind and partially sighted people find various factors inhibit their abilities to make journeys. This paper proposes that the lack of accurate, appropriate and usable geographical data is one of the reasons for this and these can be tracked back to core human factors issues such as situational awareness, mental workload and environmental ergonomics. Following a review of applicable literature a hierarchical task analysis was performed to better understand the problems in terms of the complexity of various journey types and to identify the geographical data requirements in order to make successful journeys. The task analysis produced a number of results including highlighting four underlying principles which have an impact on the data requirements during any given journey. Finally the need for accessible and accurate geographical data requirements is introduced as a result of the literature review and the task analysis. These highlight the information required in order to facilitate more accessible travel for blind and partially sighted people by providing geographical information about their surroundings in a relevant, meaningful and usable way.     

Walking strategies of visually impaired people on trapezoidal- and sinusoidal-section tactile groundsurface indicators

The visual system in walking serves to perceive feedback or feed-forward signals. Therefore, visually impaired persons (VIP) have biased motor control mechanisms. The use of leading indicators (LIs) and long canes helps toimprove their walking efficiency. The aims of this study were to compare the walking efficiency of VIP on trapezoidal- and sinusoidal-section LIs using an optoelectronic motion analysis system. VIP displayed a significantly longer stance phase, a shorter swing phase and shorter step and stride lengths when they walked on the sinusoidal LIthan when they walked on the trapezoidal LI. Compared with the trapezoidal LI, VIP walking on the sinusoidalLIdisplayed significantly lower joint ranges of motion. The centre of mass lateral displacement was wider for VIP walking on the sinusoidal LI than on the trapezoidal LI. Some significant differences were also found in sighted persons walking on both LIs. In conclusion, the trapezoidal shape enabled visually impaired subjects to walk more efficiently, whereas the sinusoidal shape caused dynamic balance problems.

Statement of Relevance: These findings suggest that VIP can walk more efficiently, with a lower risk of falls, on trapezoidal-section than on sinusoidal-section LIs. These results should be considered when choosing the most appropriate ground tactile surface indicators for widespread use.     

An enhanced text detection technique for the visually impaired to read text

An enhanced text detection technique (ETDT) is proposed, which is expected to aid the visually impaired to overcome their reading challenges. This work enhances the edge-preserving maximally stable extremal regions (eMSER) algorithm using the pyramid histogram of oriented gradients (PHOG). Histogram of oriented gradients (HOG) derived from different pyramid levels is important while detecting maximally stable extremal regions (MSER) in the ETDT approach because it gives more spatial information when compared to HOG information from a single level. To group text, a four-line, text-grouping method is newly designed for this work. Also, a new text feature, Shapeness Score is proposed, which significantly identifies text regions when combined with the other features based on morphology and stroke widths. Using the feature vector of dimension 10, the J48 decision tree and AdaBoost machine learning algorithms identify the text regions in the images. The algorithm yields better results than the existing benchmark algorithms for the ICDAR 2011 born-digital dataset and must be improved with respect to the scene text dataset.     

Accessible smartphones for blind users: A case study for a wayfinding system

While progress on assistive technologies have been made, some blind users still face several problems opening and using basic functionalities when interacting with touch interfaces. Sometimes, people with visual impairments may also have problems navigating autonomously, without personal assistance, especially in unknown environments. This paper presents a complete solution to manage the basic functions of a smartphone and to guide users using a wayfinding application. This way, a blind user could go to work from his home in an autonomous way using an adaptable wayfinding application on his smartphone. The wayfinding application combines text, map, auditory and tactile feedback for providing the information. Eighteen visually impaired users tested the application. Preliminary results from this study show that blind people and limited vision users can effectively use the wayfinding application without help. The evaluation also confirms the usefulness of extending the vibration feedback to convey distance information as well as directional information. The validation was successful for iOS and Android devices.     

Discrete Event Systems for autonomous mobile agents

Discrete Event Systems (DES) are a special type of dynamic system. The ‘state’ of these systems changes at discrete instants in time and the term ‘event’ represents the occurrence of discontinuous change (at possibly unknown intervals). Different Discrete Event Systems models are currently used for specification, verification, synthesis as well as for analysis and evaluation of different qualitative and quantitative properties of existing physical systems. The focus of this paper is the presentation of the automata and formal language model for DES introduced by Ramadge and Wonham and its application to the domain of mobile manipulator/observer agents. We demonstrate the feasibility of the DES framework for modeling, analysis and synthesis of some visually guided behaviors of agents engaged in navigational tasks and address synchronization issues between different components of the system. The use of DES formalism allows us to synthesize complex behaviors in a systematic fashion and guarantee their controllability.     

Development of a wearable guide device based on convolutional neural network for blind or visually impaired persons

Multimedia Tools and Applications - This study proposes a design for a wearable guide device for blind or visually impaired persons on the basis of video streaming and deep learning. This work...     

Real-time obstacle detection for legged robots using the Kinect sensor

Autonomous navigation of legged robots in complex environments poses a great deal of challenges compared with ground vehicles because of their different terrain traverse capabilities. An obstacle for vehicles may be traversable for legged robots. This paper proposes a real-time obstacle detection algorithm for legged robots using the Microsoft Kinect sensor. First, the elevation map of a reference grid is calculated. Then an obstacle definition for legged robots is proposed, which makes it possible for a legged robot to discriminate traversable areas from non-traversable areas. To reduce computational cost, sometimes, efficient judging rules are developed to identify obstacles. A spiral search strategy is proposed to find the most ground-like point as the starting point for graph-based traversal. Breadth-First-Traversal of the graph is used to label all traversable areas connecting to the starting point. Experimental results demonstrate that our algorithm is reliable and efficient. The proposed algorithm can be employed in real-time obstacle detection for legged robots in complex environments.     

Developing a navigation aid for the frail and visually impaired

This paper describes the development of a new navigational aid for the frail, elderly, and visually impaired person. The users were involved both in the user requirements study and in the evaluation of different prototypes. The results show that the users were able to provide information on their current aid, the use situation, and their preference regarding different solutions, but they had difficulties to provide the detailed answers on technical solutions required by the technical development team. Further, prototype evaluations with users enabled the technical team to understand the users and their use situation.     

Self-localization and obstacle avoidance for a mobile robot

This article presents a fast self-localization method based on ZigBee wireless sensor network and laser sensor, an obstacle avoidance algorithm based on ultrasonic sensors for a mobile robot. The positioning system and positioning theory of ZigBee which can obtain a rough global localization of the mobile robot are introduced. To realize accurate local positioning, a laser sensor is used to extract the features from environment, then the environmental features and global reference map can be matched. From the matched environmental features, the position and orientation of the mobile robot can be obtained. To enable the mobile robot to avoid obstacle in real-time, a heuristic fuzzy neural network is developed by using heuristic fuzzy rules and the Kohonen clustering network. The experiment results show the effectiveness of the proposed method.     

一种简单便携的存储介质信息检测及备份装置研究

随着信息时代的飞速发展,数据信息在生活中的作用越来越显重要,数据恢复在社会生活中的应用越来越广,这便需要对存储介质的信息进行必要的检测和备份。虽然一些常见的软件可以通过PC端实现检测和备份功能,但也就是针对常见USB设备和一些专业性比较强的硬盘,不方便备份且不易携带,因此能提供一种兼容各种存储介质,并且方便使用、易于携带且不依赖于电脑就能实现检测和备份的设备非常重要。本文阐述了一种简单便携的存储介质信息检测及备份装置的研究设计．．     

A multimedia speech learning system for the hearing impaired

The lack (or limited amount) of audio feedback for many persons with hearing disabilities greatly reduces the quality of speech development and use. Current methodologies for the teaching speech techniques are relatively primitive, and generally require the active participation of a therapist. This makes essential practice difficult, time consuming, and expensive for the student. However, using multimedia technology, it is possible to amplify the efforts of the teachers and students for improving the quality of speech. This paper presents a multimedia visualization system that augments the limited hearing capabilities of hearing impaired persons using visual and tactile feedback approaches and provides alternate representations of sound for increasing speech capabilities and reduced learning time. The proposed system has the capability of not only decreasing the learning time for many persons, but can also be adopted for teaching young children correct speech patterns through the use of speech learning games based on the system.     

Robot-assisted wayfinding for the visually impaired in structured indoor environments

We present a robot-assisted wayfinding system for the visually impaired in structured indoor environments. The system consists of a mobile robotic guide and small passive RFID sensors embedded in the environment. The system is intended for use in indoor environments, such as office buildings, supermarkets and airports. We describe how the system was deployed in two indoor environments and evaluated by visually impaired participants in a series of pilot experiments. We analyze the system’s successes and failures and outline our plans for future research and development.