An iterative integrated framework for thermal–visible image registration,sensor fusion,and people tracking for video surveillance applications

In this work, we propose a new integrated framework that addresses the problems of thermal–visible video registration, sensor fusion, and people tracking for far-range videos. The video registration is based on a RANSAC trajectory-to-trajectory matching, which estimates an affine transformation matrix that maximizes the overlapping of thermal and visible foreground pixels. Sensor fusion uses the aligned images to compute sum-rule silhouettes, and then constructs thermal–visible object models. Finally, multiple object tracking uses blobs constructed in sensor fusion to output the trajectories. Results demonstrate the advantage of our proposed framework in obtaining better results for both image registration and tracking than separate image registration and tracking methods.     

Fourier–Mellin registration of two hyperspectral images

Hyperspectral images contain a great amount of information which can be used to more robustly register such images. In this article, we present a phase correlation method to register two hyperspectral images that takes into account their multiband structure. The proposed method is based on principal component analysis, the multilayer fractional Fourier transform, a combination of log-polar maps, and peak processing. The combination of maps is aimed at highlighting some peaks in the log-polar map using information from different bands. The method is robust and has been successfully tested for any rotation angle with commonly used hyperspectral scenes in remote sensing for scales of up to 7.5× and with pairs of hyperspectral images taken on different dates by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor for scales of up to 6.0×.     

Older people’s production and appropriation of digital videos: an ethnographic study

While most of today’s children, young people, and adults are both consumers and producers of digital content, very little is known about older people as digital content creators. Drawing on a three-year ethnographic study, this paper reports on the digital video production and appropriation of approximately 200 older people (aged 60–85). They generated 320 videos over the course of the study. We show their motivations for engaging in digital video production, discuss their planned video making, and highlight their creativity while editing videos. We show the different meanings they ascribed to digital videos in their social appropriation of these objects, the meaningful strategies they adopted to share videos, and the impact on their perceived wellbeing. Furthermore, we outline the solutions the participants developed to overcome or cope with interaction issues they faced over time. We argue that the results portray older people as active and creative makers of digital videos with current video capturing, editing, and sharing technologies. We contend that this portrayal both encourages us to re-consider how older people should be seen within human–computer interaction and helps to frame future research/design activities that bridge the grey digital divide.     

Mapping canopy height using a combination of digital stereo‐photogrammetry and lidar

Ranging techniques such as lidar (LIght Detection And Ranging) and digital stereo‐photogrammetry show great promise for mapping forest canopy height. In this study, we combine these techniques to create hybrid photo‐lidar canopy height models (CHMs). First, photogrammetric digital surface models (DSMs) created using automated stereo‐matching were registered to corresponding lidar digital terrain models (DTMs). Photo‐lidar CHMs were then produced by subtracting the lidar DTM from the photogrammetric DSM. This approach opens up the possibility of retrospective mapping of forest structure using archived aerial photographs. The main objective of the study was to evaluate the accuracy of photo‐lidar CHMs by comparing them to reference lidar CHMs. The assessment revealed that stereo‐matching parameters and left–right image dissimilarities caused by sunlight and viewing geometry have a significant influence on the quality of the photo DSMs. Our study showed that photo‐lidar CHMs are well correlated to their lidar counterparts on a pixel‐wise basis (r up to 0.89 in the best stereo‐matching conditions), but have a lower resolution and accuracy. It also demonstrated that plot metrics extracted from the lidar and photo‐lidar CHMs, such as height at the 95th percentile of 20 m×20 m windows, are highly correlated (r up to 0.95 in general matching conditions).     

A comparison of muscle activity in using touchscreen smartphone among young people with and without chronic neck–shoulder pain

This study aimed to examine differences in muscle activity between young people with and without neck–shoulder pain (n = 20 in each group), when they performed texting on a smartphone. Texting was compared between using both hands (‘bilateral texting’) and with only one hand (‘unilateral texting’). Texting tasks were also compared with computer typing. Surface electromyography from three proximal postural muscles and four distal hand/thumb muscles on the right side was recorded. Compared with healthy controls, young people with neck–shoulder pain showed altered motor control consisting of higher muscle activity in the cervical erector spinae and upper trapezius when performing texting and typing tasks. Generally, unilateral texting was associated with higher muscle loading compared with bilateral texting especially in the forearm muscles. Compared with computer typing, smartphone texting was associated with higher activity in neck extensor and thumb muscles but lower activity in upper and lower trapezius as well as wrist extensors.     

An intuitionistic view of the Dempster–Shafer belief structure

We discuss the Dempster–Shafer belief theory and describe its role in representing imprecise probabilistic information. In particular, we note its use of intervals for representing imprecise probabilities. We note in fuzzy set theory that there are two related approaches used for representing imprecise membership grades: interval-valued fuzzy sets and intuitionistic fuzzy sets. We indicate the first of these, interval-valued fuzzy sets, is in the same spirit as Dempster–Shafer representation, both use intervals. Using a relationship analogous to the type of relationship that exists between interval-valued fuzzy sets and intuitionistic fuzzy sets, we obtain from the interval-valued view of the Dempster–Shafer model an intuitionistic view of the Dempster–Shafer model. Central to this view is the use of an intuitionistic statement, pair of values, (Bel(A) Dis(A)), to convey information about the value of a variable lying in the set A. We suggest methods for combining intuitionistic statements and making inferences from these type propositions.     

Time–frequency masking based supervised speech enhancement framework using fuzzy deep belief network

In recent years, deep learning based supervised speech enhancement methods have gained a considerable amount of research attention over the statistical signal processing based methods. In this study, we have considered the time–frequency masking based deep learning framework for speech enhancement and investigated how the performance of these methods can be improved further. We have mainly established that significant performance improvement can be achieved if the deep neural network (DNN) is pre-trained by using Fuzzy Restricted Boltzmann Machines (FRBM) rather than using regular Restricted Boltzmann Machines (RBM). This is mainly because of the fact that the performance of FRBM is more robust and effective when the training data is noisy. In order to train an FRBM, we have adopted a defuzzification method based on the crisp probabilistic mean value of fuzzy numbers. The detailed theory of training strategy of an FRBM with different fuzzy membership functions such as Symmetric Triangular Fuzzy Numbers (STFN) and Asymmetric Triangular Fuzzy Numbers (ATFN) is presented. Furthermore, we have evaluated the performance of the proposed training strategies on different DNN based Speech Enhancement Systems (SES) which are developed based on different training targets such as Complex Ideal Ratio Mask (cIRM), Ideal Ratio Mask (IRM) and Phase-Sensitive Mask (PSM). Experimental results on various noise scenarios have shown that the DNN-based speech enhancement system trained by the proposed approach ensures a consistent improvement in various objective measure scores of perceived speech quality and intelligibility while compared to the conventional DNN-based speech enhancement methods which use regular RBM for unsupervised pre-training.     

An approach to fault diagnosis of reciprocating compressor valves using Teager–Kaiser energy operator and deep belief networks

This paper presents an approach to implement vibration, pressure, and current signals for fault diagnosis of the valves in reciprocating compressors. Due to the complexity of structure and motion of such compressor, the acquired vibration signal normally involves transient impacts and noise. This causes the useful information to be corrupted and difficulty in accurately diagnosing the faults with traditional methods. To reveal the fault patterns contained in this signal, the Teager–Kaiser energy operation (TKEO) is proposed to estimate the amplitude envelopes. In case of pressure and current, the random noise is removed by using a denoising method based on wavelet transform. Subsequently, statistical measures are extracted from all signals to represent the characteristics of the valve conditions. In order to classify the faults of compressor valves, a new type of learning architecture for deep generative model called deep belief networks (DBNs) is applied. DBN employs a hierarchical structure with multiple stacked restricted Boltzmann machines (RBMs) and works through a greedy layer-by-layer learning algorithm. In pattern recognition research areas, DBN has proved to be very effective and provided with high performance for binary values. However, for implementing DBN to fault diagnosis where most of signals are real-valued, RBM with Bernoulli hidden units and Gaussian visible units is considered in this study. The proposed approach is validated with the signals from a two-stage reciprocating air compressor under different valve conditions. To confirm the superiority of DBN in fault classification, its performance is compared with that of relevant vector machine and back propagation neuron networks. The achieved accuracy indicates that the proposed approach is highly reliable and applicable in fault diagnosis of industrial reciprocating machinery.     

A Gauss–Newton approach to joint image registration and intensity correction

We develop a new efficient numerical methodology for automated simultaneous registration and intensity correction of images. The approach separates the intensity correction term from the images being registered in a regularized expression. Our formulation is consistent with the existing non-parametric image registration techniques, however, an extra additive intensity correction term is carried throughout. An objective functional is formed for which the corresponding Hessian and Jacobian is computed and employed in a multi-level Gauss–Newton minimization approach. In this paper, our experiments are based on elastic regularization on the transformation and total variation on the intensity correction. Validations on dynamic contrast enhanced MR abdominal images for both real and simulated data verified the efficacy of the model.     

Multimodality image registration by maximization of quantitative–qualitative measure of mutual information

This paper presents a novel image similarity measure, referred to as quantitative–qualitative measure of mutual information (Q-MI), for multimodality image registration. Conventional information measures, e.g., Shannon's entropy and mutual information (MI), reflect quantitative aspects of information because they only consider probabilities of events. In fact, each event has its own utility to the fulfillment of the underlying goal, which can be independent of its probability of occurrence. Thus, it is important to consider both quantitative (i.e., probability) and qualitative (i.e., utility) measures of information in order to fully capture the characteristics of events. Accordingly, in multimodality image registration, Q-MI should be used to integrate the information obtained from both the image intensity distributions and the utilities of voxels in the images. Different voxels can have different utilities, for example, in brain images, two voxels can have the same intensity value, but their utilities can be different, e.g., a white matter (WM) voxel near the cortex can have higher utility than a WM voxel inside a large uniform WM region. In Q-MI, the utility of each voxel in an image can be determined according to the regional saliency value calculated from the scale-space map of this image. Since the voxels with higher utility values (or saliency values) contribute more in measuring Q-MI of the two images, the Q-MI-based registration method is much more robust, compared to conventional MI-based registration methods. Also, the Q-MI-based registration method can provide a smoother registration function with a relatively larger capture range. In this paper, the proposed Q-MI has been validated and applied to the rigid registrations of clinical brain images, such as MR, CT and PET images.     

Aerial cooperative transporting and assembling control using multiple quadrotor–manipulator systems

In this paper, a fully distributed control scheme for aerial cooperative transporting and assembling is proposed using multiple quadrotor–manipulator systems with each quadrotor equipped with a robotic manipulator. First, the kinematic and dynamic models of a quadrotor with multi-Degree of Freedom (DOF) robotic manipulator are established together using Euler–Lagrange equations. Based on the aggregated dynamic model, the control scheme consisting of position controller, attitude controller and manipulator controller is presented. Regarding cooperative transporting and assembling, multiple quadrotor–manipulator systems should be able to form a desired formation without collision among quadrotors from any initial position. The desired formation is achieved by the distributed position controller and attitude controller, while the collision avoidance is guaranteed by an artificial potential function method. Then, the transporting and assembling tasks request the manipulators to reach the desired angles cooperatively, which is achieved by the distributed manipulator controller. The overall stability of the closed-loop system is proven by a Lyapunov method and Matrosov's theorem. In the end, the proposed control scheme is simplified for the real application and then validated by two formation flying missions of four quadrotors with 2-DOF manipulators.     

Analysis of multiple types of voice recordings in cepstral domain using MFCC for discriminating between patients with Parkinson’s disease and healthy people

In this study, we wanted to discriminate between two groups of people. The database used in this study contains 20 patients with Parkinson’s disease (PD) and 20 healthy people. Three types of sustained vowels (/a/, /o/ and /u/) were recorded from each participant and then the analyses were done on these voice samples. The technique used in this study is to extract voiceprint from each voice samples by using mel frequency cepstral coefficients (MFCCs). The extracted MFCC were compressed by calculating their average value in order to extract the voiceprint from each voice recording. Subsequently, a classification method was performed using leave one subject out (LOSO) validation scheme along with support vector machines (SVMs). We also used an independent test to validate our results by using another database which contains 28 PD patients. Based on the research result, the best obtained classification accuracy using LOSO on the first dataset was 82.50 % using MLP kernel of SVM on sustained vowel /u/. And the maximum classification accuracy using the independent test was 100 % using sustained vowel /a/ with polynomial kernel of the SVM and with MLP kernel of the SVM. This result was also achieved using sustained vowel /o/ with polynomial kernel of the SVM.     

Modelling of urban sensible heat flux at multiple spatial scales: A demonstration using airborne hyperspectral imagery of Shanghai and a temperature–emissivity separation approach

Urbanization related alterations to the surface energy balance impact urban warming (‘heat islands’), the growth of the boundary layer, and many other biophysical processes. Traditionally, in situ heat flux measures have been used to quantify such processes, but these typically represent only a small local-scale area within the heterogeneous urban environment. For this reason, remote sensing approaches are very attractive for elucidating more spatially representative information. Here we use hyperspectral imagery from a new airborne sensor, the Operative Modular Imaging Spectrometer (OMIS), along with a survey map and meteorological data, to derive the land cover information and surface parameters required to map spatial variations in turbulent sensible heat flux (Q_H). The results from two spatially-explicit flux retrieval methods which use contrasting approaches and, to a large degree, different input data are compared for a central urban area of Shanghai, China: (1) the Local-scale Urban Meteorological Parameterization Scheme (LUMPS) and (2) an Aerodynamic Resistance Method (ARM). Sensible heat fluxes are determined at the full 6 m spatial resolution of the OMIS sensor, and at lower resolutions via pixel aggregation and spatial averaging. At the 6 m spatial resolution, the sensible heat flux of rooftop dominated pixels exceeds that of roads, water and vegetated areas, with values peaking at ～ 350 W m^? 2, whilst the storage heat flux is greatest for road dominated pixels (peaking at around 420 W m^? 2). We investigate the use of both OMIS-derived land surface temperatures made using a Temperature–Emissivity Separation (TES) approach, and land surface temperatures estimated from air temperature measures. Sensible heat flux differences from the two approaches over the entire 2 × 2 km study area are less than 30 W m^? 2, suggesting that methods employing either strategy maybe practica1 when operated using low spatial resolution (e.g. 1 km) data. Due to the differing methodologies, direct comparisons between results obtained with the LUMPS and ARM methods are most sensibly made at reduced spatial scales. At 30 m spatial resolution, both approaches produce similar results, with the smallest difference being less than 15 W m^? 2 in mean Q_H averaged over the entire study area. This is encouraging given the differing architecture and data requirements of the LUMPS and ARM methods. Furthermore, in terms of mean study Q_H, the results obtained by averaging the original 6 m spatial resolution LUMPS-derived Q_H values to 30 and 90 m spatial resolution are within ～ 5 W m^? 2 of those derived from averaging the original surface parameter maps prior to input into LUMPS, suggesting that that use of much lower spatial resolution spaceborne imagery data, for example from Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) is likely to be a practical solution for heat flux determination in urban areas.     

Optimization of pin through hole connector in thermal fluid–structure interaction analysis of wave soldering process using response surface methodology

This paper presents an optimization of pin through hole (PTH) connector in the wave soldering process; the optimization was performed by using response surface methodology. The geometrical and process parameters (i.e., offset position, pin diameter, offset angle, and solder temperature) were optimized by using response surface methodology via central composite design for the wave soldering process. Thermal fluid–structure interaction aspects were considered in the optimization. A mesh-based parallel code-coupling interface was employed to connect both fluid and structural solvers. The interactive relationship between independent variables (i.e., offset position, pin diameter, offset angle, and solder temperature) and the responses (i.e., filling time at 75% volume, von Mises stress, and maximum displacement) were investigated. The generated empirical models were examined and well substantiated by the simulation results. The optimum geometrical and process parameters of the wave soldering process for the PCB and PTH connector were as follows: 0.12 mm of PTH offset position, 0.17 mm of PTH diameter, 0° of offset angle, and 473 K of molten solder temperature.     

Introducing a novel model of belief–desire–intention agent for urban land use planning

Land use planning is a potentially demanding search and optimization task that has been challenged by numerous researchers in the field of spatial planning. Agent and multi-agent systems are examples of the modern concepts, which have been gaining more attention in challenging spatial issues recently. Although the efficiency of belief, desire, and intention (BDI) architecture of agents is validated in varieties of sciences, its uses in Geospatial Information Systems (GIS) and specifically among spatial planners is still burgeoning. In this paper, we attempted to integrate the concepts of BDI agent architecture into spatial issues; as a result, a novel spatial agent model is designed and implemented to analyze the urban land use planning. The proposed approach was checked in urban land use planning problems using a case study in a municipal area. The result of implementation showed the effects of spatial agents' behaviors such as intention, commitment, and interaction on their decision.     

A framework for exploring numerical solutions of advection–reaction–diffusion equations using a GPU-based approach

In this paper we describe a general purpose, graphics processing unit (GP-GPU)-based approach for solving partial differential equations (PDEs) within advection–reaction–diffusion models. The GP-GPU-based approach provides a platform for solving PDEs in parallel and can thus significantly reduce solution times over traditional CPU implementations. This allows for a more efficient exploration of various advection–reaction–diffusion models, as well as, the parameters that govern them. Although the GPU does impose limitations on the size and accuracy of computations, the PDEs describing the advection–reaction–diffusion models of interest to us fit comfortably within these constraints. Furthermore, the GPU technology continues to rapidly increase in speed, memory, and precision, thus applying these techniques to larger systems should be possible in the future. We chose to solve the PDEs using two numerical approaches: for the diffusion, a first-order explicit forward Euler solution and a semi-implicit second order Crank–Nicholson solution; and, for the advection and reaction, a first-order explicit solution. The goal of this work is to provide motivation and guidance to the application scientist interested in exploring the use of the GP-GPU computational framework in the course of their research. In this paper, we present a rigorous comparison of our GPU-based advection–reaction–diffusion code model with a CPU-based analog, finding that the GPU model out-performs the CPU implementation in one-to-one comparisons.     

Fabrication of SU-8 photoresist micro–nanofluidic chips by thermal imprinting and thermal bonding

Micro–nanofluidic chips have been widely applied in biological and medical fields. In this paper, a simple and low-cost fabrication method for micro–nano fluidic chips is proposed. The nano-channels are fabricated by thermal nano-imprinting on an SU-8 photoresist layer followed by thermal bonding with a second SU-8 photoresist layer. The micro-channels are produced on the second layer by UV exposure and then thermal bonded by a third layer of SU-8 photoresist. The final micro–nano fluidic chip consists of micro-channels (width of 200.0 ± 0.1 μm and, depth of 8.0 ± 0.1 μm) connected by nano-channels (width of 533 ± 6 nm and, depth of 372 ± 6 nm), which has great potential in molecular filtering and detection.

     

A novel concept of a wearable information appliance using context-based human–computer interaction

In healthcare environment, different kinds of automatic solutions have been created to monitor and track patients, for example near-field imaging and low-frequency RFID. The problem has been how to use the context-based data these systems produce and how to show the related information to the nursing staff. This paper shows how hospital data can be automatically transmitted to people using location information. The information is transmitted to a name tag that has wireless connectivity and touch screen with electric paper. This concept is piloted with a test application.     

Online blind speech separation using multiple acoustic speaker tracking and time–frequency masking

Separating speech signals of multiple simultaneous talkers in a reverberant enclosure is known as the cocktail party problem. In real-time applications online solutions capable of separating the signals as they are observed are required in contrast to separating the signals offline after observation. Often a talker may move, which should also be considered by the separation system. This work proposes an online method for speaker detection, speaker direction tracking, and speech separation. The separation is based on multiple acoustic source tracking (MAST) using Bayesian filtering and time–frequency masking. Measurements from three room environments with varying amounts of reverberation using two different designs of microphone arrays are used to evaluate the capability of the method to separate up to four simultaneously active speakers. Separation of moving talkers is also considered. Results are compared to two reference methods: ideal binary masking (IBM) and oracle tracking (O-T). Simulations are used to evaluate the effect of number of microphones and their spacing.