Treetop detection using convolutional neural networks trained through automatically generated pseudo labels

ABSTRACT

Using remote sensing techniques to detect trees at the individual level is crucial for forest management while finding the treetop is an initial and important first step. However, due to the large variations of tree size and shape, traditional unsupervised treetop detectors need to be carefully designed with heuristic knowledge making an efficient and versatile treetop detection still challenging. Currently, the deep convolutional neural networks (CNNs) have shown powerful capabilities to classify and segment images, but the required volume of labelled data for the training impedes their applications. Considering the strengths and limitations of the unsupervised and deep learning methods, we propose a framework using the automatically generated pseudo labels from unsupervised treetop detectors to train the CNNs, which saves the manual labelling efforts. In this study, we use multi-view satellite imagery derived digital surface model (DSM) and multispectral orthophoto as research data and train the fully convolutional networks (FCN) with pseudo labels separately generated from two unsupervised treetop detectors: top-hat by reconstruction (THR) operation and local maxima filter with a fixed window (FFW). The experiments show the FCN detectors trained by pseudo labels, have much better detection accuracies than the unsupervised detectors (6.5% for THR and 11.1% for FFW), especially in the densely forested area (more than 20% of improvement). In addition, our comparative experiments when using manually labelled samples show the proposed treetop detection framework has the potential to significantly reduce the need for training samples while keep a comparable performance.     

Fast and robust multiple ColorChecker detection using deep convolutional neural networks

ColorCheckers are reference standards that professional photographers and filmmakers use to ensure predictable results under every lighting condition. The objective of this work is to propose a new fast and robust method for automatic ColorChecker detection. The process is divided into two steps: (1) ColorCheckers localization and (2) ColorChecker patches recognition. For the ColorChecker localization, we trained a detection convolutional neural network using synthetic images. The synthetic images are created with the 3D models of the ColorChecker and different background images. The output of the neural networks are the bounding box of each possible ColorChecker candidates in the input image. Each bounding box defines a cropped image which is evaluated by a recognition system, and each image is canonized with regards to color and dimensions. Subsequently, all possible color patches are extracted and grouped with respect to the center's distance. Each group is evaluated as a candidate for a ColorChecker part, and its position in the scene is estimated. Finally, a cost function is applied to evaluate the accuracy of the estimation. The method is tested using real and synthetic images. The proposed method is fast, robust to overlaps and invariant to affine projections. The algorithm also performs well in case of multiple ColorCheckers detection.     

Smile detection in the wild with deep convolutional neural networks

Smile or happiness is one of the most universal facial expressions in our daily life. Smile detection in the wild is an important and challenging problem, which has attracted a growing attention from affective computing community. In this paper, we present an efficient approach for smile detection in the wild with deep learning. Different from some previous work which extracted hand-crafted features from face images and trained a classifier to perform smile recognition in a two-step approach, deep learning can effectively combine feature learning and classification into a single model. In this study, we apply the deep convolutional network, a popular deep learning model, to handle this problem. We construct a deep convolutional network called Smile-CNN to perform feature learning and smile detection simultaneously. Experimental results demonstrate that although a deep learning model is generally developed for tackling “big data,” the model can also effectively deal with “small data.” We further investigate into the discriminative power of the learned features, which are taken from the neuron activations of the last hidden layer of our Smile-CNN. By using the learned features to train an SVM or AdaBoost classifier, we show that the learned features have impressive discriminative ability. Experiments conducted on the GENKI4K database demonstrate that our approach can achieve a promising performance in smile detection.     

Cloud/haze detection in airborne videos using a convolutional neural network

Multimedia Tools and Applications - In airborne videos surveillance, moving object detection and target tracking are the key steps. However, under bad weather conditions, the presence of clouds and...     

3D多尺度深度卷积神经网络肺结节检测

目的 肺结节是肺癌的早期存在形式。低剂量CT（computed tomogragphy）扫描作为肺癌筛查的重要检查手段,已经大规模应用于健康体检,但巨大的CT数据带来了大量工作,随着人工智能技术的快速发展,基于深度学习的计算机辅助肺结节检测引起了关注。由于肺结节尺寸差别较大,在多个尺度上表示特征对结节检测任务至关重要。针对结节尺寸差别较大导致的结节检测困难问题,提出一种基于深度卷积神经网络的胸部CT序列图像3D多尺度肺结节检测方法。方法 包括两阶段：1）尽可能提高敏感度的结节初检网络;2）尽可能减少假阳性结节数量的假阳性降低网络。在结节初检网络中,以组合了压缩激励单元的Res2Net网络为骨干结构,使同一层卷积具有多种感受野,提取肺结节的多尺度特征信息,并使用引入了上下文增强模块和空间注意力模块的区域推荐网络结构,确定候选区域;在由Res2Net网络模块和压缩激励单元组成的假阳性降低网络中对候选结节进一步分类,以降低假阳性,获得最终结果。结果 在公共数据集LUNA16（lung nodule analysis 16）上进行实验,实验结果表明,对于结节初检网络阶段,当平均每例假阳性个数为22时,敏感度可达到0.983,相比基准ResNet + FPN（feature pyramid network）方法,平均敏感度和最高敏感度分别提高了2.6%和0.8%;对于整个3D多尺度肺结节检测网络,当平均每例假阳性个数为1时,敏感度为0.924。结论 与现有主流方案相比,该检测方法不但提高了肺结节检测的敏感度,还有效地控制了假阳性,取得了更优的性能。     

Violence detection in videos using interest frame extraction and 3D convolutional neural network

Multimedia Tools and Applications - With the rapid development of detecting violent behaviors in surveillance cameras, requests on systems that automatically recognize violent events are expanded....     

Surface defect detection of voltage-dependent resistors using convolutional neural networks

Surface defect detection is an important way to improve the production quality of voltage-dependent resistors (VDRs). To improve the accuracy and efficiency of VDR surface quality detection, an end-to-end surface quality detection method based on deep convolutional neural networks (CNNs) was proposed. The method includes four stages: data preparation, convolution neural network design, CNN training, and testing. First, images of VDRs were acquired from three perspectives, i.e., the front, back, and side, and then training, validation and testing sets were obtained. Second, the proposed CNN models for VDR surface defect detection were constructed. Third, during the training stage, the images with class labels from the established training sets were input to the proposed network for training and validation. Finally, in the testing stage, test images from a total of 408 samples of two VDR models were used to test the trained network. The sensitivity, specificity, accuracy, precision and F measure of the proposed algorithm were compared with those of state-of-the-art methods, and the experimental results showed that the proposed method has a high recognition speed and accuracy and meets the requirements of online real-time detection.

     

Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks

The 2019 novel coronavirus disease (COVID-19), with a starting point in China, has spread rapidly among people living in other countries and is approaching approximately 101,917,147 cases worldwide according to the statistics of World Health Organization. There are a limited number of COVID-19 test kits available in hospitals due to the increasing cases daily. Therefore, it is necessary to implement an automatic detection system as a quick alternative diagnosis option to prevent COVID-19 spreading among people. In this study, five pre-trained convolutional neural network-based models (ResNet50, ResNet101, ResNet152, InceptionV3 and Inception-ResNetV2) have been proposed for the detection of coronavirus pneumonia-infected patient using chest X-ray radiographs. We have implemented three different binary classifications with four classes (COVID-19, normal (healthy), viral pneumonia and bacterial pneumonia) by using five-fold cross-validation. Considering the performance results obtained, it has been seen that the pre-trained ResNet50 model provides the highest classification performance (96.1% accuracy for Dataset-1, 99.5% accuracy for Dataset-2 and 99.7% accuracy for Dataset-3) among other four used models.

     

基于深度时空卷积神经网络的人群异常行为检测和定位

针对公共场合人群异常行为检测准确率不高和训练样本缺乏的问题,提出一种基于深度时空卷积神经网络的人群异常行为检测和定位的方法。首先针对监控视频中人群行为的特点,综合利用静态图像的空间特征和前后帧的时间特征,将二维卷积扩展到三维空间,设计面向人群异常行为检测和定位的深度时空卷积神经网络;为了定位人群异常行为,将视频分成若干子区域,获取视频的子区域时空数据样本,然后将数据样本输入设计的深度时空卷积神经网络进行训练和分类,实现人群异常行为的检测与定位。同时,为了解决深度时空卷积神经网络训练时样本数量不足的问题,设计一种迁移学习的方法,利用样本数量多的数据集预训练网络,然后在待测试的数据集中进行微调和优化网络模型。实验结果表明,该方法在UCSD和subway公开数据集上的检测准确率分别达到了99%和93%以上。     

Field-based robotic leaf angle detection and characterization of maize plants using stereo vision and deep convolutional neural networks

Maize (Zea mays L.) is one of the three major cereal crops in the world. Leaf angle is an important architectural trait of crops due to its substantial role in light interception by the canopy and hence photosynthetic efficiency. Traditionally, leaf angle has been measured using a protractor, a process that is both slow and laborious. Efficiently measuring leaf angle under field conditions via imaging is challenging due to leaf density in the canopy and the resulting occlusions. However, advances in imaging technologies and machine learning have provided new tools for image acquisition and analysis that could be used to characterize leaf angle using three-dimensional (3D) models of field-grown plants. In this study, PhenoBot 3.0, a robotic vehicle designed to traverse between pairs of agronomically spaced rows of crops, was equipped with multiple tiers of PhenoStereo cameras to capture side-view images of maize plants in the field. PhenoStereo is a customized stereo camera module with integrated strobe lighting for high-speed stereoscopic image acquisition under variable outdoor lighting conditions. An automated image processing pipeline (AngleNet) was developed to measure leaf angles of nonoccluded leaves. In this pipeline, a novel representation form of leaf angle as a triplet of keypoints was proposed. The pipeline employs convolutional neural networks to detect each leaf angle in two-dimensional images and 3D modeling approaches to extract quantitative data from reconstructed models. Satisfactory accuracies in terms of correlation coefficient (r) and mean absolute error (MAE) were achieved for leaf angle ($r>0.87,MAE<5^{°}$) and internode heights ($r>0.99,MAE<3.5\mathrm{cm}$). Our study demonstrates the feasibility of using stereo vision to investigate the distribution of leaf angles in maize under field conditions. The proposed system is an efficient alternative to traditional leaf angle phenotyping and thus could accelerate breeding for improved plant architecture.     

Detection of novel coronavirus from chest X-rays using deep convolutional neural networks

Multimedia Tools and Applications - With over 172 Million people infected with the novel coronavirus (COVID-19) globally and with the numbers increasing exponentially, the dire need of a fast...     

Classification of multiple motor imagery using deep convolutional neural networks and spatial filters

Brain–Computer Interfaces (BCI) are systems that translate brain activity patterns into commands for an interactive application, and some of them recognize patterns generated by motor imagery. Currently, these systems present performances and methodologies that still are not practical enough for realistic applications. Therefore, this paper proposes two methodologies for multiple motor imagery classification. Both methodologies use features extracted by a variant of Discriminative Filter Bank Common Spatial Pattern (DFBCSP) presented in this paper. The frequency bands selection in this variant is carried out by a novel iterative algorithm that selects the frequency band that attains the highest classification accuracy for specific binary classification. For each binary combination of classes, a frequency band is selected. The resulting samples are then set into a matrix which feeds one or many Convolutional Neural Networks previously optimized by using a Bayesian optimization. The first methodology applies a Convolutional Neural Network (CNN) for the classification of all classes and the second is a modular network composed of four expert CNNs. In this modular network, each expert CNN performs a binary classification, and a fully connected network analyzes their results. To validate both approaches two datasets were used, the BCI competition IV dataset 2a and another presented in this paper recorded from eight subjects by using the OpenBCI device. The experimental results demonstrated an improvement in the classification accuracy over many classic intelligent recognition methods, without a high computation time in order that they can be implemented in an online application.     

Localization of the slab information in factory scenes using deep convolutional neural networks

This paper proposes a novel algorithm for localizing slab identification numbers (SINs) in factory scenes. Automatic identification of product information is important for the process management, and localization of SINs in complex scenes is a major challenge for the recognition. A previous rule-based localization algorithm for SINs requires lots of prior knowledge and heuristic tuning for parameters. In this paper, a deep convolutional neural network (DCNN) is employed to overcome these limitations, and accumulated confidence is proposed to utilize neighboring outputs of the DCNN in a scene. The localization error is remarkably reduced to 1.44% by the proposed algorithm compared to 4.59% in the previous work. The proposed data-driven method can be applied to construct other automatic identification systems with minimal manual handling.     

Multifocus image fusion using convolutional neural networks in the discrete wavelet transform domain

Multimedia Tools and Applications - In this paper, a novel multifocus image fusion algorithm based on the convolutional neural network (CNN) in the discrete wavelet transform (DWT) domain is...     

Skin melanoma classification using ROI and data augmentation with deep convolutional neural networks

Multimedia Tools and Applications - Automatic classification of color images of skin helps clinicians and dermatologists in examining and investigating skin melanoma. In this paper, a new deep...     

深度全卷积网络的IVUS图像内膜与中—外膜边界检测

目的 心血管内超声（IVUS）图像内膜和中—外膜（MA）轮廓勾画是冠脉粥样硬化和易损斑块定量评估的必要过程。由于存在斑点噪声、图像伪影和各类斑块,重要组织边界的自动分割是一个非常困难的任务。为此,提出一种用于检测20 MHz心电门控IVUS图像内膜和MA边界方法。方法 首先利用深度全卷积网络（DFCN）学习原始IVUS图像与所对应手动分割图像之间映射,预测出目标或者背景的概率图,实现医学图像语义分割。然后在此基础上,结合心血管先验形状信息,采用数学形态学闭、开操作,平滑内膜和MA边界,降低分割过程中错误分类像素或区域的影响。结果 针对来自10位病人的IVUS图像及其标注信息所组成的435幅国际标准公开数据集,从线性回归、Bland-Altman分析和面积交并比（JM）、面积差异百分比（PAD）、Hausdorff距离（HD）、平均距离（AD）等性能指标上,评价本文方法。实验结果表明,算法检测结果与手动勾画结果的相关性可达到0.94,其超过94.71%的结果落在95%置信区域内,具有良好一致性。内膜和MA边界的AD指标分别为：0.07 mm和0.08 mm;HD指标分别为：0.21 mm和0.30 mm。JM指标分别为0.92和0.93;PAD指标分别为5%和4%。此外,对临床所采集的100幅IVUS图像进行了测试,证明本文学习的模型在跨数据集上具有较好的泛化能力。结论 与现有的国际算法比较,本文方法提高了各类斑块、声影区域和血管分支等因素的识别能力,不受超声斑点的影响,能准确地、可重复地检测出IVUS图像中的关键目标边界。     

深度卷积神经网络在目标检测中的研究进展

深度卷积神经网络以多层次的特征学习与丰富的特征表达能力,在目标检测领域取得了突破进展。概括了卷积神经网络在目标检测领域的研究进展,首先回顾传统目标检测的发展及存在的问题,引出卷积神经网络的目标检测基本原理和基本训练方法;然后分析了以R-CNN为代表的基于区域建议的目标检测框架,介绍以YOLO算法为代表的将目标检测归结为回归问题的目标检测框架;最后,对目前目标检测的一些问题进行简要总结,对未来深度卷积神经网络在目标检测的发展进行了展望。     

Accurate segmentation of small targets for LCD defects using deep convolutional neural networks

For the purpose of effectively solving the problems of manual inspection of thin-film transistor–liquid crystal display (TFT-LCD) with spot Mura high leakage rate and time consuming, this article proposes a visual detection method for LCD defects based on Mask R-CNN convolutional neural network. In this article, a new feature extraction network is built by fusing ResNet with efficient channel attention (ECA) channel attention mechanism, and a new feature fusion network is constructed by adding ECA channel attention mechanism to Feature Pyramid Network (FPN). In the dataset of self-made display defect images collected at the production site, the current more advanced segmentation network was compared qualitatively and quantitatively with our method, and the results showed that the proposed method was more accurate in detecting spot Mura, with a detection accuracy of 91%.     

Recent advances in efficient computation of deep convolutional neural networks

Deep neural networks have evolved remarkably over the past few years and they are currently the fundamental tools of many intelligent systems. At the same time, the computational complexity and resource consumption of these networks continue to increase. This poses a significant challenge to the deployment of such networks, especially in real-time applications or on resource-limited devices. Thus, network acceleration has become a hot topic within the deep learning community. As for hardware implementation of deep neural networks, a batch of accelerators based on a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC) have been proposed in recent years. In this paper, we provide a comprehensive survey of recent advances in network acceleration, compression, and accelerator design from both algorithm and hardware points of view. Specifically, we provide a thorough analysis of each of the following topics: network pruning, low-rank approximation, network quantization, teacher–student networks, compact network design, and hardware accelerators. Finally, we introduce and discuss a few possible future directions.     

Detection and classification of lung nodules in chest X-ray images using deep convolutional neural networks

Lung nodule classification is one of the main topics related to computer-aided detection systems. Although convolutional neural networks (CNNs) have been demonstrated to perform well on many tasks, there are few explorations of their use for classifying lung nodules in chest X-ray (CXR) images. In this work, we proposed and analyzed a pipeline for detecting lung nodules in CXR images that includes lung area segmentation, potential nodule localization, and nodule candidate classification. We presented a method for classifying nodule candidates with a CNN trained from the scratch. The effectiveness of our method relies on the selection of data augmentation parameters, the design of a specialized CNN architecture, the use of dropout regularization on the network, inclusive in convolutional layers, and addressing the lack of nodule samples compared to background samples balancing mini-batches on each stochastic gradient descent iteration. All model selection decisions were taken using a CXR subset of the Lung Image Database Consortium and Image Database Resource Initiative dataset separately. Thus, we used all images with nodules in the Japanese Society of Radiological Technology dataset for evaluation. Our experiments showed that CNNs were capable of achieving competitive results when compared to state-of-the-art methods. Our proposal obtained an area under the free-response receiver operating characteristic curve of 7.76 considering 10 false positives per image (FPPI), and sensitivity values of 73.1% and 79.6% with 2 and 5 FPPI, respectively.