多源域混淆的双流深度迁移学习

目的 针对深度学习严重依赖大样本的问题,提出多源域混淆的双流深度迁移学习方法,提升了传统深度迁移学习中迁移特征的适用性。方法 采用多源域的迁移策略,增大源域对目标域迁移特征的覆盖率。提出两阶段适配学习的方法,获得域不变的深层特征表示和域间分类器相似的识别结果,将自然光图像2维特征和深度图像3维特征进行融合,提高小样本数据特征维度的同时抑制了复杂背景对目标识别的干扰。此外,为改善小样本机器学习中分类器的识别性能,在传统的softmax损失中引入中心损失,增强分类损失函数的惩罚监督能力。结果 在公开的少量手势样本数据集上进行对比实验,结果表明,相对于传统的识别模型和迁移模型,基于本文模型进行识别准确率更高,在以DenseNet-169为预训练网络的模型中,识别率达到了97.17%。结论 利用多源域数据集、两阶段适配学习、双流卷积融合以及复合损失函数,构建了多源域混淆的双流深度迁移学习模型。所提模型可增大源域和目标域的数据分布匹配率、丰富目标样本特征维度、提升损失函数的监督性能,改进任意小样本场景迁移特征的适用性。     

一种新颖的单目视觉深度学习算法:H_SFPN

针对单目视觉目标检测,提出了一种基于single-stage深度学习的H_SFPN算法。该算法与现有的YOLOv3和CenterNet算法相比,在保证实时性能的条件下,可有效提高小目标检测的准确度。首先设计了一种新的网络架构(backbone),这种架构通过改进的沙漏(Hourglass)网络模型来提取特征图,以便充分利用底层特征的高分辨率以及高层特征的高语义信息。然后在特征图融合阶段提出了基于SFPN的特征图加权融合方法。最后,H_SFPN算法对目标位置和大小的损失函数进行了改进,可有效降低训练误差,并加快收敛速度。由MSCOCO数据集上的实验结果可知,所提H_SFPN算法明显优于Faster-RCNN,YOLOv3以及EfficientDet等现有的主流深度学习目标检测算法,其中对小目标的检测指标AP s最高,达到了32.7。     

Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions

Activity detection and classification using different sensor modalities have emerged as revolutionary technology for real-time and autonomous monitoring in behaviour analysis, ambient assisted living, activity of daily living (ADL), elderly care, rehabilitations, entertainments and surveillance in smart home environments. Wearable devices, smart-phones and ambient environments devices are equipped with variety of sensors such as accelerometers, gyroscopes, magnetometer, heart rate, pressure and wearable camera for activity detection and monitoring. These sensors are pre-processed and different feature sets such as time domain, frequency domain, wavelet transform are extracted and transform using machine learning algorithm for human activity classification and monitoring. Recently, deep learning algorithms for automatic feature representation have also been proposed to lessen the burden of reliance on handcrafted features and to increase performance accuracy. Initially, one set of sensor data, features or classifiers were used for activity recognition applications. However, there are new trends on the implementation of fusion strategies to combine sensors data, features and classifiers to provide diversity, offer higher generalization, and tackle challenging issues. For instances, combination of inertial sensors provide mechanism to differentiate activity of similar patterns and accurate posture identification while other multimodal sensor data are used for energy expenditure estimations, object localizations in smart homes and health status monitoring. Hence, the focus of this review is to provide in-depth and comprehensive analysis of data fusion and multiple classifier systems techniques for human activity recognition with emphasis on mobile and wearable devices. First, data fusion methods and modalities were presented and also feature fusion, including deep learning fusion for human activity recognition were critically analysed, and their applications, strengths and issues were identified. Furthermore, the review presents different multiple classifier system design and fusion methods that were recently proposed in literature. Finally, open research problems that require further research and improvements are identified and discussed.     

基于多特征融合的医学图像识别研究*

以医学图像为研究对象,针对任何一类特征都不能很好地表达医学图像的缺点以及进一步提高医学图像的识别率,提出了一种基于特征级数据融合与决策级数据融合相结合的分类方法。实验结果表明,采用特征级数据融合,融合后的特征可以较好地表达医学图像,且减少了后期分类的计算量;采用决策级数据融合,取得了比单个分类器更高的识别率。     

基于深度序列加权核极限学习的入侵检测算法

针对海量多源异构且数据分布不平衡的网络入侵检测问题以及传统深度学习算法无法根据实时入侵情况在线更新其输出权重的问题,提出了一种基于深度序列加权核极限学习的入侵检测算法（DBN-WOS-KELM算法）。该算法先使用深度信念网络DBN对历史数据进行学习,完成对原始数据的特征提取和数据降维,再利用加权序列核极限学习机进行监督学习完成入侵识别,结合了深度信念网络提取抽象特征的能力以及核极限学习机的快速学习能力。最后在部分KDD99数据集上进行了仿真实验,实验结果表明DBN-WOS-KELM算法提高了对小样本攻击的识别率,并且能够根据实际情况在线更新输出权重,训练效率更高。     

基于小样本不均衡数据的供水管道泄漏智能检测算法

针对能源电厂供水管道泄漏视觉检测存在数据样本少、不均衡等问题,提出一种基于小样本不 均衡数据的供水管道泄漏智能检测算法。首先,提出一种基于多掩码混合 Multi-mask mix 的数据增强方法,通 过随机生成掩码层对原始图像进行区域提取与混合,在 Multi-mask mix 中引入支持向量机(SVM)获取管道正常 和泄漏特征,为混合掩码块提供更准确的先验标签;其次,提出一种均衡化策略并应用于图像层面和掩码层面, 以实现数据均衡化;最后,基于深度学习的 Resnet18 网络模型实现管道泄漏检测与识别。实验结果表明,该算 法处理图像数据后可使 Resnet18 模型对管道泄漏识别准确率提升 1.1% ~ 4.4%,说明深度学习模型能有效提升 管道泄漏检测的分类精度,优于现有其他算法。此外,该算法现已成功应用于能源电厂供水管道泄漏检测。     

Dynamic Routing-based Multimodal Neural Network for Multi-sensory Fault Diagnosis of Induction Motor

Induction motor is the main drive power in modern manufacturing, and timely fault diagnosis of induction motor is of significance to production safety, part quality and maintenance cost control. Data fusion-based diagnosis is attractive for effective utilization of multi-source monitoring information of motors with the development of industrial internet of things. A new multi-sensory fusion model is proposed, named dynamic routing-based multimodal neural network (DRMNN), following the paradigm of multimodal deep learning (MDL). Specifically, the fusion of vibration and stator current signals are investigated. A multimodal feature extraction scheme is designed for dimensionality reduction and invariant features capturing based on multi-source information. Since it is necessary to determine the importance of each modality, a dynamic routing algorithm is introduced in the decision layer to adaptively assign proper weights to different modalities. The effectiveness and robustness of developed DRMNN is demonstrated in the experimental studies performed on a motor test rig. In comparison with similar neural networks without data fusion and other state-of-art fusion techniques, the proposed DRMNN yields better performance.     

深度学习背景下视觉显著性物体检测综述

视觉显著性物体检测是对人类视觉和认知系统的模拟,而深度学习则是对人类大脑计算方式的模拟,将两者有机结合可以有效推动计算机视觉的发展。视觉显著性物体检测的任务是从图像中定位并提取具有明确轮廓的显著性物体实例。随着深度学习的发展,视觉显著性物体检测的精度和效率都得到巨大提升,但仍然面临改进主流算法性能、减少对像素级标注样本的依赖等主要挑战。针对上述挑战,本文从视觉显著性物体检测思想与深度学习方法融合策略的角度对相关论述进行分类总结。1）分析传统显著性物体检测方法带来的启示及其缺点,指出视觉显著性物体检测的核心思路为多层次特征的提取、融合与修整;2）从改进特征编码方式与信息传递结构、提升边缘定位精度、改善注意力机制、提升训练稳定性和控制噪声的角度对循环卷积神经网络、全卷积神经网络和生成对抗网络3种主流算法的性能提升进行分析,从优化弱监督样本处理模块的角度分析了减少对像素级标注样本依赖的方法;3）对协同显著性物体检测、多类别图像显著性物体检测以及未来的研究问题和方向进行介绍,并给出了可能的解决思路。     

A hybrid deep learning model of process-build interactions in additive manufacturing

Laser powder bed fusion (LPBF) is a technique of additive manufacturing (AM) that is often used to construct a metal object layer-by-layer. The quality of AM builds depends to a great extent on the minimization of different defects such as porosity and cracks that could occur by process deviation during machine operation. Therefore, there is a need to develop new analytical methods and tools to equip the LPBF process with the inspection frameworks that assess the process condition and monitor the porosity defect in real-time. Advanced sensing is recently integrated with the AM machines to cope with process complexity and improve information visibility. This opportunity lays the foundation for online monitoring and assessment of the in-process build layer. This study presents the hybrid deep neural network structure with two types of input data to monitor the process parameters that result in porosity defect in cylinders’ layers. Results demonstrate that statistical features extracted by wavelet transform and texture analysis along with original powder bed images, assist the model in reaching a robust performance. In order to illustrate the fidelity of the proposed model, the capability of the main pipeline is examined and compared with different machine learning models. Eventually, the proposed framework identified the process conditions with an F-score of 97.14%. This salient flaw detection ability is conducive to repair the defect in real-time and assure the quality of the final part before the completion of the process.     

深度学习典型目标检测算法的改进综述

目标检测是机器视觉领域内最具挑战性的任务之一,深度学习则是目标检测最主流的实现方法.近年来,深度学习理论及技术的快速发展,使得基于深度学习的目标检测算法取得了巨大进展,学者从数据处理、网络结构、损失函数等多方面入手,提出了一系列对于目标检测算法的改进方式.针对典型目标检测算法的改进方式进行综述.归纳了常用数据集和性能评...     

基于特征级和决策级融合的人脸吸引力评价方法

在个性化的人脸吸引力的研究中,由于特征缺失和对于大众审美的影响因素考虑不足,导致预测个人偏好无法到达很高的预测精度。为了提高预测精度,提出了一个基于特征级和决策级信息融合的个性化人脸吸引力预测框架。首先,将代表不同人脸美丽特征的客观特性融合到一起,利用特征选择算法挑选出具有代表性的人脸吸引力特征,并利用不同的信息融合策略将人脸局部、全局特征融合起来;然后,将传统的人脸特征与通过深度网络自动提取的特征融合起来。同时,提出多种融合策略进行对比,将代表着大众审美偏好的评分信息与代表个人偏好的个性化评分信息进行决策级融合,最终实现个性化的人脸吸引力预测评分。实验结果表明,相比现有针对个性化人脸吸引力评价研究的算法,所提的多层次融合方法在预测精度方面有显著的提升,能够达到Pearson相关系数0.9以上。该方法可用于个性化推荐、人脸美化等领域。     

Feature-level and decision-level fusion of noncoincidently sampledsensors for land mine detection

We present and compare methods for feature-level (predetection) and decision-level (postdetection) fusion of multisensor data. This study emphasizes fusion techniques that are suitable for noncommensurate data sampled at noncoincident points. Decision-level fusion is most convenient for such data, but it is suboptimal in principle, since targets not detected by all sensors will not obtain the full benefits of fusion. A novel algorithm for feature-level fusion of noncommensurate, noncoincidently sampled data is described, in which a model is fitted to the sensor data and the model parameters are used as features. Formulations for both feature-level and decision-level fusion are described, along with some practical simplifications. A closed-form expression is available for feature-level fusion of normally distributed data and this expression is used with simulated data to study requirements for sample position accuracy in multisensor data. The performance of feature-level and decision-level fusion algorithms are compared for experimental data acquired by a metal detector, a ground-penetrating radar, and an infrared camera at a challenging test site containing surrogate mines. It is found that fusion of binary decisions does not perform significantly better than the best available sensor. The performance of feature-level fusion is significantly better than the individual sensors, as is decision-level fusion when detection confidence information is also available (“soft-decision” fusion)     

基于特征级融合神经网络的磁共振成像前列腺肿瘤CAD模型

针对磁共振成像(MRI)前列腺肿瘤感兴趣区域(ROI)在高维特征表示下存在特征相关和维数灾难问题,提出了一种基于主成分分析(PCA)的特征级融合神经网络(NN)的MRI前列腺肿瘤CAD模型。首先提取MRI前列腺肿瘤ROI的6维几何特征、6维统计特征、7维Hu不变矩特征、56维灰度共生矩阵的纹理特征、3维Tamura纹理特征和24维频域特征,得到102维特征矢量;然后通过PCA进行特征级融合得到累计贡献率达到89.62%的8维变换特征,降低特征矢量的维数;再次利用经典的神经网络(四种训练算法BFGS拟牛顿算法、BP算法、最速梯度下降算法和Levenberg-Marquardt算法)作为分类器进行分类识别;最后以180幅前列腺患者的MRI图像为原始数据,采用基于特征级融合神经网络(NN)的计算机辅助诊断模型对前列腺肿瘤进行辅助诊断。实验结果表明:经过特征级融合的神经网络识别前列腺良恶性肿瘤的能力至少提高10%左右,这种特征级融合策略是有效的,一定程度上提高了特征之间的不相关性。     

Deep feature learning versus shallow feature learning systems for joint use of airborne thermal hyperspectral and visible remote sensing data

Recently, the development of various remote sensing sensors has provided more reliable information and data for identification of different ground classes. Accordingly, multisensory fusion techniques are applied to enhance the process of information extraction from complementary airborne and spaceborne remote sensing data. Most of previous research in the literature has focused on the extraction of shallow features from a specific sensor and on classification of the resulted feature space using decision fusion systems. In recent years, Deep Learning (DL) algorithms have drawn a lot of attention in the machine learning area and have had different remote sensing applications, especially on data fusion. This study presents two different feature-learning strategies for the fusion of hyperspectral thermal infrared (HTIR) and visible remote sensing data. First, a Deep Convolutional Neural Network (DCNN)-Support Vector Machine (SVM) was utilized on the features of two datasets to provide the class labels. To validate the results with other learning strategies, a shallow feature model was used, as well. This model was based on feature fusion and decision fusion that classified and fused the two datasets. A co-registered thermal infrared hyperspectral (HTIR) and Fine Resolution Visible (Vis) RGB imagery was available from Quebec of Canada to examine the effectiveness of the proposed method. Experimental results showed that, except for the computational time, the proposed deep learning model outperformed shallow feature-based strategies in the classification performance that was based on its accuracy.     

Multiple features learning for ship classification in optical imagery

The sea surface vessel/ship classification is a challenging problem with enormous implications to the world’s global supply chain and militaries. The problem is similar to other well-studied problems in object recognition such as face recognition. However, it is more complex since ships’ appearance is easily affected by external factors such as lighting or weather conditions, viewing geometry and sea state. The large within-class variations in some vessels also make ship classification more complicated and challenging. In this paper, we propose an effective multiple features learning (MFL) framework for ship classification, which contains three types of features: Gabor-based multi-scale completed local binary patterns (MS-CLBP), patch-based MS-CLBP and Fisher vector, and combination of Bag of visual words (BOVW) and spatial pyramid matching (SPM). After multiple feature learning, feature-level fusion and decision-level fusion are both investigated for final classification. In the proposed framework, typical support vector machine (SVM) classifier is employed to provide posterior-probability estimation. Experimental results on remote sensing ship image datasets demonstrate that the proposed approach shows a consistent improvement on performance when compared to some state-of-the-art methods.     

目标跟踪综述

随着深度学习与人工智能技术的不断发展,视频目标跟踪已经成为了计算机视觉的重要研究内容,在公安布控、人机交互、交通管制、军事等各个领域起到越来越重要的作用。尽管现在国内外学者提出了多种目标跟踪算法,也搭建了较为完善的目标跟踪系统,但是算法的鲁棒性依然是一个比较大的挑战。本文对运动目标跟踪系统结构进行了简要介绍,并从特征提取及融合、外观模型、目标搜索等方面详细阐述了目前主流运动目标跟踪算法。然后对目标跟踪算法在深度学习大环境下的新发展进行了分析,从基于深度学习的目标跟踪及目标检测算法角度分析了深度学习在提高目标检测算法鲁棒性方面的有效性,最后概述了深度学习在视频目标检测算法中的具体应用并对其未来发展进行了展望。     

金字塔频率特征融合目标检测网络

针对深度学习网络在特征提取过程中运用上采样操作而致使细节纹理等高频特征缺失的问题,提出一种金字塔频率特征融合目标检测网络.网络由3个深度学习金字塔网络构成,输入图像经初级金字塔提取深度特征后,分别通过高频、低频增强金字塔形成不同的频率特征,利用特征融合来凸显深度学习网络在信息逐层传递过程中对细节信息的保护能力,提高目标...     

基于Transformer的跨尺度交互学习伪装目标检测

伪装目标检测(COD)旨在精确且高效地检测出与背景高度相似的伪装物体, 其方法可为物种保护、医学病患检测和军事监测等领域提供助力, 具有较高的实用价值. 近年来, 采用深度学习方法进行伪装目标检测成为一个比较新兴的研究方向. 但现有大多数COD算法都是以卷积神经网络(CNN)作为特征提取网络, 并且在结合多层次特征时, 忽略了特征表示和融合方法对检测性能的影响. 针对基于卷积神经网络的伪装目标检测模型对被检测目标的全局特征提取能力较弱问题, 提出一种基于Transformer的跨尺度交互学习伪装目标检测方法. 该模型首先提出了双分支特征融合模块, 将经过迭代注意力的特征进行融合, 更好地融合高低层特征; 其次引入了多尺度全局上下文信息模块, 充分联系上下文信息增强特征; 最后提出了多通道池化模块, 能够聚焦被检测物体的局部信息, 提高伪装目标检测准确率. 在CHAMELEON、CAMO以及COD10K数据集上的实验结果表明, 与当前主流的伪装物体检测算法相比较, 该方法生成的预测图更加清晰, 伪装目标检测模型能取得更高精度.     

AI-enabled and multimodal data driven smart health monitoring of wind power systems: A case study

The development of AI has enabled the fault detection of industrial components to be achieved through the combination with deep learning. A detection method combined with deep learning has also emerged for the fault detection of fan blades, such as models based on neural networks using the appearance or sound of the blades. However, the detection model obtained from a single data type often has limitations, such as low accuracy and overfitting. This is also the problem with fan blade detection. In contrast, multimodal data fusion detection models are often more stable. The modality diversity of blade diagnosis is strong, and it can be achieved from multiple modalities such as image, sound, and vibration. To improve the accuracy of fault diagnosis of fan blades, this article proposes a multimodal double-layer detection system (MTDS) based on decision-level and feature-level fusion. The system includes a wind turbine simulation platform and a multimodal detection system. It mainly obtains different modal data of the simulated wind turbine from the image, sound, and vibration signals, including blade images through unmanned aerial vehicle photography, blade vibration signals through electronic vibrators, and blade sound signals through microphones. The highly correlated sound and vibration modal data are fused at the feature level, and a detection model based on the sound and vibration mixed mode is implemented using a sound-vibration-CNN (SV-CNN) proposed in this case. Then, a detection model of the image mode is trained based on the blade image using a Convolution Block Attention Module ResNet (CBAM-ResNet) network. Finally, the detection input of the two modal models is fed into a perceptron to obtain the final prediction result, and the decision-level fusion is implemented to achieve fan blade detection based on multimodal, namely the implementation of MTDS.