Knowledge discovery through directed probabilistic topic models: a survey

Graphical models have become the basic framework for topic based probabilistic modeling. Especially models with latent variables have proved to be effective in capturing hidden structures in the data. In this paper, we survey an important subclass Directed Probabilistic Topic Models (DPTMs) with soft clustering abilities and their applications for knowledge discovery in text corpora. From an unsupervised learning perspective, “topics are semantically related probabilistic clusters of words in text corpora; and the process for finding these topics is called topic modeling”. In topic modeling, a document consists of different hidden topics and the topic probabilities provide an explicit representation of a document to smooth data from the semantic level. It has been an active area of research during the last decade. Many models have been proposed for handling the problems of modeling text corpora with different characteristics, for applications such as document classification, hidden association finding, expert finding, community discovery and temporal trend analysis. We give basic concepts, advantages and disadvantages in a chronological order, existing models classification into different categories, their parameter estimation and inference making algorithms with models performance evaluation measures. We also discuss their applications, open challenges and future directions in this dynamic area of research.     

Understanding digital transformation in advanced manufacturing and engineering: A bibliometric analysis,topic modeling and research trend discovery

Digital transformation (DT) is the process of combining digital technologies with sound business models to generate great value for enterprises. DT intertwines with customer requirements, domain knowledge, and theoretical and empirical insights for value propagations. Studies of DT are growing rapidly and heterogeneously, covering the aspects of product design, engineering, production, and life-cycle management due to the fast and market-driven industrial development under Industry 4.0. Our work addresses the challenge of understanding DT trends by presenting a machine learning (ML) approach for topic modeling to review and analyze advanced DT technology research and development. A systematic review process is developed based on the comprehensive DT in manufacturing systems and engineering literature (i.e., 99 articles). Six dominant topics are identified, namely smart factory, sustainability and product-service systems, construction digital transformation, public infrastructure-centric digital transformation, techno-centric digital transformation, and business model-centric digital transformation. The study also contributes to adopting and demonstrating the ML-based topic modeling for intelligent and systematic bibliometric analysis, particularly for unveiling advanced engineering research trends through domain literature.     

A survey of modeling for prognosis and health management of industrial equipment

Prognosis and health management plays an important role in the control of costs associated with operating large industrial equipment, such as wind turbines and aircraft. It is only fair that engineers and scientists have vastly researched modeling approaches to support decision making. Motivated by the growing availability of data and computational power as well as the advances in algorithms and methods, modeling frameworks often merge elements of physics, machine learning, and statistical learning. In this paper, we present a review on modeling in support of prognosis and health management of industrial equipment. This survey complements the existing prognosis and health management literature by discussing how modeling strategies are influenced by industry-specific aspects such as maintenance approaches (e.g., reactive, proactive, and predictive), implementation factors (e.g., industry, business model, purpose, development, and deployment), as well as supporting technologies (sensing, repair, and modeling itself). We use the onshore wind energy and civil aviation industries to illustrate how these aforementioned aspects can influence modeling and implementation of prognosis and health management. The literature review is broad and covers contributions over the past 40 years. We close the paper with few topics that can motive research going forward.     

Joint sentiment/topic modeling on text data using a boosted restricted Boltzmann Machine

Recently by the development of the Internet and the Web, different types of social media such as web blogs become an immense source of text data. Through the processing of these data, it is possible to discover practical information about different topics, individual’s opinions and a thorough understanding of the society. Therefore, applying models which can automatically extract the subjective information from documents would be efficient and helpful. Topic modeling methods and sentiment analysis are the raised topics in natural language processing and text mining fields. In this paper a new structure for joint sentiment-topic modeling based on a Restricted Boltzmann Machine (RBM) which is a type of neural networks is proposed. By modifying the structure of RBM as well as appending a layer which is analogous to sentiment of text data to it, we propose a generative structure for joint sentiment topic modeling based on neural networks. The proposed method is supervised and trained by the Contrastive Divergence algorithm. The new attached layer in the proposed model is a layer with the multinomial probability distribution which can be used in text data sentiment classification or any other supervised application. The proposed model is compared with existing models in the experiments such as evaluating as a generative model, sentiment classification, information retrieval and the corresponding results demonstrate the efficiency of the method.

     

Exploring service-oriented system engineering challenges: a systematic literature review

Service-oriented system engineering (SOSE) has drawn increasing attention since service-oriented computing was introduced in the beginning of this decade. A large number of SOSE challenges that call for special software engineering efforts have been proposed in the research community. Our goal is to gain insight into the current status of SOSE research issues as published to date. To this end, we conducted a systematic literature review exploring SOSE challenges that have been claimed between January 2000 and July 2008. This paper presents the results of the systematic review as well as the empirical research method we followed. In this review, of the 729 publications that have been examined, 51 were selected as primary studies, from which more than 400 SOSE challenges were elicited. By applying qualitative data analysis methods to the extracted data from the review, we proved our hypotheses about the classification scheme. We are able to conclude that the SOSE challenges can be classified along two dimensions: (a) based on themes (or topics) that they cover and (b) based on characteristics (or types) that they reveal. By analyzing the distribution of the SOSE challenges on the topics and types in the years 2000–2008, we are able to point out the trend in SOSE research activities. The findings of this review further provide empirical evidence for establishing future SOSE research agendas.     

Hierarchical classification of data streams: a systematic literature review

The classification task usually works with flat and batch learners, assuming problems as stationary and without relations between class labels. Nevertheless, several real-world problems do not assume these premises, i.e., data have labels organized hierarchically and are made available in streaming fashion, meaning that their behavior can drift over time. Existing studies on hierarchical classification do not consider data streams as input of their process, and thus, data is assumed as stationary and handled through batch learners. The same can be said about works on streaming data, as the hierarchical classification is overlooked. Studies concerning each area individually are promising, yet, do not tackle their intersection. This study analyzes the main characteristics of the state-of-the-art works on hierarchical classification for streaming data concerning five aspects: (i) problems tackled, (ii) datasets, (iii) algorithms, (iv) evaluation metrics, and (v) research gaps in the area. We performed a systematic literature review of primary studies and retrieved 3,722 papers, of which 42 were identified as relevant and used to answer the aforementioned research questions. We found that the problems handled by hierarchical classification of data streams include mainly classification of images, human activities, texts, and audio; the datasets are mostly created or synthetic data; the algorithms and evaluation metrics are well-known techniques or based on those; and research gaps are related to dynamic context, data complexity, and computational resources constraints. We also provide implications for future research and experiments to consider common characteristics shared amongst hierarchical classification and data stream classification.

     

Simulation for manufacturing system design and operation: Literature review and analysis

This paper provides a comprehensive review of discrete event simulation publications published between 2002 and 2013 with a particular focus on applications in manufacturing. The literature is classified into three general classes of manufacturing system design, manufacturing system operation, and simulation language/package development. The paper further categorizes the literature into 11 subclasses based on the application area. The current review contributes to the literature in three significant ways: (1) it provides a wide coverage by reviewing 290 papers; (2) it provides a detailed analysis of different aspects of the literature to identify research trends through innovative data mining approaches as well as insights derived from the review process; and (3) it updates and extends the existing classification schemes through identification and inclusion of recently emerged application areas and exclusion of obsolete categories. The results of the literature analysis are then used to make suggestions for future research.     

From event logs to goals: a systematic literature review of goal-oriented process mining

Process mining helps infer valuable insights about business processes using event logs, whereas goal modeling focuses on the representation and analysis of competing goals of stakeholders and systems. Although there are clear benefits in mining the goals of existing processes, goal-oriented approaches that consider logs during model construction are still rare. Process mining techniques, when generalizing large instance-level data into process models, can be considered as a data-driven complement to use case/scenario elicitation. Requirements engineers can exploit process mining techniques to find new system or process requirements in order to align current practices and desired ones. This paper provides a systemic literature review, based on 24 papers rigorously selected from four popular search engines in 2018, to assess the state of goal-oriented process mining. Through two research questions, the review highlights that the use of process mining in association with goals does not yet have a coherent line of research, whereas intention mining (where goal models are mined) shows a meaningful trace of research. Research about performance indicators measuring goals associated with process mining is also sparse. Although the number of publications in process mining and goal modeling is trending up, goal mining and goal-oriented process mining remain modest research areas. Yet, synergetic effects achievable by combining goals and process mining can potentially augment the precision, rationality and interpretability of mined models and eventually improve opportunities to satisfy system stakeholders.

     

基于Fisher 判别分析的过程运行状态在线评价

为了实时掌握生产过程运行状态, 提出一种基于Fisher 判别分析(FDA) 的过程运行状态在线评价方法. 提出 离线数据分类与识别算法, 以识别不同稳定运行状态的建模数据及其对应的状态等级; 利用FDA提取各个稳定运行 状态的特征属性, 建立评价模型; 在线评价时, 通过“时间窗口”数据特征与各个状态等级的相似度, 实时评价过程运行状态. 将所提出的方法应用于某湿法冶金过程的仿真结果验证了该方法的有效性.

     

Predicting emotions in online social networks: challenges and opportunities

Online social networking has become a popular means of information exchange and social interactions. Users of these platforms generate massive amounts of data about their relationships, behaviors, interests, opinions, locations visited, items purchased, and subjective experiences of various aspects of life. Moreover, these platforms enable people from wide-ranging social and cultural backgrounds to synergize and interact. One interesting area of research is the emotional dimensions contained in this user-generated content, specifically, emotion detection and prediction, which involve the extraction and analysis of emotions in social network data. This study aimed to provide a comprehensive overview and better understanding of the current state of research regarding emotion detection in online social networks by performing a systematic literature review (SLR). SLRs help identify the gaps, challenges, and opportunities in a field of study through a careful examination of current research to understand the methods and results, ultimately highlighting methodological concerns that can be used to improve future work in the field. Hence, we collected and analyzed studies that focused on emotion in social network posts and discussed various topics published in digital databases between 2010 and December 2020. Over 239 articles were initially included in the collection, and after the selection process and application of our quality criteria, 104 articles were examined, and the results showed a robust extant body of literature on the text-based emotion analysis model, while the image-based requires more attention as well as the multiple modality emotion analysis.

     

Application of data mining techniques in customer relationship management: A literature review and classification

Despite the importance of data mining techniques to customer relationship management (CRM), there is a lack of a comprehensive literature review and a classification scheme for it. This is the first identifiable academic literature review of the application of data mining techniques to CRM. It provides an academic database of literature between the period of 2000–2006 covering 24 journals and proposes a classification scheme to classify the articles. Nine hundred articles were identified and reviewed for their direct relevance to applying data mining techniques to CRM. Eighty-seven articles were subsequently selected, reviewed and classified. Each of the 87 selected papers was categorized on four CRM dimensions (Customer Identification, Customer Attraction, Customer Retention and Customer Development) and seven data mining functions (Association, Classification, Clustering, Forecasting, Regression, Sequence Discovery and Visualization). Papers were further classified into nine sub-categories of CRM elements under different data mining techniques based on the major focus of each paper. The review and classification process was independently verified. Findings of this paper indicate that the research area of customer retention received most research attention. Of these, most are related to one-to-one marketing and loyalty programs respectively. On the other hand, classification and association models are the two commonly used models for data mining in CRM. Our analysis provides a roadmap to guide future research and facilitate knowledge accumulation and creation concerning the application of data mining techniques in CRM.     

A systematic review and meta-analysis of artificial neural network application in geotechnical engineering: theory and applications

Artificial neural network (ANN) aimed to simulate the behavior of the nervous system as well as the human brain. Neural network models are mathematical computing systems inspired by the biological neural network in which try to constitute animal brains. ANNs recently extended, presented, and applied by many research scholars in the area of geotechnical engineering. After a comprehensive review of the published studies, there is a shortage of classification of study and research regarding systematic literature review about these approaches. A review of the literature reveals that artificial neural networks is well established in modeling retaining walls deflection, excavation, soil behavior, earth retaining structures, site characterization, pile bearing capacity (both skin friction and end-bearing) prediction, settlement of structures, liquefaction assessment, slope stability, landslide susceptibility mapping, and classification of soils. Therefore, the present study aimed to provide a systematic review of methodologies and applications with recent ANN developments in the subject of geotechnical engineering. Regarding this, a major database of the web of science has been selected. Furthermore, meta-analysis and systematic method which called PRISMA has been used. In this regard, the selected papers were classified according to the technique and method used, the year of publication, the authors, journals and conference names, research objectives, results and findings, and lastly solution and modeling. The outcome of the presented review will contribute to the knowledge of civil and/or geotechnical designers/practitioners in managing information in order to solve most types of geotechnical engineering problems. The methods discussed here help the geotechnical practitioner to be familiar with the limitations and strengths of ANN compared with alternative conventional mathematical modeling methods.

     

A mapping study about the standard ISO/IEC29110

ContextThe ISO/IEC29110 has been studied since its release in 2011, and its impact and evaluation over the recent years have been quite diverse. This standard is structured in five parts describing the business terms, the main Very Small Entities (VSE) profile concepts, process assessment guidelines, specification of all the generic profile group, and implementation management and engineering guide for entry and basic profiles.ObjectiveThe main purpose of this work is to provide an analysis of the research carried out about the ISO/IEC 29110 during the last ten years, and the literature that has developed around it. Literature is analyzed by using the traditional mapping study of the ISO/IEC29110 and its parts. All these studies are categorized in a set of topics where authors have been contributing. This work helps us on the identification of the main research topics within the primary studies.MethodThe mapping study is conducted as a traditional systematic mapping with a categorization of the primary studies. The main search is enhanced with additional searches for each member of the ISO/IEC 29110 series.ResultsA search strategy is defined to conduct this mapping study. 184 papers were retrieved from the literature and selected as primary studies. Our study identifies the reference studies in this area, it characterizes them, and identifies which aspects have been treated.ConclusionThe results of this mapping reveal that ISO/IEC 29110 has been used in a broad range of small contexts, and the main contributions are basically from research experiences during the recent last ten years. The literature around this standard is classified based on a well-known classification schema, the activity around this standard, and what types of studies have been carried out. Research topics are diverse, and we have identified the research methods used by the primary studies. As conclusion, more research and experimental outcomes are needed in order to observe how VSEs behave under specific circumstances.     

A comprehensive system for image scene classification

In recent years, image scene classification based on low/high-level features has been considered as one of the most important and challenging problems faced in image processing research. The high-level features based on semantic concepts present a more accurate and closer model to the human perception of the image scene content. This paper presents a new multi-stage approach for image scene classification based on high-level semantic features extracted from image content. In the first stage, the object boundaries and their labels that represent the content are extracted. For this purpose, a combined method of a fully convolutional deep network and a combined network of a two-class SVM-fuzzy and SVR are used. Topic modeling is used to represent the latent relationships between the objects. Hence in the second stage, a new combination of methods consisting of the bag of visual words, and supervised document neural autoregressive distribution estimator is used to extract the latent topics (topic modeling) in the image. Finally, classification based on Bayesian method is performed according to the extracted features of the deep network, objects labels and the latent topics in the image. The proposed method has been evaluated on three datasets: Scene15, UIUC Sports, and MIT-67 Indoor. The experimental results show that the proposed approach achieves average performance improvement of 12%, 11% and 14% in the accuracy of object detection, and 0.5%, 0.6% and 1.8% in the mean average precision criteria of the image scene classification, compared to the previous state-of-the-art methods on these three datasets.

     

Analysis of data for the carbon dioxide capture domain

To tackle the global concern for adverse impact of greenhouse gas (GHG) emissions, the post combustion carbon dioxide (CO₂) capture technology is commonly adopted for reducing industrial CO₂ emissions, for example, from power generation plants. The research on post combustion CO₂ capture has been ongoing in the last two decade, and its primary objective is to improve efficiency of the CO₂ capture process while reducing specific operating problems such as solvent degradation and corrosion. This objective requires a good understanding of the intricate relationships among parameters involved in the CO₂ capture process. From a review of the relevant literature, we observed that the most significant parameters influencing the CO₂ production rate include: heat duty, circulation rate of the solvent, CO₂ lean loading, and solvent concentration. To study the nature of relationships among the key parameters, we conducted data modeling and analysis based on the amine-based post combustion CO₂ capture process at the International Test Centre for Carbon Dioxide Capture (ITC) located in Regina, Saskatchewan of Canada. In our study, the experimental data collected from ITC from year 2003 to 2006 were analyzed using the combined approach of neural network modeling and sensitivity analysis. The neural network was trained for modeling the relationships among parameters, and the sensitivity analysis method illustrated the order of significance among the parameters. The modeling results were validated by the process experts. This paper describes the procedure of our work, and discusses the results of the analysis.     

基于空间结构统计建模的图像分类方法

提出一种基于图像空间结构统计建模的复杂纹理图像模式识别方法。从理论上分析了复杂纹理图像空间结构的韦伯分布过程,通过构造多尺度全向高斯导数滤波器,获得复杂纹理图像在不同观测尺度上的全方向空间结构统计建模表征结果。基于偏最小二乘-判决分析原理构建分类器,实现了复杂纹理图像的分类识别。实验结果表明,所提出的图像空间结构统计建模方法能获得复杂纹理图像关键性的视觉感知特性,基于该方法的图像分类准确率高且性能稳定。     

Social stress and performance in human-machine interaction: a neglected research field

Abstract

Given the increasing capabilities of highly automated systems, the article argues for a need to address the issue of social stress in human-machine interaction. It suggests a classification system of subordinate concepts found in the research literature under the heading of social stress. A review of the literature revealed a paucity of studies examining the effects of social stressors on performance. In particular, the review showed a shortage of experimental lab-based work, needed to establish clear cause-effect relationships. The article examined the suitability of different social stressors for lab-based research, not only when humans are the source of stress but also in so-called hybrid teams where social stress is caused by machine agents. The review shows that a closer link is needed between the separate literature on social stress and automation. Finally, three mechanisms are proposed that may predict how social stress may affect performance: ‘blank-out’-mechanism, ‘rumination’-mechanism, and ‘increased-motivation’-mechanism.

Practitioner summary: Theories of ergonomics and human factors may benefit from better integration of research and theoretical work in the domain of social stress. This is due to the increasing capabilities of machines to induce social stress.

Abbreviations: HMI: human-machine interaction; TSST: trier social stress test