Natural Language Processing and User Modeling: Synergies and Limitations

The fields of user modeling and natural language processing have been closely linked since the early days of user modeling. Natural language systems consult user models in order to improve their understanding of users' requirements and to generate appropriate and relevant responses. At the same time, the information natural language systems obtain from their users is expected to increase the accuracy of their user models. In this paper, we review natural language systems for generation, understanding and dialogue, focusing on the requirements and limitations these systems and user models place on each other. We then propose avenues for future research.     

一种用户故事需求质量提升方法

敏捷开发采用用户故事表达用户需求.一般采用格式受限的自然语言编写,但在用户故事编写过程中经常出现一些表述上的缺陷.典型的缺陷包括缺失必要信息、意思表达含糊不清、故事间有重复或存在冲突等.这很大程度上影响了需求的质量,影响软件开发项目的进行.提出一种用户故事需求质量提升方法.从故事缺陷定位的角度出发,该方法构建了用户故事概念模型,并根据实际案例总结并提出11条用户故事应遵循的质量准则.从而提出故事结构分析、句法模式分析以及语法分析等技术,用于自动构建带场景用户故事的实例层模型,并根据准则进行故事缺陷检测,进而提升用户故事质量.在包含36个用户故事84个场景的实际项目中进行实验,自动检测出173个缺陷,缺陷检测的准确率和召回率分别达到88.79%和95.06%.     

User story clustering in agile development: a framework and an empirical study

Agile development aims at rapidly developing software while embracing the continuous evolution of user requirements along the whole development process. User stories are the primary means of requirements collection and elicitation in the agile development. A project can involve a large amount of user stories, which should be clustered into different groups based on their functionality’s similarity for systematic requirements analysis, effective mapping to developed features, and efficient maintenance. Nevertheless, the current user story clustering is mainly conducted in a manual manner, which is time-consuming and subjective to human bias. In this paper, we propose a novel approach for clustering the user stories automatically on the basis of natural language processing. Specifically, the sentence patterns of each component in a user story are first analysed and determined such that the critical structure in the representative tasks can be automatically extracted based on the user story meta-model. The similarity of user stories is calculated, which can be used to generate the connected graph as the basis of automatic user story clustering. We evaluate the approach based on thirteen datasets, compared against ten baseline techniques. Experimental results show that our clustering approach has higher accuracy, recall rate and F1-score than these baselines. It is demonstrated that the proposed approach can significantly improve the efficacy of user story clustering and thus enhance the overall performance of agile development. The study also highlights promising research directions for more accurate requirements elicitation.     

Automating test oracles from restricted natural language agile requirements

Manual testing of software requirements written in natural language for agile or any other methodology requires more time and human resources. This leaves the testing process error prone and time consuming. For satisfied end users with bug‐free software delivered on time, there is a need to automate the test oracle process for natural language or informal requirements. The automation of the test oracle is relatively easier with formal requirements, but this task is difficult to achieve with natural language requirements. This study proposes an approach called Restricted Natural Language Agile Requirements Testing (ReNaLART) to automate the test oracle from restricted natural language agile requirements. For this purpose, it uses an existing user story template with some modifications for writing user stories. This helps in identifying test input and expected output for a user story. For comparison of expected and observed outputs it makes use of a regex pattern and string distance functions. It is capable of assigning different types of verdicts automatically depending upon the similarity/dissimilarity between observed and expected outputs of user stories. ReNaLART is validated using several case studies of different domains, namely, OLX Pakistan, Mental Health Tests, McDelivery Pakistan, BlueStacks, Power Searching with Google, TensorFlow Playground, w3Schools 2018 offline and Touch'D. It revealed several faults in five of the above listed eight applications. Plus, the proposed test oracle on an average took 0.02 s for test data generation, expected output generation and verdict assignment. Both these facts show the fault revealing effectiveness and efficiency of ReNaLART.     

Improving agile requirements: the Quality User Story framework and tool

User stories are a widely adopted requirements notation in agile development. Yet, user stories are too often poorly written in practice and exhibit inherent quality defects. Triggered by this observation, we propose the Quality User Story (QUS) framework, a set of 13 quality criteria that user story writers should strive to conform to. Based on QUS, we present the Automatic Quality User Story Artisan (AQUSA) software tool. Relying on natural language processing (NLP) techniques, AQUSA detects quality defects and suggest possible remedies. We describe the architecture of AQUSA, its implementation, and we report on an evaluation that analyzes 1023 user stories obtained from 18 software companies. Our tool does not yet reach the ambitious 100 % recall that Daniel Berry and colleagues require NLP tools for RE to achieve. However, we obtain promising results and we identify some improvements that will substantially improve recall and precision.     

Story plot generation based on CBR

In this paper we present a system for automatic story generation that reuses existing stories to produce a new story that matches a given user query. The plot structure is obtained by a case-based reasoning (CBR) process over a case base of tales and an ontology of explicitly declared relevant knowledge. The resulting story is generated as a sketch of a plot described in natural language by means of natural language generation (NLG) techniques.     

Situating natural language understanding within experience-based design

Building useful systems with an ability to understand "real" natural language input has long been an elusive goal for Artificial Intelligence. Well-known problems such as ambiguity, indirectness, and incompleteness of natural language inputs have thwarted efforts to build natural language interfaces to intelligent systems. In this article, we report on our work on a model of understanding natural language design specifications of physical devices such as simple electrical circuits. Our system, called KA, solves the classical problems of ambiguity, incompleteness and indirectness by exploiting the knowledge and problem-solving processes in the situation of designing simple physical devices. In addition, KA acquires its knowledge structures (apart from a basic ontology of devices) from the results of its problem-solving processes. Thus, KA can be bootstrapped to understand design specifications and user feedback about new devices using the knowledge structures it acquired from similar devices designed previously.In this paper, we report on three investigations in the KA project. Our first investigation demonstrates that KA can resolve ambiguities in design specifications as well as infer unarticulated requirements using the ontology, the knowledge structures, and the problem-solving processes provided by its design situation. The second investigation shows that KA's problem-solving capabilities help ascertain the relevance of indirect design specifications, and identify unspecified relations between detailed requirements. The third investigation demonstrates the extensibility of KA's theory of natural language understanding by showing that KA can interpret user feedback as well as design requirements. Our results demonstrate that situating language understanding in problem solving, such as device design in KA, provides effective solutions to unresolved problems in natural language processing.     

The generic model query language GMQL – Conceptual specification,implementation, and runtime evaluation

The generic model query language GMQL is designed to query collections of conceptual models created in arbitrary graph-based modelling languages. Querying conceptual models means searching for particular model subgraphs that comply with a predefined pattern query. Such a query specifies the structural and semantic properties of the model fragment to be returned. In this paper, we derive requirements for a generic model query language from the literature and formally specify the language’s syntax and semantics. We conduct an analysis of GMQL׳s theoretical and practical runtime performance concluding that it returns query results within satisfactory time. Given its generic nature, GMQL contributes to a broad range of different model analysis scenarios ranging from business process compliance management to model translation and business process weakness detection. As GMQL returns results with acceptable runtime performance, it can be used to query large collections of hundreds or thousands of conceptual models containing not only process models, but also data models or organizational charts. In this paper, we furthermore evaluate GMQL against the backdrop of existing query approaches thereby carving out its advantages and limitations as well as pointing toward future research.     

Generating Validation Feedback for Automatic Interpretation of Informal Requirements

This paper describes a method for validating conceptual models of digital systems derived automatically from requirements expressed in natural language. Because natural language is ambiguous and vague, most statements have multiple interpretations. The approach here is to feed back to the requirements authors visualizations of the interpretations of the requirements that have been translated to semantic networks. The visualization task is a component (the Model Generator) of the ASPIN system for automatically interpreting requirements expressed in natural language and diagrams, analyzing the requirements for consistency and completeness, and automatically generating engineering models in the VHDL language. Visualization is performed in two steps: mapping the semantic networks to compound digraphs followed by placement of the nodes of the digraphs to generate a display in terms of icons representing devices, values, actions and events; and connectives indicating carriers, data flow and control dependency.     

On the nature of models for time-sensitive remote sensing

Remote sensing employs a range of conceptual and mechanistic models. Several conceptual models have been proposed to explain remote sensing systems (RSSs) and either directly prescribe or elucidate the configuration of such systems based on scene conditions and information requirements. This study interrogates the utility of these models for the design and practical implementation of RSSs to address time-sensitive information requirements and proposes a novel conceptual model, the remote sensing communication model, that places remote sensing within a decision support context.

Three pertinent remote sensing conceptual models are critically assessed based on their satisfaction of three basic requirements: (1) prescribe or explicate RSS configuration based on user information requirements, (2) elucidate dependencies between information type, accuracy, and timeliness, and (3) describe the effect of RSS configuration on the effectiveness of users’ decisions. The conceptual remote sensing models that are evaluated are not found to be appropriate for the design and configuration of time-sensitive RSSs (TSRSSs).

The remote sensing communication model employs the (information theory) communication model developed by Shannon and Weaver to elucidate the acquisition, transmission, processing, interpretation, and effectiveness of information derived from a TSRSS. Weaver’s three levels of effective communication are used to explain the varying value of information as a function of time and user characteristics and to elucidate the effect of RSS configuration on the ability of remote sensing-derived information to inform decisions in a timely manner. The concept of communication channel capacity is used to estimate the timeliness of RSSs, and a brief example of its implementation is presented.     

基于元路径嵌入的移动应用需求偏好分析方法

随着互联网和移动应用平台的快速发展,围绕移动应用所产生的海量用户数据已经成为精确分析用户需求偏好的重要数据源.尽管已有不少学者从这些数据中分析和挖掘用户需求,但现有的方法通常只研究了数据的少数维度的特征,未能有效地挖掘多维移动应用信息以及他们之间的关联.提出一种基于元路径嵌入的移动应用需求偏好分析方法,能够为用户进行个性化移动应用推荐.具体地,首先分析移动应用的文本信息中的语义主题,挖掘用户需求偏好的分析维度.其次,将移动应用信息的语义特征构建了一个融合移动应用多维信息的概念模型,涵盖了能够表征用户需求偏好的多维度数据.基于概念模型的语义,设计了一组有意义的元路径集合,以精确地捕捉用户需求偏好的语义.最后,通过使用元路径嵌入技术进行用户行为画像,进而实现个性化的移动应用推荐.使用苹果应用商店包括1507个移动应用和153501条用户评论的真实数据集进行实验评估.实验结果表明所提的方法在各指标上均优于现有模型,其中平均F1值提升0.02,平均归一化折损累计增益(normalized discounted cumulative gain,NDCG)提升0.1.     

Modeling semantic information in engineering applications: a review

Due to the latest advances of information technology and the increasing complexity of engineering applications, it is becoming more and more important to model semantic information. There are many modeling methodologies to do the work of modeling semantic information instead of natural language processing. Since this field is very broad, the comparison discussed here is not an exhaustive study but rather the partial views of the coauthors from our own perspectives. In the present paper we give a review of the literature of conceptual models especially static one and then classify them into four type models namely structure-based model, object-oriented model, knowledge semantic-based model, and web semantic-based model. Based on the classification given above, a hierarchy structured criteria is given. According to the criteria we pick one or two representative conceptual models from each type to conduct the comparison. We compare the following five aspects of conceptual models: expressivity, clarity, semantics, formal foundation, and application fields. The comparative study shows that different models have different features and fit different fields of engineering applications. The present comparison study is useful for users to understand and choose right conceptual models combining with specific requirements of engineering applications.     

Reconciling requirement-driven data warehouses with data sources via multidimensional normal forms

Successful data warehouse (DW) design needs to be based upon a requirement analysis phase in order to adequately represent the information needs of DW users. Moreover, since the DW integrates the information provided by data sources, it is also crucial to take these sources into account throughout the development process to obtain a consistent reconciliation of data sources and information needs. In this paper, we start by summarizing our approach to specify user requirements for data warehouses and to obtain a conceptual multidimensional model capturing these requirements. Then, we make use of the multidimensional normal forms to define a set of Query/View/Transformation (QVT) relations to assure that the conceptual multidimensional model obtained from user requirements agrees with the available data sources that will populate the DW. Thus, we propose a hybrid approach to develop DWs, i.e., we firstly obtain the conceptual multidimensional model of the DW from user requirements and then we verify and enforce its correctness against data sources by using a set of QVT relations based on multidimensional normal forms. Finally, we provide some snapshots of the CASE tool we have used to implement our QVT relations.     

Conceptual modeling of natural language functional requirements

Requirements analysts consider a conceptual model to be an important artifact created during the requirements analysis phase of a software development life cycle (SDLC). A conceptual, or domain model is a visual model of the requirements domain in focus. Owing to its visual nature, the model serves as a platform for the deliberation of requirements by stakeholders and enables requirements analysts to further refine the functional requirements. Conceptual models may evolve into class diagrams during the design and execution phases of the software project. Even a partially automated conceptual model can save significant time during the requirements phase, by quickening the process of graphical communication and visualization.This paper presents a system to create a conceptual model from functional specifications, written in natural language in an automated manner. Classes and relationships are automatically identified from the functional specifications. This identification is based on the analysis of the grammatical constructs of sentences, and on Object Oriented principles of design. Extended entity-relationship (EER) notations are incorporated into the class relationships. Optimizations are applied to the identified entities during a post-processing stage, and the final conceptual model is rendered.The use of typed dependencies, combined with rules to derive class relationships offers a granular approach to the extraction of the design elements in the model. The paper illustrates the model creation process using a standard case study, and concludes with an evaluation of the usefulness of this approach for the requirements analysis. The analysis is conducted against both standard published models and conceptual models created by humans, for various evaluation parameters.     

Reconciling requirement-driven data warehouses with data sources via multidimensional normal forms

Successful data warehouse (DW) design needs to be based upon a requirement analysis phase in order to adequately represent the information needs of DW users. Moreover, since the DW integrates the information provided by data sources, it is also crucial to take these sources into account throughout the development process to obtain a consistent reconciliation of data sources and information needs. In this paper, we start by summarizing our approach to specify user requirements for data warehouses and to obtain a conceptual multidimensional model capturing these requirements. Then, we make use of the multidimensional normal forms to define a set of Query/View/Transformation (QVT) relations to assure that the conceptual multidimensional model obtained from user requirements agrees with the available data sources that will populate the DW. Thus, we propose a hybrid approach to develop DWs, i.e., we firstly obtain the conceptual multidimensional model of the DW from user requirements and then we verify and enforce its correctness against data sources by using a set of QVT relations based on multidimensional normal forms. Finally, we provide some snapshots of the CASE tool we have used to implement our QVT relations.