Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data

ObjectiveTo synthesize data quality (DQ) dimensions and assessment methods of real-world data, especially electronic health records, through a systematic scoping review and to assess the practice of DQ assessment in the national Patient-centered Clinical Research Network (PCORnet).Materials and MethodsWe started with 3 widely cited DQ literature—2 reviews from Chan et al (2010) and Weiskopf et al (2013a) and 1 DQ framework from Kahn et al (2016)—and expanded our review systematically to cover relevant articles published up to February 2020. We extracted DQ dimensions and assessment methods from these studies, mapped their relationships, and organized a synthesized summarization of existing DQ dimensions and assessment methods. We reviewed the data checks employed by the PCORnet and mapped them to the synthesized DQ dimensions and methods.ResultsWe analyzed a total of 3 reviews, 20 DQ frameworks, and 226 DQ studies and extracted 14 DQ dimensions and 10 assessment methods. We found that completeness, concordance, and correctness/accuracy were commonly assessed. Element presence, validity check, and conformance were commonly used DQ assessment methods and were the main focuses of the PCORnet data checks.DiscussionDefinitions of DQ dimensions and methods were not consistent in the literature, and the DQ assessment practice was not evenly distributed (eg, usability and ease-of-use were rarely discussed). Challenges in DQ assessments, given the complex and heterogeneous nature of real-world data, exist.ConclusionThe practice of DQ assessment is still limited in scope. Future work is warranted to generate understandable, executable, and reusable DQ measures.     

A Document Clustering and Ranking System for Exploring MEDLINE Citations

ObjectiveA major problem faced in biomedical informatics involves how best to present information retrieval results. When a single query retrieves many results, simply showing them as a long list often provides poor overview. With a goal of presenting users with reduced sets of relevant citations, this study developed an approach that retrieved and organized MEDLINE citations into different topical groups and prioritized important citations in each group.DesignA text mining system framework for automatic document clustering and ranking organized MEDLINE citations following simple PubMed queries. The system grouped the retrieved citations, ranked the citations in each cluster, and generated a set of keywords and MeSH terms to describe the common theme of each cluster.MeasurementsSeveral possible ranking functions were compared, including citation count per year (CCPY), citation count (CC), and journal impact factor (JIF). We evaluated this framework by identifying as “important” those articles selected by the Surgical Oncology Society.ResultsOur results showed that CCPY outperforms CC and JIF, i.e., CCPY better ranked important articles than did the others. Furthermore, our text clustering and knowledge extraction strategy grouped the retrieval results into informative clusters as revealed by the keywords and MeSH terms extracted from the documents in each cluster.ConclusionsThe text mining system studied effectively integrated text clustering, text summarization, and text ranking and organized MEDLINE retrieval results into different topical groups.     

Extracting social determinants of health from electronic health records using natural language processing: a systematic review

ObjectiveSocial determinants of health (SDoH) are nonclinical dispositions that impact patient health risks and clinical outcomes. Leveraging SDoH in clinical decision-making can potentially improve diagnosis, treatment planning, and patient outcomes. Despite increased interest in capturing SDoH in electronic health records (EHRs), such information is typically locked in unstructured clinical notes. Natural language processing (NLP) is the key technology to extract SDoH information from clinical text and expand its utility in patient care and research. This article presents a systematic review of the state-of-the-art NLP approaches and tools that focus on identifying and extracting SDoH data from unstructured clinical text in EHRs.Materials and MethodsA broad literature search was conducted in February 2021 using 3 scholarly databases (ACL Anthology, PubMed, and Scopus) following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A total of 6402 publications were initially identified, and after applying the study inclusion criteria, 82 publications were selected for the final review.ResultsSmoking status (n = 27), substance use (n = 21), homelessness (n = 20), and alcohol use (n = 15) are the most frequently studied SDoH categories. Homelessness (n = 7) and other less-studied SDoH (eg, education, financial problems, social isolation and support, family problems) are mostly identified using rule-based approaches. In contrast, machine learning approaches are popular for identifying smoking status (n = 13), substance use (n = 9), and alcohol use (n = 9).ConclusionNLP offers significant potential to extract SDoH data from narrative clinical notes, which in turn can aid in the development of screening tools, risk prediction models, and clinical decision support systems.     

Contemporary use of real-world data for clinical trial conduct in the United States: a scoping review

ObjectiveReal-world data (RWD), defined as routinely collected healthcare data, can be a potential catalyst for addressing challenges faced in clinical trials. We performed a scoping review of database-specific RWD applications within clinical trial contexts, synthesizing prominent uses and themes.Materials and MethodsQuerying 3 biomedical literature databases, research articles using electronic health records, administrative claims databases, or clinical registries either within a clinical trial or in tandem with methodology related to clinical trials were included. Articles were required to use at least 1 US RWD source. All abstract screening, full-text screening, and data extraction was performed by 1 reviewer. Two reviewers independently verified all decisions.ResultsOf 2020 screened articles, 89 qualified: 59 articles used electronic health records, 29 used administrative claims, and 26 used registries. Our synthesis was driven by the general life cycle of a clinical trial, culminating into 3 major themes: trial process tasks (51 articles); dissemination strategies (6); and generalizability assessments (34). Despite a diverse set of diseases studied, <10% of trials using RWD for trial process tasks evaluated medications or procedures (5/51). All articles highlighted data-related challenges, such as missing values.DiscussionDatabase-specific RWD have been occasionally leveraged for various clinical trial tasks. We observed underuse of RWD within conducted medication or procedure trials, though it is subject to the confounder of implicit report of RWD use.ConclusionEnhanced incorporation of RWD should be further explored for medication or procedure trials, including better understanding of how to handle related data quality issues to facilitate RWD use.     

Toolkits for implementing and evaluating digital health: A systematic review of rigor and reporting

ObjectiveToolkits are an important knowledge translation strategy for implementing digital health. We studied how toolkits for the implementation and evaluation of digital health were developed, tested, and reported.Materials and MethodsWe conducted a systematic review of toolkits that had been used, field tested or evaluated in practice, and published in the English language from 2009 to July 2019. We searched several electronic literature sources to identify both peer-reviewed and gray literature, and records were screened as per systematic review conventions.ResultsThirteen toolkits were eventually identified, all of which were developed in North America, Europe, or Australia. All reported their intended purpose, as well as their development process. Eight of the 13 toolkits involved a literature review, 3 did not, and 2 were unclear. Twelve reported an underlying conceptual framework, theory, or model: 3 cited the normalization process theory and 3 others cited the World Health Organization and International Telecommunication Union eHealth Strategy. Seven toolkits were reportedly evaluated, but details were unavailable. Forty-three toolkits were excluded for lack of field-testing.DiscussionDespite a plethora of published toolkits, few were tested, and even fewer were evaluated. Methodological rigor was of concern, as several did not include an underlying conceptual framework, literature review, or evaluation and refinement in real-world settings. Reporting was often inconsistent and unclear, and toolkits rarely reported being evaluated.ConclusionGreater attention needs to be paid to rigor and reporting when developing, evaluating, and reporting toolkits for implementing and evaluating digital health so that they can effectively function as a knowledge translation strategy.     

Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review

Background ObjectiveElectronic health records (EHRs) are linked with documentation burden resulting in clinician burnout. While clear classifications and validated measures of burnout exist, documentation burden remains ill-defined and inconsistently measured. We aim to conduct a scoping review focused on identifying approaches to documentation burden measurement and their characteristics.Materials and MethodsBased on Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Extension for Scoping Reviews (ScR) guidelines, we conducted a scoping review assessing MEDLINE, Embase, Web of Science, and CINAHL from inception to April 2020 for studies investigating documentation burden among physicians and nurses in ambulatory or inpatient settings. Two reviewers evaluated each potentially relevant study for inclusion/exclusion criteria.ResultsOf the 3482 articles retrieved, 35 studies met inclusion criteria. We identified 15 measurement characteristics, including 7 effort constructs: EHR usage and workload, clinical documentation/review, EHR work after hours and remotely, administrative tasks, cognitively cumbersome work, fragmentation of workflow, and patient interaction. We uncovered 4 time constructs: average time, proportion of time, timeliness of completion, activity rate, and 11 units of analysis. Only 45.0% of studies assessed the impact of EHRs on clinicians and/or patients and 40.0% mentioned clinician burnout.DiscussionStandard and validated measures of documentation burden are lacking. While time and effort were the core concepts measured, there appears to be no consensus on the best approach nor degree of rigor to study documentation burden.ConclusionFurther research is needed to reliably operationalize the concept of documentation burden, explore best practices for measurement, and standardize its use.     

Informative presence and observation in routine health data: A review of methodology for clinical risk prediction

ObjectiveInformative presence (IP) is the phenomenon whereby the presence or absence of patient data is potentially informative with respect to their health condition, with informative observation (IO) being the longitudinal equivalent. These phenomena predominantly exist within routinely collected healthcare data, in which data collection is driven by the clinical requirements of patients and clinicians. The extent to which IP and IO are considered when using such data to develop clinical prediction models (CPMs) is unknown, as is the existing methodology aiming at handling these issues. This review aims to synthesize such existing methodology, thereby helping identify an agenda for future methodological work.Materials and MethodsA systematic literature search was conducted by 2 independent reviewers using prespecified keywords.ResultsThirty-six articles were included. We categorized the methods presented within as derived predictors (including some representation of the measurement process as a predictor in the model), modeling under IP, and latent structures. Including missing indicators or summary measures as predictors is the most commonly presented approach amongst the included studies (24 of 36 articles).DiscussionThis is the first review to collate the literature in this area under a prediction framework. A considerable body relevant of literature exists, and we present ways in which the described methods could be developed further. Guidance is required for specifying the conditions under which each method should be used to enable applied prediction modelers to use these methods.ConclusionsA growing recognition of IP and IO exists within the literature, and methodology is increasingly becoming available to leverage these phenomena for prediction purposes. IP and IO should be approached differently in a prediction context than when the primary goal is explanation. The work included in this review has demonstrated theoretical and empirical benefits of incorporating IP and IO, and therefore we recommend that applied health researchers consider incorporating these methods in their work.     

High-throughput phenotyping with temporal sequences

ObjectiveHigh-throughput electronic phenotyping algorithms can accelerate translational research using data from electronic health record (EHR) systems. The temporal information buried in EHRs is often underutilized in developing computational phenotypic definitions. This study aims to develop a high-throughput phenotyping method, leveraging temporal sequential patterns from EHRs.Materials and MethodsWe develop a representation mining algorithm to extract 5 classes of representations from EHR diagnosis and medication records: the aggregated vector of the records (aggregated vector representation), the standard sequential patterns (sequential pattern mining), the transitive sequential patterns (transitive sequential pattern mining), and 2 hybrid classes. Using EHR data on 10 phenotypes from the Mass General Brigham Biobank, we train and validate phenotyping algorithms.ResultsPhenotyping with temporal sequences resulted in a superior classification performance across all 10 phenotypes compared with the standard representations in electronic phenotyping. The high-throughput algorithm’s classification performance was superior or similar to the performance of previously published electronic phenotyping algorithms. We characterize and evaluate the top transitive sequences of diagnosis records paired with the records of risk factors, symptoms, complications, medications, or vaccinations.DiscussionThe proposed high-throughput phenotyping approach enables seamless discovery of sequential record combinations that may be difficult to assume from raw EHR data. Transitive sequences offer more accurate characterization of the phenotype, compared with its individual components, and reflect the actual lived experiences of the patients with that particular disease.ConclusionSequential data representations provide a precise mechanism for incorporating raw EHR records into downstream machine learning. Our approach starts with user interpretability and works backward to the technology.     

The burden of the digital environment: a systematic review on organization-directed workplace interventions to mitigate physician burnout

ObjectiveTo conduct a systematic review identifying workplace interventions that mitigate physician burnout related to the digital environment including health information technologies (eg, electronic health records) and decision support systems) with or without the application of advanced analytics for clinical care.Materials and MethodsLiterature published from January 1, 2007 to June 3, 2020 was systematically reviewed from multiple databases and hand searches. Subgroup analysis identified relevant physician burnout studies with interventions examining digital tool burden, related workflow inefficiencies, and measures of burnout, stress, or job satisfaction in all practice settings.ResultsThe search strategy identified 4806 citations of which 81 met inclusion criteria. Thirty-eight studies reported interventions to decrease digital tool burden. Sixty-eight percent of these studies reported improvement in burnout and/or its proxy measures. Burnout was decreased by interventions that optimized technologies (primarily electronic health records), provided training, reduced documentation and task time, expanded the care team, and leveraged quality improvement processes in workflows.DiscussionThe contribution of digital tools to physician burnout can be mitigated by careful examination of usability, introducing technologies to save or optimize time, and applying quality improvement to workflows.ConclusionPhysician burnout is not reduced by technology implementation but can be mitigated by technology and workflow optimization, training, team expansion, and careful consideration of factors affecting burnout, including specialty, practice setting, regulatory pressures, and how physicians spend their time.     

Factors associated with nurse well-being in relation to electronic health record use: A systematic review

ObjectiveAlthough nurses comprise the largest group of health professionals and electronic health record (EHR) user base, it is unclear how EHR use has affected nurse well-being. This systematic review assesses the multivariable (ie, organizational, nurse, and health information technology [IT]) factors associated with EHR-related nurse well-being and identifies potential improvements recommended by frontline nurses.Materials and MethodsWe searched MEDLINE, Embase, CINAHL, PsycINFO, ProQuest, and Web of Science for literature reporting on EHR use, nurses, and well-being. A quality appraisal was conducted using a previously developed tool.ResultsOf 4583 articles, 12 met inclusion criteria. Two-thirds of the studies were deemed to have a moderate or low risk of bias. Overall, the studies primarily focused on nurse- and IT-level factors, with 1 study examining organizational characteristics. That study found worse nurse well-being was associated with EHRs compared with paper charts. Studies on nurse-level factors suggest that personal digital literacy is one modifiable factor to improving well-being. Additionally, EHRs with integrated displays were associated with improved well-being. Recommendations for improving EHRs suggested IT-, organization-, and policy-level solutions to address the complex nature of EHR-related nurse well-being.ConclusionsThe overarching finding from this synthesis reveals a critical need for multifaceted interventions that better organize, manage, and display information for clinicians to facilitate decision making. Our study also suggests that nurses have valuable insight into ways to reduce EHR-related burden. Future research is needed to test multicomponent interventions that address these complex factors and use participatory approaches to engage nurses in intervention development.     

Automated Acquisition of Disease–Drug Knowledge from Biomedical and Clinical Documents: An Initial Study

Objective

Explore the automated acquisition of knowledge in biomedical and clinical documents using text mining and statistical techniques to identify disease-drug associations.

Design

Biomedical literature and clinical narratives from the patient record were mined to gather knowledge about disease-drug associations. Two NLP systems, BioMedLEE and MedLEE, were applied to Medline articles and discharge summaries, respectively. Disease and drug entities were identified using the NLP systems in addition to MeSH annotations for the Medline articles. Focusing on eight diseases, co-occurrence statistics were applied to compute and evaluate the strength of association between each disease and relevant drugs.

Results

Ranked lists of disease-drug pairs were generated and cutoffs calculated for identifying stronger associations among these pairs for further analysis. Differences and similarities between the text sources (i.e., biomedical literature and patient record) and annotations (i.e., MeSH and NLP-extracted UMLS concepts) with regards to disease-drug knowledge were observed.

Conclusion

This paper presents a method for acquiring disease-specific knowledge and a feasibility study of the method. The method is based on applying a combination of NLP and statistical techniques to both biomedical and clinical documents. The approach enabled extraction of knowledge about the drugs clinicians are using for patients with specific diseases based on the patient record, while it is also acquired knowledge of drugs frequently involved in controlled trials for those same diseases. In comparing the disease-drug associations, we found the results to be appropriate: the two text sources contained consistent as well as complementary knowledge, and manual review of the top five disease-drug associations by a medical expert supported their correctness across the diseases.     

Automatic discourse connective detection in biomedical text

Objective

Relation extraction in biomedical text mining systems has largely focused on identifying clause-level relations, but increasing sophistication demands the recognition of relations at discourse level. A first step in identifying discourse relations involves the detection of discourse connectives: words or phrases used in text to express discourse relations. In this study supervised machine-learning approaches were developed and evaluated for automatically identifying discourse connectives in biomedical text.

Materials and Methods

Two supervised machine-learning models (support vector machines and conditional random fields) were explored for identifying discourse connectives in biomedical literature. In-domain supervised machine-learning classifiers were trained on the Biomedical Discourse Relation Bank, an annotated corpus of discourse relations over 24 full-text biomedical articles (∼112 000 word tokens), a subset of the GENIA corpus. Novel domain adaptation techniques were also explored to leverage the larger open-domain Penn Discourse Treebank (∼1 million word tokens). The models were evaluated using the standard evaluation metrics of precision, recall and F1 scores.

Results and Conclusion

Supervised machine-learning approaches can automatically identify discourse connectives in biomedical text, and the novel domain adaptation techniques yielded the best performance: 0.761 F1 score. A demonstration version of the fully implemented classifier BioConn is available at: http://bioconn.askhermes.org.     

The quality of social determinants data in the electronic health record: a systematic review

ObjectiveThe aim of this study was to collect and synthesize evidence regarding data quality problems encountered when working with variables related to social determinants of health (SDoH).Materials and MethodsWe conducted a systematic review of the literature on social determinants research and data quality and then iteratively identified themes in the literature using a content analysis process.ResultsThe most commonly represented quality issue associated with SDoH data is plausibility (n = 31, 41%). Factors related to race and ethnicity have the largest body of literature (n = 40, 53%). The first theme, noted in 62% (n = 47) of articles, is that bias or validity issues often result from data quality problems. The most frequently identified validity issue is misclassification bias (n = 23, 30%). The second theme is that many of the articles suggest methods for mitigating the issues resulting from poor social determinants data quality. We grouped these into 5 suggestions: avoid complete case analysis, impute data, rely on multiple sources, use validated software tools, and select addresses thoughtfully.DiscussionThe type of data quality problem varies depending on the variable, and each problem is associated with particular forms of analytical error. Problems encountered with the quality of SDoH data are rarely distributed randomly. Data from Hispanic patients are more prone to issues with plausibility and misclassification than data from other racial/ethnic groups.ConclusionConsideration of data quality and evidence-based quality improvement methods may help prevent bias and improve the validity of research conducted with SDoH data.     

Enhancing Text Categorization with Semantic-enriched Representation and Training Data Augmentation

ObjectiveAcquiring and representing biomedical knowledge is an increasingly important component of contemporary bioinformatics. A critical step of the process is to identify and retrieve relevant documents among the vast volume of modern biomedical literature efficiently. In the real world, many information retrieval tasks are difficult because of high data dimensionality and the lack of annotated examples to train a retrieval algorithm. Under such a scenario, the performance of information retrieval algorithms is often unsatisfactory, therefore improvements are needed.DesignWe studied two approaches that enhance the text categorization performance on sparse and high data dimensionality: (1) semantic-preserving dimension reduction by representing text with semantic-enriched features; and (2) augmenting training data with semi-supervised learning. A probabilistic topic model was applied to extract major semantic topics from a corpus of text of interest. The representation of documents was projected from the high-dimensional vocabulary space onto a semantic topic space with reduced dimensionality. A semi-supervised learning algorithm based on graph theory was applied to identify potential positive training cases, which were further used to augment training data. The effects of data transformation and augmentation on text categorization by support vector machine (SVM) were evaluated.Results and ConclusionSemantic-enriched data transformation and the pseudo-positive-cases augmented training data enhance the efficiency and performance of text categorization by SVM.     

基于本体论的电子健康档案知识库构建初探

电子健康档案具有明显的文献特征,有较强的研究价值和挖掘价值。在概述电子健康档案文献特点和生物医药语义知识库研究现状的基础上,论述了电子健康档案知识库构建的步骤、技术难点及解决思路。讨论了电子健康档案如何引入本体和本体技术以及进行语义抽取,在此基础上提出基于本体的数据挖掘技术应用于健康档案的构想,实现健康档案中医学知识的多维度关联与智能检索功能。     

基于文献研究的推拿外治法不良事件分析与对策

[目的] 了解国内推拿不良事件现状,促进推拿手法的标准化,从而提高推拿治疗的安全性。[方法] 采用回顾性期刊文献研究,检索中国知网中国期刊全文数据库、维普中文科技期刊全文数据库、万方数据库、中国生物医学文献数据库的推拿不良事件病例报道类文献,对使用的手法、意外情况类型、原发疾病、事故原因、病例数等进行统计分析。[结果] 检出有效文献202篇,共 709个病例。分析结果显示推拿不良事件主要包括骨折与脱位、神经损伤、晕厥、截瘫、椎间盘突出症等。扳法、按法、牵引、被动肢体屈伸等手法标准化程度低,在推拿治疗中易导致不良事件发生。[结论] 部分推拿手法的安全性尚待提高,推拿手法标准化是提高其安全性的重要途径,相关工作亟待开展。     

A systematic literature review of Native American and Pacific Islanders’ perspectives on health data privacy in the United States

BackgroundPrivacy-related concerns can prevent equitable participation in health research by US Indigenous communities. However, studies focused on these communities'' views regarding health data privacy, including systematic reviews, are lacking.MethodsWe conducted a systematic literature review analyzing empirical, US-based studies involving American Indian/Alaska Native (AI/AN) and Native Hawaiian or other Pacific Islander (NHPI) perspectives on health data privacy, which we define as the practice of maintaining the security and confidentiality of an individual’s personal health records and/or biological samples (including data derived from biological specimens, such as personal genetic information), as well as the secure and approved use of those data.ResultsTwenty-one studies involving 3234 AI/AN and NHPI participants were eligible for review. The results of this review suggest that concerns about the privacy of health data are both prevalent and complex in AI/AN and NHPI communities. Many respondents raised concerns about the potential for misuse of their health data, including discrimination or stigma, confidentiality breaches, and undesirable or unknown uses of biological specimens.ConclusionsParticipants cited a variety of individual and community-level concerns about the privacy of their health data, and indicated that these deter their willingness to participate in health research. Future investigations should explore in more depth which health data privacy concerns are most salient to specific AI/AN and NHPI communities, and identify the practices that will make the collection and use of health data more trustworthy and transparent for participants.     

Electronic health record–based disease surveillance systems: A systematic literature review on challenges and solutions

ObjectiveDisease surveillance systems are expanding using electronic health records (EHRs). However, there are many challenges in this regard. In the present study, the solutions and challenges of implementing EHR-based disease surveillance systems (EHR-DS) have been reviewed.Materials and MethodsWe searched the related keywords in ProQuest, PubMed, Web of Science, Cochrane Library, Embase, and Scopus. Then, we assessed and selected articles using the inclusion and exclusion criteria and, finally, classified the identified solutions and challenges.ResultsFinally, 50 studies were included, and 52 unique solutions and 47 challenges were organized into 6 main themes (policy and regulatory, technical, management, standardization, financial, and data quality). The results indicate that due to the multifaceted nature of the challenges, the implementation of EHR-DS is not low cost and easy to implement and requires a variety of interventions. On the one hand, the most common challenges include the need to invest significant time and resources; the poor data quality in EHRs; difficulty in analyzing, cleaning, and accessing unstructured data; data privacy and security; and the lack of interoperability standards. On the other hand, the most common solutions are the use of natural language processing and machine learning algorithms for unstructured data; the use of appropriate technical solutions for data retrieval, extraction, identification, and visualization; the collaboration of health and clinical departments to access data; standardizing EHR content for public health; and using a unique health identifier for individuals.ConclusionsEHR systems have an important role in modernizing disease surveillance systems. However, there are many problems and challenges facing the development and implementation of EHR-DS that need to be appropriately addressed.     

Can reproducibility be improved in clinical natural language processing? A study of 7 clinical NLP suites

BackgroundThe increasing complexity of data streams and computational processes in modern clinical health information systems makes reproducibility challenging. Clinical natural language processing (NLP) pipelines are routinely leveraged for the secondary use of data. Workflow management systems (WMS) have been widely used in bioinformatics to handle the reproducibility bottleneck.ObjectiveTo evaluate if WMS and other bioinformatics practices could impact the reproducibility of clinical NLP frameworks.Materials and MethodsBased on the literature across multiple researcho fields (NLP, bioinformatics and clinical informatics) we selected articles which (1) review reproducibility practices and (2) highlight a set of rules or guidelines to ensure tool or pipeline reproducibility. We aggregate insight from the literature to define reproducibility recommendations. Finally, we assess the compliance of 7 NLP frameworks to the recommendations.ResultsWe identified 40 reproducibility features from 8 selected articles. Frameworks based on WMS match more than 50% of features (26 features for LAPPS Grid, 22 features for OpenMinted) compared to 18 features for current clinical NLP framework (cTakes, CLAMP) and 17 features for GATE, ScispaCy, and Textflows.Discussion34 recommendations are endorsed by at least 2 articles from our selection. Overall, 15 features were adopted by every NLP Framework. Nevertheless, frameworks based on WMS had a better compliance with the features.ConclusionNLP frameworks could benefit from lessons learned from the bioinformatics field (eg, public repositories of curated tools and workflows or use of containers for shareability) to enhance the reproducibility in a clinical setting.     

人工韧带与自体肌腱重建前交叉韧带的meta分析

目的 系统评价LARS人工韧带及自体移植物在前交叉韧带重建中的安全性和有效性.方法 通过检索PubMed数据库、中国学术期刊全文数据库(CNKI)、中国生物医学文献数据库(CBMdisc)、万方数据库获得已公开发表的LARS人工韧带与自体移植物进行前交叉韧带重建的文献.提取相关数据进行meta分析.结果 本研究共纳入了9篇文献,总病例数456例,结果显示LARS人工韧带术后患者膝关节Lysholm评分及Tegner评分较术前有明显提高(Lysholm:MD=50.05,95 ％CI 48.41～51.68;Tegner:MD=4.41,95％CI 3.40～5.42);并且在术后2年其Lysholm评分及Tegner评分改善较自体肌腱移植仍更明显(Lysholm:MD=0.20,95％CI 0.04～0.35;Tegner:MD=0.18,95％ CI 0.04～0.32).同时,术后2年LARS人工韧带组滑膜炎等并发症发生率低,与自体移植物无明显差异.结论 LARS人工韧带具有良好的临床疗效和安全性,并且在术后2年其稳定性较自体肌腱移植仍具有明显优势,但该结论仍需长随访、高质量的临床研究进一步证实.