Routine Outcomes Monitoring to Support Improving Care for Schizophrenia: Report from the VA Mental Health QUERI

In schizophrenia, treatments that improve outcomes have not been reliably disseminated. A major barrier to improving care has been a lack of routinely collected outcomes data that identify patients who are failing to improve or not receiving effective treatments. To support high quality care, the VA Mental Health QUERI used literature review, expert interviews, and a national panel process to increase consensus regarding outcomes monitoring instruments and strategies that support quality improvement. There was very good consensus in the domains of psychotic symptoms, side-effects, drugs and alcohol, depression, caregivers, vocational functioning, and community tenure. There are validated instruments and assessment strategies that are feasible for quality improvement in routine practice.     

Quality Management Program of Stroke Rehabilitation Using Adherence to Guidelines: A Nationwide Initiative in Japan

Background and AimIn recent years, interest in the quality of medical care has rapidly increased worldwide. However, quality indicators that contribute to establishing standard treatment in stroke medicine, especially rehabilitation, are not well-developed in Japan. Japan has established Kaifukuki (convalescent) rehabilitation wards, and the development of quality indicators for stroke rehabilitation in the convalescent phase is an urgent issue.MethodsWe first reviewed the literature regarding quality indicators for stroke rehabilitation. Next, we extracted candidate indicators from identified reports and guidelines and surveyed educational hospitals certified by the Japanese Association of Rehabilitation Medicine. On the basis of the survey results, we reevaluated the suitability of the proposed indicators in discussions with an expert panel.ResultsThe questionnaire survey highlighted several important items that revealed there is room for improvement in adherence. For stroke rehabilitation in the convalescent phase, we adopted 15 indicators that were feasible as indicators to be used for comparisons between facilities, based on scoring by and opinions of the expert panel. These indicators measured structure (2 indicators), process (5 indicators), and outcome (8 indicators).ConclusionThis is the first study to establish quality indicators to standardize stroke rehabilitation in Japan. We developed this set of 15 indicators using an evidence-based approach. However, many tasks remain for continuous quality improvement.     

Association of specialist involvement and quality of care for Parkinson's disease.

Because Parkinson's disease (PD) has multiple neurological symptoms and often complex treatments, the quality of PD care may be higher when a specialist is involved. We examined the medical records, from 1998 to 2004, of 401 Los Angeles veterans with Parkinson's disease to determine whether care met key indicators of PD care quality. All care following a visit to a movement-disorder specialist or general neurologist was classified as specialty care. We compared adherence to each indicator by level of specialist involvement through logistic regression models. Over the study period, 10 indicators of PD care quality were triggered 2,227 times. Overall, movement disorder specialist involvement (78%) was associated with higher adherence to indicators than did general neurologist involvement (70%, P = 0.006) and nonneurologist involvement (52%, P < 0.001). The differences between movement disorder specialist and nonneurologist involvement were especially large for four indicators: treatment of wearing-off, assessments of falls, depression, and hallucinations. There is significant room for improving aspects of PD care quality among patients who do not have the involvement of a specialist. Quality of care interventions should involve specialists in management of motor symptoms and incorporate methods for routine assessment of nonmotor PD symptoms.     

Using global statistical tests in long‐term Parkinson's disease clinical trials

Parkinson's disease (PD) impairments are multidimensional, making it difficult to choose a single primary outcome when evaluating treatments to stop or lessen the long‐term decline in PD. We review commonly used multivariate statistical methods for assessing a treatment's global impact, and we highlight the novel Global Statistical Test (GST) methodology. We compare the GST to other multivariate approaches using data from two PD trials. In one trial where the treatment showed consistent improvement on all primary and secondary outcomes, the GST was more powerful than other methods in demonstrating significant improvement. In the trial where treatment induced both improvement and deterioration in key outcomes, the GST failed to demonstrate statistical evidence even though other techniques showed significant improvement. Based on the statistical properties of the GST and its relevance to overall treatment benefit, the GST appears particularly well suited for a disease like PD where disability and impairment reflect dysfunction of diverse brain systems and where both disease and treatment side effects impact quality of life. In future long term trials, use of GST for primary statistical analysis would allow the assessment of clinically relevant outcomes rather than the artificial selection of a single primary outcome. © 2009 Movement Disorder Society     

Quality of Care Measures for the Treatment of Bipolar Disorder

The staff of the American Psychiatric Assocition (APA), the American Psychiatric Institute for Research and Education (APIRE), and a national panel of experts in bipolar disorder and practice guideline development have collaborated to generate a set of quality of care indicators for the pharmacologic and psychosocial treatment of bipolar disorder. The indicators were derived from APA’s evidence-based Practice Guideline for the Treatment of Patients with Bipolar Disorder, 2002 (1) and the Expert Consensus Guideline Series: Medication Treatment of Bipolar Disorder, 2000 (2) These quality indicators can be used for quality monitoring, benchmarking, and quality improvement efforts across health plans, systems of care, and health care providers to improve quality and outcomes of care for patients with bipolar disorder.     

Clinical quality measures for electrodiagnosis in suspected carpal tunnel syndrome

Extensive research has documented that medical care in the United States is not of optimal quality, meaning that well‐established care processes are not consistently provided to the patients who would benefit from them. To assess and improve quality of care, specific measures are needed. The objective of this study was to develop quality measures for electrodiagnostic testing in suspected carpal tunnel syndrome (CTS). We used a variation of the well‐established RAND/UCLA Appropriateness Method to develop the measures. A physiatrist and quality measurement experts developed draft measures based on guidelines and literature. Subsequently, in a two‐round, modified‐Delphi process, a multidisciplinary panel of 11 national experts in CTS reviewed a summary of the evidence and then rated the measures for validity and feasibility. Seven draft measures were developed. The expert panel combined two, modified the others, and then judged all resulting measures to be valid and feasible. The measures cover compelling indications for testing, essential test components when CTS is suspected, skin temperature measurement and normalization, and the appropriate interpretation of test results. These measures define a minimum standard of care for the use of electrodiagnostic tests in suspected CTS and are consistent with recent guidelines developed by the American Association of Neuromuscular and Electrodiagnostic Medicine. Provider organizations, insurance companies, and professional societies can use these measures in efforts to monitor and improve quality of care for this common and disabling condition. Muscle Nerve, 2010     

Development of performance indicators for the primary care management of pediatric epilepsy: expert consensus recommendations based on the available evidence

PURPOSE: To use available evidence and expert consensus to develop performance indicators for the evaluation and management of pediatric epilepsy. METHODS: We used a three-step process to develop the performance indicators. First, research findings were compiled into evidence tables focusing on different clinical issues. Second, an advisory panel of clinicians, educational and public health experts, and families of children with epilepsy reviewed the evidence. The advisory group used the evidence to draft a preliminary set of performance indicators for pediatric epilepsy management. Third, 13 internationally recognized experts in pediatric neurology or epilepsy rated the value of these indicators on a 5-point scale [1 (essential) to 5 (not necessary)] in a two-round Delphi process. Positive consensus was reached if >or=80% of experts gave an indicator a "1" rating and negative consensus if >80% gave an indicator a "5" rating. Indicators that achieved positive consensus during either round of the Delphi process constituted the final set of indicators. RESULTS: Of the 68 draft performance indicators, the expert panel members achieved positive consensus on 30 performance indicators: eight indicators related to diagnostic strategies and seizure classification, nine related to antiepileptic drug use, six related to cognitive and behavioral issues, six related to quality of life, and three related to specialty referrals. CONCLUSIONS: We identified 30 potential indicators for evaluating the care provided to pediatric patients with epilepsy. The next step is to examine the relation of these performance indicators to clinical outcomes and health care utilization among pediatric patients with epilepsy.     

A national initiative to advance school mental health performance measurement in the US

Standardized health performance measurement has increasingly become an imperative for assuring quality standards in national health care systems. As compared to somatic health performance measures, behavioral health performance measures are less developed. There currently is no national standardized performance measurement system for monitoring comprehensive school mental health in the USA. Drawing upon related initiatives in the school mental health field and national performance standards for behavioral health care in non-school settings, the current study describes the identification, development, and refinement of quality and sustainability performance measures for comprehensive school mental health. A three-phase method involving literature review and expert consensus, nominal group decision-making, and a modified Delphi process resulted in the development of two performance assessments, which include 12 domains and 67 indicators. Future directions for national performance measurement among school mental health systems to drive improvement in quality and sustainability are considered.     

Development of quality indicators for memory clinics

OBJECTIVE: To develop and validate a set of relevant, feasible, and reliable quality indicators (QIs) for the Memory Clinics (MCs). BACKGROUND: MCs are important care providers for people with dementia and their caregivers. A set of valid QIs is needed to incorporate evidence-based guidelines into MC clinical practice, and measure adherence to guidelines. METHODS: A total of 17 MC specialists, 12 primary care physicians; and seven informal caregivers participated in several Delphi rounds to select and validate QIs. Ten MCs participated in the indicator compliance study involving the analysis of data extracted from 100 medical records. RESULTS: The initial set of 56 single QIs was reduced to a final set of 14 indicators measuring the quality of processes, structures, and outcomes of care. The panels of representatives of MCs, primary care physicians, and informal caregivers judged overall face validity to be high. The differences in compliance rates between the three indicator types were significant (p<0.001) as were the differences between the MCs (p<0.005). The compliance measures were highest for the process indicators and lowest for the outcome indicators. CONCLUSION: The final set of 14 QIs that met the psychometric requirements can be used to facilitate the implementation of guidelines and the assessment of the quality of care offered by MCs. The QIs are acceptable for a broad range of users (specialists, referring physicians, and informal caregivers), and are capable of discriminating between MCs in terms of quality.     

From quality assurance to quality improvement

As we are learning to master the methods of quality assurance, a new conceptual approach focused on quality improvement is being advocated. But this new approach is not a derailment for health care practitioners committed to improve patient care quality. Rather, it is a progressive step that builds upon the concepts and methods of quality assurance. Whereas quality assurance tends to focus on correcting problems in patient care quality--especially individual practitioners' problems--quality improvement focuses on finding opportunities to improve quality by changing systems as well as individual practitioner behavior. Whereas quality assurance tends to rely primarily on standards or guidelines for the structures and processes of care and the intensive review of individual cases with undesirable outcomes, quality improvement also utilizes statistical profiles of outcomes, structures, and processes as baselines against which improvement can be measured. Whereas quality assurance may engender defensiveness in an individual practitioner, quality improvement is based on the health care practitioner's professional ethic and pride in providing even better care to patients tomorrow than is possible today. To improve patient care quality, it is necessary to examine the structures, processes, and outcomes of care. The structures and processes can be controlled by the practitioner and organization; the outcomes are the benchmarks against which the effectiveness of the structures and processes can be evaluated. Outcomes in health care are the effects of more than just the structures and processes controlled by the practitioner and organization. Therefore, outcomes are not direct and complete measures of the quality of the care provided. They are instead indicators of performance. These indicators can focus attention on care that lies outside the normal statistical parameters of performance and can provide baselines against which improvements can be measured. The monitoring and evaluation process described in Joint Commission standards is a method for establishing priorities and using indicators to improve the quality of patient care. This method can be used to focus case-based review of care and to establish baselines for continuous improvement. This conceptual approach that emphasizes quality improvement and the use of the monitoring and evaluation process is guiding the Joint Commission's development of clinical indicators and revision of its standards and survey process to help health care organizations in the transition from quality assurance to quality improvement.     

Assessment of the Massachusetts Behavioral Health Program Year 6

After a difficult transition from the previous vendor to the Massachusetts Behavioral Health Partnership (MBHP), Year 6 was a year of stability and incremental changes for the Massachusetts Behavioral Health Program. This assessment of Year 6 is based on interviews with key players, data provided by the MBHP, a survey of providers, as well as on the fifth year of an ongoing review of the program. Results indicate that enrollment grew, and new services were developed in response to identified needs. Providers considered access, utilization, and quality of care to be the same or better than a year earlier. Coordination improved, but was not optimal. Clinical and overall decisions with MBHP were collaborative or negotiated and less hierarchical in manner than the previous year. Providers rated MBHP better than other managed care organizations on quality of care and utilization review decisions, access, flexibility, and administration.     

Improving nursing home quality of care through outcomes data: the MDS quality indicators

BACKGROUND: The quality of care provided to nursing home residents is a continuing source of concern throughout the world. In the United States, the Health Care Financing Administration mandated the use of a standardised resident assessment instrument, called the Minimum Data Set (MDS) which facilitated the development of a set of nursing home quality of care indicators. METHODS: The MDS Quality Indicators were developed by a team of researchers at the University of Wisconsin-Madison and covered 12 domains, as well as associated risk factors. This initial set of indicators then went through an iterative process of review by national clinical panels, further empirical analysis, validation studies, and pilot tests, to confirm whether they were accurate, reliable and useful tools for identifying quality problems. RESULTS: The final set of MDS Quality Indicators includes 24 variables that include both processes and outcomes of care and prevalence and incidence measures. They are defined at both the resident and facility level. Four of the indicators are risk-adjusted. CONCLUSION: The indicators have several possible applications: a source of consumer information; to help guide policy makers; and to assist providers in conducting quality improvement initiatives and to evaluate the impact of these efforts.     

The concept of quality of life: what we know and do not know

Background Over the past two decades the concept of quality of life (QOL) has increasingly become a focus for research and application in the fields of education/special education, health care (physical and behavioural), social services (disabilities and ageing), and families. Methods This article summarizes our current understanding of the construct of individual QOL as it pertains to persons with intellectual disabilities (ID). The article's three major sections discuss what we know, what we are beginning to understand, and what we still do not know about the QOL construct. Results We currently know the importance of the QOL construct as a service delivery principle, along with its current use and multidimensional nature. We are beginning to understand the importance of methodological pluralism in the assessment of QOL, the multiple uses of quality indicators, the predictors of assessed QOL, the effects of different data collection strategies, and the etic (universal) and emic (culture‐bound) properties of the construct. We have yet to understand fully the use of QOL‐related outcomes in programme change, how to best evaluate the outcomes of QOL‐related services, and how to use the concept of QOL to impact public and disability reform. Conclusion The article concludes with a brief discussion of future challenges related to demonstrating the concept's social validity and positive impact on the lives of persons with ID.     

Evaluation of the quality of the care pathway for patients with multiple sclerosis in France: Results of an original study of a cohort of 700 patients

IntroductionEvaluating the quality of the care pathway for patients with chronic diseases, such as multiple sclerosis (MS), is an important issue. Process indicators are a recognized method for evaluating professional practices. However, these tools have been little developed in the field of MS, and few data are available. The aim of this study was to describe, retrospectively, with validated indicators, the quality of the care pathway in a population-based cohort of 700 patients with the first manifestations of the disease occurring between January 1, 2000 and December 31, 2001 and during the first 10 years of disease.MethodThis assessment was based on 48 indicators specific to MS. The information required for the calculation of each indicator was collected from the source files of the 700 patients of the cohort.ResultsData for the 10 years of follow-up were collected for 80% of the patients. In total, 36 indicators were calculated. These results reveal that there is room for improvement, particularly in terms of the initial assessment, access to ophthalmological evaluation, employment, obtaining an evaluation of the need for rehabilitation and access to such care.ConclusionThe results of this survey provide access to unprecedented new data in France, that professionals and patients can appropriate to improve the targeting of actions, to improve the quality of care further for patients with MS in France. We propose to continue this process by submitting, for discussion, a targeted list of updated indicators relating to changes in guidelines, and in issues concerning the quality of patient management.     

Conjunto de indicadores de calidad asistencial en el abordaje del ictus

ObjectiveThis study proposes a set of quality indicators for care outcomes in patients with acute cerebral infarction. These indicators are understandable and relevant from a clinical viewpoint, as well as being acceptable and feasible in terms of time required, ease of data capture, and interpretability.MethodThe method consisted of reaching consensus among doctors after having reviewed the literature on quality indicators in stroke. We then designed and conducted a field study to assess the understandability and feasibility of the set of indicators.ResultsConsensus yielded 8 structural indicators, 5 process indicators, and 12 result indicators. Additionally, standards of reference were established for each indicator.ConclusionThis set of indicators can be used to monitor the quality care for stroke patients, identify strengths, and potentially to identify areas needing improvement.     

Developing tools to evaluate quality of care management for patients living with multiple sclerosis: An original French initiative

IntroductionAssessing the quality of care management for patients with a chronic disease such as multiple sclerosis (MS) is a major challenge for healthcare systems around the world. It needs to be carried out using tools that are recognized by professionals and patients alike, and should concern practices, systems, and scientific data. No such tools are currently available in Europe. The purpose of the present study was to develop indicators to contribute to assess the quality of care management for patients with MS in France.MethodsAn expert panel comprising 25 professionals from well known teams across France selected the indicators on the basis of consensus. In accordance with the Rand/UCLA Appropriateness Method, each expert had to agree with the recommendations, and there had to be agreement among the experts.ResultsThe expert panel selected 48 indicators representing seven domains of care management for patients with MS: physical and rehabilitation medicine, disease progression, access to care, magnetic resonance imaging (MRI) management, relapse management, management of disease-modifying treatments, and management of the symptoms of disability progression. Some of these quality indicators (notably pertaining to MRI management) had not previously been identified in the literature.ConclusionThese indicators may allow professionals to comprehensively assess and compare their practices and cooperation, thereby contributing to improve the quality of care management for patients with MS in France.     

Piloting the NPF data-driven quality improvement initiative

ObjectiveTo pilot a data-driven quality care program in National Parkinson Foundation (NPF) Centers of Excellence.BackgroundEvidence from comparative effectiveness research (CER) can be used to guide decisions regarding health care and to improve quality and efficiency of care. We propose to develop the infrastructure required to conduct CER across an extensive network of NPF Centers of Excellence.MethodsWe present the staged planning for a pilot study which will demonstrate the development and implementation of the infrastructure that will be needed for a large standardized patient-centered, clinical practice database for PD. This database will support CER and drive quality improvement studies.ResultsWe describe the infrastructure for the ongoing pilot feasibility testing in a subset of six NPF Centers of Excellence, and we discuss the impact that the data (available in 2010) could have in guiding PD management.ConclusionThis preliminary experience will facilitate the longitudinal tracking of therapies and of outcomes in PD clinical practice. Further, we are hopeful that the information will provide insight into PD that will extend beyond the clinical trials population (the population included in most available PD databases). This prospective standardized real-world multi-center clinical practice database will aim to identify positive health outcomes associated with treatment approaches, and to identify variations in clinical outcomes that may suggest improvements in best clinical practice patterns.     

Validity of the Depression-Arkansas (D-ARK) Scale: A Tool for Measuring Major Depressive Disorder

The results of 2 validation studies for an assessment tool designed specifically for quality improvement and outcomes assessment efforts in mental healthcare are presented in this paper. The studies evaluated a new tool to assess the patient outcomes for major depressive disorder following treatment in routine clinical settings called the Depression-Arkansas Scale (D-ARK). Study 1 included 54 patients recruited from 3 hospital-based clinics (2 mental health clinics and 1 primary care clinic). Study 2 includes 827 patients from 5 clinical settings including a university based outpatient clinic, a VA based mental health clinic, and a managed-care program. These 2 very different studies provide preliminary evidence that the D-ARK may be a useful tool for quality improvement efforts in the mental healthcare setting. Specifically, they indicate that the D-ARK has strong validity when compared to 2 different research assessments, the Structured Clinical Interview for DSM-III-R, Patient Edition (SCID-P) and the Inventory to Diagnose Depression (IDD), and compared to clinical assessments using both the clinical diagnosis and a clinician checklist     

Quality improvement for depression in primary care: do patients with subthreshold depression benefit in the long run?

OBJECTIVE: Quality improvement programs for depression can improve outcomes, but the utility of including patients with subthreshold depression in quality improvement programs is unclear. The authors examined 57-month effects of quality improvement on clinical outcomes and mental health care utilization of primary care patients with depressive disorder and subthreshold depression. METHOD: In a group-level, randomized, controlled trial, 46 primary care clinics were randomly assigned to provide usual care or care with a quality improvement intervention that included provider training and other resources for either medication management (medications quality improvement) or evidence-based psychotherapy (therapy quality improvement). Among 1,356 enrolled depressed patients, 991 completed the 57-month follow-up interview (604 patients with depressive disorder and 387 with subthreshold depression). Outcomes measured at 57 months were presence of probable depressive disorder, unmet need for appropriate care (untreated probable disorder), and mental health care utilization in the prior 6 months. RESULTS: Among patients with subthreshold depression at baseline, those seen in clinics with quality improvement programs with special resources for therapy were less likely to have probable depressive disorder and unmet need for care at follow-up, compared with those seen in clinics that provided usual care. Among patients with depressive disorder at baseline, those seen in clinics with quality improvement programs with special resources for medication management were less likely to have unmet need for care at follow-up, compared with those seen in clinics that provided usual care. Patients with subthreshold depression at baseline seen in clinics with a quality improvement intervention were less likely at follow-up to have had a mental health visit (in primary care or specialty care, depending on the intervention) in the prior 6 months. CONCLUSIONS: Relative to usual care, quality improvement interventions improved 57-month outcomes (probable depression, unmet need, or both) for primary care patients with depressive disorder and subthreshold depression and lowered use of mental health visits for those with subthreshold depression. The results highlight the feasibility and utility of including patients with subthreshold depression in such programs.     

Clinical utility as a criterion for revising psychiatric diagnoses

OBJECTIVE: Changes in DSM-IV were guided by empirical data that mostly focused on improving diagnostic validity and reliability. Although many changes were made explicitly to improve clinical utility, no formal effort was made to empirically determine actual improvements in clinical utility. The authors propose that future revisions of DSM empirically demonstrate improvement in clinical utility to clarify whether the advantages of changing the diagnostic criteria outweigh potential negative consequences. METHOD: The authors provide a formal definition of clinical utility and then suggest that the merits of a proposed change to DSM be evaluated by considering 1) its impact on the use of the diagnostic system, 2) whether it enhances clinical decision making, and 3) whether it improves clinical outcome. RESULTS: Evaluating a change based on its impact on use considers both user acceptability and accuracy in application of the diagnostic criteria. User acceptability can be measured by surveying users' reactions, assessing user acceptability in a field trial setting, and measuring the effects on ease of use. Assessment of the correct application of diagnostic criteria entails comparing the clinician's diagnostic assessment to expert diagnostic assessment. Assessments of the impact on clinical decision making use methods developed for evaluating adherence to practice guidelines. Improvement in outcome entails measuring reduction in symptom severity or improvement in functioning or in documenting the prevention of a future negative outcome. CONCLUSIONS: Empirical methods should be applied to the assessment of changes that purport to improve clinical utility in future revisions of DSM.