The incorporation of potential confounding variables in Markov models

OBJECTIVE: To improve the quality of the methods used in Markov modelling studies by increasing the external validity by means of the incorporation of confounding variables. STUDY DESIGN: The concepts were illustrated using a hypothetical Markov model for Parkinson's disease. METHODS: The methodology consisted of incorporation of an extra explanatory variable in the Markov health states by means of health state-specific relationships between this explanatory variable and costs as well as time-dependent values of the extra explanatory variable. In addition, we determined the relevance of the incorporation of an extra explanatory variable by means of various sensitivity analyses. RESULTS: The results showed that the outcomes of a health economic model may be severely biased, when a confounding effect of an extra explanatory variable is not taken into account. Hence the external validity of Markov models may be limited, and consequently the results of the model are not an accurate reflection of reality. CONCLUSION: This study proves the need for the incorporation of all relevant explanatory variables in a health economic model.     

Measuring sensitivity in pharmacoeconomic studies. Refining point sensitivity and range sensitivity by incorporating probability distributions.

OBJECTIVE: The aim of the present study is to describe a refinement of a previously presented method, based on the concept of point sensitivity, to deal with uncertainty in economic studies. DESIGN: The original method was refined by the incorporation of probability distributions which allow a more accurate assessment of the level of uncertainty in the model. In addition, a bootstrap method was used to create a probability distribution for a fixed input variable based on a limited number of data points. The original method was limited in that the sensitivity measurement was based on a uniform distribution of the variables and that the overall sensitivity measure was based on a subjectively chosen range which excludes the impact of values outside the range on the overall sensitivity. PATIENTS AND PARTICIPANTS: The concepts of the refined method were illustrated using a Markov model of depression. MAIN OUTCOME MEASURES AND RESULTS: The application of the refined method substantially changed the ranking of the most sensitive variables compared with the original method. The response rate became the most sensitive variable instead of the 'per diem' for hospitalisation. CONCLUSIONS: The refinement of the original method yields sensitivity outcomes, which greater reflect the real uncertainty in economic studies.     

Principles of pharmacoeconomics and their impact on strategic imperatives of pharmaceutical research and development

The importance of evidence-based health policy is widely acknowledged among health care professionals, patients and politicians. Health care resources available for medical procedures, including pharmaceuticals, are limited all over the world. Economic evaluations help to alleviate the burden of scarce resources by improving the allocative efficiency of health care financing. Reimbursement of new medicines is subject to their cost-effectiveness and affordability in more and more countries.There are three major approaches to calculate the cost-effectiveness of new pharmaceuticals. Economic analyses alongside pivotal clinical trials are often inconclusive due to the suboptimal collection of economic data and protocol-driven costs. The major limitation of observational naturalistic economic evaluations is the selection bias and that they can be conducted only after registration and reimbursement.Economic modelling is routinely used to predict the cost-effectiveness of new pharmaceuticals for reimbursement purposes. Accuracy of cost-effectiveness estimates depends on the quality of input variables; validity of surrogate end points; and appropriateness of modelling assumptions, including model structure, time horizon and sophistication of the model to differentiate clinically and economically meaningful outcomes. These economic evaluation methods are not mutually exclusive; in practice, economic analyses often combine data collection alongside clinical trials or observational studies with modelling.The need for pharmacoeconomic evidence has fundamentally changed the strategic imperatives of research and development (R&D). Therefore, professionals in pharmaceutical R&D have to be familiar with the principles of pharmacoeconomics, including the selection of health policy-relevant comparators, analytical techniques, measurement of health gain by quality-adjusted life-years and strategic pricing of pharmaceuticals.     

Incorporation of uncertainty in health economic modelling studies

In a recent leading article in PharmacoEconomics, Nuijten described some methods for incorporating uncertainty into health economic models and for utilising the information on uncertainty regarding the cost effectiveness of a therapy in resource allocation decision-making. His proposals are found to suffer from serious flaws in statistical and health economic reasoning.Nuijten's suggestions for incorporating uncertainty: (a) wrongly interpret the p-value as the probability that the null hypothesis is true; (b) represent this probability wrongly by truncating the input distribution; and (c) in the specific example of an antiparkinsonian drug uses a completely inappropriate p-value of 0.05 when the null hypothesis would, in reality, be emphatically disproved by the data.His suggestions regarding minimum important differences in cost effectiveness: (a) introduce areas of indifference that suggest inappropriate reliance on cost minimisation while failing to recognise that decisions should be based on expected costs versus benefits; and (b) offer no guidance on how the probabilities associated with these areas could be used in decision-making. Furthermore, Nuijten's model for Parkinson's disease is over-simplified to the point of providing a bad example of modelling practice, which may mislead the readers of PharmacoEconomics.The rationale for this paper is to ensure that readers do not apply inappropriate analyses as a result of following the proposals contained in Nuijten's paper. In addition to a detailed critique of Nuijten's proposals, we provide brief summaries of the currently accepted best practice in cost-effectiveness decision-making under uncertainty.     

Handling uncertainty in cost-effectiveness models

The use of modelling in economic evaluation is widespread, and it most often involves synthesising data from a number of sources. However, even when economic evaluations are conducted alongside clinical trials, some form of modelling is usually essential. The aim of this article is to review the handling of uncertainty in the cost-effectiveness results that are generated by the use of decision-analytic-type modelling. The modelling process is split into a number of stages: (i) a set of methods to be employed in a study are defined, which should include a 'reference case' of agreed methods to enhance the comparability of results; (ii) the clinical and demographic characteristics of the patients the model relates to should be specified as carefully as in any experimental study; and (iii) the data requirements of the model should be estimated using the principles of Bayesian statistics, such that prior distributions are specified for unknown model parameters. Monte Carlo simulation can then be employed to sample from these prior distributions to obtain a distribution of the cost effectiveness of the intervention. Such probabilistic analyses are related to parameter uncertainty. In addition, modelling uncertainty is likely to add a further layer of uncertainty to the results of particular analyses.     

Advantages of using the net-benefit approach for analysing uncertainty in economic evaluation studies

No consensus has yet been reached on how to analyse uncertainty in economic evaluation studies where individual patient data are available for costs and health effects. This paper summarises the available results regarding the analysis of uncertainty on the cost-effectiveness plane and argues for using the net-benefit approach when analysing uncertainty in cost-effectiveness studies. The net-benefit approach avoids the interpretation and statistical problems related to the incremental cost effectiveness ratio and implies several advantages. First, traditional statistical methods can be used for confidence-interval estimation and hypothesis testing. Second, calculation of the optimal sample size and the power of the study are facilitated allowing the correlation between costs and effects to vary within and between patient groups. Third, the use of a Bayesian approach to cost-effectiveness analysis is facilitated. Fourth, a formal relation between cost-effectiveness acceptability curves and statistical inference is provided. Finally, the net-benefit approach gives the Fieller's limits of the confidence interval for the incremental cost-effectiveness ratio in the cost-effectiveness plane. Based on these advantages the net-benefit approach should strongly be considered when analysing uncertainty in cost-effectiveness analyses.     

Measuring sensitivity in pharmacoeconomic studies. An integration of point-sensitivity and range-sensitivity

The level of uncertainty with regard to the outcomes of pharmacoeconomic studies cannot be completely covered by the statistical methods routinely employed to handle uncertainty in clinical research. Sensitivity analysis is the most common methodology to deal with the extra uncertainty associated with pharmacoeconomics, and has also been incorporated in recent guidelines on healthcare evaluation. However, the execution of a sensitivity analysis and the interpretation of its results have not yet been standardised, which may lead to subjectivity and consequently weaken the value of economic evaluations. This article presents a method of dealing more systematically with uncertainty and eliminating potential bias in sensitivity analysis, with regard to the measurement of sensitivity and the comparison of the degree of sensitivity between variables. An assessment of the disadvantages of using slope as a measure of sensitivity leads to 2 types of sensitivity analyses (point-sensitivity and range-sensitivity), which are integrated into one method for the measurement of sensitivity.     

A critical review of model-based economic studies of depression: modelling techniques, model structure and data sources

Depression is the most common mental health disorder and is recognized as a chronic disease characterized by multiple acute episodes/relapses. Although modelling techniques play an increasingly important role in the economic evaluation of depression interventions, comparatively little attention has been paid to issues around modelling studies with a focus on potential biases. This, however, is important as different modelling approaches, variations in model structure and input parameters may produce different results, and hence different policy decisions. This paper presents a critical review of literature on recently published model-based cost-utility studies of depression. Taking depression as an illustrative example, through this review, we discuss a number of specific issues in relation to the use of decision-analytic models including the type of modelling techniques, structure of models and data sources. The potential benefits and limitations of each modelling technique are discussed and factors influencing the choice of modelling techniques are addressed. This review found that model-based studies of depression used various simulation techniques. We note that a discrete-event simulation may be the preferred technique for the economic evaluation of depression due to the greater flexibility with respect to handling time compared with other individual-based modelling techniques. Considering prognosis and management of depression, the structure of the reviewed models are discussed. We argue that a few reviewed models did not include some important structural aspects such as the possibility of relapse or the increased risk of suicide in patients with depression. Finally, the appropriateness of data sources used to estimate input parameters with a focus on transition probabilities is addressed. We argue that the above issues can potentially bias results and reduce the comparability of economic evaluations.     

Principles of pharmacoeconomic analysis of drug therapy

Economic analyses have become increasingly important in healthcare in general and with respect to pharmaceuticals in particular. If economic analyses are to play an important and useful role in the allocation of scarce healthcare resources, then such analyses must be performed properly and with care. This article outlines some of the basic principles of pharmacoeconomic analysis. Every analysis should have an explicitly stated perspective, which, unless otherwise justified, should be a societal perspective. Cost minimisation, cost-effectiveness, cost-utility and cost-benefit analyses are a family of techniques used in economic analyses. Cost minimisation analysis is appropriate when alternative therapies have identical outcomes, but differ in costs. Cost-effectiveness analysis is appropriate when alternative therapies differ in clinical effectiveness but can be examined from the same dimension of health outcome. Cost-utility analysis can be used when alternative therapies may be examined using multiple dimensions of health outcome, such as morbidity and mortality. Cost-benefit analysis requires the benefits of therapy to be described in monetary units and is not usually the technique of choice. The technique used in an analysis should be described and explicitly defended according to the problem being examined. For each technique, the method of determining costs is the same; direct, indirect, and intangible costs can be considered. The specific costs to be used depend on the analytical perspective; a societal perspective implies the use of both direct and indirect economic costs. A modelling framework such as a decision tree, influence diagram, Markov chain, or network simulation must be used to structure the analysis explicitly. Regardless of the choice of framework, all modelling assumptions should be described. The mechanism of data collection for model inputs must be detailed and defended. Models must undergo careful verification and validation procedures. Following baseline analysis of the model, further analyses should examine the role of uncertainty in model assumptions and data.     

Lessons from trial-based cost-effectiveness analyses of mental health interventions: why uncertainty about the outcome, estimate and willingness to pay matters

The principal aim of this article is to share lessons learned by the authors while conducting economic evaluations, using clinical trial data, of mental health interventions. These lessons are quite general and have clear relevance for pharmacoeconomic studies. In addition, we explore how net benefit regression can be used to enhance consideration of key issues when conducting an economic evaluation based on clinical trial data. The first study we discuss found that cost-effectiveness results varied markedly based on the choice of both the patient outcome and the willingness to pay for more of that outcome. The importance of willingness to pay was also highlighted in the results from the second study. Even with a set willingness-to-pay value, most of the time the probability that the new treatment was cost effective was not 100%. In the third study, the cost effectiveness of the new treatment varied by patient characteristics. These observations have important implications for pharmacoeconomic studies. Namely, analysts must carefully consider choice of patient outcome, willingness to pay, patient heterogeneity and the statistical uncertainty inherent in the data. Net benefit regression is a useful technique for exploring these crucial issues when undertaking an economic evaluation using patient-level data on both costs and effects.     

Using economic evaluations to reduce the burden of asthma and chronic obstructive pulmonary disease

There is increasing interest in the use of economic evaluations in healthcare, because of the need to maximise health benefits from limited resources. The focus of most economic evaluations is on efficiency, though they may also consider the issue of equity. In an economic evaluation, it is important to consider all the relevant costs, not just the acquisition cost of the treatment. Likewise, it is important to include all the benefits in the economic appraisal, although the outcomes of relevance to decision-makers may differ according to their perspective. If an intervention costs less but delivers fewer benefits than the comparator or, more commonly, a new intervention increases benefits compared with standard therapy but at increased cost, decision-makers must consider whether the extra cost is worth the extra benefit. This depends on the opportunity cost of introducing the new intervention - i.e. the benefit forgone by doing less of something else to fund it. In other words, decision-makers need to decide on the maximum amount they are willing to pay for an additional unit of health benefit. The result of an economic evaluation will be strongly influenced by the information used in the analysis. Currently, clinical trials are the most common source of data for economic evaluations. Yet there are a number of limitations in the information generated by clinical trials, which are primarily designed for regulatory approval. Consequently, decision analytical models are being increasingly used to synthesise data from various sources and to manage uncertainty in input parameters. When using economic evaluations, decision-makers may be unwilling to take a broad perspective on costs, focusing instead on their narrow budgetary concerns. Incentives may be required within healthcare systems to ensure that decision-makers adhere more strictly to the results of formal analysis.     

Methodological issues in evaluating cost effectiveness of adjuvant aromatase inhibitors in early breast cancer: a need for improved modelling to aid decision making

The optimal adjuvant hormonal strategy in post-menopausal women with early breast cancer is a subject of ongoing debate. Aromatase inhibitors (AIs) have been successfully evaluated in clinical trials that have compared them with a standard treatment of 5 years of tamoxifen. However, several options are available in terms of treatment schedule and selected drug. Systematic reviews of clinical trials and health economic evaluations attempt to contribute to the debate. The objective of this paper is to provide a critical review of existing health economic evaluations with a focus on those parameters and assumptions with the largest impact on final outcomes.A wide range of different inputs and assumptions exist, which make a comparison of results difficult, if not impossible. In particular, the modelling of recurrence rates over longer time horizons than those observed in clinical trials, a cornerstone of health economic modelling, is subject to quite different approaches. The practice of indirect comparison of different AIs without sufficiently acknowledging population differences is also bothersome. A list of key features (related to time horizon, clinical data input, patient subtypes, budget impact and model calibration) that an ideal model should have in order to better assist decision makers in this field is proposed.     

Interspecies extrapolation in environmental exposure standard setting: A Bayesian synthesis approach

Currently the extrapolation of evidence from studies of non-human species to the setting of environmental exposure standards for humans includes the imposition of a variety of uncertainty factors reflecting unknown aspects of the procedure, including the relevance of evidence from one species to impacts in another. This paper develops and explores more flexible modelling of aspects of this extrapolation, using models proposed by DuMouchel [DuMouchel, W.H., Harris, J.E., 1983. Bayes methods for combining the results of cancer studies in humans and other species (with comment). J. Am. Statist. Assoc. 78, 293–308.] The approaches are based on Bayesian meta-analysis methods involving explicit modelling of relevance in the prior distributions, estimated using Markov chain Monte Carlo (MCMC) methods. The methods are applied to evidence relating chlorinated by-products exposure to adverse reproductive health effects. The relative merits of various approaches are discussed, and developments and next steps are outlined.     

Spatial regression models for large-cohort studies linking community air pollution and health

Cohort study designs are often used to assess the association between community-based ambient air pollution concentrations and health outcomes, such as mortality, development and prevalence of disease, and pulmonary function. Typically, a large number of subjects are enrolled in the study in each of a small number of communities. Fixed-site monitors are used to determine long-term exposure to ambient pollution. The association between community average pollution levels and health is determined after controlling for risk factors of the health outcome measured at the individual level (i.e., smoking). We present a new spatial regression model linking spatial variation in ambient air pollution to health. Health outcomes can be measured as continuous variables (pulmonary function), binary variables (prevalence of disease), or time-to-event data (survival or development of disease). The model incorporates risk factors measured at the individual level, such as smoking, and at the community level, such as air pollution. We demonstrate that the spatial autocorrelation in community health outcomes, an indication of not fully characterizing potentially confounding risk factors to the air pollution--health association, can be accounted for through the inclusion of location in the deterministic component of the model assessing the effects of air pollution on health or through a distance-decay spatial autocorrelation function in the stochastic component of the model, or both. We present a statistical approach that can be implemented for very large cohort studies. Our methods are illustrated with an analysis of the American Cancer Society cohort to determine whether the prevalence of heart disease is associated with concentrations of sulfate particles. From a statistical point of view, it appears that a location surface in the deterministic component of the model was preferred to a distance-decay autocorrelation structure in the model's stochastic component.     

Modelling and simulation of variability and uncertainty in toxicokinetics and pharmacokinetics

Two important methodological issues within the framework of the variability and uncertainty analysis of toxicokinetic and pharmacokinetic systems are discussed: (i) modelling and simulation of the existing physiologic variability in a population; and (ii) modelling and simulation of variability and uncertainty when there is insufficient or not well defined (e.g. small sample, semiquantitative, qualitative and vague) information available. Physiologically based pharmacokinetic models are especially suited for separating and characterising the physiologic variability from the overall variability and uncertainty in the system. Monte Carlo sampling should draw from multivariate distributions, which reflect all levels of existing dependencies in the intact organism. The population characteristics should be taken into account. A fuzzy simulation approach is proposed to model variability and uncertainty when there is semiquantitative, qualitative and vague information about the model parameters and their statistical distributions cannot be defined reliably.     

An introduction to the methods of cost-effectiveness analysis

The comparative cost-effectiveness of interventions is a fundamental consideration of health technology assessment (HTA) in the UK.(1) The use of modelling to extrapolate benefits to patients and costs over a specified time period is a common technique in cost-effectiveness analyses. All modelling techniques, by their nature, are subject to different levels of uncertainty. Assessment and understanding of the level and impact of this uncertainty is a fundamental part of the decision-making process. In this article, we build on our previous article A guide to health economic evaluations and discuss different modelling approaches to cost-effectiveness analysis and the importance of uncertainty.(2)     

Modelling disease progression in Alzheimer's disease: a review of modelling methods used for cost-effectiveness analysis

The literature reporting economic evaluations related to the treatment of Alzheimer's disease (AD) has developed over the last decade. Most analyses have used economic models to estimate the cost effectiveness of drugs for the treatment of AD. This review considers the range of methods used in the published cost-effectiveness literature to model AD progression and the effect of interventions on the progression of AD. The review builds on and updates an earlier systematic review of cost-effectiveness studies on drugs for AD. Systematic and rigorous methods were used to search the literature for economic evaluations estimating the cost effectiveness of donepezil, rivastigmine, galantamine or memantine in AD. The literature search covered a wide range of electronic databases (e.g. MEDLINE, EMBASE), and included literature from the inception of databases up to the end of 2005. The search identified 22 published economic evaluations. An outline and brief critical review of the identified studies is provided, and thereafter the methods used to model disease progression were considered in more detail. The review employs recent guidance on good practice in decision-analytic modelling in HTA to critically review the modelling methods used. Using this guidance, the models are assessed against the broad criteria of model structure, data inputs and assessment of uncertainty and inconsistency. Concerns were noted over the model structure employed in all models. The reliance on cognitive scores to model AD, the progression of the disease, and the effect of treatment on costs and consequences is regarded as a serious limitation in almost all of the studies identified. There are also limitations over the data used to populate published models, especially around the failure of studies to document and establish the basis for the modelling of treatment effects. It is also clear that studies modelling AD progression, and subsequently the cost effectiveness of treatment, have not addressed uncertainty or consistency (internal and/or external) in sufficient detail. Further research is required on more appropriate methods for the modelling of AD progression. In the meantime, future economic evaluations of treatment need to be more explicit on the methods used to model AD, and the data used to populate models.     

Assessing drug distribution in tissues expressing P-glycoprotein using physiologically based pharmacokinetic modeling: identification of important model parameters through global sensitivity analysis

The structural complexity of a PBPK model is usually accompanied with significant uncertainty in estimating its input parameters. In the last decade, the global sensitivity analysis, which accounts for the variability of all model input parameters simultaneously as well as their correlations, has gained a wide attention as a powerful probing technique to identify and control biological model uncertainties. However, the current sensitivity analysis techniques used in PBPK modeling often neglect the correlation between these input parameters. We introduce a new strategy in the PBPK modeling field to investigate how the uncertainty and variability of correlated input parameters influence the outcomes of the drug distribution process based on a model we recently developed to explain and predict drug distribution in tissues expressing P-glycoprotein (P-gp). As direct results, we will also identify the most important input parameters having the largest contribution to the variability and uncertainty of model outcomes. We combined multivariate random sampling with a ranking procedure. Monte–Carlo simulations were performed on the PBPK model with eighteen model input parameters. Log-normal distributions were assumed for these parameters according to literature and their reported correlations were also included. A multivariate sensitivity analysis was then performed to identify the input parameters with the greatest influence on model predictions. The partial rank correlation coefficients (PRCC) were calculated to establish the input–output relationships. A moderate variability of predicted C_last and C_max was observed in liver, heart and brain tissues in the presence or absence of P-gp activity. The major statistical difference in model outcomes of the predicted median values has been obtained in brain tissue. PRCC calculation confirmed the importance for a better quantitative characterisation of input parameters related to the passive diffusion and active transport of the unbound drug through the blood-tissue membrane in heart and brain. This approach has also identified as important input parameters those related to the drug metabolism for the prediction of model outcomes in liver and plasma. The proposed Monte–Carlo/PRCC approach was aimed to address the effect of input parameters correlation in a PBPK model. It allowed the identification of important input parameters that require additional attention in research for strengthening the physiological knowledge of drug distribution in mammalian tissues expressing P-gp, thereby reducing the uncertainty of model predictions.