Evaluation of the benchmark dose for point of departure determination for a variety of chemical classes in applied regulatory settings

Repeated-dose studies received by the New Substances Assessment and Control Bureau (NSACB) of Health Canada are used to provide hazard information toward risk calculation. These studies provide a point of departure (POD), traditionally the NOAEL or LOAEL, which is used to extrapolate the quantity of substance above which adverse effects can be expected in humans. This project explored the use of benchmark dose (BMD) modeling as an alternative to this approach for studies with few dose groups. Continuous data from oral repeated-dose studies for chemicals previously assessed by NSACB were reanalyzed using U.S. EPA benchmark dose software (BMDS) to determine the BMD and BMD 95% lower confidence limit (BMDL(05) ) for each endpoint critical to NOAEL or LOAEL determination for each chemical. Endpoint-specific benchmark dose-response levels , indicative of adversity, were consistently applied. An overall BMD and BMDL(05) were calculated for each chemical using the geometric mean. The POD obtained from benchmark analysis was then compared with the traditional toxicity thresholds originally used for risk assessment. The BMD and BMDL(05) generally were higher than the NOAEL, but lower than the LOAEL. BMDL(05) was generally constant at 57% of the BMD. Benchmark provided a clear advantage in health risk assessment when a LOAEL was the only POD identified, or when dose groups were widely distributed. Although the benchmark method cannot always be applied, in the selected studies with few dose groups it provided a more accurate estimate of the real no-adverse-effect level of a substance.     

Model Uncertainty and Bayesian Model Averaged Benchmark Dose Estimation for Continuous Data

The benchmark dose (BMD) approach has gained acceptance as a valuable risk assessment tool, but risk assessors still face significant challenges associated with selecting an appropriate BMD/BMDL estimate from the results of a set of acceptable dose‐response models. Current approaches do not explicitly address model uncertainty, and there is an existing need to more fully inform health risk assessors in this regard. In this study, a Bayesian model averaging (BMA) BMD estimation method taking model uncertainty into account is proposed as an alternative to current BMD estimation approaches for continuous data. Using the “hybrid” method proposed by Crump, two strategies of BMA, including both “maximum likelihood estimation based” and “Markov Chain Monte Carlo based” methods, are first applied as a demonstration to calculate model averaged BMD estimates from real continuous dose‐response data. The outcomes from the example data sets examined suggest that the BMA BMD estimates have higher reliability than the estimates from the individual models with highest posterior weight in terms of higher BMDL and smaller 90th percentile intervals. In addition, a simulation study is performed to evaluate the accuracy of the BMA BMD estimator. The results from the simulation study recommend that the BMA BMD estimates have smaller bias than the BMDs selected using other criteria. To further validate the BMA method, some technical issues, including the selection of models and the use of bootstrap methods for BMDL derivation, need further investigation over a more extensive, representative set of dose‐response data.     

A Comparison of Three Methods for Calculating Confidence Intervals for the Benchmark Dose

Various methods exist to calculate confidence intervals for the benchmark dose in risk analysis. This study compares the performance of three such methods in fitting nonlinear dose-response models: the delta method, the likelihood-ratio method, and the bootstrap method. A data set from a developmental toxicity test with continuous, ordinal, and quantal dose-response data is used for the comparison of these methods. Nonlinear dose-response models, with various shapes, were fitted to these data. The results indicate that a few thousand runs are generally needed to get stable confidence limits when using the bootstrap method. Further, the bootstrap and the likelihood-ratio method were found to give fairly similar results. The delta method, however, resulted in some cases in different (usually narrower) intervals, and appears unreliable for nonlinear dose-response models. Since the bootstrap method is more time consuming than the likelihood-ratio method, the latter is more attractive for routine dose-response analysis. In the context of a probabilistic risk assessment the bootstrap method has the advantage that it directly links to Monte Carlo analysis.     

Comparing Experimental Designs for Benchmark Dose Calculations for Continuous Endpoints

The BMD (benchmark dose) method that is used in risk assessment of chemical compounds was introduced by Crump (1984) and is based on dose-response modeling. To take uncertainty in the data and model fitting into account, the lower confidence bound of the BMD estimate (BMDL) is suggested to be used as a point of departure in health risk assessments. In this article, we study how to design optimum experiments for applying the BMD method for continuous data. We exemplify our approach by considering the class of Hill models. The main aim is to study whether an increased number of dose groups and at the same time a decreased number of animals in each dose group improves conditions for estimating the benchmark dose. Since Hill models are nonlinear, the optimum design depends on the values of the unknown parameters. That is why we consider Bayesian designs and assume that the parameter vector has a prior distribution. A natural design criterion is to minimize the expected variance of the BMD estimator. We present an example where we calculate the value of the design criterion for several designs and try to find out how the number of dose groups, the number of animals in the dose groups, and the choice of doses affects this value for different Hill curves. It follows from our calculations that to avoid the risk of unfavorable dose placements, it is good to use designs with more than four dose groups. We can also conclude that any additional information about the expected dose-response curve, e.g., information obtained from studies made in the past, should be taken into account when planning a study because it can improve the design.     

Potential Uncertainty Reduction in Model‐Averaged Benchmark Dose Estimates Informed by an Additional Dose Study

A methodology is presented for assessing the information value of an additional dosage experiment in existing bioassay studies. The analysis demonstrates the potential reduction in the uncertainty of toxicity metrics derived from expanded studies, providing insights for future studies. Bayesian methods are used to fit alternative dose‐response models using Markov chain Monte Carlo (MCMC) simulation for parameter estimation and Bayesian model averaging (BMA) is used to compare and combine the alternative models. BMA predictions for benchmark dose (BMD) are developed, with uncertainty in these predictions used to derive the lower bound BMDL. The MCMC and BMA results provide a basis for a subsequent Monte Carlo analysis that backcasts the dosage where an additional test group would have been most beneficial in reducing the uncertainty in the BMD prediction, along with the magnitude of the expected uncertainty reduction. Uncertainty reductions are measured in terms of reduced interval widths of predicted BMD values and increases in BMDL values that occur as a result of this reduced uncertainty. The methodology is illustrated using two existing data sets for TCDD carcinogenicity, fitted with two alternative dose‐response models (logistic and quantal‐linear). The example shows that an additional dose at a relatively high value would have been most effective for reducing the uncertainty in BMA BMD estimates, with predicted reductions in the widths of uncertainty intervals of approximately 30%, and expected increases in BMDL values of 5–10%. The results demonstrate that dose selection for studies that subsequently inform dose‐response models can benefit from consideration of how these models will be fit, combined, and interpreted.     

Issues in Qualitative and Quantitative Risk Analysis for Developmental Toxicology1

The qualitative and quantitative evaluation of risk in developmental toxicology has been discussed in several recent publications.(^1–3) A number of issues still are to be resolved in this area. The qualitative evaluation and interpretation of end points in developmental toxicology depends on an understanding of the biological events leading to the end points observed, the relationships among end points, and their relationship to dose and to maternal toxicity. The interpretation of these end points is also affected by the statistical power of the experiments used for detecting the various end points observed. The quantitative risk assessment attempts to estimate human risk for developmental toxicity as a function of dose. The current approach is to apply safety (uncertainty) factors to die no observed effect level (NOEL). An alternative presented and discussed here is to model the experimental data and apply a safety factor to an estimated risk level to achieve an “acceptable” level of risk. In cases where the dose-response curves upward, this approach provides a conservative estimate of risk. This procedure does not preclude the existence of a threshold dose. More research is needed to develop appropriate dose-response models that can provide better estimates for low-dose extrapolation of developmental effects.     

A Review of Recent Advances in Benchmark Dose Methodology

In this review, recent methodological developments for the benchmark dose (BMD) methodology are summarized. Specifically, we introduce the advances for the main steps in BMD derivation: selecting the procedure for defining a BMD from a predefined benchmark response (BMR), setting a BMR, selecting a dose–response model, and estimating the corresponding BMD lower limit (BMDL). Although the last decade has shown major progress in the development of BMD methodology, there is still room for improvement. Remaining challenges are the implementation of new statistical methods in user‐friendly software and the lack of consensus about how to derive the BMDL.     

Relation Between Benchmark Dose and No-Observed-Adverse-Effect Level in Clinical Research: Effects of Daily Alcohol Intake on Blood Pressure in Japanese Salesmen

The benchmark dose (BMD) is defined as the dose that corresponds to a specific change in an adverse response compared to the response in unexposed subjects, and the lower 95% confidence limit is termed the benchmark dose level (BMDL). In this study, the threshold of daily ethanol intake affecting blood pressure was calculated by both the BMD approach and multiple logistic regression analysis to clarify the relation between the BMDL and no-observed-adverse-effect level (NOAEL). Systolic and diastolic blood pressures (SBP and DBP) and daily ethanol intake were explored in 1,100 Japanese salesmen. The SBP and DBP were positively related to daily ethanol intake (p < 0.001) when adjusting for possible confounders such as age, body mass index, and smoking status. The adjusted risk for hypertension (SBP >or= 140 mmHg or DBP >or= 90 mmHg) increased significantly when daily ethanol intake exceeded 60 g/day, and the categorical dose of interest was 60.1-90 g/day. The BMDL and BMD of ethanol intake for increased SBP and DBP were estimated to be approximately 60 and 75 g/day, respectively. These findings suggest that the BMDL and BMD correspond to the NOAEL and lowest-observed-adverse-effect level, respectively, if the sample number of clinical data is large enough to confirm the dose-response association.     

Properties of Model-Averaged BMDLs: A Study of Model Averaging in Dichotomous Response Risk Estimation

Model averaging (MA) has been proposed as a method of accounting for model uncertainty in benchmark dose (BMD) estimation. The technique has been used to average BMD dose estimates derived from dichotomous dose-response experiments, microbial dose-response experiments, as well as observational epidemiological studies. While MA is a promising tool for the risk assessor, a previous study suggested that the simple strategy of averaging individual models' BMD lower limits did not yield interval estimators that met nominal coverage levels in certain situations, and this performance was very sensitive to the underlying model space chosen. We present a different, more computationally intensive, approach in which the BMD is estimated using the average dose-response model and the corresponding benchmark dose lower bound (BMDL) is computed by bootstrapping. This method is illustrated with TiO(2) dose-response rat lung cancer data, and then systematically studied through an extensive Monte Carlo simulation. The results of this study suggest that the MA-BMD, estimated using this technique, performs better, in terms of bias and coverage, than the previous MA methodology. Further, the MA-BMDL achieves nominal coverage in most cases, and is superior to picking the "best fitting model" when estimating the benchmark dose. Although these results show utility of MA for benchmark dose risk estimation, they continue to highlight the importance of choosing an adequate model space as well as proper model fit diagnostics.     

Benchmark Dose Analysis via Nonparametric Regression Modeling

Estimation of benchmark doses (BMDs) in quantitative risk assessment traditionally is based upon parametric dose‐response modeling. It is a well‐known concern, however, that if the chosen parametric model is uncertain and/or misspecified, inaccurate and possibly unsafe low‐dose inferences can result. We describe a nonparametric approach for estimating BMDs with quantal‐response data based on an isotonic regression method, and also study use of corresponding, nonparametric, bootstrap‐based confidence limits for the BMD. We explore the confidence limits’ small‐sample properties via a simulation study, and illustrate the calculations with an example from cancer risk assessment. It is seen that this nonparametric approach can provide a useful alternative for BMD estimation when faced with the problem of parametric model uncertainty.     

Calculation of Benchmark Doses from Continuous Data

A benchmark dose (BMD) is the dose of a substance that corresponds to a prescribed increase in the response (called the benchmark response or BMR) of a health effect. A statistical lower bound on the benchmark dose (BMDL) has been proposed as a replacement for the no-observed-adverse-effect-level (NOAEL) in setting acceptable human exposure levels. A method is developed in this paper for calculating BMDs and BMDLs from continuous data in a manner that is consistent with those calculated from quantal data. The method involves defining an abnormal response, either directly by specifying a cutoff x₀ that separates continuous responses into normal and abnormal categories, or indirectly by specifying the proportion P₀ of abnormal responses expected among unexposed subjects. The method does not involve actually dichotomizing individual continuous responses into quantal responses, and in certain cases can be applied to continuous data in summarized form (e.g., means and standard deviations of continuous responses among subjects in discrete dose groups). In addition to specifying the BMR and either x₀ or P₀ , the method requires specification of the distribution of continuous responses, including specification of the dose-response θ(d) for a measure of central tendency. A method is illustrated for selecting θ(d) to make the probability of an abnormal response any desired dose-response function. This enables the same dose-response model (Weibull, log-logistic, etc.) to be used for the probability of an abnormal response, regardless of whether the underlying data are continuous or quantal. Whenever the continuous responses are normally distributed with standard deviation σ (independent of dose), the method is equivalent to defining the BMD as the dose corresponding to a prescribed change in the mean response relative to σ.     

Model Uncertainty and Model Averaging in the Estimation of Infectious Doses for Microbial Pathogens

Food‐borne infection is caused by intake of foods or beverages contaminated with microbial pathogens. Dose‐response modeling is used to estimate exposure levels of pathogens associated with specific risks of infection or illness. When a single dose‐response model is used and confidence limits on infectious doses are calculated, only data uncertainty is captured. We propose a method to estimate the lower confidence limit on an infectious dose by including model uncertainty and separating it from data uncertainty. The infectious dose is estimated by a weighted average of effective dose estimates from a set of dose‐response models via a Kullback information criterion. The confidence interval for the infectious dose is constructed by the delta method, where data uncertainty is addressed by a bootstrap method. To evaluate the actual coverage probabilities of the lower confidence limit, a Monte Carlo simulation study is conducted under sublinear, linear, and superlinear dose‐response shapes that can be commonly found in real data sets. Our model‐averaging method achieves coverage close to nominal in almost all cases, thus providing a useful and efficient tool for accurate calculation of lower confidence limits on infectious doses.     

Benchmark Dose Calculation for Ordered Categorical Responses

The use of benchmark dose (BMD) calculations for dichotomous or continuous responses is well established in the risk assessment of cancer and noncancer endpoints. In some cases, responses to exposure are categorized in terms of ordinal severity effects such as none, mild, adverse, and severe. Such responses can be assessed using categorical regression (CATREG) analysis. However, while CATREG has been employed to compare the benchmark approach and the no‐adverse‐effect‐level (NOAEL) approach in determining a reference dose, the utility of CATREG for risk assessment remains unclear. This study proposes a CATREG model to extend the BMD approach to ordered categorical responses by modeling severity levels as censored interval limits of a standard normal distribution. The BMD is calculated as a weighted average of the BMDs obtained at dichotomous cutoffs for each adverse severity level above the critical effect, with the weights being proportional to the reciprocal of the expected loss at the cutoff under the normal probability model. This approach provides a link between the current BMD procedures for dichotomous and continuous data. We estimate the CATREG parameters using a Markov chain Monte Carlo simulation procedure. The proposed method is demonstrated using examples of aldicarb and urethane, each with several categories of severity levels. Simulation studies comparing the BMD and BMDL (lower confidence bound on the BMD) using the proposed method to the correspondent estimates using the existing methods for dichotomous and continuous data are quite compatible; the difference is mainly dependent on the choice of cutoffs for the severity levels.     

Characterizing Dose-Response I: Critical Assessment of the Benchmark Dose Concept

We present a critical assessment of the benchmark dose (BMD) method introduced by Crump⁽¹⁾ as an alternative method for setting a characteristic dose level for toxicant risk assessment. The no-observed-adverse-effect-level (NOAEL) method has been criticized because it does not use all of the data and because the characteristic dose level obtained depends on the dose levels and the statistical precision (sample sizes) of the study design. Defining the BMD in terms of a confidence bound on a point estimate results in a characteristic dose that also varies with the statistical precision and still depends on the study dose levels.⁽²⁾ Indiscriminate choice of benchmark response level may result in a BMD that reflects little about the dose-response behavior available from using all of the data. Another concern is that the definition of the BMD for the quantal response case is different for the continuous response case. Specifically, defining the BMD for continuous data using a ratio of increased effect divided by the background response results in an arbitrary dependence on the natural background for the endpoint being studied, making comparison among endpoints less meaningful and standards more arbitrary. We define a modified benchmark dose as a point estimate using the ratio of increased effect divided by the full adverse response range which enables consistent placement of the benchmark response level and provides a BMD with a more consistent relationship to the dose-response curve shape.     

Parameters of a Dose‐Response Model Are on the Boundary: What Happens with BMDL?

It is well known that, under appropriate regularity conditions, the asymptotic distribution for the likelihood ratio statistic is χ². This result is used in EPA's benchmark dose software to obtain a lower confidence bound (BMDL) for the benchmark dose (BMD) by the profile likelihood method. Recently, based on work by Self and Liang, it has been demonstrated that the asymptotic distribution of the likelihood ratio remains the same if some of the regularity conditions are violated, that is, when true values of some nuisance parameters are on the boundary. That is often the situation for BMD analysis of cancer bioassay data. In this article, we study by simulation the coverage of one- and two-sided confidence intervals for BMD when some of the model parameters have true values on the boundary of a parameter space. Fortunately, because two-sided confidence intervals (size 1–2α) have coverage close to the nominal level when there are 50 animals in each group, the coverage of nominal 1−α one-sided intervals is bounded between roughly 1–2α and 1. In many of the simulation scenarios with a nominal one-sided confidence level of 95%, that is, α= 0.05, coverage of the BMDL was close to 1, but for some scenarios coverage was close to 90%, both for a group size of 50 animals and asymptotically (group size 100,000). Another important observation is that when the true parameter is below the boundary, as with the shape parameter of a log-logistic model, the coverage of BMDL in a constrained model (a case of model misspecification not uncommon in BMDS analyses) may be very small and even approach 0 asymptotically. We also discuss that whenever profile likelihood is used for one-sided tests, the Self and Liang methodology is needed to derive the correct asymptotic distribution.     

Correlation of Noncancer Benchmark Doses in Short‐ and Long‐Term Rodent Bioassays

This study investigated whether, in the absence of chronic noncancer toxicity data, short‐term noncancer toxicity data can be used to predict chronic toxicity effect levels by focusing on the dose–response relationship instead of a critical effect. Data from National Toxicology Program (NTP) technical reports have been extracted and modeled using the Environmental Protection Agency's Benchmark Dose Software. Best‐fit, minimum benchmark dose (BMD), and benchmark dose lower limits (BMDLs) have been modeled for all NTP pathologist identified significant nonneoplastic lesions, final mean body weight, and mean organ weight of 41 chemicals tested by NTP between 2000 and 2012. Models were then developed at the chemical level using orthogonal regression techniques to predict chronic (two years) noncancer health effect levels using the results of the short‐term (three months) toxicity data. The findings indicate that short‐term animal studies may reasonably provide a quantitative estimate of a chronic BMD or BMDL. This can allow for faster development of human health toxicity values for risk assessment for chemicals that lack chronic toxicity data.     

Benchmark Dose Risk Assessment for Formaldehyde Using Airflow Modeling and a Single-Compartment, DNA-Protein Cross-Link Dosimetry Model to Estimate Human Equivalent Doses

Formaldehyde induced squamous-cell carcinomas in the nasal passages of F344 rats in two inhalation bioassays at exposure levels of 6 ppm and above. Increases in rates of cell proliferation were measured by T. M. Monticello and colleagues at exposure levels of 0.7 ppm and above in the same tissues from which tumors arose. A risk assessment for formaldehyde was conducted at the CIIT Centers for Health Research, in collaboration with investigators from Toxicological Excellence in Risk Assessment (TERA) and the U.S. Environmental Protection Agency (U.S. EPA) in 1999. Two methods for dose-response assessment were used: a full biologically based modeling approach and a statistically oriented analysis by benchmark dose (BMD) method. This article presents the later approach, the purpose of which is to combine BMD and pharmacokinetic modeling to estimate human cancer risks from formaldehyde exposure. BMD analysis was used to identify points of departure (exposure levels) for low-dose extrapolation in rats for both tumor and the cell proliferation endpoints. The benchmark concentrations for induced cell proliferation were lower than for tumors. These concentrations were extrapolated to humans using two mechanistic models. One model used computational fluid dynamics (CFD) alone to determine rates of delivery of inhaled formaldehyde to the nasal lining. The second model combined the CFD method with a pharmacokinetic model to predict tissue dose with formaldehyde-induced DNA-protein cross-links (DPX) as a dose metric. Both extrapolation methods gave similar results, and the predicted cancer risk in humans at low exposure levels was found to be similar to that from a risk assessment conducted by the U.S. EPA in 1991. Use of the mechanistically based extrapolation models lends greater certainty to these risk estimates than previous approaches and also identifies the uncertainty in the measured dose-response relationship for cell proliferation at low exposure levels, the dose-response relationship for DPX in monkeys, and the choice between linear and nonlinear methods of extrapolation as key remaining sources of uncertainty.     

Modeling Fetal Death and Malformation in Developmental Toxicity Studies

We review approaches to dose-response modeling and risk assessment for binary data from developmental toxicity studies. In particular, we focus on jointly modeling fetal death and malformation and use a continuation ratio formulation of the multinomial distribution to provide a model for risk. Generalized estimating equations are used to account for clustering of animals within litters. The fitted model is then used to calculate doses corresponding to a specified level of excess risk. Two methods of arriving at a lower confidence limit or Benchmark dose are illustrated and compared. We also discuss models based on single binary end points and compare our approach to a binary analysis of whether or not the animal was 'affected' (either dead or malformed). The models are illustrated using data from four developmental toxicity studies in EG, DEHP, TGDM, and DYME conducted through the National Toxicology Program.     

Estimation of Benchmark Dose of Lifetime Cadmium Intake for Adverse Renal Effects Using Hybrid Approach in Inhabitants of an Environmentally Exposed River Basin in Japan

The aim of this study is to estimate the reference level of lifetime cadmium intake (LCd) as the benchmark doses (BMDs) and their 95% lower confidence limits (BMDLs) for various renal effects by applying a hybrid approach. The participants comprised 3,013 (1,362 men and 1,651 women) and 278 (129 men and 149 women) inhabitants of the Cd‐polluted and nonpolluted areas, respectively, in the environmentally exposed Kakehashi River basin. Glucose, protein, aminonitrogen, metallothionein, and β₂‐microglobulin in urine were measured as indicators of renal dysfunction. The BMD and BMDL that corresponded to an additional risk of 5% were calculated with background risk at zero exposure set at 5%. The obtained BMDLs of LCd were 3.7 g (glucose), 3.2 g (protein), 3.7 g (aminonitrogen), 1.7 g (metallothionein), and 1.8 g (β₂‐microglobulin) in men and 2.9 g (glucose), 2.5 g (protein), 2.0 g (aminonitrogen), 1.6 g (metallothionein), and 1.3 g (β₂‐microglobulin) in women. The lowest BMDL was 1.7 g (metallothionein) and 1.3 g (β₂‐microglobulin) in men and women, respectively. The lowest BMDL of LCd (1.3 g) was somewhat lower than the representative threshold LCd (2.0 g) calculated in the previous studies. The obtained BMDLs may contribute to further discussion on the health risk assessment of cadmium exposure.     

Estimating Upper Confidence Limits for Extra Risk in Quantal Multistage Models

Multistage models are frequently applied in carcinogenic risk assessment. In their simplest form, these models relate the probability of tumor presence to some measure of dose. These models are then used to project the excess risk of tumor occurrence at doses frequently well below the lowest experimental dose. Upper confidence limits on the excess risk associated with exposures at these doses are then determined. A likelihood-based method is commonly used to determine these limits. We compare this method to two computationally intensive "bootstrap" methods for determining the 95% upper confidence limit on extra risk. The coverage probabilities and bias of likelihood-based and bootstrap estimates are examined in a simulation study of carcinogenicity experiments. The coverage probabilities of the nonparametric bootstrap method fell below 95% more frequently and by wider margins than the better-performing parametric bootstrap and likelihood-based methods. The relative bias of all estimators are seen to be affected by the amount of curvature in the true underlying dose-response function. In general, the likelihood-based method has the best coverage probability properties while the parametric bootstrap is less biased and less variable than the likelihood-based method. Ultimately, neither method is entirely satisfactory for highly curved dose-response patterns.