Development of a Quantitative Microbial Risk Assessment for Human Salmonellosis Through Household Consumption of Fresh Minced Pork Meat in Belgium

A quantitative microbial risk assessment (QMRA) according to the Codex Alimentarius Principles is conducted to evaluate the risk of human salmonellosis through household consumption of fresh minced pork meat in Belgium. The quantitative exposure assessment is carried out by building a modular risk model, called the METZOON-model, which covers the pork production from farm to fork. In the METZOON-model, the food production pathway is split up in six consecutive modules: (1) primary production, (2) transport and lairage, (3) slaughterhouse, (4) postprocessing, (5) distribution and storage, and (6) preparation and consumption. All the modules are developed to resemble as closely as possible the Belgian situation, making use of the available national data. Several statistical refinements and improved modeling techniques are proposed. The model produces highly realistic results. The baseline predicted number of annual salmonellosis cases is 20,513 ( SD 9061.45). The risk is estimated higher for the susceptible population (estimate  4.713 × 10⁻⁵; SD 1.466 × 10⁻⁵  ) compared to the normal population  (estimate 7.704 × 10⁻⁶; SD 5.414 × 10⁻⁶)  and is mainly due to undercooking and to a smaller extent to cross-contamination in the kitchen via cook's hands.     

Validation of Quantitative Microbial Risk Assessment Using Epidemiological Data from Outbreaks of Waterborne Gastrointestinal Disease

The assumptions underlying quantitative microbial risk assessment (QMRA) are simple and biologically plausible, but QMRA predictions have never been validated for many pathogens. The objective of this study was to validate QMRA predictions against epidemiological measurements from outbreaks of waterborne gastrointestinal disease. I screened 2,000 papers and identified 12 outbreaks with the necessary data: disease rates measured using epidemiological methods and pathogen concentrations measured in the source water. Eight of the 12 outbreaks were caused by Cryptosporidium, three by Giardia, and one by norovirus. Disease rates varied from 5.5 × 10^?6 to 1.1 × 10^?2 cases/person‐day, and reported pathogen concentrations varied from 1.2 × 10^?4 to 8.6 × 10² per liter. I used these concentrations with single‐hit dose–response models for all three pathogens to conduct QMRA, producing both point and interval predictions of disease rates for each outbreak. Comparison of QMRA predictions to epidemiological measurements showed good agreement; interval predictions contained measured disease rates for 9 of 12 outbreaks, with point predictions off by factors of 1.0–120 (median = 4.8). Furthermore, 11 outbreaks occurred at mean doses of less than 1 pathogen per exposure. Measured disease rates for these outbreaks were clearly consistent with a single‐hit model, and not with a “two‐hit” threshold model. These results demonstrate the validity of QMRA for predicting disease rates due to Cryptosporidium and Giardia.     

Modeling Logistic Performance in Quantitative Microbial Risk Assessment

In quantitative microbial risk assessment (QMRA), food safety in the food chain is modeled and simulated. In general, prevalences, concentrations, and numbers of microorganisms in media are investigated in the different steps from farm to fork. The underlying rates and conditions (such as storage times, temperatures, gas conditions, and their distributions) are determined. However, the logistic chain with its queues (storages, shelves) and mechanisms for ordering products is usually not taken into account. As a consequence, storage times—mutually dependent in successive steps in the chain—cannot be described adequately. This may have a great impact on the tails of risk distributions. Because food safety risks are generally very small, it is crucial to model the tails of (underlying) distributions as accurately as possible. Logistic performance can be modeled by describing the underlying planning and scheduling mechanisms in discrete-event modeling. This is common practice in operations research, specifically in supply chain management. In this article, we present the application of discrete-event modeling in the context of a QMRA for  Listeria monocytogenes  in fresh-cut iceberg lettuce. We show the potential value of discrete-event modeling in QMRA by calculating logistic interventions (modifications in the logistic chain) and determining their significance with respect to food safety.     

A Quantitative Microbial Risk Assessment Model for Legionnaires'' Disease: Animal Model Selection and Dose-Response Modeling

Legionnaires' disease (LD), first reported in 1976, is an atypical pneumonia caused by bacteria of the genus Legionella, and most frequently by L. pneumophila (Lp). Subsequent research on exposure to the organism employed various animal models, and with quantitative microbial risk assessment (QMRA) techniques, the animal model data may provide insights on human dose-response for LD. This article focuses on the rationale for selection of the guinea pig model, comparison of the dose-response model results, comparison of projected low-dose responses for guinea pigs, and risk estimates for humans. Based on both in vivo and in vitro comparisons, the guinea pig (Cavia porcellus) dose-response data were selected for modeling human risk. We completed dose-response modeling for the beta-Poisson (approximate and exact), exponential, probit, logistic, and Weibull models for Lp inhalation, mortality, and infection (end point elevated body temperature) in guinea pigs. For mechanistic reasons, including low-dose exposure probability, further work on human risk estimates for LD employed the exponential and beta-Poisson models. With an exposure of 10 colony-forming units (CFU) (retained dose), the QMRA model predicted a mild infection risk of 0.4 (as evaluated by seroprevalence) and a clinical severity LD case (e.g., hospitalization and supportive care) risk of 0.0009. The calculated rates based on estimated human exposures for outbreaks used for the QMRA model validation are within an order of magnitude of the reported LD rates. These validation results suggest the LD QMRA animal model selection, dose-response modeling, and extension to human risk projections were appropriate.     

Risk Assessment of Human Toxoplasmosis Associated with the Consumption of Pork Meat in Italy

Toxoplasmosis is a cosmopolitan disease and has a broad range of hosts, including humans and several wild and domestic animals. The human infection is mostly acquired through the consumption of contaminated food and pork meat has been recognized as one of the major sources of transmission. There are, however, certain fundamental differences between countries; therefore, the present study specifically aims to evaluate the exposure of the Italian population to Toxoplasma gondii through the ingestion of several types of pork meat products habitually consumed in Italy and to estimate the annual number of human infections within two subgroups of the population. A quantitative risk assessment model was built for this reason and was enriched with new elements in comparison to other similar risk assessments in order to enhance its accuracy. Sensitivity analysis and two alternative scenarios were implemented to identify the factors that have the highest impact on risk and to simulate different plausible conditions, respectively. The estimated overall average number of new infections per year among adults is 12,513 and 92 for pregnant women. The baseline model showed that almost all these infections are associated with the consumption of fresh meat cuts and preparations (mean risk of infection varied between 4.5 × 10⁻⁵ and 5.5 × 10⁻⁵) and only a small percentage is due to fermented sausages/salami. On the contrary, salt‐cured meat products seem to pose minor risk but further investigations are needed to clarify still unclear aspects. Among all the considered variables, cooking temperature and bradyzoites’ concentration in muscle impacted most the risk.     

An Ecological Model for Quantitative Risk Assessment for Schistosomiasis: The Case of a Patchy Environment in the Coastal Tropical Area of Northeastern Brazil

We developed a stochastic model for quantitative risk assessment for the Schistosoma mansoni (SM) parasite, which causes an endemic disease of public concern. The model provides answers in a useful format for public health decisions, uses data and expert opinion, and can be applied to any landscape where the snail Biomphalaria glabrata is the main intermediate host (South and Central America, the Caribbean, and Africa). It incorporates several realistic and case‐specific features: stage‐structured parasite populations, periodic praziquantel (PZQ) drug treatment for humans, density dependence, extreme events (prolonged rainfall), site‐specific sanitation quality, environmental stochasticity, monthly rainfall variation, uncertainty in parameters, and spatial dynamics. We parameterize the model through a real‐world application in the district of Porto de Galinhas (PG), one of the main touristic destinations in Brazil, where previous studies identified four parasite populations within the metapopulation. The results provide a good approximation of the dynamics of the system and are in agreement with our field observations, i.e., the lack of basic infrastructure (sanitation level and health programs) makes PG a suitable habitat for the persistence and growth of a parasite metapopulation. We quantify the risk of SM metapopulation explosion and quasi‐extinction and the time to metapopulation explosion and quasi‐extinction. We evaluate the sensitivity of the results under varying scenarios of future periodic PZQ treatment (based on the Brazilian Ministry of Health's plan) and sanitation quality. We conclude that the plan might be useful to slow SM metapopulation growth but not to control it. Additional investments in better sanitation are necessary.     

Estimating Listeria monocytogenes Growth in Ready-to-Eat Chicken Salad Using a Challenge Test for Quantitative Microbial Risk Assessment

Currently, there is a growing preference for convenience food products, such as ready-to-eat (RTE) foods, associated with long refrigerated shelf-lives, not requiring a heat treatment prior to consumption. Because Listeria monocytogenes is able to grow at refrigeration temperatures, inconsistent temperatures during production, distribution, and at consumer's household may allow for the pathogen to thrive, reaching unsafe limits. L. monocytogenes is the causative agent of listeriosis, a rare but severe human illness, with high fatality rates, transmitted almost exclusively by food consumption. With the aim of assessing the quantitative microbial risk of L. monocytogenes in RTE chicken salads, a challenge test was performed. Salads were inoculated with a three-strain mixture of cold-adapted L. monocytogenes and stored at 4, 12, and 16 °C for eight days. Results revealed that the salad was able to support L. monocytogenes’ growth, even at refrigeration temperatures. The Baranyi primary model was fitted to microbiological data to estimate the pathogen's growth kinetic parameters. Temperature effect on the maximum specific growth rate (μ_max) was modeled using a square-root-type model. Storage temperature significantly influenced μ_max of L. monocytogenes (p < 0.05). These predicted growth models for L. monocytogenes were subsequently used to develop a quantitative microbial risk assessment, estimating a median number of 0.00008726 listeriosis cases per year linked to the consumption of these RTE salads. Sensitivity analysis considering different time–temperature scenarios indicated a very low median risk per portion (<−7 log), even if the assessed RTE chicken salad was kept in abuse storage conditions.     

Inferring an Augmented Bayesian Network to Confront a Complex Quantitative Microbial Risk Assessment Model with Durability Studies: Application to Bacillus Cereus on a Courgette Purée Production Chain

The Monte Carlo (MC) simulation approach is traditionally used in food safety risk assessment to study quantitative microbial risk assessment (QMRA) models. When experimental data are available, performing Bayesian inference is a good alternative approach that allows backward calculation in a stochastic QMRA model to update the experts’ knowledge about the microbial dynamics of a given food‐borne pathogen. In this article, we propose a complex example where Bayesian inference is applied to a high‐dimensional second‐order QMRA model. The case study is a farm‐to‐fork QMRA model considering genetic diversity of Bacillus cereus in a cooked, pasteurized, and chilled courgette purée. Experimental data are Bacillus cereus concentrations measured in packages of courgette purées stored at different time‐temperature profiles after pasteurization. To perform a Bayesian inference, we first built an augmented Bayesian network by linking a second‐order QMRA model to the available contamination data. We then ran a Markov chain Monte Carlo (MCMC) algorithm to update all the unknown concentrations and unknown quantities of the augmented model. About 25% of the prior beliefs are strongly updated, leading to a reduction in uncertainty. Some updates interestingly question the QMRA model.     

Time-Dose-Response Models for Microbial Risk Assessment

While microbial risk assessment (MRA) has been used for over 25 years, traditional dose-response analysis has only predicted the overall risk of adverse consequences from exposure to a given dose. An important issue for consequence assessment from bioterrorist and other microbiological exposure is the distribution of cases over time due to the initial exposure. In this study, the classical exponential and beta-Poisson dose-response models were modified to include exponential-power dependency of time post inoculation (TPI) or its simplified form, exponential-reciprocal dependency of TPI, to quantify the time of onset of an effect presumably associated with the kinetics of in vivo bacterial growth. Using the maximum likelihood estimation approach, the resulting time-dose-response models were found capable of providing statistically acceptable fits to all tested pooled animal survival dose-response data. These new models can consequently describe the development of animal infectious response over time and represent observed responses fairly accurately. This is the first study showing that a time-dose-response model can be developed for describing infections initiated by various pathogens. It provides an advanced approach for future MRA frameworks.     

A Poultry-Processing Model for Quantitative Microbiological Risk Assessment

A poultry-processing model for a quantitative microbiological risk assessment (QMRA) of campylobacter is presented, which can also be applied to other QMRAs involving poultry processing. The same basic model is applied in each consecutive stage of industrial processing. It describes the effects of inactivation and removal of the bacteria, and the dynamics of cross-contamination in terms of the transfer of campylobacter from the intestines to the carcass surface and the environment, from the carcasses to the environment, and from the environment to the carcasses. From the model it can be derived that, in general, the effect of inactivation and removal is dominant for those carcasses with high initial bacterial loads, and cross-contamination is dominant for those with low initial levels. In other QMRA poultry-processing models, the input-output relationship between the numbers of bacteria on the carcasses is usually assumed to be linear on a logarithmic scale. By including some basic mechanistics, it is shown that this may not be realistic. As nonlinear behavior may affect the predicted effects of risk mitigations; this finding is relevant for risk management. Good knowledge of the variability of bacterial loads on poultry entering the process is important. The common practice in microbiology to only present geometric mean of bacterial counts is insufficient: arithmetic mean are more suitable, in particular, to describe the effect of cross-contamination. The effects of logistic slaughter (scheduled processing) as a risk mitigation strategy are predicted to be small. Some additional complications in applying microbiological data obtained in processing plants are discussed.     

A Practical Framework for the Construction of a Biotracing Model: Application to Salmonella in the Pork Slaughter Chain

A novel purpose of the use of mathematical models in quantitative microbial risk assessment (QMRA) is to identify the sources of microbial contamination in a food chain (i.e., biotracing). In this article we propose a framework for the construction of a biotracing model, eventually to be used in industrial food production chains where discrete numbers of products are processed that may be contaminated by a multitude of sources. The framework consists of steps in which a Monte Carlo model, simulating sequential events in the chain following a modular process risk modeling (MPRM) approach, is converted to a Bayesian belief network (BBN). The resulting model provides a probabilistic quantification of concentrations of a pathogen throughout a production chain. A BBN allows for updating the parameters of the model based on observational data, and global parameter sensitivity analysis is readily performed in a BBN. Moreover, a BBN enables “backward reasoning” when downstream data are available and is therefore a natural framework for answering biotracing questions. The proposed framework is illustrated with a biotracing model of Salmonella in the pork slaughter chain, based on a recently published Monte Carlo simulation model. This model, implemented as a BBN, describes the dynamics of Salmonella in a Dutch slaughterhouse and enables finding the source of contamination of specific carcasses at the end of the chain.     

Characterizing the Risk of Infection from Mycobacterium tuberculosis in Commercial Passenger Aircraft Using Quantitative Microbial Risk Assessment

Quantitative microbial risk assessment was used to predict the likelihood and spatial organization of Mycobacterium tuberculosis ( Mtb ) transmission in a commercial aircraft. Passenger exposure was predicted via a multizone Markov model in four scenarios: seated or moving infectious passengers and with or without filtration of recirculated cabin air. The traditional exponential ( k  = 1) and a new exponential ( k  = 0.0218) dose-response function were used to compute infection risk. Emission variability was included by Monte Carlo simulation. Infection risks were higher nearer and aft of the source; steady state airborne concentration levels were not attained. Expected incidence was low to moderate, with the central 95% ranging from 10⁻⁶ to 10⁻¹ per 169 passengers in the four scenarios. Emission rates used were low compared to measurements from active TB patients in wards, thus a "superspreader" emitting 44 quanta/h could produce 6.2 cases or more under these scenarios. Use of respiratory protection by the infectious source and/or susceptible passengers reduced infection incidence up to one order of magnitude.     

Quantitative Risk Assessment of Vibrio parahaemolyticus in Finfish: A Model of Raw Horse Mackerel Consumption in Japan

The aim of this study was to evaluate the effects of implemented control measures to reduce illness induced by Vibrio parahaemolyticus (V. parahaemolyticus) in horse mackerel (Trachurus japonicus), seafood that is commonly consumed raw in Japan. On the basis of currently available experimental and survey data, we constructed a quantitative risk model of V. parahaemolyticus in horse mackerel from harvest to consumption. In particular, the following factors were evaluated: bacterial growth at all stages, effects of washing the fish body and storage water, and bacterial transfer from the fish surface, gills, and intestine to fillets during preparation. New parameters of the beta‐Poisson dose‐response model were determined from all human feeding trials, some of which have been used for risk assessment by the U.S. Food and Drug Administration (USFDA). The probability of illness caused by V. parahaemolyticus was estimated using both the USFDA dose‐response parameters and our parameters for each selected pathway of scenario alternatives: washing whole fish at landing, storage in contaminated water, high temperature during transportation, and washing fish during preparation. The last scenario (washing fish during preparation) was the most effective for reducing the risk of illness by about a factor of 10 compared to no washing at this stage. Risk of illness increased by 50% by exposure to increased temperature during transportation, according to our assumptions of duration and temperature. The other two scenarios did not significantly affect risk. The choice of dose‐response parameters was not critical for evaluation of control measures.     

Regional Risk Assessment for Point Source Pollution Based on a Water Quality Model of the Taipu River,China

Point source pollution is one of the main threats to regional environmental health. Based on a water quality model, a methodology to assess the regional risk of point source pollution is proposed. The assessment procedure includes five parts: (1) identifying risk source units and estimating source emissions using Monte Carlo algorithms; (2) observing hydrological and water quality data of the assessed area, and evaluating the selected water quality model; (3) screening out the assessment endpoints and analyzing receptor vulnerability with the Choquet fuzzy integral algorithm; (4) using the water quality model introduced in the second step to predict pollutant concentrations for various source emission scenarios and analyzing hazards of risk sources; and finally, (5) using the source hazard values and receptor vulnerability scores to estimate overall regional risk. The proposed method, based on the Water Quality Analysis Simulation Program (WASP), was applied in the region of the Taipu River, which is in the Taihu Basin, China. Results of source hazard and receptor vulnerability analysis allowed us to describe aquatic ecological, human health, and socioeconomic risks individually, and also integrated risks in the Taipu region, from a series of risk curves. Risk contributions of sources to receptors were ranked, and the spatial distribution of risk levels was presented. By changing the input conditions, we were able to estimate risks for a range of scenarios. Thus, the proposed procedure may also be used by decisionmakers for long‐term dynamic risk prediction.     

Ensuring the Quality of Occupational Safety Risk Assessment

In work environments, the main aim of occupational safety risk assessment (OSRA) is to improve the safety level of an installation or site by either preventing accidents and injuries or minimizing their consequences. To this end, it is of paramount importance to identify all sources of hazards and assess their potential to cause problems in the respective context. If the OSRA process is inadequate and/or not applied effectively, it results in an ineffective safety prevention program and inefficient use of resources. An appropriate OSRA is an essential component of the occupational safety risk management process in industries. In this article, we performed a survey to elicit the relative importance for identified OSRA tasks to enable an in‐depth evaluation of the quality of risk assessments related to occupational safety aspects on industrial sites. The survey involved defining a questionnaire with the most important elements (tasks) for OSRA quality assessment, which was then presented to safety experts in the mining, electrical power production, transportation, and petrochemical industries. With this work, we expect to contribute to the main question of OSRA in industries: “What constitutes a good occupational safety risk assessment?” The results obtained from the questionnaire showed that experts agree with the proposed OSRA process decomposition in steps and tasks (taxonomy) and also with the importance of assigning weights to obtain knowledge about OSRA task relevance. The knowledge gained will enable us, in the near future, to build a framework to evaluate OSRA quality for industrial sites.     

Quantitative Risk Assessment of Norovirus Transmission in Food Establishments: Evaluating the Impact of Intervention Strategies and Food Employee Behavior on the Risk Associated with Norovirus in Foods

We developed a quantitative risk assessment model using a discrete event framework to quantify and study the risk associated with norovirus transmission to consumers through food contaminated by infected food employees in a retail food setting. This study focused on the impact of ill food workers experiencing symptoms of diarrhea and vomiting and potential control measures for the transmission of norovirus to foods. The model examined the behavior of food employees regarding exclusion from work while ill and after symptom resolution and preventive measures limiting food contamination during preparation. The mean numbers of infected customers estimated for 21 scenarios were compared to the estimate for a baseline scenario representing current practices. Results show that prevention strategies examined could not prevent norovirus transmission to food when a symptomatic employee was present in the food establishment. Compliance with exclusion from work of symptomatic food employees is thus critical, with an estimated range of 75–226% of the baseline mean for full to no compliance, respectively. Results also suggest that efficient handwashing, handwashing frequency associated with gloving compliance, and elimination of contact between hands, faucets, and door handles in restrooms reduced the mean number of infected customers to 58%, 62%, and 75% of the baseline, respectively. This study provides quantitative data to evaluate the relative efficacy of policy and practices at retail to reduce norovirus illnesses and provides new insights into the interactions and interplay of prevention strategies and compliance in reducing transmission of foodborne norovirus.     

A Risk Assessment Model for Campylobacter in Broiler Meat

A quantitative microbiological risk assessment model describes the transmission of Campylobacter through the broiler meat production chain and at home, from entering the processing plant until consumption of a chicken breast fillet meal. The exposure model is linked to a dose-response model to allow estimation of the incidence of human campylobacteriosis. The ultimate objective of the model is to serve as a tool to assess the effects of interventions to reduce campylobacteriosis in the Netherlands. The model describes some basic mechanistics of processing, including the nonlinear effects of cross-contamination between carcasses and their leaking feces. Model input is based on the output of an accompanying farm model and Dutch count data of Campylobacters on the birds' exterior and in the feces. When processing data are lacking, expert judgment is used for model parameter estimation. The model shows that to accurately assess of the effects of interventions, numbers of Campylobacter have to be explicitly incorporated in the model in addition to the prevalence of contamination. Also, as count data usually vary by several orders of magnitude, variability in numbers within and especially between flocks has to be accounted for. Flocks with high concentrations of Campylobacter in the feces that leak from the carcasses during industrial processing seem to have a dominant impact on the human incidence. The uncertainty in the final risk estimate is large, due to a large uncertainty at several stages of the chain. Among others, more quantitative count data at several stages of the production chain are needed to decrease this uncertainty. However, this uncertainty is smaller when relative risks of interventions are calculated with the model. Hence, the model can be effectively used by risk management in deciding on strategies to reduce human campylobacteriosis.     

A Probabilistic Transmission Model of Salmonella in the Primary Broiler Production Chain

Annual data from the Finnish National Salmonella Control Programme were used to build up a probabilistic transmission model of salmonella in the primary broiler production chain. The data set consisted of information on grandparent, parent, and broiler flock populations. A probabilistic model was developed to describe the unknown true prevalences, vertical and horizontal transmissions, as well as the dynamical model of infections. By combining these with the observed data, the posterior probability distributions of the unknown parameters and variables could be derived. Predictive distributions were derived for the true number of infected broiler flocks under the adopted intervention scheme and these were compared with the predictions under no intervention. With the model, the effect of the intervention used in the programme, i.e., eliminating salmonella positive breeding flocks, could be quantitatively assessed. The 95% probability interval of the posterior predictive distribution for (broiler) flock prevalence under current (1999) situation was [1.3%-17.4%] (no intervention), and [0.9%-5.8%] (with intervention). In the scenario of one infected grandparent flock, these were [2.8%-43.1%] and [1.0%-5.9%], respectively. Computations were performed using WinBUGS and Matlab softwares.     

Global Sensitivity Analysis Applied to a Contamination Assessment Model of Listeria monocytogenes in Cold Smoked Salmon at Consumption

In this study, a variance‐based global sensitivity analysis method was first applied to a contamination assessment model of Listeria monocytogenes in cold smoked vacuum packed salmon at consumption. The impact of the choice of the modeling approach (populational or cellular) of the primary and secondary models as well as the effect of their associated input factors on the final contamination level was investigated. Results provided a subset of important factors, including the food water activity, its storage temperature, and duration in the domestic refrigerator. A refined sensitivity analysis was then performed to rank the important factors, tested over narrower ranges of variation corresponding to their current distributions, using three techniques: ANOVA, Spearman correlation coefficient, and partial least squares regression. Finally, the refined sensitivity analysis was used to rank the important factors.     

Quantitative Uncertainty Analysis for a Weed Risk Assessment System

Weed risk assessments (WRA) are used to identify plant invaders before introduction. Unfortunately, very few incorporate uncertainty ratings or evaluate the effects of uncertainty, a fundamental risk component. We developed a probabilistic model to quantitatively evaluate the effects of uncertainty on the outcomes of a question‐based WRA tool for the United States. In our tool, the uncertainty of each response is rated as Negligible, Low, Moderate, or High. We developed the model by specifying the likelihood of a response changing for each uncertainty rating. The simulations determine if responses change, select new responses, and sum the scores to determine the risk rating. The simulated scores reveal potential variation in WRA risk ratings. In testing with 204 species assessments, the ranges of simulated risk scores increased with greater uncertainty, and analyses for most species produced simulated risk ratings that differed from the baseline WRA rating. Still, the most frequent simulated rating matched the baseline rating for every High Risk species, and for 87% of all tested species. The remaining 13% primarily involved ambiguous Low Risk results. Changing final ratings based on the uncertainty analysis results was not justified here because accuracy (match between WRA tool and known risk rating) did not improve. Detailed analyses of three species assessments indicate that assessment uncertainty may be best reduced by obtaining evidence for unanswered questions, rather than obtaining additional evidence for questions with responses. This analysis represents an advance in interpreting WRA results, and has enhanced our regulation and management of potential weed species.