Nocito, J.A., H.A. Walker, J.F. Paul, and C.A. Menzie. 1989. Application
of a Risk Assessment Framework for Marine Disposal of Sewage Sludge at
Midshelf and Offshelf Sites. In: Aquatic Toxicology and Environmental
Fate: Eleventh Volume, ASTM STP 1007, G.W. Suter II and M.A. Lewis
(eds.), American Society for Testing and Materials, Philadelphia, pp.
101-120.

Paul, J.F., H.A. Walker, and V.J. Bierman, Jr. 1983. Probabilistic
Approach for the Determination of the Potential Area of Influence for
Waste Disposed at the 106-Mile Ocean Disposal Site. In: 106-Mile Waste
Disposal Site Characterization Update Report, J.B. Pearce, D.C. Miller,
and C. Berman (eds.), NOAA Technical Memorandum NMFS-F/NEC-26, National
Marine Fisheries Service, Woods Hole, MA, pp. A-1 to A-8.

Paul, J.F., V.J. Bierman, Jr., W.R. Davis, G.L. Hoffman, W.R. Munns,
C.E. Pesch, P.F. Rogerson, and S.C. Schimmel. 1988. The Application of a
Hazard Assessment Research Strategy to the Ocean Disposal of a Dredged
Material: Exposure Assessment Component. In: Oceanic Processes in Marine
Pollution, Vol. 5, D.A. Wolfe and T.P. O’Connor (eds.), Kreiger
Publishing Co., Melbourne, FL, pp. 123-135.

Prager, J.C., V.J. Bierman, Jr., J.F. Paul, and J.S. Bonner. 1986.
Sampling the Oceans for Pollution: A Risk Assessment Approach to
Evaluating Low-Level Radioactive Waste Disposal at Sea. Dangerous
Properties of Industrial Materials Report, Vol. 6, No. 3, pp. 2-26.

Walker, H.A., J.F. Paul, J.A. Nocito, and J.H. Gentile. 1988. Ecological
and Human Health Risks for Sewage Sludge Disposal at the 106-Mile Site.
In: Proceedings of the Seventh International Ocean Disposal Symposium,
Environment Canada, pp. 625-643.

Appendix D: Case Study Examples of the Application of Probabilistic Risk
Analysis in U.S. Environmental Protection Agency Regulatory
Decision-Making

Prepared by Risk Assessment Forum

PRA Working Group 2

Allison Hess, David Hrdy, John Langstaff, Elizabeth Margosches, Michael
Messner, and Marian Olsen

Disclaimer

This document is a preliminary draft. It has not been formally released
by the U.S. Environmental Protection Agency and should not at this stage
be construed to represent Agency policy. It is being circulated for
comments on its technical merit and policy implications. Mention of
trade names or commercial products does not constitute endorsement or
recommendation for use.

Foreword

The U.S. Environmental Protection Agency’s (EPA’s) Risk Assessment
Forum was directed by the Science Policy Council in the Office of the
Science Advisor to consider how to better and more fully implement
probabilistic risk analysis (PRA) and related tools in the EPA
decision-making process. A technical panel of senior scientists gathered
and identified several ways of moving the agenda to fully implement PRA.
This Appendix focuses on examples of how PRA approaches have been used
at EPA to inform regulatory decisions.

This White Paper was prepared by representatives from various EPA
program offices and regions currently involved in the development and
application of PRA techniques. The workgroup selected the case study
examples based on the workgroup’s knowledge of the specific PRA
procedures, the types of techniques demonstrated, availability to the
reader through the internet, peer-reviewed, and illustrative of a
spectrum of PRA used in EPA. The case studies are not designed to
provide an exhaustive discussion of the wide variety of applications of
PRA used within the Agency but to highlight specific examples reflecting
the range of approaches currently applied within EPA.

Acknowledgments

	We would like to acknowledge the scientists and risk assessors who
performed the original analyses on which the summaries of these case
studies are based.  The names of the points of contact are included for
each study, and many more contributors were involved and acknowledged in
the original work.

	Contributors and Reviewers to this Document

Gary Bangs, Jonathan Chen, Lisa Conner, Kathryn Gallagher, and Valerie
Zartarian

The PRA Technical Panel

The EPA Risk Assessment Forum

Executive Summary

This Appendix is designed to serve as a resource for risk managers faced
with decisions regarding when to apply PRA techniques to inform
environmental decisions and for exposure and risk assessors who may not
be familiar with the wide range of available PRA approaches. The
document outlines categories of PRAs classified by the complexity of
analysis to aid the decision-making process. This approach identifies
various PRA tools that include techniques ranging from a simple
sensitivity analysis (e.g., identification of key exposure parameters or
data visualization) requiring limited time, resources, and expertise to
develop (Group 1); to probabilistic approaches, including Monte Carlo
analysis, that provide tools for evaluating variability and uncertainty
separately and require more resources and specialized expertise (Group
2); and to sophisticated techniques of expert elicitation that generally
require significant investment of employee time, additional expertise,
and external peer-review (Group 3).

This document describes case studies wherein PRA techniques have been
used within this ranked framework to provide additional information for
risk managers. PRA is a scientific tool to help describe the data or the
risk and is one of many inputs considered by the risk manager in the
decision-making process. The case study summaries are provided in a
format designed to highlight how the results of the PRAs were considered
in decision-making. These summaries include specific information on the
conduct of the analyses as an aid to determining what tools might be
appropriate for developing specific exposure or risk assessments for
other assessments.

The case studies range from examples of less resource-intensive analyses
that might assist in identifying key exposure parameters or the need for
more data to more detailed and resource-intensive approaches. Tools
include Monte Carlo modeling, sensitivity analyses, and application of
expert elicitation. Examples of applications in human health and
ecological risk assessment include the exposure of children to chromated
copper arsenate treated wood, the relation between particulates in air
and health, dietary exposures to pesticides, modeling sea level change,
sampling watersheds, and modeling bird and animal exposures.

1. Introduction

 methods typically used in EPA deterministic risk assessments rely on a
combination of point values―some conservative and some
typical―yielding a point estimate of exposure that is at some unknown
point in the range of possible risks (EPA, 2004).

The development of sophisticated computational tools over the past 10
years has prompted an increased interest in analyses that evaluate the
variability and uncertainty in the risk assessments; these include the
use of tools such as probabilistic risk analysis, or PRA (EPA, 2001,
2004). These analyses provide the results of the risk assessment as a
probability or likelihood of different risk levels in a population
(describing variability) or to characterize uncertainty in risk
estimates.

This Appendix presents case studies of PRA conducted by EPA over the
past 10 to 15 years. Table 1 summarizes the case studies by title,
technique demonstrated, classification based on Human Health and
Ecological Risk Assessment, and the program office responsible for
developing the case studies. This document, by illustration, provides a
“snapshot” of utilization of PRA across various programs in EPA.

2. Overall Approach to Probabilistic Risk Analysis at the U.S.
Environmental Protection Agency

2.1. U.S. Environmental Protection Agency Guidance and Policies on
Probabilistic Risk Analysis

The case studies presented here build on the principles of PRA outlined
in EPA’s 1996 Policy (EPA, 1996) and Guiding Principles for Monte
Carlo Analysis (EPA, 1997b) and subsequent guidance documents on
developing and using PRA.  Guidance has been developed for the Agency as
well as for individual programs that refers to the use of PRA, including
the Risk Assessment Guidance for Superfund Part III (EPA, 2001); Risk
Assessment Forum Framework for Ecological Risk Assessment (EPA, 1992b);
Guidelines for Ecological Risk Assessment (EPA, 1998); Guidance for Risk
Characterization (EPA, 1995a); Policy for Risk Characterization (EPA,
1995a); Policy on Evaluating Health Risks to Children (EPA, 1995b);
Policy for Use of Probabilistic Analysis in Risk Assessment (EPA,
1997a); Guidance on Cumulative Risk Assessment. Part 1. Planning and
Scoping (EPA, 1997c ; and Risk Characterization Handbook (EPA, 2000).

As shown in the individual case studies, the range and scope of the PRA
will depend on the overall objectives of the decision that the analysis
will inform. The Guiding Principles for Monte Carlo Analysis lay out the
general approach that should be taken in all cases, beginning with
defining the problem and scope of the assessment, so that the best tools
and approach may be selected.  The Guiding Principles also describe the
process of estimating and characterizing variability and uncertainty
around the risk estimates. Stahl and Cimorelli (2005) and the Risk
Assessment Guidance for Superfund Volume III (EPA, 2001) highlight the
importance of communication between risk assessor and manager. Stahl and
Cimorelli (2005) and Jamieson (1996) indicate it is important to
determine whether a particular level of uncertainty is acceptable or
not. The authors also suggest this decision is a matter of context,
values, and regulatory policy. The Risk Assessment Guidance for
Superfund Part III (Chapter 2 and Appendix F in EPA, 2001) describes a
process for determining the appropriate level of PRA using a ranked
approach from the less resource- and time-intensive approaches to more
sophisticated analyses (Chapter 2 in EPA, 2001). Further, the Risk
Assessment Guidance for Superfund Part III  outlines a process for
developing a PRA work plan and a checklist for PRA reviewers (Chapter 2
and Appendix F in EPA, 2001). This guidance also provides information
regarding how to communicate PRA results to risk managers and
stakeholders (Chapter 6 in EPA, 2001).

The guidance and policies on uncertainty and variability, and
application of the principles of PRA, all highlight the ongoing need for
communication between the risk assessor and risk manager. The ongoing
communication is important in determining the appropriate levels of
analysis for the specific decision.

2.2. Categorizing Case Studies

The ranked approach used for categorization is a process for a
systematic, informed progression to increasingly more complex risk
assessment methods of PRA that is outlined in the Risk Assessment
Guidance for Superfund (EPA, 2001). The use of categories provides a
framework for evaluating the various techniques of PRA.  Higher
categories reflect increasing complexity and, in many cases, will
require more time and resources. Higher categories also reflect
increasing characterization of variability and uncertainty in the risk
estimate, which may be important for making specific risk management
decisions. Central to the approach is a systematic, informed progression
using an iterative process of evaluation, deliberation, data collection,
planning and scoping, development and updates to the work plan, and
communication. All of these steps focus on deciding

whether or not the risk assessment, in its current state (i.e.,
deterministic risk analysis), is sufficient to support risk management
decisions (a clear path to exiting the process is available at each
step); and

if the assessment is determined to be insufficient, whether or not
progression to a higher group of complexity (or refinement of the
current analyses) would provide a sufficient benefit to warrant the
additional effort of performing a PRA.

This paper groups case studies according to level of effort and
complexity of the analysis, and the increasing sophistication of the
methods used (Table 1).  Although each group generally represents
increasing effort and cost, this may not always be true.  The groups are
intended to also reflect the progression from simple to complex analysis
that is determined by the interactive planning and scoping efforts of
the risk assessors and managers.  The use of particular terms to
describe the groups, including tiers, was avoided due to specific
programmatic and regulatory connotations.

2.2.1. Group 1 Case Studies

Assessments within this group typically involve a sensitivity analysis
and serve as an initial screening step in the risk assessment.
Sensitivity analyses identify important parameters in the assessment
where additional investigation may be helpful (Kurowicka and Cooke,
2006). Sensitivity analysis can be simple or involve more complex
mathematical and statistical techniques such as correlation and
regression analysis to determine which factors in a risk model
contribute most to the variance in the risk estimate.

Within the sensitivity analyses, a range of techniques is available:
simple, “back of the envelope” calculation, where the risk
parameters are evaluated using a range of exposure parameters to
determine the parameter that contributes most significantly to the risk
(Case Study 1); analyses to rank relative contributions of variables to
the overall risk (Case Study 2); and data visualization using graphical
techniques to array the data or Monte Carlo simulations (e.g., scatter
plots).

More sophisticated analyses may include sensitivity ratios (i.e.,
elasticity); sensitivity scores (i.e., weighted sensitivity ratios);
correlation coefficient or coefficient of determination, r2 (e.g.,
Pearson product moment, Spearman rank); normalized multiple regression
coefficient; and goodness-of-fit tests for subsets of the risk
distribution (EPA, 2001).

The sensitivity analyses typically require limited resources and time to
conduct. Results of the sensitivity analyses are useful in identifying
key parameters where additional Group 2 or 3 analyses may be
appropriate. Sensitivity analyses are also helpful in identifying key
parameters where additional research will have the most impact on the
risk assessment.

2.2.2. Group 2 Case Studies

Case studies within this group include more sophisticated application of
probabilistic tools, including PRA of specific exposure parameters (Case
Studies 3 and 4), one-dimensional analyses (Case Study 5), and
probabilistic sensitivity analysis (Case Studies 6 and 7).

The Group 2 case studies require larger time commitments for
development, specialized expertise, and additional analysis of exposure
parameter data sources. Depending on the nature of the analysis, peer
involvement or peer review may be appropriate to the evaluation of the
products of the analysis.

2.2.3. Group 3 Case Studies

Assessments within this group are the most resource- and time-intensive
analyses of the three categories. Risk analyses include two-dimensional
Monte Carlo analysis that evaluate model variability and uncertainty
(Case Studies 8 through 10); Microexposure Event Analysis, in which
long-term exposure of an individual is simulated as the sum of separate
short-term exposure events (Case Study 11); and Probabilistic Analysis
(Case Studies 12 and 13).

Other types of analyses within this group include the expert elicitation
method that is a systematic process of formalizing and quantifying, in
terms of probabilities, experts’ judgments about uncertain quantities
(Case Studies 14 and 15); Bayesian statistics that is a specialized
branch of statistics that views the probability of an event occurring as
the degree of belief or confidence in that occurrence (Case Study 16);
and geostatistical analysis, which is another specialized branch of
statistics that explicitly takes into account the geo-referenced context
of data and the information (i.e., attributes) attached to the data.

The Group 3 analyses require additional time and expertise in the
planning and  analysis of the assessment. Within this group, the level
of expertise and resource commitments may vary with techniques such as
expert elicitation requiring significantly longer time for planning,
identification of experts, and meetings, when compared with the other
techniques.

.

Table 1.  Case Study Titles, Description, Type of Assessment (Human
Health or Ecological Risk Assessment) and Program Office that Developed
Assessment.

Study

No.	Title and Case Study Description	HH/

Eco1	Program

oup 2―Probabilistic Risk Analysis, One-Dimensional Monte Carlo
Analysis, and

Probabilistic Sensitivity Analysis

Probabilistic Risk Analysis

3	Probabilistic Assessment of Angling Duration Used in Assessment of
Exposure to Hudson River Sediments via Consumption of Contaminated Fish.
A probabilistic analysis of one parameter in an exposure
assessment―the time an individual fishes in a large river system.
Development of site-specific information regarding exposure, with an
existing data set for this geographic area, was needed to represent this
exposed population. This information was used in the one-dimensional PRA
described in Case Study 5.	HH	Region 2/

Superfund

4	Probabilistic Analysis of Dietary Exposure to Pesticides for Use in
Setting Tolerance Levels. The probabilistic Dietary Exposure Evaluation
Model (DEEM) provides more accurate information on the range and
probability of possible exposures.	HH	OPP

1 HH = Human Health Risk Assessment; Eco = Ecological Risk Assessment.

2 OPP, Office of Pesticides Programs; ORD, Office of Research and
Development; OAR, Office of Air and Radiation; OW, Office of Water.

Study

No.

Title and Case Study Description	HH/

Eco	Program

Office

Group 2 (cont’d.)

One-Dimensional Monte Carlo Analysis

5	One-Dimensional Probabilistic Risk Analysis of Exposures to
Polychlorinated Biphenyls (PCBs) via Consumption of Fish from a
Contaminated Sediment Site. An example of a one-dimensional PRA (1-D
Monte Carlo analysis of the variability of exposure as a function of the
variability of individual exposure factors.) to evaluate the risks to
anglers who consume recreationally caught fish from a PCB-contaminated
river.	HH	Region 2/

Group 3―Advanced Probabilistic Risk Analysis―Two-Dimensional Monte
Carlo Analysis Including Microexposure Modeling, Bayesian Statistics,
Geostatistics, and Expert Elicitation

Two- Dimensional Probabilistic Risk Analysis

8	Two-Dimensional Probabilistic Risk Analysis of Cryptosporidium in
Public Water Supplies, with Bayesian Approaches to Uncertainty Analysis.
An analysis of variability in the occurrence of Cryptosporidium in raw
water supplies and in the treatment efficiency, as well as the
uncertainty in these inputs. This case study includes an analysis of the
dose-response relationship for Cryptosporidium infection.	HH	OW

9	Two-Dimensional Probabilistic Model of Children’s Exposure to
Arsenic in Chromated Copper Arsenate (CCA) Pressure-Treated Wood. A
two-dimensional model that addresses both variability and uncertainty in
the exposures of children to CCA pressure-treated wood. The analysis was
built on the sensitivity analysis described in Case Study 2.	HH	OPP/ORD

Study

No.

Title and Case Study Description	HH/

Eco	Program

Office

Group 3 (cont’d.)

10	Two-Dimensional Probabilistic Exposure Assessment of Ozone

A probabilistic exposure assessment that addresses short-term exposures
to ozone.  Population exposure to ambient ozone levels was evaluated
using EPA’s Air Pollutants Exposure (APEX) model, also referred to as
the Total Risk Integrated Methodology/Exposure (TRIM.Expo) model.	HH	OAR

Group 3―Microexposure Event Modeling

Microexposure Event Analysis

11	Analysis of Microenivironmental Exposures to Particulate Matter
(PM2.5) for a Population Living in Philadelphia, PA. A microexposure
event analysis to simulate individual exposures to PM2.5 in specific
microenvironments including the outdoors, indoor residences, offices,
schools, stores, and a vehicle.	HH	Region 3

Group 3―Expert Elicitation and Bayesian Belief Network

Expert Elicitation

14	Expert Elicitation of Concentration-Response Relationship Between
Particulate Matter (PM2.5) Exposure and Mortality. An expert elicitation
used to derive probabilistic estimates of the uncertainty in one element
of a cost-benefit analysis used to support the PM2.5 regulations.	HH
ORD/

OAR

15	Expert Elicitation of Sea-Level Change Resulting from Global Climate
Change. An example of a PRA that describes the probability of sea level
rise and parameters that predict sea level change.	Eco

	16	Expert Elicitation for Bayesian Belief Network Model of Stream
Ecology. An example of a Bayesian belief network model of the effect of
increased fine-sediment load in a stream on macroinvertebrate
populations.	Eco	ORD

3. Case Study Summaries

Group 1 Case Studies

Case Study 1: Sensitivity Analysis of Key Variables in Probabilistic
Assessment of Children’s Exposure to Arsenic in Chromated Copper
Arsenate (CCA) Pressure-Treated Wood

This case study provides an example of the application of sensitivity
analysis to identify important variables for population exposure
variability for a Group 2 assessment (Case Study 9) and to indicate
areas for further research.  Specifically, EPA’s Office of Research
and Development (ORD), in collaboration with the Office of Pesticide
Programs (OPP) used sensitivity analyses to identify the key variables
in children’s exposure to CCA treated wood.

Approach.  The sensitivity analyses used two approaches.  The first
approach estimated baseline exposure by running the exposure model with
each input variable set to its median (50th percentile) value.  Next,
alternative exposure estimates were made by setting each input to its
25th or 75th percentile value while holding all other inputs at their
median values.  The ratio of the exposure estimate calculated when an
input was estimated at its 25th or 75th percentile to the exposure
estimate calculated when the input was at its median value provided a
measure of that input’s importance to the overall exposure assessment.
 The second approach applied multiple stepwise regression analysis to
the data points generated from the first approach.  The correlation
between the input variables and the exposure estimates provided an
alternative measure of the input variable’s relative importance in the
exposure assessment.  These two approaches were used in tandem to
identify the critical inputs to the exposure assessment model.

Results of Analysis.  The two sensitivity analyses together identified
six critical input variables that most influenced the exposure
assessment.  The critical input variables were: wood surface
residue-to-skin transfer efficiency, wood surface residue levels,
fraction of hand surface area mouthed per mouthing event, average
fraction of nonresidential outdoor time spent playing on a CCA-treated
playset, frequency of hand-washing, and frequency of hand-to-mouth
activity.

Management Considerations:  The results of the sensitivity analyses were
used to identify the most important input parameters in the treated wood
risk assessments.  The process also identified critical areas for future
research.  In particular, the assessment pointed to a need to collect
data on the amount of dislodgeable residue that is transferred from the
wood surface to a child’s hand upon contact, and to better
characterize the amount of dislodgeable residue that exists on the wood
surface.

Document Availability.

Title:  The final report on the probabilistic exposure assessment of
CCA-treated wood.

Zartarian, V.G., J. Xue, H. A. Ozkaynak, W. Dang, G. Glen, L. Smith, and
C. Stallings. A Probabilistic Exposure Assessment for Children Who
Contact CCA-treated Playsets and Decks Using the Stochastic Human
Exposure and Dose Simulation Model for the Wood Preservative Scenario
(SHEDS-WOOD), Final Report. U.S. Environmental Protection Agency,
Washington, DC, EPA/600/X-05/009.   HYPERLINK
"http://www.epa.gov/heasd/sheds/cca_treated.htm"
http://www.epa.gov/heasd/sheds/cca_treated.htm

See also: Xue, J., Zartarian, V.G., Özkaynak, H., Dang, W., Glen, G.,
Smith, L., and Stallings, C.   A probabilistic arsenic exposure
assessment for children who contact chromated copper arsenate
(CAA)-treated playsets and decks, Part 2:  Sensitivity and uncertainty
analyses.  Risk Analysis 26:533, 2006.

Contact Person: Dr. Jianping Xue, EPA's Office of Research and
Development,<PHONE_NUMBER><EMAIL_ADDRESS>
Case Study 2: Assessment of Relative Contribution of Atmospheric
Deposition to Watershed Contamination

Watershed contamination can result from several different sources,
including direct release into a water body, input from upstream water
bodies, and deposition from airborne sources.  Efforts to control water
body contamination begin with an analysis of the environmental sources
in order to identify those parameters providing the greatest
contribution and to determine where mitigation and/or analysis resources
should be directed.

Approach. This case study provides an example of a back-of-the envelope
analysis of the contribution of air deposition to overall watershed
contamination to identify uncertainties and/or data gaps as well as to
target resource expenditures. (Group 1: Deterministic Analysis).
Nitrogen inputs have been studied in several east and Gulf Coast
estuaries due to concerns about eutrophication.  Nitrogen from
atmospheric deposition is estimated to be as high as 10 to 40% of the
total input of nitrogen to many of these estuaries and perhaps higher in
a few cases. For a watershed that has not already been studied, a
back-of-the envelope calculation could be prepared based on information
based on nitrogen deposition rates measured in a similar area.  To
estimate the deposition load directly to the waterbody, one would
multiply the nitrogen deposition rate by the area of the waterbody.
The analyst could then estimate the nitrogen load from other sources,
(e.g., point source discharges and runoff) to estimate a total nitrogen
load for the waterbody.  The estimate of loading due to atmospheric
deposition could then be divided by the total nitrogen load for the
waterbody to estimate the percent contribution directly to the waterbody
from atmospheric deposition.

The May 2003 report by the Casco Bay Air Deposition Study Team titled
“Estimating Pollutant Loading from Atmospheric Deposition Using Casco
Bay, Maine” is an analysis using the methodology described above.  The
Casco Bay Estuary, located in the southwestern Maine, is used as a case
study. The paper also includes the results of a field air deposition
monitoring program conducted in Casco Bay (1998 - 2000) and favorably
compares the estimates developed for rate of deposition of nitrogen,
mercury and PAHs to the field monitoring results. The estimation
approach is a useful starting point for understanding the sources of
pollutants entering water bodies that cannot be accounted for through
run-off or point source discharges.

Results of Analysis. The approach outlined above was applied to the
Casco Bay Estuary in Maine.  Resources, tools and strategies for
pollution abatement can be effectively targeted at priority sources if
estuaries are to be protected. Understanding the sources and annual
loading of contaminants to an estuary guides good water quality
management by defining the range of controls of both air and water
pollution needed to achieve a desired result. The cost of conducting
monitoring to determine atmospheric loading to a water body can be
prohibitively high. Also, collection of monitoring data is a long-term
undertaking, since a minimum of three
years of data is advisable in order to “smooth out” inter-annual
variability. The estimation techniques described in this paper can serve
as a useful and inexpensive “first-cut” at understanding the
importance of the atmospheric as a pollution source, and can help to
pinpoint those areas where field measurements are needed to guide future
management decisions.

Management Consideration: If a review of information on air deposition
available for the analysis indicates a wide range of potential
deposition rates, then further study of this input would lead to better
characterization of the air contribution to overall contamination.  If
the back-of-the envelope analysis suggests that air deposition is very
small relative to other inputs, then resources should be targeted at
studying or reducing other inputs before proceeding with further
analysis of the air inputs.

Document Availability.  The back-of-the envelope calculation is outlined
in Frequently Asked Questions about Atmospheric Deposition: A Handbook
for Watershed Managers (available at   HYPERLINK
"http://www.epa.gov/air/oaqps/gr8water/handbook/airdep_sept.pdf"
http://www.epa.gov/air/oaqps/gr8water/handbook/airdep_sept.pdf  ).

Further analysis is available in Deposition of Air Pollutants to the
Great Waters - Third Report to Congress (available at   HYPERLINK
"http://www.epa.gov/air/oaqps/gr8water/3rdrpt/index.html"
http://www.epa.gov/air/oaqps/gr8water/3rdrpt/index.html  )

The Casco Bay Estuary examples is available at:    HYPERLINK
"http://www.epa.gov/owow/airdeposition/index.html"
www.epa.gov/owow/airdeposition/index.html .

Contact Person.  Gail Lacy at<PHONE_NUMBER> at   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>.

Contact for Casco Site is:  Diane Gould at<EMAIL_ADDRESS>
Group 2 Case Studies

Case Study 3: Probabilistic Assessment of Angling Duration Used in
Assessment of Exposure to Hudson River Sediments via Consumption of
Contaminated Fish

In assessing the health impact of contaminated Superfund sites, exposure
duration typically is assumed to be the same as the length of time an
individual lives in a specific area (i.e., residence duration). In
conducting the human health risk assessment for the Hudson River PCB
Superfund Site, however, there was concern that exposure duration based
on residence duration may underestimate the time spent fishing (i.e.,
angling duration).

Risk Analysis. An individual may move from one residence to another and
continue to fish in the same location, or an individual may choose to
stop fishing irrespective of the location of his or her home. EPA Region
2 developed a site-specific distribution of angling duration using the
fishing patterns reported in a New York State-wide angling survey
(Connelly et al., 1991) and migration data for the five counties
surrounding the 40-plus miles of the Upper Hudson River collected as
part of the U.S. Census.

Results of Analysis. The 50th and 95th percentile values from the
distribution of angling durations were higher than the default values
based on residence duration using standard default exposure assumptions
for residential scenarios and were used as bases for the central
tendency and reasonable maximum exposure point estimates, respectively,
in the deterministic assessment.

Management Considerations. The information provided in this analysis was
used in the point estimate analysis. The full distribution was used in
conducting a Group 2 PRA for the fish consumption pathway, which is
presented as Case Study 6.

Document Availability. The final risk assessment was released in
November 2000 (available at   HYPERLINK
"http://www.epa.gov/hudson/reports.htm"
http://www.epa.gov/hudson/reports.htm ).

Contacts. Remedial Project Manager, Alison Hess,<PHONE_NUMBER>; Risk
Assessor, Marian Olsen,<PHONE_NUMBER>

Case Study 4: Probabilistic Analysis of Dietary Exposure to Pesticides
for use in Setting Tolerance Levels

Under the Federal Food, Drug, and Cosmetic Act (FFDCA), EPA may
authorize a tolerance or exemption from the requirement of a tolerance,
to allow a pesticide residue in food, only if the Agency determines that
such residues would be “safe”.  This determination is made by
estimating exposure to the pesticide and comparing the estimated
exposure to a toxicological benchmark dose (i.e., a dose where there is
reasonable certainty of no harm).  Until 1998, Office of Pesticide
Programs (OPP) used a software program called the Dietary Risk
Evaluation System (DRES) to conduct its acute dietary risk assessments
for pesticide residues in foods. Acute assessments conducted with DRES
assumed that 100% of a given crop with registered uses of a pesticide
was treated with that pesticide and that all such treated crop items
contained pesticide residues at the maximum legal (tolerance) level
matching this to a reasonably high consumption value (around 95th
percentile). The resulting DRES acute risk estimates were considered
"high-end" or "bounding" estimates.   However, it was not possible to
know where the pesticide exposure estimates from the DRES software fit
in the overall distribution of exposures due to the limits of the tools
being used.

Approach:  To address these deficiencies, OPP has developed an acute
probabilistic dietary exposure guidance in order to use a model to
estimate exposure to pesticides in the food supply.  Rather than the
crude "high-end," single point estimates provided by deterministic
assessments, the probabilistic Dietary Exposure Evaluation Model (DEEM)
provides specific information on the range and probability of possible
exposures and depending upon the characterization of the input, 95th
percentile regulation generally for lower tiers that do not include
percent crop treated, to the 99.9th percentile for the more refined
assessments which would include percent of crop treated information.

Probabilistic Analysis.  This case study provides an example of a
one-dimensional probabilistic risk assessment of dietary exposure to
pesticides (Group 2).  The DEEM generates acute, probabilistic dietary
exposure assessments using data on (1) the distribution of daily
consumption of specific commodities (e.g., wheat, corn, apples, etc.) by
specific individuals, and (2) the distribution of concentrations of a
specific pesticide in those food commodities.  Data on commodity
consumption are collected by USDA in its Continuing Survey of Food
Intake by Individuals (CSFII). Pesticide residue concentrations on food
commodities are generally obtained from crop field trials, USDA’s
Pesticide Data Program (PDP) data, Food and Drug Administration (FDA)
monitoring data, or market basket surveys conducted by the registrants.
Using these data, DEEM is able to calculate an estimate of the risk to
the general U.S. population in addition to 26 population subgroups,
including five subgroups for infants and children (infants less than 1,
children 1-2, children 3-5, youth 6-12 and teen 13-19).

Results of Analysis.  DEEM has been used in risk assessments to support
tolerance levels for several pesticides (e.g., phosalone) and as part of
cumulative risk assessments for organophosphorus compounds (see Case
Study 11) and other pesticides.

Management Considerations:  Using the DRES, risk management decisions
were being made without a full picture of the distribution of risk among
the population, and also without full knowledge of where in the
distribution of risk the DRES risk estimate lay.  This was of concern
not only for regulators interested in public health protection, but also
for the pesticide registrants who could argue that the Agency was being
arbitrary in selecting the level at which to regulate. For most cases
reviewed by OPP to date, estimated exposure at the 99.9th percentile
calculated by DEEM probabilistic techniques is significantly lower than
exposure calculated using DRES-type deterministic assumptions at the
unknown percentile.

Document Availability.

Link to DEEM model available at   HYPERLINK
"http://www.epa.gov/oppsrrd1/cumulative/methods_tools.htm"
http://www.epa.gov/oppsrrd1/cumulative/methods_tools.htm

Contact Person.

David Hrdy at<PHONE_NUMBER> or<EMAIL_ADDRESS>
Case Study 5: One-Dimensional Probabilistic Risk Analysis of Exposure
to Polychlorinated Biphenyls (PCBs) via Consumption of Fish from a
Contaminated Sediment Site

EPA Region 2 conducted a preliminary deterministic human health risk
assessment at the Hudson River PCBs Superfund site. The deterministic
risk analysis showed that consumption of recreationally caught fish
provided the highest exposure among relevant exposure pathways and
resulted in cancer risks and noncancer health hazards that exceeded
regulatory benchmarks.

Probabilistic Analysis. Because of the size, complexity, and high level
of public interest in this site, EPA Region 2 implemented a Group 2
probabilistic assessment to characterize the variability in risks
associated with the fish consumption exposure pathway. The analysis was
a 1-Dimensional Monte Carlo analysis of the variability of exposure as a
function of the variability of individual exposure factors.  Uncertainty
was assessed using sensitivity analysis of the input variables. Data to
characterize distributions of exposure parameters were drawn from the
published literature (e.g., fish consumption rate) or from existing
databases such as the U.S. Census data (e.g., angling duration, see Case
Study 3). Mathematical models of the environmental fate, transport, and
bioaccumulation of PCBs in the Hudson River previously developed were
used to forecast changes in PCB concentration over time.

Results of Analysis. The results of the PRA were in line with the
deterministic results. For the Central Tendency individual, point
estimates were near the median (50th percentile). For the Reasonable
Maximum Exposure individual, point estimate values were at or above the
95th percentile of the probabilistic analysis. The deterministic and
probabilistic risk analyses were the subject of a formal peer review by
a panel of independent experts.

The Monte Carlo base case scenario is the one from which point estimate
exposure factors for

fish ingestion were drawn, thus the point estimate RMEs and the Monte
Carlo base case estimates can be compared. Similarly, the point estimate
central tendency (average) and the Monte Carlo base case midpoint (50th
percentile) are comparable. For cancer risk, the point estimate RME for
fish ingestion (1 x 10-3) falls approximately at the 95th percentile
from the Monte Carlo base case analysis. The point estimate central
tendency value (3 x10-5) and the Monte Carlo base case 50th
percentile value (6 x10-5) are similar. For non-cancer health
hazards, the point estimate RME for fish ingestion (104 for young child)
falls between the 95th and 99th percentiles of the Monte Carlo base
case. The point estimate central tendency hazard index (HI) (12 for
young child) is approximately equal to the 50th percentile of the Monte
Carlo base case HI of 11.  Figures 1 and 2 provide a comparison of
results from the probabilistic analysis with that of the deterministic
risk analysis for cancer risks and non-cancer health hazards.

Management Considerations. Early and continued involvement of the
community improved public acceptance of the results. In addition,
careful consideration of the methods used to present the probabilistic
results to the public lead to greater understanding of the findings.

Document Availability. The final risk assessment was released in
November 2000 (available at   HYPERLINK
"http://www.epa.gov/hudson/reports.htm"
http://www.epa.gov/hudson/reports.htm ).

Contacts. Remedial Project Manager, Alison Hess,<PHONE_NUMBER>; Risk
Assessor, Marian Olsen,<PHONE_NUMBER>

A comparison of results from the probabilistic analysis with that of the
deterministic risk analysis for cancer risks and non-cancer health
hazards.  Figures 1 and 2 plot percentiles for 72 combinations of
exposure variables (e.g., distributions from creel angler surveys;
residence duration; fishing locations; cooking losses, etc.) of the
non-cancer Hazard Index values and the cancer risks, respectively.  In
each of these figures, the variability of cancer risk or non-cancer HIs
for anglers within the exposed population is plotted on the y axis for
particular percentiles within the population.  This variability is a
function of the variations in fish consumption rates, fishing duration,
differences in fish species ingested, etc.  The uncertainty in the
estimates is indicated by the range of either cancer risk or non-cancer
HI values plotted on the x-axis.  This uncertianty is a function of the
72 combinations of the exposure factor inputs examined in the
sensitivity analysis.  This analysis provides a semi-quantitative
confidence interval for the cancer risks and HI values at any
particulate percentile.  As these figures show, the intervals span
somewhat less than two orders of magnitude (e.g., < 100 fold). The
vertical lines indicate the deterministic endpoints.

Figure 2.  Monte Carlo Non-Cancer Hazard Index Summary Based on a
One-Dimensional Probabilistic Risk Analysis of Exposure to
Polychlorinated Biphenyls (PCBs) via Consumption of Fish from a
Contaminated Sediment Site.  From Phase 2 Reprot:  Further Site
Characterization and Analysis.  Volume 2F – Revised Human Health Risk
Assesment, Hudson River PCBs Reassessment RI/FS.  U.S. EPA, November
2000.

Case Study 6: Probabilistic Sensitivity Analysis of Expert Elicitation
of Concentration-Response Relationship Between Particulate Matter
(PM2.5) Exposure and Mortality

In 2002, the National Research Council (NRC) recommended that EPA
improve its characterization of uncertainty in the benefits assessment
for proposed regulations of air pollutants. NRC recommended that
probability distributions for key sources of uncertainty be developed
using available empirical data or through formal elicitation of expert
judgments. In response to this recommendation, EPA conducted an expert
elicitation evaluation of the concentration-response relationship
between PM2.5 exposure and mortality, a key component of the benefits
assessment of the PM2.5 regulation. Further information on the expert
elicitation procedure and results is provided in Case Study 12. To
evaluate the degree to which the results of the assessment depended on
individual experts’ judgments or on the methods of expert elicitation,
a probabilistic sensitivity analysis was performed of the results

concentrations from 12 to 11 μg/m3. The 12 individual distributions of
estimated avoided deaths were then pooled using equal weights to create
a single overall distribution reflecting input from each expert. This
distribution served as the baseline for the sensitivity analysis, which
compared the means and standard deviations of the baseline distribution
with several variants.

Results of Analysis. The first analysis examined sensitivity of the mean
and standard deviation of the overall mortality distribution to the
removal of individual experts’ distributions. In general, the results
suggested a fairly equal split between those experts whose removal
shifted the distribution mean up and those who shifted it down and
relatively modest impacts of individual experts. The standard deviation
of the combined distribution also was not affected strongly by removal
of individual experts. The second analysis evaluated whether the use of
parametric or nonparametric approaches affected the overall results. The
results suggested that the use of parametric distributions led to
distributions with similar or slightly increased uncertainty compared
with distributions provided by experts who offered percentiles of a
nonparametric distribution. The last analyses evaluated whether
participation in the Pre- or Post-elicitation Workshops impacted the
results. Participation in either workshop did not appear to have a
significant effect on experts’ judgments, based on measures of change
in the baseline distribution. Overall, the sensitivity analyses
demonstrated that the assessment was robust, with little dependence on
individual experts’ judgments or on the specific elicitation methods
evaluated.

Management Considerations. The sensitivity analysis demonstrated the
robustness of the PM 2.5 expert elicitation-based assessment by showing
that the panel of experts was generally well balanced and that
alternative elicitation methods would not have markedly altered the
overall results.

Document Availability. The details of this analysis are provided in the
IEC document titled: “Expanded Expert Judgment Assessment of the
Concentration-Response Relationship Between PM2.5 Exposure and
Mortality” Final Report, September 21, 2006 (  HYPERLINK
"http://www.epa.gov/ttn/ecas/regdata/uncertainty/pm_ee_report.pdf"
www.epa.gov/ttn/ecas/regdata/uncertainty/pm_ee_report.pdf ).

The expert elicitation assessment, along with the Regulatory Impact
Analysis (RIA) of the PM2.5 standard, is available at   HYPERLINK
"http://www.epa.gov/ttn/ecas/ria.html"
http://www.epa.gov/ttn/ecas/ria.html .

Contact. Lisa Conner,<PHONE_NUMBER>,   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Case Study 7: Environmental Monitoring and Assessment Program (EMAP):
Using Probabilistic Sampling to Evaluate the Condition of the Nation’s
Aquatic Resources

Monitoring is a key tool used to identify where the environment is in
healthy biological condition and requires protection, and where
environmental problems are occurring and need remediation.  However,
most monitoring is not currently done in a way that allows for
statistically-valid assessments of water quality conditions in
unmonitored waters (GAO 2000).  States thus cannot adequately measure
the status and trends in water quality in their waters as required by
Clean Water Act Section 305(b).

EMAP’s focus has been to develop unbiased statistical survey design
frameworks, and sensitive indicators that allow the condition of aquatic
ecosystems to be assessed at state, regional, and national scales.  A
cornerstone of EMAP has been the use of probabilistic sampling to allow
representative, unbiased, cost-effective condition assessments for
aquatic resources over large areas.  EMAP’s statistical survey methods
are very efficient, requiring relatively few sample locations to make
valid scientific statements about the condition of aquatic resources
over large areas (e.g., the condition of all the wadeable streams in the
Western US).

Probabilistic Analysis. This research program had a number of case
studies using probabilistic sampling designs for different aquatic
resources (estuaries, streams, and rivers). An EMAP probability-based
sampling program provides an unbiased estimate of the condition of an
aquatic resource over a large geographic area from a small number of
samples. The principal characteristics of a probabilistic sampling
design are:  the population being sampled is unambiguously described;
every element in the population has the opportunity to be sampled with a
known probability; and sample selection is carried out by a random
process. This approach allows statistical confidence levels to be placed
on the estimates and provides the potential to detect statistically
significant changes and trends in condition with repeated sampling. In
addition, this approach permits the aggregation of data collected from
smaller areas to predict the condition of a large geographic area.

The EMAP design framework allows the selection of unbiased,
representative sampling sites and specifies the information to be
collected at these sites. The validity of the overall inference rests on
the design and subsequent analysis to produce regionally representative
information. The EMAP uses   HYPERLINK
"http://www.epa.gov/nheerl/arm/documents/presents/grts_ss.pdf"
Generalized Random Tessellation Stratified (GRTS) Spatially-Balanced
Survey Designs for Aquatic Resources .  The spatially-balanced aspect
spreads out the sampling locations geographically, but still ensures
that each element has an equal chance of being selected.

Results of the Analysis.  Data collected using the EMAP approach has
allowed the Agency to make scientifically defensible assessments of the
ecological condition of large geographic areas for reporting to Congress
under CWA 305(b).  The EMAP approach has been used to provide the first
reports on the condition of the nation’s estuaries, streams, rivers
and lakes, and it is scheduled to used for wetlands.  EMAP findings have
been included in EPA’s Report on the Environment, and the Heinz
Center’s The State of the Nation’s Ecosystems.  Data collected
through an EMAP approach improve the ability to assess ecological
progress in environmental protection and restoration, and provide
valuable information for decision-makers and the public. The use of
probabilistic analysis methods allows meaningful assessment and regional
comparisons of aquatic ecosystem conditions across the United States.
Finally, the probabilistic approach provides scientific credibility for
the monitoring network and aids in identifying data gaps.

Management Considerations.  Use of an EMAP approach addresses criticisms
from the General Accounting Office, the National Academies of Sciences
(NAS), the Heinz Center (a nonprofit environmental policy institution),
and others who noted the nation lacked the data to make scientifically
valid characterizations of water quality regionally and across the
United States. The program provides cost-effective, scientifically
defensible, and representative data, to permit the evaluation of
quantifiable trends in ecosystem condition, to support performance-based
management, and to facilitate better public decisions regarding
ecosystem management. EMAP’s approach has now been adopted by the
EPA’s Office of Water (OW) to collect data on the condition of all the
nation’s aquatic resources.  OW, Office of Air and Radiation and
Office of Prevention, Pesticides, and Toxic Substances now have
environmental accountability endpoints using EMAP approaches in their
Agency performance goals.

Document Availability. Available at   HYPERLINK
"http://www.epa.gov/emap/index.html"  http://www.epa.gov/emap/index.html
.

U. S. EPA. 2002. Research Strategy. Environmental Monitoring and
Assessment Program. U.S. EPA, Office of Research and Development,
National Health and Environmental Effects Research Laboratory. U.S. EPA,
Research Triangle Park, NC. Available at   HYPERLINK
"http://www.epa.gov/emap/html/pubs/docs/resdocs/emap_research_strategy.p
df"  www.epa.gov/emap/html/pubs/docs/resdocs/emap_research_strategy.pdf
.

Information on EMAP monitoring designs is available at

http://www.epa.gov/nheerl/arm/designpages/monitdesign/monitoring_design_
info.htm

Contacts. Michael McDonald,<PHONE_NUMBER>,   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>; Tony
Olsen,<PHONE_NUMBER>,   HYPERLINK<EMAIL_ADDRESS><EMAIL_ADDRESS>
Group 3 Case Studies

Case Study 8: Two-Dimensional Probabilistic Risk Analysis of
Cryptosporidium in Public Water Supplies, with Bayesian Approaches to
Uncertainty Analysis

Probabilistic assessment of the occurrence and health effects associated
with Cryptosporidium bacteria in public drinking water supplies was used
to support the economic analysis of the final Long-Term 2 Enhanced
Surface Water Treatment Rule (LT2).  EPA’s Office of Ground Water and
Drinking Water (OGWDW) conducted this analysis and established a
baseline disease burden attributable to Cryptosporidium in Public Water
supplies that use surface water sources.  Next, it models the source
water monitoring and finished water improvements that will be realized
as a result of the Rule.  Post-Rule risk is estimated and the Rule’s
health benefit is the result of subtracting this from the baseline
disease burden.

Probabilistic Risk Analysis.  Probabilistic assessment was used for this
analysis as a means of addressing the variability in the occurrence of
Cryptosporidium in raw water supplies, the variability in the treatment
efficiency, as well as the uncertainty in these inputs and in the
dose-response relationship for Cryptosporidium infection.  This case
study provides an example of a PRA that evaluates both variability and
uncertainty at the same time and is referred to as a two-dimensional
probabilistic risk assessment.    The analysis also included
probabilistic treatments of uncertain dose-response and occurrence
parameters.  Markov Chain Monte Carlo samples of parameter sets filled
this function.  This  Bayesian approach (treating the unknown parameters
as random variables) differs from classical treatments, which would
regard the parameters as unknown, but fixed (Group 3: Advanced PRA).
The risk analysis used existing datasets (e.g., occurrence of
Cryptosporidium and treatment efficacy) to inform the variability of
these inputs.  Uncertainty distributions were characterized based on
professional judgment or by analyzing data using Bayesian statistical
techniques.

Results of Analysis. The risk analysis identified the Cryptosporidium
dose-response relationship as the most critical model parameters in the
assessment, followed by the occurrence of the pathogen and treatment
efficiency.  By simulating implementation of the Rule using imprecise,
biased measurement methods, the assessment provided estimates of the
number of public water supply systems that would require corrective
action and the nature of the actions likely to be implemented.  This
information afforded a realistic measure of the benefits (in reduced
disease burden) expected with the LT2 rule.  In response to SAB
comments, additional Cryptosporidium dose-response models were added to
more fully reflect uncertainty in this element of the assessment.

Management Considerations. The rule underwent external peer review,
review by EPA’s Science Advisory Board (SAB) and intense review by the
Office of Management and Budget (OMB).  Occurrence and dose-response
components of the risk analysis model were communicated to stakeholders
during the Rule’s Federal Advisory Committee Act (FACA) process.  Due
to the rigor of the analysis and the signed FACA “Agreement in
Principle”, the OMB review was straight-forward.

Document Availability. The final assessment of occurrence and exposure
to Cryptosporidium was released in December 2005 (available at
HYPERLINK
"http://www.epa.gov/safewater/disinfection/lt2/regulations.html"
http://www.epa.gov/safewater/disinfection/lt2/regulations.html ).

Contact. Michael Messner,<PHONE_NUMBER>,   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Case Study 9: Two-Dimensional Probabilistic Model of Children’s
Exposure to Arsenic in Chromated Copper Arsenate (CCA) Pressure-Treated
Wood

Probabilistic models were developed in response to EPA’s October 2001
Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) Scientific
Advisory Panel (SAP) recommendations to use probabilistic modeling to
estimate children’s exposures to arsenic from chromated copper
arsenate (CCA) treated playsets and home decks.

Probabilistic Risk Analysis.  EPA’s Office of Research and Development
(ORD), in collaboration with the Office of Pesticide Programs (OPP)
developed and applied a probabilistic exposure assessment of
children’s exposure to arsenic and chromium from contact with
CCA-treated wood playsets and decks.  This case study provides an
example of the use of two-dimensional (i.e., addressing both variability
and uncertainty) probabilistic exposure assessment (Group 3: Advanced
PRA).  The two-dimensional assessment employed a modification of the
ORD’s SHEDS (Stochastic Human Exposure and Dose Simulation) model to
simulate children’s exposure to arsenic and chromium from CCA-treated
wood playsets and decks, and surrounding soil.  Staff from both ORD and
OPP collaborated in the development of the SHEDS-Wood model.

Results of Analysis.  A draft of the probabilistic exposure assessment
received SAP review in December, 2003; the final report was released in
2005.  The results of the probabilistic exposure assessment were
consistent with or in the range of the results of deterministic exposure
assessments conducted by several other organizations.  The model results
were used to compare exposures under a variety of scenarios, including
cold vs. warm weather activity patterns, use of wood sealants to reduce
the availability of contaminants on the surface of the wood, and
different hand-washing frequencies.  The modeling of alternative
mitigation scenarios indicated that the use of sealants could result in
the greatest exposure reduction, while increased frequency of
hand-washing could also reduce exposure.

OPP used the SHEDS-Wood exposure results in their probabilistic
children’s risk assessment for CCA (EPA, 2008).  This included
recommendations for risk reduction (use of sealants and careful
attention to children’s hand-washing) to homeowners with existing CCA
wood structures.  In addition, the exposure assessment was used to
identify areas for further research, including: the efficacy of wood
sealants in reducing dislodgeable contaminant residues, the frequency
with which children play on or around CCA wood, and transfer efficiency
and residue concentrations.  In order to better characterize the
efficacy of sealants in reducing exposure, EPA and the Consumer Product
Safety Commission conducted a 2-year study of how dislodgeable
contaminant residue levels changed with the use of various types of
commercially-available wood sealants.

Management Considerations.  The SHEDS-wood model was one of Agency’s
first probabilistic modeling assessments for regulatory purposes.  The
OPP used SHEDS results directly in their final risk assessment for
children playing on CCA treated playground equipment and decks. The
model enhanced risk assessment and management decisions and prioritized
data needs related to wood preservatives.  The modeling product was
pivotal in the risk management and re-registration eligibility decisions
for CCA, and in advising the public how to minimize health risks from
existing treated wood structures. Industry is also using SHEDS to
estimate exposures to CCA and other wood preservatives. Some states are
using the risk assessment as guidance in setting their regulations for
CCA related playground equipment.

Document Availability.

The model results were included in the final report on the probabilistic
exposure assessment of CCA-treated wood surfaces:

Zartarian, V.G., J. Xue, H. A. Ozkaynak, W. Dang, G. Glen, L. Smith, and
C. Stallings. A Probabilistic Exposure Assessment for Children Who
Contact CCA-treated Playsets and Decks Using the Stochastic Human
Exposure and Dose Simulation Model for the Wood Preservative Scenario
(SHEDS-WOOD), Final Report. U.S. Environmental Protection Agency,
Washington, DC, EPA/600/X-05/009.   HYPERLINK
"http://www.epa.gov/heasd/sheds/cca_treated.htm"
http://www.epa.gov/heasd/sheds/cca_treated.htm

The final probabilistic risk assessment based on the SHEDS-Wood exposure
assessment can be found at:
http://www.epa.gov/oppad001/reregistration/cca/final_cca_factsheet.htm

Results of the sealant studies were released in January, 2007 (available
at   HYPERLINK
"http://www.epa.gov/oppad001/reregistration/cca/index.htm#reviews"
http://www.epa.gov/oppad001/reregistration/cca/index.htm#reviews  ).

The results of the analysis were published as:

Zartarian, V.G., Xue, J., Özkaynak, H., Dang, W., Glen, G., Smith, L.,
and Stallings, C.   A probabilistic arsenic exposure assessment for
children who contact chromated copper arsenate (CAA)-treated playsets
and decks, Part 1:  Model methodology, variability results, and model
evaluation.  Risk Analysis 26:515, 2006.

Contact.  Valerie G. Zartarian, Ph.D., EPA's Office of Research and
Development,<PHONE_NUMBER>,   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>

Case Study 10: Two-Dimensional Probabilistic Exposure Assessment of
Ozone

As part of EPA’s recent review of the ozone National Ambient Air
Quality Standards (NAAQS), the Office of Air Quality Planning and
Standards (OAQPS) conducted detailed probabilistic exposure and risk
assessments in evaluating potential alternative standards for ozone. At
different stages of this review, the Clean Air Scientific Advisory
Committee (CASAC) Ozone Panel (an independent scientific review
committee of EPA’s SAB) and the public reviewed and provided comments
on analyses and documents prepared by EPA. A scope and methods plan for
the exposure and risk assessments was developed in 2005 (EPA, 2005).
This plan was intended to facilitate consultation with CASAC, as well as
public review, and to obtain advice on the overall scope, approaches,
and key issues in advance of the completion of the analyses. This case
study describes the probabilistic exposure assessment, addressing
short-term exposures to ozone. The exposure estimates were used as an
input to the health risk assessment for lung function decrements in all
children and asthmatic school-aged children based on exposure-response
relationships derived from controlled human exposure studies.

Probabilistic Exposure Analysis. Population exposure to ambient ozone
levels was evaluated using EPA’s APEX model, also referred to as the
Total Risk Integrated Methodology/Exposure (TRIM.Expo) model. Exposure
estimates were developed for recent ozone levels, based on 2002 to 2004
air quality data, and for ozone levels simulated to just meet the
existing 0.08 ppm, 8-h ozone NAAQS and several alternative ozone
standards, based on adjusting 2002 to 2004 air quality data. Exposure
estimates were modeled for 12 urban areas located throughout the United
States for the general population, all school-age children, and
asthmatic school-age children. This exposure assessment is described in
a technical report (EPA, 2007b). The exposure model, APEX, is documented
in a user’s guide and technical document (EPA, 2006a,b). A Monte Carlo
approach was used to produce quantitative estimates of the uncertainty
in the APEX model results, based on estimates of the uncertainties for
the most important model inputs. The quantification of model input
uncertainties, a discussion of model structure uncertainties, and the
results of the uncertainty analysis are documented in Langstaff (2007).

Results of Analysis. Uncertainty in the APEX model predictions results
from uncertainties in the spatial interpolation of measured
concentrations, the microenvironment models and parameters, people’s
activity patterns, and, to a lesser extent, model structure. The
predominant sources of uncertainty appear to be the human activity
pattern information and the spatial interpolation of ambient
concentrations from monitoring sites to other locations. The primary
policy-relevant finding was that the uncertainty in the exposure
assessment is small enough to lend confidence to the use of the model
results for the purpose of informing the Administrator’s decision on
the ozone standard.

The following figure illustrates the uncertainty distribution for one
model result, the percent of children with exposures above 0.08 ppm-8hr
while at moderate exertion.  The “point estimate” of 20 percent is
the result from the APEX simulation using the best estimates of the
model inputs. The corresponding result from the Monte Carlo simulations
ranges from 17 to 26 percent, with a 95 percent uncertainty interval
(UI) of 19 to 24 percent.  Note that the uncertainty intervals are not
symmetric since the distributions are skewed.

Management Considerations. The exposure analysis also provided
information on the frequency with which population exposures exceeded
several potential health effect benchmark levels that were identified
based on evaluation of health effects in clinical studies.

The exposure and risk assessments are summarized in Chapters 4 and 5,
respectively, of the Ozone Staff Paper (EPA, 2007a). The exposure
estimates over these potential health effect benchmarks were part of the
basis for the Administrator’s March 27, 2008, decision to revise the
ozone NAAQS from 0.08 to 0.075 ppm, 8-h average (see 73 FR 16436).

Document Availability.

 Langstaff, J. E. (2007). Analysis of Uncertainty in Ozone Population
Exposure Modeling. OAQPS Staff Memorandum to Ozone NAAQS Review Docket
(OAR-2005-0172). Available at   HYPERLINK
"http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_td.html"
http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_td.html .

EPA (2007a). Review of National Ambient Air Quality Standards for Ozone:
Assessment of Scientific and Technical Information - OAQPS Staff Paper.
OAQPS, U.S. EPA, RTP, NC. Available at   HYPERLINK
"http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_sp.html"
http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_sp.html .

EPA (2007b). Ozone Population Exposure Analysis for Selected Urban
Areas. OAQPS, U.S. EPA, RTP, NC. Available at   HYPERLINK
"http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_td.html"
http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_td.html .

EPA (2006a,b). Total Risk Integrated Methodology (TRIM) - Air Pollutants
Exposure Model Documentation (TRIM.Expo / APEX, Version 4) Volume I:
User’s Guide; Volume II: Technical Support Document. OAQPS, U.S. EPA,
RTP, NC. June 2006. Available at   HYPERLINK
"http://www.epa.gov/ttn/fera/human_apex.html"
http://www.epa.gov/ttn/fera/human_apex.html .

EPA (2005). Ozone Health Assessment Plan: Scope and Methods for Exposure
Analysis and Risk Assessment. OAQPS, U.S. EPA, RTP, NC. Available at
HYPERLINK "http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_pd.html"
 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_pd.html .

Contact. John E. Langstaff, EPA’s Office of Air and Radiation,
919-541-1449,   HYPERLINK<EMAIL_ADDRESS><EMAIL_ADDRESS>
Case Study 11: Analysis of Microenvironmental Exposures to Particulate
Matter (PM2.5) for a Population Living in Philadelphia, PA

This case study used the Stochastic Human Exposure and Dose Simulation
model for particulate matter (SHEDS-PM) developed by EPA’s National
Exposure Research Laboratory (NERL) to prepare a probabilistic
assessment of population exposure to particulate matter (PM) in
Philadelphia, PA.  This case study simulation was prepared to accomplish
three goals:  1) to estimate the contribution of PM of ambient (outdoor)
origin to total PM exposure, 2) to determine the major factors that
influence personal exposure to PM, and 3) to identify factors that
contribute the greatest uncertainty to model predictions.

Probabilistic Risk Analysis.  The two-dimensional probabilistic
assessment used a microexposure event technique to simulate individual
exposures to PM in specific microenvironments (outdoors, indoor
residence, office school, store, restaurant or bar, and in a vehicle).
The assessment used spatially and temporally interpolated ambient PM2.5
measurements from 1992-93 and 1990 U.S. Census data for each census
tract in Philadelphia.  This case study provides an example of both
two-dimensional (variability and uncertainty) probabilistic assessment
and microexposure event assessment (Group 3: Advanced PRA).

Results of Analysis.  Results of the analysis showed that that human
activity patterns did not have as strong an influence on ambient PM2.5
exposures as was observed for exposure to indoor PM2.5 sources.
Exposure to PM2.5 of ambient origin contributed a significant percent of
the daily total PM2.5 exposures, especially for the segment of the
population without exposure to environmental tobacco smoke in the
residence. Development of the SHEDS-PM model using the Philadelphia
PM2.5 case study also provided useful insights into data needs for
improving inputs to the SHEDS-PM model, reducing uncertainty and further
refinement of the model structure.  Some of the important data needs
identified from the application of the model include: daily PM
measurements over multiple seasons and across multiple sites within an
urban area, improved capability of dispersion models to predict ambient
PM concentrations at greater spatial resolution and over a year time
period, measurement studies to better characterize the physical factors
governing infiltration of ambient PM2.5 into residential
microenvironments, further information on particle-generating sources
within the residence, and data for the other indoor microenvironments
not specified in the model.

Management Considerations:  The application of the SHEDS-PM model to the
Philadelphia population gave insights into data needs and areas for
model refinement.  The continued development and evaluation of the
SHEDS-PM population exposure model are being conducted as part of
EPA/ORD’s effort to develop a source-to-dose modeling system for PM
and air toxics. This type of exposure and dose modeling system is
considered to be important for scientific and policy evaluation of the
critical pathways, as well as exposure factors and source types
influencing human exposures to a variety of environmental pollutants,
including particulate matter.

Document Availability.

The assessment is available at   HYPERLINK
"http://www.epa.gov/heasd/pm/pdf/exposure-model-for-pm.pdf"
http://www.epa.gov/heasd/pm/pdf/exposure-model-for-pm.pdf .

The results of the analysis were published as:

Burke, J., Zufall, M., and Özkaynak, H.  A population exposure model
for particulate matter: Case study results for PM2.5 in Philadelphia,
PA.  Journal of Exposure Analysis and Environmental Epidemiology 11:
470, 2001.

Georgepoulos,  P.G., Wang, S.-W., Vyas, V.M., Sun, Q., Burke, J.,
Vedantham, R., McCurdy, T., and Ozkayanak, H.  A source-to-dose
assessment of poulation exposure to fine PM and ozone in Philadelphia,
PA, during a summer 1999 episode.  J. Expos. Analysis and Environm. Epi.
1-19, 2005.

Contact:.  Janet M. Burke, EPA’s Office of Research and Development,
919-541-0820,   HYPERLINK<EMAIL_ADDRESS><EMAIL_ADDRESS>
Case Study 12: Probabilistic Analysis in Cumulative Risk Assessment of
Organophosphorus Pesticides

In 1996, Congress enacted the Food Quality Protection Act (FQPA), which
requires EPA to consider “available evidence concerning the cumulative
effects on infants and children of such residues and other substances
that have a common mechanism of toxicity” when setting pesticide
tolerances (i.e., the maximum amount of pesticide residue legally
allowed to remain on food products). FQPA also mandated that EPA
completely reassess the safety of all existing pesticide tolerances
(those in effect since August 1996) to ensure that they are supported by
up-to-date scientific data and meet current safety standards. Because
organophosphorus pesticides (OPs) were assigned priority for tolerance
reassessment, these pesticides were the first “common mechanism”
group identified by EPA’s OPP. The ultimate risk management goal
associated with this cumulative risk assessment (CRA) was to establish
safe tolerance levels for this group of pesticides, while meeting the
FQPA standard for protecting infants and children.

Probabilistic Risk Analysis. This case study provides an example of an
advanced probabilistic assessment (Group 3). In 2006, EPA analyzed
exposures to 30 OPs through food consumption, consumption of drinking
water, and exposure via pesticide application. EPA used Calendex, a
probabilistic computer software program (available at   HYPERLINK
"http://www.exponent.com/calendex_software"
http://www.exponent.com/calendex_software ), to integrate various
pathways, while simultaneously incorporating the time dimensions of the
input data. Based on the results of the exposure assessment, EPA
calculated margins of exposure (MOEs) for the total cumulative risk from
all pathways.

The food component of the OPs CRA was highly refined, as it was based on
residue monitoring data from the USDA’s PDP and supplemented with
information from the FDA’s Surveillance Monitoring Programs and Total
Diet Study. The residue data were combined with actual consumption data
from USDA’s Continuing Survey of Food Intakes by Individuals using
probabilistic techniques. The CRA evaluated drinking water exposures on
a regional basis. The assessment focused on areas where combined OP
exposure is likely to be highest within each region. Primarily, the
analysis used probabilistic modeling to estimate the co-occurrence of OP
residues in water. Monitoring data were not available with enough
consistency to be the sole basis for the assessment; however, they were
used to corroborate the modeling results. Data sources for the water
component of the assessment included USDA Agricultural Usage Reports for
Field Crops, Fruits, and Vegetables; USDA Typical Planting and
Harvesting Dates for Field Crops and Fresh Market and Processing
Vegetables; local sources for refinements; and monitoring studies from
the U.S. Geological Survey and other sources. Finally, exposure via the
oral, dermal, and inhalation routes resulting from applications of OPs
in and around homes, schools, offices, and other public areas were
assessed probabilistically for each of the seven regions. The data
sources for this part of the assessment included information from
surveys and task forces, special studies and reports from published
scientific literature, EPA’s Exposure Factors Handbook (USEPA, 1997),
and other sources.

Results of Analysis. The OPs CRA presented potential risk from
single-day (acute) exposures across a year and from a series of 21-day
rolling averages across the year. MOEs at the 99.9th percentile were
approximately 100 or greater for all populations for the 21-day average
results. The only exception is a brief period (roughly 2 weeks) in which
drinking water exposures resulting from phorate use on sugarcane result
in MOEs near 80 for children aged 1 to 2 years. Generally, exposures
through the food pathway dominated total MOEs, and exposures through
drinking water were substantially lower throughout most of the year.
Residential exposures were substantially smaller than exposures through
both food and drinking water.

The OPs CRA was very resource intensive. Work began in 1997 with the
preparation of guidance documents and the development of a CRA
methodology. Over 2 to 3 years, more than 25 people spent 50 to 100% of
their time working on the assessment, with up to 50 people working on
the CRA at critical periods. EPA has spent approximately $1 million on
this assessment (e.g., for computers, models, and contractor support).

Management Considerations.  The OP CRA was a landmark demonstration of
the feasibility of a regulatory level assessment of the risk from
multiple chemicals.  On its completion, EPA presented the CRA at
numerous public technical briefings and FIFRA SAP meetings, and made all
of the data inputs available to the public. OPP’s substantial effort
to communicate methodologies, approaches, and results to the
stakeholders aided in the success of the OPs CRA. The stakeholders
expressed appreciation for the transparent nature of the OPs CRA and
recognized the innovation and hard work that went into developing the
assessments.

Document Availability. The 2006 assessment and related documents are
available at   HYPERLINK
"http://www.epa.gov/pesticides/cumulative/common_mech_groups.htm#op"
http://www.epa.gov/pesticides/cumulative/common_mech_groups.htm#op .

Contact.  David Miller,<PHONE_NUMBER>;   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>.

Case Study 13: Probabilistic Ecological Effects Risk Assessment Models
for Evaluating Pesticide Uses

As part of the process of developing and implementing a probabilistic
approach for ecological risk assessment, an illustrative case was
completed in 1996. The illustrative case involved both deterministic and
probabilistic risk analysis for effects of a hypothetical chemical X on
birds and aquatic species. Following the recommendations of the SAP and
in response to issues raised by OPP risk managers, the Agency began an
initiative to refine the ecological risk assessment process for
evaluating the effects of pesticides to terrestrial and aquatic species
within the context of FIFRA, the main statutory authority for regulating
pesticides at the Federal level. Among the key goals and objectives of
EPA’s initiative were to:

incorporate probabilistic tools and methods to provide an estimate on
the magnitude and probability of effects;

build on existing data requirements for registration;

utilize, wherever possible, existing databases and create new ones from
existing data sources to minimize the need to generate additional data;
and

focus additional data requirements on reducing uncertainty in key areas.

After proposing a four-level risk assessment scheme, with higher levels
reflecting more refined risk assessment techniques, the Agency developed
pilot models for both terrestrial and aquatic species. Refined risk
assessment models (Level II) were then developed and used in a generic
chemical case study that was presented to the SAP in 2001.

Probabilistic Analysis. This case study describes an advanced
probabilistic model for ecological effects of pesticides (Group 3). The
terrestrial Level II model (version 2.0) is a multimedia
exposure/effects model that evaluates acute mortality levels in generic
or specific avian species over a user-defined exposure window. The
spatial scale is at the field level, which includes the field and
surrounding area. Both areas are assumed to meet the habitat
requirements for each species, and contamination of edge or adjacent
habitat from drift is assumed to be zero. For each individual bird
considered in a run of the Level II model, a random selection of values
is made for the major exposure input parameters to estimate an external
oral dose equivalent for that individual. The estimated dose equivalent
is compared to a randomly assigned tolerance for the individual
preselected from the dose/response distribution. If the dose is greater
than the tolerance, the individual is scored “dead,” if not, the
individual is scored “not dead.” After multiple iterations of
individuals, a probability density function of percent mortality is
generated.

May 29-31, 1996, the Agency presented two ecological risk assessment
case studies to SAP for review and comment. Although recognizing and
generally reaffirming the utility of EPA’s current deterministic
assessment process, SAP offered a number of suggestions for improvement.
Foremost among their suggestions was a recommendation to move beyond the
existing deterministic assessment approach by developing the tools and
methodologies necessary to conduct a probabilistic assessment of
effects. Such an assessment would estimate the magnitude and probability
of the expected impact and define the level of certainty and variation
involved in the estimate, information that risk managers within EPA’s
OPP also had requested in the past.

The aquatic Level II model is a two-dimensional Monte Carlo risk model
consisting of three main components: (1) exposure, (2) effects, and (3)
risk. The exposure scenarios used at Level II are intended to provide
estimates for vulnerable aquatic habitats across a wide range of
geographical conditions under which a pesticide product is used. The
Level II risk evaluation process yields estimates of likelihood and
magnitude of effects for acute exposures. For the estimate of acute
risks, a distribution of estimated exposure and a distribution of lethal
effects are combined through a two-dimensional Monte Carlo analysis to
obtain a distribution of joint probability functions. For the estimate
of chronic risks, a distribution of exposure concentrations is compared
to a chronic measurement endpoint. The risk analysis for chronic effects
provides information only on the probability that the chronic end point
assessed will be exceeded, not on the magnitude of the chronic effect
expected.

Results of Analysis. As part of the process of developing and
implementing a probabilistic approach for ecological risk assessment, a
case study was completed. The case study involved both deterministic and
probabilistic risk analyses for effects of ChemX on birds and aquatic
species. The deterministic screen conducted on ChemX concluded
qualitatively that it could pose a high risk to both freshwater fish and
invertebrates and showed that PRA was warranted. Based on the
probabilistic analysis, it was concluded that the use of ChemX was
expected to infrequently result in significant freshwater fish
mortalities but routinely result in reduced growth and other chronic
effects in exposed fish. Substantial mortalities and chronic effects to
sensitive aquatic invertebrates were predicted to routinely occur after
peak exposures.

Management Considerations.  In its review of the case study, the FIFRA
SAP congratulated the Agency on the effort made to conduct PRA of
pesticide effects in ecosystems. The panel commented that the approach
had progressed greatly from earlier efforts, and that the intricacy of
the models was surprisingly good, given the time interval in which the
Agency had to complete the task.  Following the case study, the Agency
refined the models based on the SAP comments. In addition, the
terrestrial Level II model was refined to include dermal and inhalation
exposure.

Document Availability. An overview of the models is available at

  HYPERLINK
"http://www.epa.gov/oppefed1/ecorisk/fifrasap/rra_exec_sum.htm#Terrestri
al"
http://www.epa.gov/oppefed1/ecorisk/fifrasap/rra_exec_sum.htm#Terrestria
l .

Contact: Donna Randall,<PHONE_NUMBER>,   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Case Study 14: Expert Elicitation of Concentration-Response
Relationship Between Particulate Matter (PM2.5) Exposure and Mortality

In 2002, the NRC recommended that EPA improve its characterization of
uncertainty in the benefits assessment for proposed regulations of air
pollutants. NRC recommended that probability distributions for key
sources of uncertainty be developed using available empirical data or
through formal elicitation of expert judgments.  A key component of
EPA’s approach for assessing potential health benefits associated with
air quality regulations targeting emissions of PM2.5 and its precursors
is the effect of changes in ambient PM2.5 levels on mortality. Avoided
premature deaths constitute, on a dollars basis, between 85 and 95% of
the monetized benefits reported in EPA’s retrospective and prospective
Section 812A benefit-cost analyses of the Clean Air Act (EPA, 1997 and
1999) and in Regulatory Impact Analysis (RIAs) for rules such as the
Heavy Duty Diesel Engine/Fuel Rule (EPA, 2000) and the Non-road Diesel
Engine Rule (EPA, 2004). In response to the National Research Council
(NRC) recommendation, EPA conducted an expert elicitation evaluation of
the concentration-response relationship between PM2.5 exposure and
mortality.

Probabilistic Risk Analysis. This case study provides an example of the
use of expert elicitation (Group 3) to derive probabilistic estimates of
the uncertainty in one element of a cost-benefit analysis. Expert
elicitation uses carefully structured interviews to elicit from each
expert a best estimate of the true value for an outcome or variable of
interest, as well as their uncertainty about the true value. This
uncertainty is expressed as a subjective probabilistic distribution of
values and reflects each expert’s interpretation of theory and
empirical evidence from relevant disciplines, as well as their beliefs
about what is known and not known about the subject of the study. For
the PM2.5 expert elicitation, the process consisted of development of an
elicitation protocol, selection of experts, development of a briefing
book, conducting elicitation interviews, the use of expert workshops
prior to and following individual elicitation of judgments, and the
expert judgments themselves. The elicitation involved personal
interviews with 12 health experts who have conducted research on the
relationship between PM2.5 exposures and mortality.

The main quantitative question asked each expert to provide a
probabilistic distribution for the average expected decrease in U.S.
annual, adult, and all-cause mortality associated with a 1-μg/m3
decrease in annual average PM2.5 levels. When answering the main
quantitative question, each expert was instructed to consider that the
total mortality effect of a 1-μg/m3 decrease in ambient annual average
PM2.5 may reflect reductions in both short-term peak and long-term
average exposures to PM2.5. Each expert was asked to aggregate the
effects of both types of changes in their answers.

The experts were given the option to integrate their judgments about the
likelihood of a causal relationship or threshold in the
concentration-response function into their own distributions or to
provide a distribution “conditional on” one or both of these
factors.

Results of Analysis. The project team developed the interview protocol
between October 2004 and January 2006. Development of the protocol was
informed by an April 2005 symposium held by the project team, where
numerous health scientists and analysts provided feedback; by detailed
pretesting with independent EPA scientists in November 2005; and by
discussion with the participating experts at a pre-elicitation workshop
in January 2006. The elicitation interviews were conducted between
January and April 2006. Following the interviews, the experts reconvened
for a post-elicitation workshop in June 2006, in which the project team
anonymously shared the results of all experts with the group.

The median estimates for the PM2.5 mortality relationship were generally
similar to estimates derived from the two epidemiological studies most
often used in benefits assessment. However, in almost all cases, the
spread of the uncertainty distributions elicited from the experts
exceeded the statistical uncertainty bounds reported by the most
influential epidemiologic studies, suggesting that the expert
elicitation process was successful in developing more comprehensive
estimates of uncertainty for the PM2.5 mortality relationship. The
uncertainty distributions for PM2.5 concentration-response resulting
from the expert elicitation process were used in the RIA for the revised
NAAQS standard for PM2.5 (promulgated on September 21, 2006). Because
the NAAQS are exclusively health based standards, this RIA played no
part in EPA’s determination to revise the Pm2.5 NAAQS.  Benefits
estimates in the RIA were presented as ranges and included additional
information on the quantified uncertainty distributions surrounding the
points on the ranges, derived from both epidemiological studies and the
expert elicitation results. OMB’s review of the RIA was completed in
March 2007.

Management Considerations.  The NAAQS are exclusively health-based
standards, so these analyses were not used in any manner by EPA in
determining whether to revise the NAAQS for PM2.5.  Moreover, the
request to engage in the expert elicitation did not come from the Clean
Air Scientific Advisory Committee, or CASAC, the official peer review
body for the NAAQS, so that a decision to conduct the analyses does not
reflect CASAC advice that such analyses inform NAAQS determinations.
The analyses addressed comments from the National Research Council that
recommended that probability distributions for key sources of
uncertainty be addressed. The analyses were used in EPA’s
retrospective and prospective Section 812A benefit-cost analyses of the
Clean Air Act (EPA, 1997 and 1999) and in RIAs for rules such as the
Heavy Duty Diesel Engine/Fuel Rule (EPA, 2000) and the Non-road Diesel
Engine Rule (EPA, 2004). In response to the NRC recommendation, EPA
conducted an expert elicitation evaluation of the concentration-response
relationship between PM2.5 exposure and mortality.

Document Availability. The assessment is available at   HYPERLINK
"http://www.epa.gov/ttn/ecas/ria.html"
http://www.epa.gov/ttn/ecas/ria.html .

Contact: Lisa Conner,<PHONE_NUMBER>,   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Case Study 15: Expert Elicitation of Sea-Level Rise Resulting from
Global Climate Change

The United Nations Framework Convention on Climate Change requires
nations to implement measures for adapting to rising sea level and other
effects of changing climate. To decide on an appropriate response,
coastal planners and engineers weigh the cost of these measures against
the likely cost of failing to prepare, which depends on the probability
of the sea rising a particular amount. The U.S. National Academy of
Engineering recommended that assessments of sea level rise should
provide probability estimates. Coastal engineers regularly employ
probability information when designing structures for floods, and courts
use probabilities to determine the value of land expropriated by
regulations. This case study describes the development of a probability
distribution for sea level rise, using models employed by previous
assessments, as well as the expert opinions of 20 climate and glaciology
reviewers about the probability distributions for particular model
coefficients.

Probabilistic Analysis. This case study provides an example both of an
analysis describing the probability of sea level rise, as well as an
expert elicitation of the likelihood of particular models and
probability distributions of the coefficients used by those models to
predict future sea level rise (Group 3). The assessment of the
probability of sea level rise used existing models describing the change
in five components of sea level rise associated with
greenhouse-gas-related climate change (thermal expansion, small
glaciers, polar precipitation, melting and ice discharge from Greenland,
ice discharge from Antarctica). To provide a starting point for the
expert elicitation, initial probability distributions were assigned to
each model coefficient based on the published literature.

Once the initial probabilistic assessment was completed, the draft
report was circulated to expert reviewers considered most qualified to
render judgments on particular processes for revised estimates of the
likelihood of particular models and the model coefficients’
probability distributions. Experts representing both extremes of climate
change science (those who predicted trivial consequences and those who
predicted catastrophic effects; those whose thinking had been excluded
from previous assessments) were included. The experts were asked to
provide subjective assessments of the probabilities of various models
and model coefficients. These subjective probability estimates were used
in place of the initial probabilities in the final model simulations.
Different reviewer opinions were not combined to produce a single
probability distribution for each parameter, but, rather, each
reviewer’s opinions were used in independent iterations of the
simulation. The group of simulations resulted in the probability
distribution of sea level rise.

Results of Analysis. The analysis, completed with a budget of $100,000,
provided a probabilistic estimate of sea level rise for use by coastal
engineers and regulators. The results suggested that there is a 65%
chance that sea level will rise 1 mm/year more rapidly in the next 30
years than it has been rising in the last century. Under the assumption
that nonclimatic factors do not change, the projections suggested that
there is a 50% chance that global sea level will rise 45 cm, and a 1%
chance of a 112-cm rise by the year 2100. The median prediction of sea
level rise was similar to the midpoint estimate of 48 cm published by
the Intergovernmental Panel on Climate Change (IPCC, 2006) shortly
thereafter.. The report also found a 1% chance of a 4-5 meter rise over
the next two centuries.

Management Considerations:  Both reports (EPA 1995; Titus and Narayanan
1996) discuss several uses of the results of this study.  By providing a
probabilistic representation of sea level rise, the assessment allows
coastal residents to make decisions with recognition of the uncertainty
associated with predicted change.   Rolling easements that vest when the
sea rises to a particular level can be properly valued in both
arms-length transaction sales or when calculating the allowable
deduction for a charitable contribution of the easement to a
conservancy.   Cost-benefit assessments of alternative infrastructure
designs—which already consider flood probabilities—can also consider
sea level rise uncertainty.  Assessments of the benefits of preventing
an acceleration of sea level rise can more readily include
low-probability outcomes, which can provide a better assessment of the
true risk when the damage function is nonlinear, which is often the
case.

Document Availability.

 EPA 1995.   The Probability of Sea Level Rise.    Washington, D.C.:
Climate Change Division.

http://epa.gov/climatechange/effects/coastal/slrmaps_probability.html

IPCC (1996).   Climate Change 1995: The Science of Climate Change.
Contribution of Working Group I to the Second Assessment of the
Intergovernmental Panel on Climate Change.

  HYPERLINK "http://uk.cambridge.org/"  Cambridge University Press ,
Cambridge CB2 2RU ENGLAND.

Titus, J. G. and V Narayanan.  1996.    HYPERLINK
"http://epa.gov/climatechange/effects/coastal/Risk_of_rise.html"  The
Risk of Sea Level Rise: A Delphic Monte Carlo Analysis in which Twenty
Researchers Specify Subjective Probability Distributions for Model
Coefficients within their Respective Areas of Expertise  - Climatic
Change, 33: 151-212 (1996).
http://epa.gov/climatechange/effects/coastal/Risk_of_rise.html

Contact:  James G. Titus,<PHONE_NUMBER>;   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Case Study 16: Expert Elicitation for Bayesian Belief Network Model of
Stream Ecology

The identification of the causal pathways leading to stream impairment
is a central challenge to  understanding ecological relationships.
Bayesian belief networks (BBNs) are a promising tool for modeling
presumed causal relationships, providing a modeling structure within
which different factors describing the ecosystem can be causally linked,
and uncertainties expressed for each linkage.

BBNs can be used to model complex systems that involve several
interdependent or interrelated variables. In general, a BBN is a model
that evaluates situations where some information is already known, and
incoming data are uncertain or partially unavailable. The information is
depicted with influence diagrams that present a simple visual
representation of a decision problem, for which quantitative estimates
of effect probabilities are assigned. As such, BBNs have the potential
for representing ecological knowledge and uncertainty in a format that
is useful for predicting outcomes from management actions or for
diagnosing the causes of observed conditions. Generally, specification
of a BBN can be performed using available experimental data, literature
review information (secondary data), and expert elicitation. Attempts to
specify a BBN for the linkage between fine sediment load and
macroinvertebrate populations using data from literature reviews have
failed because of the absence of consistent conceptual models and lack
of quantitative data or summary statistics needed for the model. In
light of these deficiencies, an effort was made to use expert
elicitation to specify a BBN for the relationship between fine sediment
load resulting from human activity and populations of
macroinvertebrates. The goals of this effort were to examine whether
BBNs might be useful for modeling stream impairment and to assess
whether expert opinion could be elicited to make the BBN approach useful
as a management tool.

Probabilistic Risk Analysis. This case study provides an example of
expert elicitation in the development of a BBN model of the effect of
increased fine sediment load in a stream on macroinvertebrate
populations (Group 3). For the purpose of this study, a stream setting
(a Midwestern, low-gradient stream) and a scenario of impairment
(introduction of excess fine sediment) were used. Five stream ecologists
with experience in the specified geographic setting were invited to
participate in an elicitation workshop. An initial model was depicted
using influence diagrams, with the goals of structuring and specifying
the model using expert elicitation. The ecologists were guided through a
knowledge elicitation session in which they defined factors that
described relevant chemical, physical, and biological aspects of the
ecosystem. The ecologists then described how these factors were
connected and were asked to provide subjective, quantitative estimates
of how different attributes of the macroinvertebrate assemblage would
change in response to increased levels of fine sediment. Elicited input
was used to restructure the model diagram and to develop probabilistic
estimates of the relationships among the variables.

Results of Analysis. The elicited input was compiled and presented in
terms of the model as structured by the stream ecologists and their
model specifications. The results were presented both as revised
influence diagrams and with Bayesian probabilistic terms representing
the elicited input. The study yielded several important lessons. Among
these were the elicitation process takes time (including an initial
session by teleconference as well as a 3-day workshop), defining a
scenario with an appropriate degree of detail is critical, and
elicitation of complex ecological relationships is feasible.

Management Considerations. The study was considered successful for
several reasons. First, the feasibility of the elicitation approach to
building stream ecosystem models was demonstrated. The study also
resulted in the development of new graphical techniques to perform the
elicitation. The elicited input was interpreted in terms of predictive
distributions to support fitting a complete Bayesian model. Further, the
study was successful in bringing together a group of experts in a
particular subject area for the purpose of sharing information and
learning about expert elicitation in support of model building. The
exercise provided insights into how best to adapt knowledge elicitation
methods to ecological questions and informed the assembled stream
ecologists on the elicitation process and on the potential benefits of
this modeling approach. The explicit quantification of uncertainty in
the model not only enhances the utility of the model predictions but
also can help guide future research.

Document Availability. Black, P and Stockton, T. 2005. Using Knowledge
Elicitation to Inform a Bayesian Belief Network Model of a Stream
Ecosystem. Neptune and Company, Inc. July.

Yuan, L. TI: A Bayesian Approach for Combining Data Sets to Improve
Estimates of Taxon Optima, AGU, 86(18), Jt. Assem. Suppl., Abstract #
NB41E-04, 2005.

Contact:  Lester Yuan,<PHONE_NUMBER>,   HYPERLINK
<EMAIL_ADDRESS><EMAIL_ADDRESS>

4. References to Case Studies

Jamieson, D. (1996). Scientific uncertainty and the political process.
Ann. Am. Acad. Pol. Soc. Sci., 545:35-43.

Kurowicka, D. and Cooke, R. (2006). “Uncertainty Analysis With High
Dimensional Dependent Modeling.” Wiley Series in Probability and
Statistics, May, 2006.

Stahl, C.H., Cimorelli, A.J. (2005). How much uncertainty is too much
and how do we know? A case example of the assessment of ozone monitor
network options. Risk Anal, 25:1109-1120.

U. S. Environmental Protection Agency (EPA). (1992a). Guidelines for
exposure assessment. EPA 600Z-92/001. Risk Assessment Forum, Washington,
DC, 170 pp. (  HYPERLINK
"http://cfpub.epa.gov/ncea/raf/recordisplay.cfm?deid=559070"
http://cfpub.epa.gov/ncea/raf/recordisplay.cfm?deid=559070 ).

U. S. Environmental Protection Agency (EPA). (1992b). Framework for
ecological risk assessment. EPA/630/R-92/001. Washington, DC.

U. S. Environmental Protection Agency (EPA). (1995a). Policy for risk
characterization. Science Policy Council, Washington, DC (  HYPERLINK
"http://www.epa.gov/osp/spc/rcpolicy.htm"
http://www.epa.gov/osp/spc/rcpolicy.htm ).

U. S. Environmental Protection Agency (EPA). (1995b). Policy on
evaluating health risks to children. Science Policy Council, Washington,
DC (  HYPERLINK "http://www.epa.gov/osp/spc/memohlth.htm"
http://www.epa.gov/osp/spc/memohlth.htm ).

U. S. Environmental Protection Agency (EPA). (1997a). Policy for use of
probabilistic analysis in risk assessment at the U.S. Environmental
Protection Agency. Fred Hansen, Deputy Administrator. Science Policy
Council, Washington, DC (  HYPERLINK
"http://www.epa.gov/osp/spc/probpol.htm"
http://www.epa.gov/osp/spc/probpol.htm ).

w

x

y

z

.

/

8

9

L

q

x

y

z

/

8

9

ꐔx␷㠀$⑈愀Ĥ摧塉		̀Ĥ␱愀Ĥ摧塉

j¸

j;

j¬

j/

j

㜀$␸䠀$

h

㜀$␸䠀$

਀&䘋

h

ЀS摧৻G

ž

Ÿ

hþv

 h*@

摧৻G

h*@

h0

 h0

h0

hþv

B*

h

  h

h

h

–

—

™

«

a

y

˜

™

␃愁Ĥ摧⶙ÿᔀ™

«

¬

摧⶙ÿᘀ«

¬

±

º

 hÿ

	㜀$␸䠀$摧⶙ÿ

x

‚

OJ

QJ

摧⶙ÿ	㜀$␸䠀$摧⶙ÿ

^J

01. Risk Assessment Forum, Office of Research and Development,
Washington, DC.

U. S. Environmental Protection Agency (EPA). (1997c). Guidance on
cumulative risk assessment. Part 1: Planning and scoping. Science Policy
Council, Washington, DC (  HYPERLINK
"http://www.epa.gpv/osp/spc/cumrisk2.htm"
http://www.epa.gpv/osp/spc/cumrisk2.htm ).

U. S. Environmental Protection Agency (EPA). (1998). Guidelines for
ecological risk assessment. EPA/630/R-95/002F. Risk Assessment Forum,
Washington, DC, 171 pp. (  HYPERLINK
"http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12460"
http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12460 ).

U. S. Environmental Protection Agency (EPA). (2000). Science Policy
Council Handbook: Risk Characterization Handbook. EPA 100-B00-002.
Science Policy Council, Washington, DC, December (  HYPERLINK
"http://www.epa.gov/osp/spc/rchandbk.pdf"
http://www.epa.gov/osp/spc/rchandbk.pdf ).

U. S. Environmental Protection Agency (EPA). (2001). Risk assessment
guidance for Superfund: Volume III—Part A, Process for conducting
probabilistic risk assessment. EPA 540-R-02-002. Office of Emergency and
Remedial Response, Washington, DC, December (  HYPERLINK
"http://www.epa.gov/superfund/programs/risk/rags3a/index.htm"
http://www.epa.gov/superfund/programs/risk/rags3a/index.htm ).

U. S. Environmental Protection Agency (EPA). (2004). An Examination of
EPA Risk Assessment Principles and Practices. U.S. EPA, Office of the
Science Advisor, Office of Research and Development, Washington, DC.
EPA/100/B-04/001 March 2004 (  HYPERLINK
"http://www.epa.gov/OSA/ratf.htm"  http://www.epa.gov/OSA/ratf.htm ).

List of Acronyms and Abbreviations

APEX		Air Pollutants Exposure Model

BBN		Bayesian belief network

CASAC	Clean Air Scientific Advisory Committee

CCA		chromated copper arsenate

CRA		cumulative risk assessment

CSFII		Continuing Survey of Food Intake by Individuals

DEEM		Dietary Exposure Evaluation Model

DRES		Dietary Risk Evaluation System

Eco		ecological risk assessment

EMAP		Environmental Monitoring and Assessment Program

EPA		U.S. Environmental Protection Agency

FACA		Federal Advisory Committee Act

FDA		Food and Drug Administration

FIFRA		Federal Insecticide, Fungicide, and Rodenticide Act

FQPA		Food Quality Protection Act

HH		human health

LT		long-term

MOEs		margins of exposure

NAAQS	National Ambient Air Quality Standards

NRC		National Research Council

OAQPS	Office of Air Quality Planning and Standards

OAR		Office of Air and Radiation

OGWDW	Office of Groundwater and Drinking Water

OMB		Office of Management and Budget

OP		organophosphorous pesticide

ORD		Office of Risk Analysis

OW		Office of Water

PCB		polychlorinated biphenyl

PDP		Pesticide Data Program

PM		particulate matter

PRA		probabilistic risk analysis

RIA		Regulatory Impact Analysis

SAB		Science Advisory Board

SAP		Scientific Advisory Panel

SHEDS	Stochastic Human Exposure and Dose Simulation

TRIM		Total Risk Integrated Methodology

TRIM.Expo	Total Risk Integrated Methology/Exposure Model

UI		Uncertainty Interval

USDA		U.S. Department of Agriculture

EXTERNAL REVIEW DRAFT – DO NOT CITE OR QUOTE

 PAGE

  PAGE  i

DRAFT INTERNAL TO EPA RISK ASSESSMENT FORUM DO NOT CITE OR QUOTE v
SAVEDATE  \@ "M/d/yyyy h:mm:ss am/pm"  \* MERGEFORMAT  6/18/2009 9:24:00
AM

 PAGE

 PAGE   37

DRAFT INTERNAL TO EPA DO NOT CITE OR QUOTE

Levels of Analyses

Sensitivity analysis

Monte Carlo analysis of variability in

Exposure data

Human health or ecological effect data

Monte Carlo analysis of uncertainty

“Cumulative” PRA(multi-pathway or multi-chemical

Two-dimensional PRA of uncertainty and variability

Decision uncertainty analysis

Geospatial analysis

Expert elicitation

Central Tendency

 Value

RME Value

Range of Non-Cancer Hazard Index (HI) Estimates for Fish Ingestion

Fraction of Anglers with

HI (  Indicated Value

Central Tendency

 Value

RME

Value

Range of Cancer Risk Estimates

for Fish Ingestion

Fraction of Anglers with

Risk ( than Indicated Value