Appendix A

Table A1. Respondents and Other Assumptions

Description	Number	Source

Annual Number of Construction Starts > 1 acre	84,472 	Table 1.
Distribution of Model Projects for each Model Project Size and Duration
Category for three Project Types (Residential Construction,
Nonresidential Construction, and Transportation). [Construction ELG
Technical Development Document]

Annual Number of Construction Starts 1 - 4.99 acres	47,969 	Table 1.
Distribution of Model Projects for each Model Project Size and Duration
Category for three Project Types (Residential Construction,
Nonresidential Construction, and Transportation). [Construction ELG
Technical Development Document]

Annual Number of Construction Starts > 5 acres	36,503 	Table 1.
Distribution of Model Projects for each Model Project Size and Duration
Category for three Project Types (Residential Construction,
Nonresidential Construction, and Transportation). [Construction ELG
Technical Development Document]

Total Number of Small MS4s Issued by EPA	361 	Stormwater Rulemaking MS4
Analysis - EPA

Total Number of Small MS4s Issued by Authorized States	6,271 	Stormwater
Rulemaking MS4 Analysis - EPA

Total Number of Industrial Facilities eligible for the "no exposure" to
stormwater waiver from Phase I regulations.	36,377 	Economic Analysis of
the Final Phase II Stormwater Rule (10/01/1999) as cited in NPDES
Stormwater Program Phase II (EPA ICR No 1820.04, OMB Control No.
2040-0211) corrected by Jack Faulk, personal communication. Th
04/08/2008: "I believe the no exposure number is five times higher than
our current thinking.  I would use 36377 as the number of sites covered
by a no exposure certification." Reaffirmed by Brian Rittenhouse
9/22/2011.

Total Number of Existing Major NPDES Permits - Municipal	4,123 	PCS or
ICIS Query: Standard (individual) NPDES Municipal

Total Number of Existing Minor NPDES Permits - Municipal	9,471 	PCS or
ICIS Query: Standard (individual) NPDES Municipal

Total Number of Existing Sludge Only NPDES Permits - Municipal	954 	PCS
or ICIS Query: Sludge Only Municipal

Total Number of Existing Major NPDES Permits - Nonmunicipal	2,036 	PCS
or ICIS Query: Standard (individual) NPDES Nonmunicipal

Total Number of Existing Minor NPDES Permits - Nonmunicipal	28,133 	PCS
or ICIS Query: Standard (individual) NPDES Nonmunicipal

Total Number of Existing Sludge Only NPDES Permits - Nonmunicipal	329
PCS or ICIS Query: Sludge Only Nonmunicipal

Total Number of Existing Major NPDES Permits - Federal	80 	PCS or ICIS
Query: Standard (individual) NPDES Federal

Total Number of Existing Minor NPDES Permits - Federal	510 	PCS or ICIS
Query: Standard (individual) NPDES Federal

Annual Number of New Major NPDES Permits - Nonmunicipal Industrial	19
PCS or ICIS Query: New STANDARD NPDES in the previous 3 years
Nonmunicipal

Annual Number of New Minor NPDES Permits - Nonmunicipal Industrial	785
PCS or ICIS Query: New STANDARD NPDES in the previous 3 years
Nonmunicipal

Annual Number of New NPDES Permits - Nonmunicipal Stormwater	86 	PCS or
ICIS Query: New STORMWATER NPDES in last 3 years Nonmunicipal

Total Number of Existing NPDES Permits - Nonmunicipal Industrial	30,169
PCS or ICIS Query: Standard (individual) NPDES Nonmunicipal

Total Number of Existing NPDES Permits - Nonmunicipal Stormwater
(individual Permit)	783 	PCS or ICIS Query: STORMWATER (individual)
NPDES Nonmunicipal

Total Number of NPDES General Permits Issued by EPA (Not permittees
covered by general permits, but general permits)	27 	PCS or ICIS Query:
General Permits (not permittees)

Total Number of NPDES General Permits Issued by Authorized States (Not
permittees covered by general permits, but general permits)	497 	PCS or
ICIS Query: General Permits (not permittees)

Total Number of Phase I MS4s Permittees (Issued by States)	758
Stormwater Rulemaking MS4 Analysis - EPA

Total Number of Phase I MS4s Permittees (Issued by EPA)	12 	MS4 Program
Implementation - Status of Permit Issuance

Total Number of Phase I MS4s Permits (Issued by States)	171 	Stormwater
Rulemaking MS4 Analysis - EPA

Total Number of Phase I MS4s Permits (Issued by EPA)	5 	MS4 Program
Implementation - Status of Permit Issuance

Annual Number of New Permits-Major	89 	PCS or ICIS Query: New Standard
(individual) NPDES in the previous 3 years

Annual Number of New Permits-Minor	1,518 	PCS or ICIS Query: New
Standard (individual) NPDES in the previous 3 years

Annual Number of New State Issued Permits-General	15 	PCS or ICIS Query:
New general permits (not permittees) in the previous 3 years

Total Number of Existing POTWs  ≽1.0 mgd	3,958 	PCS or ICIS Query: SIC
4952 POTW

Total Number of Existing POTWs  0.1 - 1.0 mgd	5,354 	PCS or ICIS Query:
SIC 4952 POTW

Total Number of Existing POTWs  ≺0.1 mgd	3,020 	PCS or ICIS Query: SIC
4952 POTW

Total Number of Existing POTWs Unknown Flow	1,353 	PCS or ICIS Query:
SIC 4952 POTW

Total Adjusted Number of Existing POTWs  ≽1.0 mgd	4,391 	PCS or ICIS
Query: SIC 4952; adjusted to account for facilities without flow
information in PCS

Total Adjusted Number of Existing POTWs  0.1 - 1.0 mgd	5,942 	PCS or
ICIS Query: SIC 4952; adjusted to account for facilities without flow
information in PCS

Total Adjusted Number of Existing POTWs  ≺0.1 mgd	3,352 	PCS or ICIS
Query: SIC 4952; adjusted to account for facilities without flow
information in PCS

Total Number of Existing PrOTWs	3,462 	PCS or ICIS Query: SIC 4952 PrOTW

Total Number of Industrial stormwater dischargers covered by a
Multi-Sector General Permit (MSGP)	89,415 	Jack Faulk, personal
communication. Tuesday 5/21/2008 5:57 PM.  Reaffirmed by Brian
Rittenhouse 9/22/2011.

Annual Number of Notice of Termination (NOT) from Industrial stormwater
dischargers under the Multi-Sector General Permit (MSGP)	6,857 	Estimate
from previous ICR.

Annual Number of Municipal Sludge Facilities required under their sewage
sludge permit conditions to submit Permittee Reports of Planned Facility
Changes and Anticipated Noncompliance.	153 	Estimate from previous ICR.

Annual Number of Nonmunicipal Sludge Facilities required under their
sewage sludge permit conditions to submit Permittee Reports of Planned
Facility Changes and Anticipated Noncompliance.	10 	Estimate from
previous ICR.

Annual Number of sludge facilities required to submit Permittee Reports
of Inaccurate Previous Information under sewage sludge use or disposal
conditions	30 	Estimate from previous ICR.

Total Number of Non-Stormwater General Permittees (State-issued)	57,058
GPCFs September 2011, excluding GM O&G Permit. From Jackie Clark

Total Number of Non-Stormwater General Permittees (EPA-issued)	5,367
GPCFs September 2011, excluding GM O&G Permit. From Jackie Clark

Uniform Federal Transportation/Utility System Application Form (Alaskan
Lands Application)	3 	Estimate from previous ICR.

MS4 Petition for Individual Permit	24 	Estimate from previous ICR.

Permit Consolidation Request	0 	Estimate from previous ICR was 2.
Changed by Dave Hair.

Notification of Construction Prior to Wastewater Permit Issuance	3
Estimate from previous ICR.

NPDES Ocean Discharge Information	30 	Estimate from previous ICR.

Total Number of CAFO Permittees	0 	Burden for this activity moved to
Supporting Statement for the Information Collection Request for Animal
Sectors. OMB Control No. 2040-0250, EPA ICR No. 1989.07. May 2010

Total Number of CAAP Permittees	0 	Burden for this activity moved to
Supporting Statement for the Information Collection Request for Animal
Sectors. OMB Control No. 2040-0250, EPA ICR No. 1989.07. May 2010

Annual Number of POTWs required to submit compliance schedule reports
regarding sewage sludge permit conditions	100 	Estimate from previous
ICR.

Annual Number of PrOTWs required to submit compliance schedule reports
regarding sewage sludge permit conditions	32 	Estimate from previous
ICR.

Annual Number of POTWs required to submit non-compliance reports
regarding sewage sludge permit conditions	31 	Estimate from previous
ICR.

Annual Number of PrOTWs required to submit non-compliance  reports
regarding sewage sludge permit conditions	10 	Estimate from previous
ICR.

Annual Number of permittees required to respond to Section 308(a)
letters	1,200 	Estimate from previous ICR.

Total Number of Pesticides Packaging and Repackaging facilities with
certification potential	326 	Estimate from previous ICR.

Total Number of Aluminum Forming facilities with certification potential
76 	Estimate from previous ICR.

Total Number of Coil Coating facilities with certification potential	101
	Estimate from previous ICR.

Total Number of Can Making (subcategory of coil coating) facilities with
certification potential	18 	Estimate from previous ICR.

Total Number of Porcelain Enameling* facilities with certification
potential	54 	Estimate from previous ICR.

Total Number of Pharmaceutical Manufacturing facilities with
certification potential	130 	Estimate from previous ICR.

Total Number of Pulp, Paper, and Paperboard facilities with
certification potential	361 	Estimate from previous ICR.

Total Number of Subpart B, Bleached Papergrade Kraft and Soda facilities
with certification potential	74 	Estimate from previous ICR.

Total Number of Building Paper and Board Mills facilities with
certification potential	174 	Estimate from previous ICR.

Total Number of Steam Electric facilities with certification potential
1,372 	Estimate from previous ICR.

Total Number of Electroplating** facilities with certification potential
0 	Estimate from previous ICR.

Total Number of Metal Finishing facilities with certification potential
2,032 	Estimate from previous ICR.

Total Number of Electrical and Electronic Components facilities with
certification potential	86 	Estimate from previous ICR.

Annual Number of Part 435 Certification of Oil and Gas Extraction	68
Estimate from previous ICR.

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (AL)	0 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (IL)	0 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (IN)	5 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (KY)	7 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (MD)	1 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (OH)	21 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (PA)	26 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (TN)	9 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (VA)	9 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Annual Number of sites to be permitted Coal Remining at AML with
pre-existing discharges (WV)	0 	Economic and Environmental Impact
Assessment of Effluent Limitations Guidelines and Standards for the Coal
Mining Industry: Remining and Western Alkaline Subcategories
(EPA-821-B-01-013).

Total Number of Western Alkaline Coal Mining with Sediment Control	46
Need to get from EPA. Original Source: 1998 Keystone Coal Industry
Manual (as cited in EPA ICR No. 1944.03). These mines are in AZ, CO, MT,
NM, ND, UT and WY. All except NM have NPDES authorization.

Total number of Subpart B, Bleached Papergrade Kraft and Soda Enrolled
in Milestone Plans	29 	EPA ICR No. 1877.03

Total Number of Subpart B, Bleached Papergrade Kraft and Soda
anticipated to Enrolled in Milestone Plans (Tier One Mills)	0 	Estimate
from previous ICR.

Total Number of Subpart B, Bleached Papergrade Kraft and Soda
anticipated to Enrolled in Milestone Plans (Tier Two Mills)	0 	Estimate
from previous ICR.

Total Number of Subpart B, Bleached Papergrade Kraft and Soda
anticipated to Enrolled in Milestone Plans (Tier Three Mills)	0
Estimate from previous ICR.

Total Number of Subpart B Bleached Papergrade Kraft & Soda, Direct
Discharging facilities	75 	Estimate from previous ICR.

Total Number of Subpart E Ca / Sodium / Mg Sulfite, Direct Discharging
facilities	5 	Estimate from previous ICR.

Total Number of Subpart E Ammonium Sulfite, Direct Discharging
facilities	2 	Estimate from previous ICR.

Total Number of Subpart E Specialty Grade, Direct Discharging facilities
2 	Estimate from previous ICR.

Total number of Minor (Municipal) Dischargers not required to report
1,719 	PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor
reporting frequency

Total number of Minor (nonmunicipal) Dischargers not required to report
10,751 	PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor
reporting frequency

Total number of Minor (Municipal) Dischargers that report Monthly	6,569
PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor reporting
frequency

Total number of Minor (nonmunicipal) Dischargers that report Monthly
11,996 	PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor
reporting frequency

Total number of Minor (Municipal) Dischargers that report Bimonthly	32
PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor reporting
frequency

Total number of Minor (nonmunicipal) Dischargers that report Bimonthly
119 	PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor
reporting frequency

Total number of Minor (Municipal) Dischargers that report Quarterly	940
PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor reporting
frequency

Total number of Minor (nonmunicipal) Dischargers that report Quarterly
3,843 	PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor
reporting frequency

Total number of Minor (Municipal) Dischargers that report Semi-Annually
9 	PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor
reporting frequency

Total number of Minor (nonmunicipal) Dischargers that report
Semi-Annually	219 	PCS or ICIS Query: STANDARD (municipal and
nonmunicipal) minor reporting frequency

Total number of Minor (Municipal) Dischargers that report Annually	202
PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor reporting
frequency

Total number of Minor (nonmunicipal) Dischargers that report Annually
1,205 	PCS or ICIS Query: STANDARD (municipal and nonmunicipal) minor
reporting frequency

Percent of Non-stormwater General Permittees not required to report
72.6%	PCS or ICIS Query: General Permits (non-stormwater) reporting
frequency

Percent of Non-stormwater General Permittees that report Monthly	7.8%
PCS or ICIS Query: General Permits (non-stormwater) reporting frequency

Percent of Non-stormwater General Permittees that report Bimonthly	0.4%
PCS or ICIS Query: General Permits (non-stormwater) reporting frequency

Percent of Non-stormwater General Permittees that report Quarterly	9.3%
PCS or ICIS Query: General Permits (non-stormwater) reporting frequency

Percent of Non-stormwater General Permittees that report Semi-Annually
6.4%	PCS or ICIS Query: General Permits (non-stormwater) reporting
frequency

Percent of Non-stormwater General Permittees that report Annually	3.6%
PCS or ICIS Query: General Permits (non-stormwater) reporting frequency

Total Number of Domestic Septage Haulers	4,784 	Estimate from previous
ICR.

Sludge Only, Class I Mgmt Facilities (Municipal Biosolids)	113 	PCS or
ICIS Query: Sludge Only Municipal by Sludge Type

Sludge Only, Non-Class I Mgmt Facilities  (Municipal Biosolids)	1 	PCS
or ICIS Query: Sludge Only Municipal by Sludge Type

Sludge Only, Unknown Class Mgmt Facilities  (Municipal Biosolids)	840
PCS or ICIS Query: Sludge Only Municipal by Sludge Type

Sludge Only, Class I Mgmt Facilities (Industrial)	73 	PCS or ICIS Query:
Sludge Only Nonmunicipal by Sludge Type

Sludge Only, Non-Class I Mgmt Facilities (Industrial)	39 	PCS or ICIS
Query: Sludge Only Nonmunicipal by Sludge Type

Sludge Only, Unknown Class Mgmt Facilities  (Industrial Biosolids)	217
PCS or ICIS Query: Sludge Only Nonmunicipal by Sludge Type

Sludge / NPDES POTWs, Class I Sludge Mgmt Facilities	989 	PCS or ICIS
Query: SIC 4952 (POTW) by sludge type

Sludge / NPDES POTWs, Non-Class I Sludge Mgmt Facilities	1,070 	PCS or
ICIS Query: SIC 4952 (POTW) by sludge type

Sludge / NPDES POTWs,  Unknown Class Sludge Mgmt Facilities	11,626 	PCS
or ICIS Query: SIC 4952 (POTW) by sludge type

Sludge / NPDES PrOTWs, Class I Sludge Mgmt Facilities	113 	PCS or ICIS
Query: SIC 4952 (PrOTW) by sludge type

Sludge / NPDES PrOTWs, Non-Class I Sludge Mgmt Facilities	73 	PCS or
ICIS Query: SIC 4952 (PrOTW) by sludge type

Sludge / NPDES PrOTWs,  Unknown Class Sludge Mgmt Facilities	3,276 	PCS
or ICIS Query: SIC 4952 (PrOTW) by sludge type

Total number of Major Municipal facilities in the Great Lakes Watershed
269 	PCS or ICIS Great Lakes Queries

Total number of Major Nonmunicipal facilities in the Great Lakes
Watershed	190 	PCS or ICIS Great Lakes Queries

Total number of Minor Municipal facilities in the Great Lakes Watershed
796 	PCS or ICIS Great Lakes Queries

Total number of Minor Nonmunicipal facilities in the Great Lakes
Watershed	1,068 	PCS or ICIS Great Lakes Queries

Additional facilities that will require WQBEL monitoring under the Great
Lakes Program	176 	Assessment of Compliance Costs Resulting from
Implementation of the Final Great Lakes Water Quality Guidance (March
13, 1995)

Municipal facilities required to develop approvable control strategies
(Pollutant Minimization Programs) under the Great Lakes Program	20 	EPA
estimate

Municipal facilities required to Implement control strategies (Pollutant
Minimization Programs) under the Great Lakes Program	170 	EPA estimate

Nonmunicipal facilities required to develop approvable control
strategies (Pollutant Minimization Programs) under the Great Lakes
Program	40 	EPA estimate

Nonmunicipal facilities required to implement approvable control
strategies (Pollutant Minimization Programs) under the Great Lakes
Program	250 	EPA estimate

Annual modification and variance request under the Great Lakes Program
18 	Assessment of Compliance Costs Resulting from Implementation of the
Final Great Lakes Water Quality Guidance (March 13, 1995)

Total number of Municipalities with Combined Sewer Systems	746	Personal
communication, Mohammed Billah 10/17/2011

Total number of CSO Permits	853 	Personal communication, Mohammed Billah
10/17/2011, 11/30/11

CSO permits with LTCP Permit Requirements	734 	Personal communication,
Mohammed Billah 10/17/2011, 11/30/11

Municipalities that will submit NMC documentation in the next 3 years -
Small	0	Personal communication, Mohammed Billah 10/17/2011, 11/30/11

Municipalities that will submit NMC documentation in the next 3 years -
Medium	0	Personal communication, Mohammed Billah 10/17/2011, 11/30/11

Municipalities that will submit NMC documentation in the next 3 years -
Large	0	Personal communication, Mohammed Billah 10/17/2011, 11/30/11

Percentage of Municipalities that Choose Public Notification - Option 1
(Signs at outfalls)	43%	Previous CSO Control Policy ICR (OMB No.
2040-0170; EPA ICR No. 1680.04)

Percentage of Municipalities that Choose Public Notification - Option 2
(Signs at beaches, other public exposure areas)	43%	Previous CSO Control
Policy ICR (OMB No. 2040-0170; EPA ICR No. 1680.04)

Percentage of Municipalities that Choose Public Notification - Option 3
(Press or radio advisories)	30%	Previous CSO Control Policy ICR (OMB No.
2040-0170; EPA ICR No. 1680.04)

Small Permittees Writing LTCP that have existing studies	8	6.6% of CSO
permits that will submit LTCP (OMB No. 2040-0170; EPA ICR No. 1680.04)

Medium Permittees Writing LTCP that have existing studies	16	13.1% of
CSO permits that will submit LTCP (OMB No. 2040-0170; EPA ICR No.
1680.04)

Large Permittees Writing LTCP that have existing studies	5	4.3% of CSO
permits that will submit LTCP (OMB No. 2040-0170; EPA ICR No. 1680.04)

Small Permittees Writing LTCP that do not have existing studies	71	59.3%
of CSO permits that will submit LTCP (OMB No. 2040-0170; EPA ICR No.
1680.04)

Medium Permittees Writing LTCP that do not have existing studies	15
12.8% of CSO permits that will submit LTCP (OMB No. 2040-0170; EPA ICR
No. 1680.04)

Large Permittees Writing LTCP that do not have existing studies	5	3.9%
of CSO permits that will submit LTCP (OMB No. 2040-0170; EPA ICR No.
1680.04)

Table A2. Unique Respondents

Ò

Ó

Å

Æ

Ò

Ò

Ó

ü

kd

©欀⍤

©欀

kd

kd

kd

Total Number of Non-Stormwater General Permittees	57,058

	Total Number of Domestic Septage Haulers	4,784

	Total number of Municipalities with Combined Sewer Systems	746

	Total number of Vessels	71,030

	Total non-state respondents	397,703

	Authorized States	47

States. Tribes, territories and DC	637 *

* 590 of these 637 states respond to only one information item
(certification of EPA-issued permits) [C40]

A-  PAGE  11


[Federal Register Volume 86, Number 31 (Thursday, February 18, 2021)]
[Notices]
[Page 10120]
[FR Doc No: 2021-03304]

-----------------------------------------------------------------------

DEPARTMENT OF HOMELAND SECURITY

Federal Emergency Management Agency

[Internal Agency Docket No. FEMA-4564-DR; Docket ID FEMA-2021-0001]

Florida; Amendment No. 5 to Notice of a Major Disaster
Declaration

AGENCY: Federal Emergency Management Agency, DHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: This notice amends the notice of a major disaster declaration
for the State of Florida (FEMA-4564-DR), dated September 23, 2020, and
related determinations.

DATES: This change occurred on January 15, 2021.

FOR FURTHER INFORMATION CONTACT: Dean Webster, Office of Response and
Recovery, Federal Emergency Management Agency, 500 C Street SW,
Washington, DC 20472,<PHONE_NUMBER>.

SUPPLEMENTARY INFORMATION: The Federal Emergency Management Agency
(FEMA) hereby gives notice that pursuant to the authority vested in the
Administrator, under Executive Order 12148, as amended, Kevin A.
Wallace, of FEMA is appointed to act as the Federal Coordinating
Officer for this disaster.
    This action terminates the appointment of Jeffrey L. Coleman as
Federal Coordinating Officer for this disaster.

The following Catalog of Federal Domestic Assistance Numbers (CFDA)
are to be used for reporting and drawing funds: 97.030, Community
Disaster Loans; 97.031, Cora Brown Fund; 97.032, Crisis Counseling;
97.033, Disaster Legal Services; 97.034, Disaster Unemployment
Assistance (DUA); 97.046, Fire Management Assistance Grant; 97.048,
Disaster Housing Assistance to Individuals and Households In
Presidentially Declared Disaster Areas; 97.049, Presidentially
Declared Disaster Assistance--Disaster Housing Operations for
Individuals and Households; 97.050, Presidentially Declared Disaster
Assistance to Individuals and Households--Other Needs; 97.036,
Disaster Grants--Public Assistance (Presidentially Declared
Disasters); 97.039, Hazard Mitigation Grant.

Robert J. Fenton,
Senior Official Performing the Duties of the Administrator, Federal
Emergency Management Agency.
[FR Doc. 2021-03304 Filed 2-17-21; 8:45 am]
BILLING CODE 9111-23-P


[Federal Register Volume 82, Number 200 (Wednesday, October 18, 2017)]
[Notices]
[Pages 48516-48519]
[FR Doc No: 2017-22550]

-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Food and Drug Administration

[Docket No. FDA-2017-N-5569]

Agency Information Collection Activities; Proposed Collection;
Comment Request; Medical Devices; Device Tracking

AGENCY: Food and Drug Administration, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Food and Drug Administration (FDA or Agency) is

announcing an opportunity for public comment on the proposed collection
of certain information by the Agency. Under the Paperwork Reduction Act
of 1995 (PRA), Federal Agencies are required to publish notice in the
Federal Register concerning each proposed collection of information,
including each proposed extension of an existing collection of
information, and to allow 60 days for public comment in response to the
notice. This notice solicits comments on information collection
requirements for the tracking of medical devices.

DATES: Submit either electronic or written comments on the collection
of information by December 18, 2017.

ADDRESSES: You may submit comments as follows. Please note that late,
untimely filed comments will not be considered. Electronic comments
must be submitted on or before December 18, 2017. The https://www.regulations.gov electronic filing system will accept comments until
midnight Eastern Time at the end of December 18, 2017. Comments
received by mail/hand delivery/courier (for written/paper submissions)
will be considered timely if they are postmarked or the delivery
service acceptance receipt is on or before that date.

Electronic Submissions

    Submit electronic comments in the following way:
     Federal eRulemaking Portal: https://www.regulations.gov.
Follow the instructions for submitting comments. Comments submitted
electronically, including attachments, to https://www.regulations.gov
will be posted to the docket unchanged. Because your comment will be
made public, you are solely responsible for ensuring that your comment
does not include any confidential information that you or a third party
may not wish to be posted, such as medical information, your or anyone
else's Social Security number, or confidential business information,
such as a manufacturing process. Please note that if you include your
name, contact information, or other information that identifies you in
the body of your comments, that information will be posted on https://www.regulations.gov.
     If you want to submit a comment with confidential
information that you do not wish to be made available to the public,
submit the comment as a written/paper submission and in the manner
detailed (see ``Written/Paper Submissions'' and ``Instructions'').

Written/Paper Submissions

    Submit written/paper submissions as follows:
     Mail/Hand delivery/Courier (for written/paper
submissions): Dockets Management Staff (HFA-305), Food and Drug
Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852.
     For written/paper comments submitted to the Dockets
Management Staff, FDA will post your comment, as well as any
attachments, except for information submitted, marked and identified,
as confidential, if submitted as detailed in ``Instructions.''
    Instructions: All submissions received must include the Docket No.
FDA-2017-N-5569 for ``Agency Information Collection Activities;
Proposed Collection; Comment Request; Medical Devices; Device
Tracking.'' Received comments, those filed in a timely manner (see
ADDRESSES), will be placed in the docket and, except for those
submitted as ``Confidential Submissions,'' publicly viewable at https://www.regulations.gov or at the Dockets Management Staff between 9 a.m.
and 4 p.m., Monday through Friday.
     Confidential Submissions--To submit a comment with
confidential information that you do not wish to be made publicly
available, submit your comments only as a written/paper submission. You
should submit two copies total. One copy will include the information
you claim to be confidential with a heading or cover note that states
``THIS DOCUMENT CONTAINS CONFIDENTIAL INFORMATION.'' The Agency will
review this copy, including the claimed confidential information, in
its consideration of comments. The second copy, which will have the
claimed confidential information redacted/blacked out, will be
available for public viewing and posted on https://www.regulations.gov.
Submit both copies to the Dockets Management Staff. If you do not wish
your name and contact information to be made publicly available, you
can provide this information on the cover sheet and not in the body of
your comments and you must identify this information as
``confidential.'' Any information marked as ``confidential'' will not
be disclosed except in accordance with 21 CFR 10.20 and other
applicable disclosure law. For more information about FDA's posting of
comments to public dockets, see 80 FR 56469, September 18, 2015, or
access the information at: https://www.gpo.gov/fdsys/pkg/FR-2015-09-18/pdf/2015-23389.pdf.
    Docket: For access to the docket to read background documents or
the electronic and written/paper comments received, go to https://www.regulations.gov and insert the docket number, found in brackets in
the heading of this document, into the ``Search'' box and follow the
prompts and/or go to the Dockets Management Staff, 5630 Fishers Lane,
Rm. 1061, Rockville, MD 20852.

FOR FURTHER INFORMATION CONTACT: Amber Sanford, Office of Operations,
Food and Drug Administration, Three White Flint North, 10A63, 11601
Landsdown St., North Bethesda, MD 20852,<PHONE_NUMBER>,
<EMAIL_ADDRESS>
SUPPLEMENTARY INFORMATION: Under the PRA (44 U.S.C. 3501-3520), Federal
Agencies must obtain approval from the Office of Management and Budget
(OMB) for each collection of information they conduct or sponsor.
``Collection of information'' is defined in 44 U.S.C. 3502(3) and 5 CFR
1320.3(c) and includes Agency requests or requirements that members of
the public submit reports, keep records, or provide information to a
third party. Section 3506(c)(2)(A) of the PRA (44 U.S.C. 3506(c)(2)(A))
requires Federal Agencies to provide a 60-day notice in the Federal
Register concerning each proposed collection of information, including
each proposed extension of an existing collection of information,
before submitting the collection to OMB for approval. To comply with
this requirement, FDA is publishing notice of the proposed collection
of information set forth in this document.
    With respect to the following collection of information, FDA
invites comments on these topics: (1) Whether the proposed collection
of information is necessary for the proper performance of FDA's
functions, including whether the information will have practical
utility; (2) the accuracy of FDA's estimate of the burden of the
proposed collection of information, including the validity of the
methodology and assumptions used; (3) ways to enhance the quality,
utility, and clarity of the information to be collected; and (4) ways
to minimize the burden of the collection of information on respondents,
including through the use of automated collection techniques, when
appropriate, and other forms of information technology.

Medical Devices; Device Tracking--21 CFR Part 821

OMB Control Number 0910-0442--Extension

    Section 211 of the Food and Drug Administration Modernization Act
of 1997 (FDAMA) (Pub. L. 105-115) became effective on February 19,
1998. FDAMA amended the previous medical device tracking provisions
under section 519(e)(1) and (2) of the Federal Food,

Drug, and Cosmetic Act (the FD&C Act) (21 U.S.C. 360i(e)(1) and (2))
that were added by the Safe Medical Devices Act of 1990 (SMDA) (Pub. L.
101-629). Unlike the tracking provisions under SMDA, which required
tracking of any medical device meeting certain criteria, FDAMA allows
FDA discretion in applying tracking provisions to medical devices
meeting certain criteria and provides that tracking requirements for
medical devices can be imposed only after FDA issues an order. In the
Federal Register of February 8, 2002 (67 FR 5943), FDA issued a final
rule that conformed existing tracking regulations to changes in
tracking provisions effected by FDAMA under part 821 (21 CFR part 821).
    Section 519(e)(1) of the FD&C Act, as amended by FDAMA, provides
that FDA may require by order that a manufacturer adopt a method for
tracking a class II or III medical device, if the device meets one of
the three following criteria: (1) The failure of the device would be
reasonably likely to have serious adverse health consequences, (2) the
device is intended to be implanted in the human body for more than 1
year (referred to as a ``tracked implant''), or (3) the device is life-
sustaining or life-supporting (referred to as a ``tracked l/s-l/s
device'') and is used outside a device user facility.
    Tracked device information is collected to facilitate identifying
the current location of medical devices and patients possessing those
devices, to the extent that patients permit the collection of
identifying information. Manufacturers and FDA (where necessary) use
the data to: (1) Expedite the recall of distributed medical devices
that are dangerous or defective and (2) facilitate the timely
notification of patients or licensed practitioners of the risks
associated with the medical device.
    In addition, the regulations include provisions for: (1) Exemptions
and variances; (2) system and content requirements for tracking; (3)
obligations of persons other than device manufacturers, e.g.,
distributors; (4) records and inspection requirements; (5)
confidentiality; and (6) record retention requirements.
    Respondents for this collection of information are medical device
manufacturers, importers, and distributors of tracked implants or
tracked l/s-l/s devices used outside a device user facility.
Distributors include multiple and final distributors, including
hospitals.
    The annual hourly burden for respondents involved with medical
device tracking is estimated to be 615,380 hours per year. The burden
estimates cited in tables 1 through 3 are based on the number of device
tracking orders issued in the last 3 years, an average of 12 tracking
orders annually. FDA estimates that approximately 22,000 respondents
may be subject to tracking reporting requirements.
    Under Sec.  821.25(a), device manufacturers subject to FDA tracking
orders must adopt a tracking method which can provide certain device,
patient, and distributor information to FDA within 3 to 10 working
days. Assuming one occurrence per year, FDA estimates it would take a
firm 20 hours to provide FDA with location data for all tracked devices
and 56 hours to identify all patients and/or multiple distributors
possessing tracked devices.
    Under Sec.  821.25(d) manufacturers must notify FDA of distributor
noncompliance with reporting requirements. Based on the number of
audits manufacturers conduct annually, FDA estimates it would receive
no more than one notice in any year, and that it would take 1 hour per
incident.
    Under Sec.  821.30(c)(2), multiple distributors must provide data
on current users of tracked devices, current device locations, and
other information, upon request from a manufacturer or FDA. FDA has not
made such a request and is not aware of any manufacturer making a
request. Assuming one multiple distributor receives one request in a
year from either a manufacturer or FDA, and that lists may be generated
electronically, the Agency estimates a burden of 1 hour to comply.
    Under Sec.  821.30(d) distributors must verify data or make
required records available for auditing, if a manufacturer provides a
written request. FDA's estimate of the burden for distributor audit
responses assumes that manufacturers audit database entries for 5
percent of tracked devices distributed. Each audited database entry
prompts one distributor audit response. Because lists may be generated
electronically, FDA estimates a burden of 1 hour to comply.
    FDA estimates the burden of this collection of information as
follows:

                                 Table 1--Estimated Annual Reporting Burden \1\
----------------------------------------------------------------------------------------------------------------
                                                     Number of
     Activity/21 CFR section         Number of     responses per   Total annual   Average burden    Total hours
                                    respondents     respondent       responses     per response
----------------------------------------------------------------------------------------------------------------
Discontinuation of business--                  1               1               1               1               1
 821.1(d).......................
Exemption or variance--821.2 and               1               1               1               1               1
 821.30(e)......................
Notification of failure to                     1               1               1               1               1
 comply--821.25(d)..............
Multiple distributor data--                    1               1               1               1               1
 821.30(c)(2)...................
                                 -------------------------------------------------------------------------------
    Total.......................  ..............  ..............  ..............  ..............               4
----------------------------------------------------------------------------------------------------------------
\1\ There are no capital costs or operating and maintenance costs associated with this collection of
  information.

                               Table 2--Estimated Annual Recordkeeping Burden \1\
----------------------------------------------------------------------------------------------------------------
                                                     Number of                    Average burden
     Activity/21 CFR section         Number of      records per    Total annual         per         Total hours
                                   recordkeepers   recordkeeper       records      recordkeeping
----------------------------------------------------------------------------------------------------------------
Tracking information--821.25(a).              12               1              12              76             912
Record of tracking data--                     12          46,260         555,120               1         555,120
 821.25(b)......................
Standard operating procedures--               12               1              12              63             756
 821.25(c) \2\..................
Manufacturer data audit--                     12           1,124          13,488               1          13,488
 821.25(c)(3)...................
Multiple distributor data and             22,000               1          22,000               1          22,000
 distributor tracking records--
 821.30(c)(2) and (d)...........
                                 -------------------------------------------------------------------------------

    Total.......................  ..............  ..............  ..............  ..............         592,276
----------------------------------------------------------------------------------------------------------------
\1\ There are no capital costs or operating and maintenance costs associated with this collection of
  information.
\2\ One-time burden.

                           Table 3--Estimated Annual Third-Party Disclosure Burden \1\
----------------------------------------------------------------------------------------------------------------
                                                     Number of
     Activity/21 CFR section         Number of      disclosures    Total annual   Average burden    Total hours
                                    respondents   per respondent    disclosures   per disclosure
----------------------------------------------------------------------------------------------------------------
Acquisition of tracked devices            22,000               1          22,000               1          22,000
 and final distributor data--
 821.30(a) and (b)..............
Multiple distributor data and              1,100               1           1,100               1           1,100
 distributor tracking records--
 821.30(c)(2) and (d)...........
                                 -------------------------------------------------------------------------------
    Total.......................  ..............  ..............  ..............  ..............          23,100
----------------------------------------------------------------------------------------------------------------
\1\ There are no capital costs or operating and maintenance costs associated with this collection of
  information.

    The burden estimate for this information collection has not changed
since the last OMB approval.
    This document also refers to previously approved collections of
information found in FDA regulations. These collections of information
are subject to review by the Office of Management and Budget under the
PRA (44 U.S.C. 3501-3520). The collections of information found in
Sec. Sec.  821.2(b), 821.25(e), and 821.30(e) have been approved under
OMB control number 0910-0191.

    Dated: October 11, 2017.
Leslie Kux,
Associate Commissioner for Policy.
[FR Doc. 2017-22550 Filed 10-17-17; 8:45 am]
 BILLING CODE 4164-01-P


[Federal Register: December 20, 2010 (Volume 75, Number 243)]
[Notices]
[Page 79378-79379]
[DOCID:fr20de10-940]

=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Food and Drug Administration

[Docket No. FDA-2010-N-0420]

Agency Information Collection Activities; Submission for Office
of Management and Budget Review; Comment Request; Testing
Communications on FDA-Regulated Products Used in Animals

AGENCY: Food and Drug Administration, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Food and Drug Administration (FDA) is announcing that a
proposed collection of information has been submitted to the Office of
Management and Budget (OMB) for review and clearance under the
Paperwork Reduction Act of 1995.

DATES: Fax written comments on the collection of information by January
19, 2011.

ADDRESSES: To ensure that comments on the information collection are
received, OMB recommends that written comments be faxed to the Office
of Information and Regulatory Affairs, OMB, Attn: FDA Desk Officer,
FAX:<PHONE_NUMBER>, or e-mailed to<EMAIL_ADDRESS>All
comments should be identified with the OMB control number 0910-New and
title ``Testing Communications on FDA-Regulated Products Used in
Animals.'' Also include the FDA docket number found in brackets in the
heading of this document.

FOR FURTHER INFORMATION CONTACT: Johnny Vilela, Office of Information
Management, Food and Drug Administration, 1350 Piccard Dr., PI50-400B,
Rockville, MD 20850,<PHONE_NUMBER><EMAIL_ADDRESS>
SUPPLEMENTARY INFORMATION: In compliance with 44 U.S.C. 3507, FDA has
submitted the following proposed collection of information to OMB for
review and clearance.

Testing Communications on FDA-Regulated Products Used In Animals--(OMB
Control Number 0910-New)

    FDA's Center for Veterinary Medicine (CVM) has authorization under
section 903(d)(2)(D) of the Federal Food, Drug, and Cosmetic Act (21
U.S.C. 393(d)(2)(D)) to conduct educational and public information
programs relating to the safety of CVM-regulated products. Further, CVM
is authorized to conduct this needed research to ensure that these
programs have the highest likelihood of being effective. Thus, CVM
concludes that improving communications about the safety of regulated
animal drugs, feed, food additives, and devices will involve many
research methods, including individual in-depth interviews, mall-
intercept interviews, focus groups, self-administered surveys,
gatekeeper reviews, and omnibus telephone surveys.
    The information collected will serve three major purposes. First,
as formative research, it will provide critical knowledge needed about
target audiences to develop messages and campaigns about the use of
animal drugs, feed, food additives, and devices. Knowledge of both the
consumer and the veterinary professional decision-making processes will
provide a better understanding of target audiences that FDA will need
in order to design effective communication strategies, messages, and
labels. These communications will aim to improve public understanding
of the risks and benefits of using regulated animal drugs, feed, food
additives, and devices by providing users with a better context in
which to place risk information more

completely. Second, as initial testing, it will allow FDA to assess the
potential effectiveness of messages and materials in reaching and
successfully communicating with their intended audiences. Testing
messages with a sample of the target audience will allow FDA to refine
messages while still in the developmental stage. Respondents will be
asked to give their reaction to the messages in either individual or
group settings. Third, as evaluative research, it will allow FDA to
ascertain the effectiveness of the messages and the distribution method
of these messages in achieving the objectives of the message campaign.
Evaluation of campaigns is a vital link in continuous improvement of
communications at FDA.
    In the Federal Register of August 19, 2010 (75 FR 51271), FDA
published a 60-day notice requesting public comment on the proposed
collection of information. FDA received comments from two individuals
and one trade association. FDA acknowledges one request for additional
details on the necessity and purpose of the information to be
collected, but notes that comments were invited on FDA's request for a
generic clearance related to the formative testing of communications
about veterinary products and products for animals. Under this generic
clearance, details of individual studies (research questions, target
audiences, methodologies, and consultants) will be tailored to specific
communications-related questions. For each study FDA requests under
this clearance, FDA will provide OMB with these details on the
information collection. The communication development process will
inform the purpose of the data collection and the means by which the
data will be collected. For very early message development, qualitative
research such as in-depth interviews or focus groups will be
appropriate. At later communication development stages, more
quantitative data collection would be more useful. FDA plans to use the
data collected under this generic clearance to inform its
communications campaigns. The data will not be used for the purposes of
making policy or regulatory decisions.
    Audience targets are also informed by the specific research
question. Nonetheless, FDA provided more information by specifying some
of the groups more likely to be targeted in tasks under this generic
clearance, including: Consumers, pet owners, large animal producers,
veterinarians, animal distributors, pet shop owners, stockyards staff
and owners, abattoir owners or staff, grocery meat purchasers,
agricultural extension agents, and professors of food science and
related fields.
    Furthermore, comments related to ways to enhance the data
collection and to assess FDA's estimate of burden indicated that FDA
should not limit itself to in-house expertise. FDA acknowledges that
assistance may be requested from experts in other Government agencies.
Depending on the specific research question to be addressed, FDA may
consult experts in the United States Department of Agriculture and the
United States Environmental Protection Agency.
    FDA received a comment relating to the cruelty and sadism of animal
testing. In response to this comment, FDA notes that its notice was for
public comment on data collection related to communication studies. No
animal testing is involved.
    FDA received a comment that made a series of complaints against the
Agency unrelated to its notice for public comment. Accordingly, those
comments are not addressed in this document.
    FDA estimates the burden of this collection of information as
follows:

                                 Table 1--Estimated Annual Reporting Burden \1\
----------------------------------------------------------------------------------------------------------------
                                       No. of       Annual frequency     Total annual        Hours per     Total
     21 U.S.C. 393(d)(2)(D)          respondents      per response        responses          response      hours
----------------------------------------------------------------------------------------------------------------
Individual in-depth interviews..               360                 1             360                  .75  270
General public focus group                     288                 1             288                 1.50  432
 interviews.....................
Intercept interviews: Central                  200                 1             200                  .25   50
 location.......................
Intercept interviews: Telephone              2,000                 1           2,000                  .08  160
 \2\............................
Self-administered surveys.......             2,400                 1           2,400                  .25  600
Gatekeeper reviews..............               300                 1             300                  .50  150
Omnibus surveys.................             1,200                 1           1,200                  .17  204
Total (general public)..........  ................  ................  .................  ................  1,866
Veterinarian/scientific expert                 288                 1             288                 1.50  432
 focus group interviews.........
                                 -------------------------------------------------------------------------------
    Total (overall).............  ................  ................  .................  ................  2,298
----------------------------------------------------------------------------------------------------------------
\1\ There are no capital costs or operating and maintenance costs associated with this collection of
  information.
\2\ These are brief interviews with callers to test message concepts and strategies following their call-in
  request to an FDA Center 1-800 number.

    Dated: December 7, 2010.
Leslie Kux,
Acting Assistant Commissioner for Policy.
[FR Doc. 2010-31891 Filed 12-17-10; 8:45 am]
BILLING CODE 4160-01-P


Lydia Wegman/RTP/USEPA/US

03/06/2008 11:12 AM

To

<EMAIL_ADDRESS> cc

Susan Stone/RTP/USEPA/US@EPA, Tricia Crabtree/RTP/USEPA/US@EPA

Subject

Fw: Follow up items on Ozone NAAQS review

Margo,

Here are the articles about the studies that we believe were mentioned during your meeting on March 3 with ALA and other groups.

Lydia

----- Forwarded by Lydia Wegman/RTP/USEPA/US on 03/06/2008 11:01 AM -----

Susan Stone/RTP/USEPA/US

03/06/2008 10:12 AM

To

Lydia Wegman/RTP/USEPA/US@EPA

cc

Subject

Re: Fw: Follow up items on Ozone NAAQS review

Lydia:  The articles that Margo requested are attached.  I'm not sure if this Kim et al 2007 is the one she wanted, since I didn't see what information was presented at the meeting, but I'm presuming this is correct.  Susan

Susan Lyon Stone

Office of Air Quality Planning and Standards

US EPA (C504-06)

Research Triangle Park, NC 27711

Phone:<PHONE_NUMBER>; Fax:<PHONE_NUMBER>

Lydia Wegman/RTP/USEPA/US

03/04/2008 07:20 PM

To

"Karen Martin"<EMAIL_ADDRESS>"Harvey Richmond" <EMAIL_ADDRESS>"Susan Stone"<EMAIL_ADDRESS>"Dave Mckee"<EMAIL_ADDRESS>"Bryan Hubbell" <EMAIL_ADDRESS>"John Hannon"<EMAIL_ADDRESS> cc

Subject

Fw: Follow up items on Ozone NAAQS review

More requests from OMB.

 ----- Original Message -----

  From: "King, Heidi R."<EMAIL_ADDRESS>   Sent: 03/04/2008 05:31 PM EST

  To: Lydia Wegman

  Cc: "Schwab, Margo"<EMAIL_ADDRESS>"Johansson, Robert" <EMAIL_ADDRESS>   Subject: Follow up items on Ozone NAAQS review

Lydia, my apologies, I neglected to list items that we ask that EPA to send to OMB:

1.    EPA response to NAM’s request for correction

2.    Paragraph by Harvey and Brian on the sensitivity of Risk Assessment to Model Specification(s)

    (Harvey and Brian were to draft a paragraph on the sensitivity of the risk assessment to the change of 8-hour average to 24-hour average; agreement following review of the L&M rule)

3.    Copies of the following articles (these were referenced by the EO 12866 mtg attendees on 3/3/08):

a.    Yang and Chen (2007)

b.    Medina and Ra… (2006)

c.     Kim et al., (2007)

d.    Dales (2006)

e.    Tritche et al. (2006)

OMB owes to EPA a list of attendees at the EO 12866 meeting 3/3/08, at which Erika Sasser represented EPA on the phone.

Thanks all -- best regards,

heidi


C 1 Site MCDD1, August  Sept. 93, pH frequency 6.5 7 7.5 8 8.5 9 9.5 .00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

frequency pH Site MCDD1, July 94 pH frequency 6.5 7 7.5 8 8.5 9 9.5 .00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

frequency pH C 2 Site MCDD1, July Sept. 95 pH frequency 6.5 7 7.5 8 8.5 9 9.5 .00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

frequency pH Site SJB, August  Sept. 93 pH frequency 6.5 7 7.5 8 8.5 9 9.5 .00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

frequency pH C 3 SIte SJB, June July 94 pH frequency 6.5 7 7.5 8 8.5 9 9.5 .00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

frequency pH SIte SJB, July  August 95 pH frequency 6.5 7 7.5 8 8.5 9 9.5 .00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

frequency pH C 4 C 5 C 6 C 7 C 8 C 9 C 10 C 11 C 12 C 13 C 14 C 15 Site SJB, July  Sept. 1992 DO 0 2 4 6 8 10 12 14 16 18 20 7/ 28/ 92 0: 00 8/ 2/ 92 0: 00 8/ 7/ 92 0: 00 8/ 12/ 92 0: 00 8/ 17/ 92 0: 00 8/ 22/ 92 0: 00 8/ 27/ 92 0: 00 9/ 1/ 92 0: 00 9/ 6/ 92 0: 00 Date DO, mg/ L C 16 SIte MCDD, DO summer 1992 0 2 4 6 8 10 12 14 16 18 7/ 23/ 92 0: 00 8/ 2/ 92 0: 00 8/ 12/ 92 0: 00 8/ 22/ 92 0: 00 9/ 1/ 92 0: 00 9/ 11/ 92 0: 00 Date DO, mg/ L C 17 C 18


Using Probabilistic Methods to Enhance the Role of

Risk Analysis in Decision Making

Managers' Summary

Prepared by the EPA Risk Assessment Forum Working Group

Co-leads:

Bob Hetes, Marian Olsen, Kathryn Gallagher, Cynthia Stahl, Rita Schoeny

And the RAF PRA Technical Panel

This document is a draft for peer review. It has not been formally
released by the U.S. Environmental Protection Agency and should not at
this stage be construed to represent Agency policy. It is being
circulated for comments on its technical merit and policy implications.
Mention of trade names or commercial products does not constitute
endorsement or recommendation for use.

This document was produced by a Technical Panel of the EPA Risk
Assessment Forum.  The authors drew on their experience in doing
probabilistic assessments and interpreting them to improve risk
management of environmental and health hazards.  Interviews,
presentations, and dialogues with risk managers conducted by the
Technical Panel have contributed to the insights and recommendations in
this summary and the associated White Papers.Using Probabilistic
Methods to Enhance the Role of

Risk Analysis in Decision Making

Manager’s Summary

EPA has been called upon by numerous advisory bodies such as the Science
Advisory Board and U.S. National Academy of Sciences to incorporate
probabilistic risk information into the Agency decision making process.
Probabilistic risk assessment (PRA) is a group of techniques that
provide a range and likelihood estimate for one or more steps of hazard,
exposure or risk, rather than a single point estimate.  A Risk
Assessment Forum PRA technical panel has been formed from
representatives of Offices and Regions who conduct PRA.  The panel has
developed several products to promote enhanced use of PRA, including a
white paper describing PRA and its utility and application in Agency
decisions, and a compendium of case studies.  The panel is developing
Agency resources such as a clearinghouse of case studies, best
practices, and resources.  Seminars are being developed to raise general
knowledge of how these tools can be used, and act as a precursor for
future training.  The purpose of this paper is to present general
concepts and principles of PRA, to describe how PRA can improve the
bases of Agency decisions, and to provide illustrations of how PRA has
been used in risk estimation and to describe the uncertainty in risk
decision making.

Why should I care about PRA?  Why is it important to risk managers?

The use of PRA is often a major recommendation in reviews of EPA
products and procedures (e.g., Science Advisory Board review of EPA
practices in 2006, NAS review of the Dioxin Reassessment, OMB’s
Circular A-4 and Updated Principles for Risk Analysis).  The Agency has
some basic guidance, as well as some program-specific procedures and
applications for PRA.  The enhanced use of PRA and characterization of
uncertainty would respond to outside recommendations and potentially
enhance the overall transparency and quality of EPA assessments.  These
approaches would provide additional tools to address specific challenges
faced by managers and improve confidence in Agency decisions.
Specifically, PRA can inform decision makers about specific segments of
the population at risk, not just the mean (average) or extreme values.
A PRA can also confirm or support the conclusions of a deterministic
risk estimate.  Having this information can be important to risk
managers if a different decision might be made when the upper or lower
ends of the range of estimated exposures, doses, or risks are used.

What is PRA?  How does it compare with current practice?

A basic characteristic of PRA is that it does not generate a single
point estimate but rather produces a range and likelihood that a
particular exposure, dose, or effect will occur.  Probabilistic risk
assessment, in its simplest form, is a group of statistical techniques
that allow analysis of variability and uncertainty to be incorporated
into exposure and/or risk assessments.  As mentioned above, this kind of
information can be critical in risk management decisions - especially if
one were interested in a specific portion of the population and the
likelihood of exposure or risk is known.

What are common challenges facing EPA decision makers?

  SEQ CHAPTER \h \r 1 EPA Offices and Regions are faced with similar
mandates; basic attributes critical to decisions are (1) understanding
whom or what we are protecting, and (2) what is the appropriate degree
of confidence in the estimated protection provided by a particular
decision.  A further complication is the fact that decisions are often
time-sensitive and need to be made based on the current state of
knowledge.  Health and environmental impacts of environmental exposures
cannot be isolated and directly measured.  Therefore, risk assessment
methods have been developed to estimate health and ecological risks
based on available data and information.  The risk decision making may
also consider other assessments conducted to address other factors, such
as economic impacts of risk management actions.  Uncertainty can be
introduced into any assessment at any step in the process, even when
using the most accurate data with the most sophisticated models.  As a
result, EPA must always make decisions in the presence of uncertainty.

How can PRA and enhanced characterization of uncertainty and variability
help?

There are several ways in which various types of PRA can enhance risk
management decision making.  First, a sensitivity analysis (either
deterministic or probabilistic) can determine if more refined
information about the distribution and range of data can have a
substantial effect on the choice of decision options.  If so, there are
two ways that PRA can improve decisions.  First, using PRA one can
explicitly address the elements of a risk-based decision – whom or
what we are protecting and with what degree of confidence.  Secondly,
the use of PRA can characterize the inherent uncertainties and the
impact of those uncertainties on the decision.  When uncertainty is
present, PRA can better inform a decision, increase the transparency of
the inputs to the decision, and assist in selecting among various
management options.  Below we describe how PRA can:

1) enhance EPA decisions by providing more information about the
possible impacts of alternative regulatory decisions;

2) provide clarity on whom we are protecting and the confidence in the
estimates of protection provided by a given regulatory decision;

3) allow for more detailed comparison of alternative risk management
options in terms of estimated impacts on both protection and costs; and

4) improve the overall confidence in specific decisions.

As a manager what do I need to know about PRA?

	We all have common baseline experiences with probability, uncertainty
and variability such as weather forecasting, political polls, or climate
change predictions.   PRA can be used on many levels or degrees of
sophistication to support or improve decisions.  The following sections
describe the basics of PRA in more detail, how it can be used to support
decisions, and what to consider in pursuing a PRA or using PRA results.

How does EPA typically address variability and scientific uncertainty?

EPA cannot perform a time- and resource-intensive risk assessment for
every situation and EPA decision, and therefore, must be strategic in
determining whether more intensive assessments are needed.  When EPA
does not explicitly quantify the degree of confidence in a risk
estimate, the Agency attempts to increase the confidence that risk is
not underestimated by using default options to deal with uncertainty and
variability.  As depicted below, EPA most often uses risk assessment
methods that rely on default assumptions using a combination of point
values -- some conservative (high parameter values that are more likely
to overestimate risk) and some typical or average.  These values are put
into a model or single model structure and the risks are calculated as
shown below in Equation 1.

Equation 1:

This approach typically produces a single estimate of risks (e.g., 10-6
cancer risk; that is increased risk of 1 in a million additional
cancers); these may be referred to as deterministic assessments or point
estimates of risk.  These point estimates are useful, particularly in
screening assessments, but the inherent uncertainties are not fully
quantified.  These inherent limitations can affect EPA decisions in the
following ways:

Inability to explicitly characterize the basic elements of EPA decisions
-- whom or what are we protecting or with what degree of confidence

Inability to more realistically or accurately compare across alternative
risk management choices (e.g., cleanup levels, permit levels,
regulations, actions) or risks due to different levels of conservatism

Decreased ability to make tradeoffs or appropriate balance between
benefits and costs

Liability to criticism and debate of being overly conservative and
unrealistic, or of providing inadequate protection; these criticisms
frequently cannot be directly answered, which reduces the credibility of
EPA decisions.

What are variability and uncertainty and their relevance in risk
assessment?

	The 2004 Staff Paper Risk Assessment Principles and Practices (
HYPERLINK "http://www.epa.gov/osa"  www.epa.gov/osa ) and the PRA White
Paper #1 describe uncertainty and variability in some detail and can be
referred to for more information.

	Variability refers to the inherent natural variation, diversity, and
heterogeneity across time and/or space, or individuals within a
population.  While we can better describe and understand variability in
the world, or a particular system, it is unavoidable and cannot be
reduced.  Variability is present in all aspects of the source to effect
continuum (Figure 1 below):

Figure 1. Source-to-Effect Continuum

in how pollutants are released (e.g., effectiveness of emission
controls),

influenced by environmental conditions once released (e.g., meteorology
– wind and precipitation),

exposure to receptors  (e.g., inhalation or ingestion rates),

effect (e.g., endpoint, health status, genetic susceptibility).

An example of variability is the amount of water consumed by a
population.  For example, if we conducted a survey of 1,000 people and
asked them how much water they consumed, we might have the following
distribution (i.e., plot of the data in Figure 2) which shows that 5% of
the population consumes 2.41 liters of water/day or higher while the
average individual consumes 1.4 liters of water/day , based on studies
identified in the Exposure Factors Handbook (EPA, 1997).

Figure 2.  Probability Distribution of Drinking Water Intake

	Uncertainty refers to imperfect knowledge or lack of precise knowledge
of the real world, either for specific values of interest or in the
description of the system.  While numerous schemes for classifying
uncertainty have been proposed, most focus on two broad categories.

Parameter uncertainty refers to uncertainties in specific estimates or
values used in a model.

Model uncertainty refers to the gaps in scientific knowledge or theory
that is required to make accurate predictions.

From a risk manager’s perspective, both are important in that
variability is related to our understanding of whom or what we are
protecting and uncertainty relates to our confidence in the estimate and
the level of protection.

How does PRA address variability and uncertainty?

By contrast, as depicted below PRA uses distributions of values that
reflect variability and/or uncertainty in parameters and/or models; the
result is an overall probability statement of the risk (e.g., what are
the risks to the average or mean individual and high end individual such
as the 95th percentile as illustrated in Equation 2 below).  This
hypothetical example illustrates the approach for estimating individual
risk.  In some cases exposure and health risk are estimated for the
entire population or particular subgroups of the population.

Equation 2:

What is the impact of uncertainty on decisions?

When uncertainty is present -- where data and information are incomplete
or are inadequate -- making informed decisions is more difficult and
there is greater potential for decision errors.  In the case of
environmental regulations, specific decisions may either lead to over-
or under-regulation compared to decisions that could be made with
perfect information.  Setting an environmental standard that is too lax
may threaten public health, while a standard that is unnecessarily
stringent may impose a significant economic cost for a marginal gain in
public health and environmental protection.

What key questions can be asked or considered by decision makers?

The PRA Technical Panel conducted several dialogues with EPA decision
makers, asking them what questions arise when they are faced with the
task of making decisions in the presence of uncertainty.  The following
questions represent typical concerns.

How representative or conservative is the estimate, (e.g., what is the
variability around an estimate)?

What are the major gaps in knowledge, and what are the major assumptions
used in the assessment? How reasonable are the assumptions?

Would my decision be different if the data were different?  Would
additional data collection and research likely lead to a different
decision? How long will it take to collect the information, how much
would it cost, and would the resulting decision be significantly
different?

Will the use of additional resources, such as a probabilistic approach,
impact the decision making in a timely manner (i.e., better characterize
uncertainties, better identify variability, impact timelines, etc.)?

What are the liabilities/consequences of making a decision under the
current level of knowledge and uncertainty?

What is the percentile of the population to be protected?

How do the different alternative decision choices and the interpretation
of uncertainty and variability impact the target population?

How can PRA help inform decisions?

PRA can provide information to decision makers on specific questions
related to uncertainty and variability.  For questions of uncertainty
and to minimize the likelihood of unintended consequences, PRA can help
provide the following types of information:

Characterize uncertainty in estimates (what is the degree of confidence
in the estimate?).  That is, could the prediction be off by a factor of
2, a factor of 10, or a factor of 1,000?

Identify the critical parameters and assumptions which most impact or
influence a decision and the risk assessment;

Identify the “tipping points” where the decision option chosen would
be different if the risk estimates were different, or a different
assumption were valid;

Estimate the likelihood that critical data values exist or the validity
of assumptions;

Estimate the degree of confidence in a particular decision and/or the
likelihood of specific decision errors

Estimate, (in conjunction with other techniques, such as sensitivity
analysis and value of information) the possibility of alternative
outcomes with additional information, or estimate trade-offs related to
different risks or risk management decisions;

Identify impact of additional information on decision making considering
the cost and time to obtain the information and resulting change in
decision (that is, value of information).

For consideration of variability, PRA for example can provide the
following types of information for exposures:

Explicitly define the exposures for various sectors of the population
(whom are we trying to protect?) That is, will the regulatory action
keep 50% of the population, 90% of the population 99.9%  or some other
fraction of the population below a specified exposure, dose, or risk
target?

Provide information including the variability in the exposures among the
population, and information on the percentile of the population that is
being evaluated in the risk assessment (i.e., people who consume a glass
of water/ day or people who consume a gallon of water/day).  This
information is helpful in addressing comments:

from the regulatory community on conservatism of EPA’s risk
assessments;

from the community regarding concerns whether their particular exposures
were assessed in the risk assessment,

about whom or what is being protected by a risk management action, and

whether and what additional research may be needed to reduce
uncertainty.

PRA helps inform decisions by identifying the alternatives available to
the   HYPERLINK "http://en.wikipedia.org/wiki/Decision_maker" \o
"Decision maker"  decision maker , the   HYPERLINK
"http://en.wikipedia.org/wiki/Uncertainty" \o "Uncertainty"  uncertainty
 they face, and by providing evaluation measures of outcomes (often
referred to as decision analysis). Uncertainties are often represented
as   HYPERLINK "http://en.wikipedia.org/wiki/Probabilities" \o
"Probabilities"  probabilities  or   HYPERLINK
"http://en.wikipedia.org/wiki/Probability_distributions" \o "Probability
distributions"  probability distributions , in graphs or numerically.

A few hypothetical examples of the types of risk management questions
which can explicitly be addressed through PRA are illustrated in Figure
3 below.

Figure 3.  What Questions Can PRA Address?

What are some of the limitations of PRA?

	PRA relates to the application of probabilistic techniques to one or
more phases of the risk assessment paradigm, including hazard
characterization, exposure, toxicity, and/or risk assessment.  Data may
not be available to support probabilistic techniques at all of these
stages in the same assessment, requiring the risk assessor to continue
to apply some deterministic science-policy assumptions and conversions.
If science-policy assumptions, or default values for parameters are
applied to a PRA, they should to be clearly articulated in the
dissemination of results. PRA typically requires more time to develop
than a deterministic assessment, but these techniques fit into a
graduated, or tiered approach, to risk analysis.  Additional limitations
are that:

PRA is generally more data intensive, requiring additional financial,
time and analytic resources to obtain the necessary statistical
distribution input data for each aspect of the risk assessment.  It is
anticipated that more routine incorporation of probabilistic designs in
risk assessment and its supporting research could reduce this cost
differential.

PRA techniques have been most successful on the exposure aspect of human
health risk assessment.

	The dissemination of a statistical distribution or probability output
number should be carefully related to the quality and coverage of the
input statistical distribution data, otherwise the PRA results could
lead to a false sense of accuracy.

PRA can be used to characterize the uncertainty and variability in
situations with limited data.   As yet, there is not extensive
experience using PRA to characterize the range of effects or the
dose-response for populations, including sensitive populations and
life-stages.

What is EPA’s experience in PRA?

	In the past, EPA has usually, but not always, relied on deterministic
or point estimates to evaluate risk; (e.g., 10-6 or one in a million
risk of cancer).  However, the use of PRA to evaluate uncertainty and
variability in risk assessments is increasing.  These efforts are varied
across Programs and Regions, as well as in complexity and applications.
Many PRA applications focused on specific elements of a risk assessment
(e.g., exposure), variability, or uncertainty.  The document Case Study
Examples of the Application of Probabilistic Risk Assessment in EPA
Regulatory Decision Making contains summary examples of PRAs that have
been conducted to support regulatory decisions and/or regulatory impact
analyses.  A few examples of PRA use in EPA include:

EMAP program:  The Office of Research and Development (ORD) developed
and Office of Water (OW) adopted applied probabilistic sampling
techniques to evaluate nation’s aquatic resources under CWA Section
305(b)

Hudson River PCB-Contaminated Sediment Site: Region 2 evaluated the
variability in risks to anglers who consume recreationally caught fish
contaminated with PCBs from sediment contamination in the Hudson River.

Chromated Copper Arsenate Risk Assessment: ORD and the Office of
Pesticide Programs (OPP) conducted a probabilistic exposure assessment
of children’s exposure (addressing both variability and uncertainty)
to arsenic and chromium from contact with CCA-treated wood playsets and
decks.

Evaluating Ecological Effects of Pesticide Uses: OPP developed a
probabilistic model which evaluates acute mortality levels in generic
and specific ecological species for user-defined pesticide uses and
exposures.

PM2.5 Health Impacts: The Office of Air and Radiation (OAR) used expert
elicitation to more completely characterize, both qualitatively and
quantitatively, the uncertainties associated with the relationship
between reduction in PM2.5 and benefits of reduced PM2.5-related
mortality.

EPA’s experience with PRA includes not only individual assessments or
applications but also the development of general guidance and policies
such as these:

Policy for Use of Probabilistic Analysis in Risk Assessment (1997),

Guiding Principles for Monte Carlo Analysis (1997), and

Risk Assessment Guidance for Superfund, Volume III – Part A. Process
for Conducting Probabilistic Risk Assessment.

How should a risk manager approach a PRA?  What should a manager
consider?

There is a range of probabilistic risk analysis techniques that may be
useful to support environmental decisions.  Communication between the
risk managers and risk assessors is critical for clear definition of the
specific needs of the decision maker and the questions to be addressed
by the PRA. The risk assessor and risk manager should evaluate the types
of techniques appropriate to meet the goals of the assessment and
establish a process for completing and reviewing the PRA in a
cost-effective and timely manner. The dialogue should continue until the
PRA is completed to everyone’s satisfaction.

When should we consider doing PRA?

	Conducting a sensitivity analysis within the context of the decision
can help managers determine whether having such information is critical
and that the time and resources spent to perform PRA are warranted in
specific cases.  PRA may not be needed when the decision is routine,
legislatively mandated, or a standard methodology is prescribed.
Furthermore, PRA may not be needed when there is high confidence in the
data and models used to support the decision.  On the other hand
planning and scoping discussions or a preliminary analysis may indicate
that information from a PRA may be critical or influence the risk
management decision.  Some examples include :

A specified target level of protection in a population is identified
(e.g., 95th percentile), and it is necessary to demonstrate that this
goal is met;

Significant equity issues are raised by variation in risks among the
exposed population of concern;

Screening level point estimates of risk are higher than the level of
concern;

Uncertainty is high, and decisions are contentious or have large
resource implications;

Specific critical risk estimates and assumptions point to different risk
management alternatives;

Scientific rigor and quality of the assessment is critical to
credibility of the EPA decision.

What is the right level of analysis?

As is the case for risk assessment in general, approaches to PRA and
specific analytical methods may vary dramatically in terms of complexity
and resource implications.  The concept of iterative or tiered analyses
to address this continuum is widely accepted in risk assessment, and the
same principle applies to PRA as well.  There is a wide range of methods
and approaches to PRA, of varying complexity and rigor, which can be
applied for different purposes ranging from sensitivity analysis to
integrated analysis of uncertainty and variability.  The goal is to
choose a level of detail and refinement for an analysis appropriate to
the overall objectives of the decision and the types of available data
and analyses needed to support decisions.  Early and continued dialogue
between risk manager and risk assessor is critical to developing a clear
understanding of overall project objectives, needs of the decision
maker, timing, and how PRA may play a role.  These discussions should
focus on deciding the following:

(1) whether or not the risk assessment, in its current state, is
sufficient to support risk management decisions (a clear path to exiting
the process is available); and

(2) if the assessment is determined to be insufficient, whether or not
progression to a higher level of complexity would provide a sufficient
benefit to warrant the additional effort of performing a PRA.

If I am going to use PRA what are things to consider?

	If one decides that the use of PRA would provide valuable information
in support of a decision, some other things to consider in moving
forward include:

Resources needed to develop the PRA and review the document,

Expertise of EPA staff to develop a PRA or review a PRA submitted by a
contractor or member of the regulated community;

Data availability and format (e.g., electronic or paper copy) to develop
distributions to include in the PRA,

Time needed for the development and review of the analyses,

Funding, either intramural or extramural that may be necessary for
development and review of the document,

Peer-review including either internal and/or external review which has
time and cost implications, and

Communicating results to the scientific community, Agency executives,
stakeholders and the general public.

What are the resources and costs needed to conduct a PRA?

PRA can be expected to require more time, effort and resources than
standard default-based deterministic assessments.  The costs and
resources will vary depending on the tool or approach that is selected.
That is, there is a continuum of PRA approaches to choose from, ranging
from simple approaches such as sensitivity analyses to complex
approaches such as two-dimensional Monte Carlo analyses   In some cases,
simple sensitivity analyses, which may require limited time and risk
assessor resources, can be conducted in-house.  More sophisticated
analyses may require specific expertise or use of specific tools or
models.  Proper application of probabilistic methods requires not only
software and data, but also guidance and training for both analysts
using the tools as well as for managers and decision makers tasked with
interpreting and communicating the results.  While increases in
resources needed to conduct a probabilistic assessment can be expected,
the development of standardized approaches and/or methods can lead to
the routine incorporation of PRA in Agency approaches and greatly
reduced costs in future applications.

Does PRA require more data than conventional approaches?

	In general, PRA requires more data than conventional approaches because
distributions of values rather than single values are used.  How much
more data is required is often the topic of debate in the technical
community.  Minimum data needs vary depending on the analytical approach
used; empirical-based (observational or frequentist) methods have
significant data requirements compared to so called subjective methods.
However, some of the data that would be applied in a frequentist
approach may already be available as part of the underlying data set
used in standard deterministic analyses.  As a result, PRA can be
applied in most cases, as long as methods used are appropriate for the
available body of evidence and data.

Communication of PRA Results to the Manager and Community.  Does
presentation of results matter?

	The lack of familiarity with PRA presents a challenge in effectively
presenting results to managers, stakeholders, and the public.  Many view
PRA as a highly technical discipline utilizing sophisticated mathematics
and requiring extensive training to understand.  Single point estimates
are easy to grasp for most people, based in part on familiarity with the
approach over the history of EPA. While some people initially have
difficulty interpreting probability distributions of values, we all have
common baseline experiences with probability, uncertainty and
variability (e.g., weather forecasting); these could be used to frame
discussion of results.  It is not necessary to understand the underlying
mathematics or even to include results as full distributions.  Results
can be distilled to the critical essence or decision-meaningful value of
interest.

The audience and its range of knowledge and expertise must be considered
in developing materials for effective communication.  It is helpful when
a decision is made to conduct a PRA to consider early explanation or
training of the community, managers, and others in the basic principles
before the final decision is presented.  Alternatively, it may be
helpful to present the results of the PRA with the point estimate to
provide context for the results.

How can I get more information on PRA?

	This document provides a general overview and basic concepts to
establish some familiarity and a foundation for further education on
PRA.  The white paper entitled "Using Probabilistic Methods to Enhance
the Role of Risk Analysis in Decision making – Uses and Case Studies
in EPA" provides more of a detailed discussion of PRA and EPA's
experience with it. There are numerous additional resources for more
detail on PRA.  Additionally, the RAF PRA technical panel has been
tasked with developing resources to facilitate the understanding and
implementation of PRA.  It is developing an electronic clearinghouse of
resources (policies, guidance, tools, case studies) as well as specific
training seminars which will soon be available.  More information on the
PRA technical panel and the clearinghouse can be found at   HYPERLINK
"http://www.epa.gov/raf"  www.epa.gov/raf  or on the Environmental
Science Connector.  See also the EPA source for links to risk assessment
methods and policies:   HYPERLINK "http://www.epa.gov/risk"
www.epa.gov/risk .

GLOSSARY

Analysis. Examination of anything complex to understand its nature or to
determine its essential features (WHO IPCS Risk Assessment Terminology)

Assessment. The analysis and transformation of data into policy-relevant
information that can assist decision making and action.

Assessment end point. 1. Quantitative or qualitative expression of a
specific factor or metric with which a risk may be associated, as
determined through an appropriate risk assessment. 2. An explicit
expression of the environmental value that is to be protected,
operationally defined by an ecological entity and its attributes. For
example, salmon are valued ecological entities; reproduction and age
class structure are some of their important attributes. Together, salmon
“reproduction and age class structure” form an assessment end point.

Ecological risk assessment. An ecological risk assessment evaluates the
potential adverse effects that human activities have on the plants and
animals that make up ecosystems. The risk assessment process provides a
way to develop, organize, and present scientific information, so that it
is relevant to environmental decisions. When conducted for a particular
place, such as a watershed, the ecological risk assessment process can
be used to identify vulnerable and valued resources, prioritize data
collection activity, and link human activities with their potential
effects.

Ecosystem. The interacting system of a biological community (plants and
animals) and its nonliving environment.

Environment. The sum of all external conditions affecting the life,
development, and survival of an organism.

Expert elicitation.  Expert elicitation (EE) is a systematic process of
formalizing and quantifying, typically in probabilistic terms, expert
judgments about uncertain quantities.

Frequentist (or frequency) probability. A view of probability that
concerns itself with the frequency of events in a long series of trials,
or is based upon a data set.

Inputs. Quantities that are input to a model.

Model. 1. A set of constraints restricting the possible joint values of
several quantities. 2. A hypothesis or system of belief regarding how a
system works or responds to changes in its inputs. 3. A mathematical
function with parameters that can be adjusted so the function closely
describes a set of empirical data. A mechanistic model usually reflects
observed or hypothesized biological or physical mechanisms and has model
parameters with real-world interpretation. In contrast, statistical or
empirical models selected for particular numerical properties are best
fits to data; model parameters may or may not have real-world
interpretation. When data quality is otherwise equivalent, extrapolation
from mechanistic models (e.g., biologically based dose-response models)
often carries higher confidence than extrapolation using empirical
models (e.g., logistic models).

Modeling. 1. Development of a mathematical or physical representation of
a system or theory that accounts for all or some of its known
properties. Models often are used to test the effect of changes of
components on the overall performance of the system. 2. Use of
mathematical equations to simulate and predict real events and
processes. 3. Development or application of conceptual or graphical
methods to depict the structure and organization among major elements of
the system to be modeled.

Parameter. 1. A variable, measurable property whose value is a
determinant of the characteristics of a system (e.g., Temperature,
pressure, and density are parameters of the atmosphere.). 2. A constant
or variable term in a function that determines the specific form of the
function but not its general nature, as “a” in f(x) = ax, where
“a” determines only the slope of the line described by f(x). 3. A
variable entering into the mathematical form of any probability
distribution model such that the possible values of the variable
correspond to different distributions.

Probability. 1. Frequentist approach/ The frequency with which samples
are obtained within a specified range or for a specified category (e.g.,
the probability that an average individual with a particular mean dose
will develop an illness). 2. Bayesian approach. The degree of belief
regarding the different possible values of a quantity or event.

Probabilistic risk analysis. Application of a computational method,
based on a randomized sampling of available data or information or
probabilities obtained from experts, to produce a probability
distribution to more fully describe the data than selecting a single
point in the distribution, e.g., the mean.

Risk. 1. Risk includes consideration of exposure to the possibility of
an adverse outcome, the frequency with which one or more types of
adverse outcomes may occur, and the severity or consequences of the
adverse outcomes if such occur. 2. The potential for realization of
unwanted, adverse consequences to human life, health, property, or the
environment. 3. The probability of adverse effects resulting from
exposure to an environmental agent or mixture of agents. 4. The combined
answers to (1) What can go wrong? (2) How likely is it? and (3) What are
the consequences?

Risk analysis. 1. A process for identifying, characterizing,
controlling, and communicating risks in situations where an organism,
system, subpopulation, or population could be exposed to a hazard. Risk
analysis is a process that includes risk assessment, risk management,
and risk communication (WHO). 2. A detailed examination, including risk
assessment, risk evaluation, and risk management alternatives, performed
to understand the nature of unwanted, negative consequences to human
life, health, property, or the environment; an analytical process to
provide information regarding undesirable events; the process of
quantification of the probabilities and expected consequences for
identified risks.

Risk assessment. 1. A process intended to calculate or estimate the risk
to a given target organism, system, subpopulation, or population,
including the identification of attendant uncertainties following
exposure to a particular agent, taking into account the inherent
characteristics of the agent of concern, as well as the characteristics
of the specific target system (WHO). 2. The evaluation of scientific
information on the hazardous properties of environmental agents (hazard
characterization), the dose-response relationship (dose-response
assessment), and the extent of human exposure to those agents (exposure
assessment) (NRC, 1983). The product of the risk assessment is a
statement regarding the probability that populations or individuals so
exposed will be harmed and to what degree (risk characterization)
(USEPA, 2000). 3. Qualitative and quantitative evaluation of the risk
posed to human health or the environment by the actual or potential
presence or use of specific pollutants.

Risk-informed decision making. An approach to decision making in which
insights from probabilistic risk analyses are considered with other
insights and factors.

Risk management. A decision-making process that takes into account
environmental laws, regulations, political, social, economic,
engineering, and scientific information, including a risk assessment, to
weigh policy alternatives associated with a hazard.

Scenario. 1. An outline or model of an expected or supposed sequence of
events. 2. A set of facts, assumptions, and inferences about how
exposure takes place and regarding how exposures translate into adverse
effects that aides the analyst in evaluating, estimating, or quantifying
exposures and risks. Scenarios might include identification of
pollutants, pathways, exposure routes, and modes of action, among
others.

Sensitivity analysis. A study of how the variation in data inputs
(including inputs to models) affect the outputs of a model or choice
among potential decision options.

Levels. Refers to various hierarchical levels of complexity and
refinement for different types of modeling approaches that can be used
in risk assessment. A deterministic risk assessment with conservative
assumptions is an example of a lower level type of analysis that can be
used to determine whether exposures and risks are below levels of
concern. Examples of progressively higher levels include the use of
deterministic risk assessment coupled with sensitivity analysis, the use
of probabilistic techniques to characterize either variability or
uncertainty only, and the use of two-dimensional probabilistic
techniques to distinguish between but simultaneously characterize both
variability and uncertainty.

Two-dimensional probabilistic analysis. A modeling approach in which
inter-individual variability in exposure and risk is characterized using
frequency distributions, and in which uncertainty in the estimates of
statistics of the frequency distributions (e.g., the mean, median,
standard deviation, percentiles) are characterized using probability
distributions.

Uncertainty. Occurs because of a lack of knowledge. It is not the same
as variability. For example, a risk assessor may be very certain that
different people drink different amounts of water but may be uncertain
about how much variability there is in water intakes within the
population. Uncertainty often can be reduced by collecting more and
better data, whereas variability is an inherent property of the
population being evaluated. Variability can be better characterized with
more data but it cannot be reduced or eliminated. Efforts to clearly
distinguish between variability and uncertainty are important for both
risk assessment and risk characterization, although they both may be
incorporated into an assessment.

Uncertainty analysis. A detailed examination of the systematic and
random errors of a measurement or estimate; an analytical process to
provide information regarding uncertainty.

Value of information. A quantitative measure of the value of knowing the
outcome of an uncertain variable prior to making a decision. Decision
theory provides a means for calculating the value of both perfect and
imperfect information. The former value, informally known as the value
of clairvoyance, is an upper bound for the latter. Obtaining meaningful
value-of-information measurements requires an awareness of important
restrictions (concerning the nature of free will) on the validity of
this kind of information.

Variability. Refers to true heterogeneity or diversity, as exemplified
in natural variation . For example, among a population that drinks water
from the same source and with the same contaminant concentration, the
risks from consuming the water may vary. This may result from
differences in exposure (i.e., different people drinking different
amounts of water and having different body weights, different exposure
frequencies, and different exposure durations), as well as differences
in response (e.g., genetic differences in resistance to a chemical
dose). Those inherent differences are referred to as variability.
Differences among individuals in a population are referred to as
inter-individual variability, differences for one individual over time
is referred to as intra-individual variability.

EXTERNAL PEER REVIEW DRAFT – DO NOT CITE OR QUOTE

 PAGE   1