[Federal Register Volume 79, Number 98 (Wednesday, May 21, 2014)]
[Notices]
[Page 29181]
[FR Doc No: 2014-11715]

-----------------------------------------------------------------------

DEPARTMENT OF ENERGY

Federal Energy Regulatory Commission

[Docket No. EL14-49-000]

Southwest Power Pool, Inc.; Notice of Petition for Declaratory
Order

    Take notice that on May 9, 2014, Southwest Power Pool, Inc. (SPP),
pursuant to Rule 207 of the Federal Energy Regulatory Commission's
(Commission) Rules of Practice and Procedure, 18 CFR 385.207, filed a
petition for declaratory order requesting that the Commission confirm
that: (1) Section 7.4 of the SPP Open Access Transmission Tariff does
not limit SPP's right to seek contract damages in an appropriate court
for the nonpayment default for the entire term of the service
contracted under the PTP Service Agreement with AES Shady Point, LLC
(AES PTP Agreement); (2) nothing in the Commission's regulations
requiring that transmission providers submit notices of termination
before terminating service limits SPP's ability to seek damages for
breach of contract in an appropriate court; and (3) the Commission's
order \1\ accepting cancellation of the AES PTP Agreement does not
limit SPP's right to seek damages in an appropriate court for AES's
nonperformance of the AES PTP Agreement.
---------------------------------------------------------------------------

    \1\ Southwest Power Pool, Inc., Docket No. ER13-989-000
(unpublished letter order issued on April 23, 2013).
---------------------------------------------------------------------------

    Any person desiring to intervene or to protest this filing must
file in accordance with Rules 211 and 214 of the Commission's Rules of
Practice and Procedure (18 CFR 385.211, 385.214). Protests will be
considered by the Commission in determining the appropriate action to
be taken, but will not serve to make protestants parties to the
proceeding. Any person wishing to become a party must file a notice of
intervention or motion to intervene, as appropriate. Such notices,
motions, or protests must be filed on or before the comment date. On or
before the comment date, it is not necessary to serve motions to
intervene or protests on persons other than the Applicant.
    The Commission encourages electronic submission of protests and
interventions in lieu of paper using the ``eFiling'' link at http://www.ferc.gov. Persons unable to file electronically should submit an
original and 5 copies of the protest or intervention to the Federal
Energy Regulatory Commission, 888 First Street NE., Washington, DC
20426.
    This filing is accessible on-line at http://www.ferc.gov, using the
``eLibrary'' link and is available for review in the Commission's
Public Reference Room in Washington, DC. There is an ``eSubscription''
link on the Web site that enables subscribers to receive email
notification when a document is added to a subscribed docket(s). For
assistance with any FERC Online service, please email
<EMAIL_ADDRESS>or call<PHONE_NUMBER> (toll free). For
TTY, call<PHONE_NUMBER>.
    Comment Date: 5:00 p.m. Eastern Time on June 9, 2014.

    Dated: May 14, 2014.
Kimberly D. Bose,
Secretary.
[FR Doc. 2014-11715 Filed 5-20-14; 8:45 am]
BILLING CODE 6717-01-P


--------------------------------------------------------------------------------

	GUIDANCE ON USE OF MODELED RESULTS

	TO DEMONSTRATE ATTAINMENT OF THE OZONE NAAQS

	 	EPA-454/B-95-007

	U.S. Environmental Protection Agency 	Office of Air Quality Planning and Standards 	Research Triangle Park, NC 27711

	June 1996

DISCLAIMER

This report has been reviewed by the Office of Air Quality Planning and Standards, U.S. Environmental Protection Agency, and has been approved for publication.  Any mention of trade names or commercial products is not intended to constitute endorsement or recommendation for use.

	TABLE OF CONTENTS

EXECUTIVE SUMMARY	S-1

1.0 INTRODUCTION	1 1.1 Purpose	1 1.2 Background	1

2.0 OVERVIEW OF THE ATTAINMENT DEMONSTRATION PROCESS	3

3.0 EPISODE SELECTION	5

4.0 APPROACH 1: STATISTICAL APPROACH	9 4.1 The Statistical Test	9 4.1.1 Benchmark 1	9 4.1.2 Benchmark 2	13 4.1.3 Benchmark 3	18 4.1.4 Summary of the Statistical Test	19 4.2 Weight of Evidence Determination	19 4.2.1 Model Performance and Results	19 4.2.2 Transport	20 4.2.3 Other Analyses	21 4.3 Further Discussion of the Statistical Test	21 4.3.1 Derivation of the Default Ranking Procedure	21 4.3.2 Sensitivity of the Ranking Procedure	22 4.3.3 Performance of Ranking Procedures	22 4.3.4 Use of Alternative Ranking Procedures	23

5.0 APPROACH 2: DETERMINISTIC APPROACH	25 5.1 The Deterministic Test	25 5.2 Exception Through Use of a Focused Model Performance Evaluation	25 5.3 Weight of Evidence Determination	26 5.3.1 Factors Affecting Confidence in Modeled  Results	26 5.3.2 Severity of Modeled Episodes	27 5.3.3 Trend Analysis	28 5.3.4 Use of Observational Models	30 5.3.5 Considering Incremental Costs and Benefits	30 5.3.6 Other Optional Analyses	31 5.3.7 Transport	31 5.3.8 Summary, Weight of Evidence Determinations  in the Deterministic Approach	31 5.4 Summary of Deterministic Approach	33

6.0 MULTI-STAGE NATURE OF THE ATTAINMENT DEMONSTRATION        PROCESS	35

7.0 SUMMARY	39

8.0 REFERENCES CITED	43

APPENDIX A: EXAMPLE APPLICATION	45

APPENDIX B: GLOSSARY	51

APPENDIX C: RELATED ISSUES	55

EXECUTIVE SUMMARY

S.1 Purpose

The purpose of this document is to revise the modeled test for demonstrating attainment of the national ambient air quality standard (NAAQS) for ozone.  We also refine guidance for selecting episodes to model so that greater consideration is given to an area's ozone design value and the severity of meteorological conditions accompanying observed exceedances.

The recommended revisions make the modeled attainment test more closely reflect the form of the NAAQS.  Like the NAAQS, the test now permits occasional exceedances at any location.  The recommended revisions are also intended to take account of uncertainties inherent in available models and in estimating future emissions.  These uncertainties have become better appreciated since intensive efforts to apply the Urban Airshed Model (UAM) for regulatory purposes began in 1991.

S.2 Overview

Two acceptable approaches are identified for demonstrating attainment of the ozone NAAQS.  These are shown in Figure S.1.  The first of these is called the "Statistical Approach".  This approach contains a "Statistical Test" and a weight of evidence determination.  We refer to the second acceptable approach as the "Deterministic Approach".  This approach also consists of two parts:  a "Deterministic Test" and a weight of evidence determination.

If the test in the selected approach is not passed, there is an option to perform a weight of evidence determination using additional information, such as air quality data.  If this leads to compelling evidence that attainment is likely, attainment is demonstrated.  In a weight of evidence determination, model results are weighed heavily.  The further results are from passing the test, the more difficult it is to develop compelling supplementary evidence that attainment is likely.

Projecting ozone concentrations several years into the future has attendant uncertainties.  Uncertainty about growth, location of new sources and effectiveness of future technologies are major components of uncertainty about model predictions.  A technically viable attainment demonstration for "severe" and "extreme" nonattainment areas should include provision for at least one mid-course review of air quality, emissions and modeled data.  A second review, shortly before the statutory attainment date, is also required.  To make the mid-course review as insightful as possible, we recommend that the attainment demonstration include projections to some intermediate year (e.g., ~1999-2000).  When combined with a mid-course review of updated air quality,

--------------------------------------------------------------------------------

emissions and modeled data, this should prove helpful in assessing whether refinements are needed in the current control strategy.

Figure S.2 shows the 3-stage process (current review, mid-course review and review at the statutory date) we require.  Note that as the attainment date nears, increased reliance on observed data is anticipated.  This results from anticipated improvements in the data base as well as from use of shorter projection periods.

S.3 The Statistical Approach for Demonstrating Attainment       The Statistical Approach consists of a test and an optional weight of evidence determination.  The statistical test has three parts.

(1) It allows up to 3 exceedances at every location, depending on severity of modeled episodes.  Exceedances are only allowed on episode days which are "severe".

(2) It limits the magnitude of each allowed exceedance.  Limits depend on severity of each episode, and are calculated so as to be consistent with observed ozone patterns at sites currently attaining the NAAQS.

(3) If the model underpredicts observed ozone, the test requires at least an 80% reduction in the predicted incidence of ozone greater than 124 ppb.

The preceding test depends critically on our ability to rank severity of days selected for modeling.  We have developed a default procedure for doing this.  In the default procedure, days are ranked using a regression equation relating highest daily maximum ozone concentration to several meteorological variables.  If the regression equation is unable to explain at least 65% of the observed variation in the highest daily maxima, the Statistical Approach should not be used.

A weight of evidence determination is included as a second, optional part of the Statistical Approach.  This determination entails use of supplementary analyses to determine whether attainment is likely despite model results which do not pass the statistical test.  The further results are from passing the test, the more difficult it becomes to demonstrate attainment through use of a weight of evidence determination.  The weight of evidence concept is described more fully in Section S.4.

S.4 The Deterministic Approach for Demonstrating Attainment

The Deterministic Approach consists of a test plus an optional weight of evidence determination.  The deterministic test is

--------------------------------------------------------------------------------

passed if daily maximum concentrations predicted in every surface grid cell are < 124 ppb for all primary episode days.  A primary  episode day is generally every modeled day except for the first day of each episode.

A weight of evidence determination may be undertaken to demonstrate attainment despite results which do not pass the deterministic test.  As with the statistical test, the further results are from passing the deterministic test, the more difficult it becomes to demonstrate attainment through use of a weight of evidence determination.  However, the deterministic test is more conservative than the statistical test.  Thus, the burden of proof which weight of evidence from supplementary analyses needs to overcome is less in the Deterministic Approach.

In a weight of evidence determination, each of the analyses shown in Table S.1 should be considered and the results should be documented.  The degree to which each additional analysis is pursued is decided on a case by case basis.  Documentation accompanying weight of evidence results should include explanations of why excluded analyses were not pursued.  The middle column in Table S.1 outlines factors which would lead one to assign greater weight to the indicated analysis.  The right-hand column describes outcomes which would lend support to concluding that attainment is demonstrated, even though model results do not quite pass the established test.

Table S.1.  Factors Affecting Weight of Evidence and Acceptance of Model Results Nearly Passing the Attainment Test

Type of Analysis

Factors Increasing Weight of Evidence

Factors Supporting Deviation from Test Benchmark(s)

Photochemical Grid Model

-good performance -extensive data          base -short projection        period -confidence in          inventories &        projections

-overpredictions -major improvement     in predicted AQ      using a variety      of indicators -results come very     close to meeting     the benchmark(s) -other peer-          reviewed grid        models predict       comparable or        better improvement   in ozone

Trend Data

-extensive            monitoring network -precursor & ozone     trends avail. -statistical model     normalizing trend    explains much        variance -little bias in        statistically        predicted highest    ozone -short projection      period -pronounced, stat.     significant          normalized trend -continued,            comparable           relative             reductions in        emissions            provided for

-pronounced            downward             normalized trend     exceeding that       anticipated with     grid model

Observational Models

-extensive            monitoring network -QA'd, self-          consistent results -plausible,           physical             explanations for     findings

-indicates sources    other than those     in modeled           strategies play      significant roles

Selected Episodes

-all met.regimes       corresponding w.     high obs. O3         considered -met.ozone             potential of         episodes exceeded    ~ 1/year

-observed O3 >>        design value -Severity of met.      conditions           expected to be       exceeded << 1/yr

Incremental Costs/Benefits

-good documentation   for cost estimates -lack of              alternatives for     reducing emissions -lack of model        responsiveness for   variety of           strategies as        benchmark is         approached

-lack of model        responsiveness       accompanied by       high incremental     costs

Other (optional) Analyses

-rationale            documented

1.0 INTRODUCTION 	 1.1 Purpose

This document updates portions of the Guideline for Regulatory Application of the Urban Airshed Model (U.S.EPA, 1991) so that guidance better reflects experience gained in model applications since 1991.  The updated guidance supports ongoing revisions to State implementation plans (SIP's) to meet the national ambient air quality standard (NAAQS) for ozone.  More specifically, guidance described herein will be used along with the 1991 Guideline to implement phase II of the SIP revisions described in a March 2, 1995 U.S. EPA policy memorandum (Nichols, 1995).

Changes described herein focus on the modeled attainment test assessing whether a proposed control scenario will likely lead to attainment of the NAAQS by statutory dates.  Thus, guidance described in Section 6.4 of the 1991 Guideline is superseded.  We also refine earlier guidance on episode selection (Section 3.1 and Appendix B of the 1991 Guideline).  By adding these latter refinements, we do not mean to imply that States should consider replacement episodes in their ongoing SIP revisions.  However, if a State elects to simulate a new episode, we recommend that efforts be made to follow the refined guidance.

1.2 Background

As States and the EPA gained experience applying the Urban Airshed Model (UAM), several things became apparent.  First, photochemical grid models require a great deal of information.  Much of this information is uncertain.  Further, model formulation reflects limits imposed by existing scientific knowledge as well as by computational necessities.  Uncertainties in model inputs and limitations in model formulation lead to uncertainties in model predictions.  This implies that a revised modeled attainment test should provide means to take better account of uncertainty.

A second finding from recent model applications is that controls estimated as necessary to attain the NAAQS can be very high.  Despite such estimates, monitored ozone data reflect downward trends in many areas over the past 10 years (U.S.EPA, 1994).  Monitored data are the definitive means for classifying an area's attainment status.  This has led to a number of redesignations from "nonattainment" to "attainment" status.  It has also led to concerns among some that the existing modeled attainment test may be too conservative.

The monitored attainment test allows 1.0 expected exceedance per year of a daily maximum ozone concentration of 124 ppb at all monitoring sites.  The previous modeled test required estimated daily maxima to be 120 ppb or less in all surface grid cells for all selected primary episode days.  The apparent contrast between the two tests causes concern that the modeled test may lead one to prescribe controls beyond those necessary to pass the monitored test.  The guidance described herein provides an opportunity to more closely replicate the monitored test.  This is done by considering severity of selected episodes more explicitly and allowing modeled exceedances on "severe" days (i.e., days having meteorological conditions which are unusually conducive to high ozone formation or transport).

2.0 OVERVIEW OF THE ATTAINMENT DEMONSTRATION PROCESS

In the preceding section, we noted two goals for a revised attainment test:  (1) it should provide a means to consider uncertainty, and (2) it should replicate the monitored test more closely by allowing an occasional modeled exceedance.  These two goals must be consistent with the overriding purpose of an attainment demonstration:  to provide a reasonable expectation that the measures and procedures outlined will result in attainment of the NAAQS by statutory dates.

We identify two approaches for meeting these goals.  Both approaches need to include documentation that the episodes considered include days with high observed ozone concentrations. Both procedures also include provisions for performing one or more periodic reassessments of observed and predicted air quality data prior to the statutory attainment date.  This is to ensure that additional control measures can be invoked before the statutory date should a need be identified later.  We refer to these reassessments as "mid-course reviews".

The first acceptable approach is called the "Statistical Approach".  It includes a test which allows modeled exceedances on up to 3 days at different locations, depending on the severity of the selected episode days.  In addition to limits on allowable number of modeled exceedances, the test includes a limit on the magnitude of daily maximum concentrations exceeding 124 ppb and a minimum required reduction in the occurrence of modeled values in excess of 124 ppb.  Thus, the test includes three benchmarks.  The first of these limits the number of days with allowed exceedances.  The second restricts the magnitude of an allowed exceedance.  The third benchmark requires a minimum level of improvement in air quality to be exceeded.

The Statistical Approach uses ranked severity of episode days in a quantitative manner.  Ranking severity of episode days  introduces additional uncertainties.  If the ranking scheme fails to meet defined performance standards,  the Statistical Approach may not be used.  In this case, it is necessary to revert to the second acceptable approach (i.e., the "Deterministic Approach", to be discussed later).  The Deterministic Approach may consider severity of  selected episodes also.  However, it does so more qualitatively, in concert with other analyses, in a weight of evidence determination.

If one or more of the statistical test's benchmarks is failed, a weight of evidence determination may be performed using corroborative information.  If the corroborative information is consistent with the likelihood that a proposed strategy will lead to attainment of the NAAQS by statutory dates, attainment has been demonstrated.  The further model results are from passing the test's benchmarks, the more difficult it is to show that a

strategy is, nevertheless, adequate in a weight of evidence determination.      We call the second acceptable approach for demonstrating attainment the "Deterministic Approach".  This approach consists of a deterministic test and an optional weight of evidence determination.  The deterministic test is passed if predicted daily maximum ozone concentrations are < 124 ppb in all surface grid cells on all modeled primary episode days.  Thus, the test contains a single benchmark.  Exceptions may be considered if modeled exceedances are few, and are likely attributable to an artifact introduced by the model.  In addition, if the test is not passed, a weight of evidence determination may be used to show that attainment of the NAAQS is still likely.  Because the deterministic test is more conservative than the statistical test, the burden of proof needed in a weight of evidence determination to permit small deviations from the test's benchmark is less than in the case for the statistical test.

The remainder of this document is organized as follows.  In Section 3.0, we offer some supplementary guidance on episode selection which can be used with information in Section 3.1 and Appendix B of Guideline for Regulatory Application of the Urban Airshed Model (U.S.EPA, 1991).  In Section 4.0, we describe the Statistical Approach for demonstrating attainment.  In Section 5.0, we discuss the Deterministic Approach for demonstrating attainment.  Because weight of evidence determinations are potentially more important in the Deterministic Approach, use of weight of evidence is also described in Section 5.0.  Section 6.0 addresses the multi-stage nature of attainment assessments.  Section 7.0 summarizes key points in this guidance.  Finally, we include several appendices.  Appendix A contains a complete example illustrating use of this guidance with a set of hypothetical model results.  Appendix B is a glossary of new or frequently used terms in this document.  Appendix C presents issues related to use of this guidance.  These issues are discussed in question/answer format.

3.0 EPISODE SELECTION

In this section, we recommend procedures for selecting new episodes to model.  Selection of new episodes for modeling is optional.  The procedure we recommend differs from that described in Section 3.1 and Appendix B of the 1991 Guideline in only minor respects.  The principal differences are:  (1) we now recommend that areas strive to model days with observed daily maxima close to the design value for the most severely classified nonattainment area being modeled; and (2) ranked severity of meteorological conditions accompanying the air quality observations should be considered as a factor in choosing the episode days, where reliable rankings exist.  The issue of reliability and performance standards for ranking procedures is discussed further in Section 4.3.  The goal of the episode selection process is to choose episodes containing days with observations near but slightly above the design value and meteorological ozone forming potential likely to be exceeded about once per year.  Other factors affecting choice of episodes remain as stated in the 1991 Guideline.

The following step by step example illustrates the recommended procedure.

Step 1.  Identify Distinctive Meteorological Regimes and List Days  Within Each Regime According to the Observed Highest Daily Maximum Ozone Concentration

This step is identical to the current procedure.  The primary means of distinguishing meteorological regimes is the source/receptor orientation implied by the windfield.  Table 3.1 illustrates the results of this step.

Table 3.1.  Example Identifying Candidate Days For Modeling

Met.Regime 1

Met.Regime 2

Met.Regime 3

C11

C12

C13

C21

C22

C23

C31

C32

No more obs. above 124 ppb

C41

C42

        -

C51

No more obs. above 124 ppb

        -

C61

         -

        -

No more obs. above 124 ppb

         -

        -

where C12 is the day with the highest observed daily maximum ozone concentration occurring with meteorological regime 2.

Step 2.  Use Data Compiled by the EPA (or Calculated Using an Approved Alternate Procedure) to Assign a Ranking in the Meteorological Ozone Forming Potential for Each Day Identified in Step 1.

Using a default ranking procedure, we have compiled ranked lists of ozone forming potential for days during the ozone season for each day over a 41- year period of record for nearly every modeled area.  The ranking is based on meteorological ozone forming potential using a regression model developed by Cox and Chu (Cox and Chu, 1993; Cox and Chu, 1996).  Thus, this effort should be minimal for the States if the default procedure is used for the rankings.  One merely identifies the assigned rank for each of the days listed in Step 1.  Use of alternate ranking procedures is discussed further in Section 4.3.

As a result of Step 2, Table 3.1 becomes Table 3.2.

Table 3.2.  Candidate Days with Associated Ranked Ozone Forming Potentials

Met. Regime 1

Met. Regime 2

Met. Regime 3

C11=190 ppb    (1)

C12=162 ppb    (33)

C13=145 ppb   (150)

C21=177 ppb    (5)

C22=155 ppb    (57)

C23=135 ppb   (55)

C31=167 ppb    (79)

C32=134 ppb    (110)

        -

C41=145 ppb    (12)

C42=129 ppb    (104)

        -

C51=141 ppb    (32)

         -

        -

C61=141 ppb    (45)

         -

        -

        -

         -

        -

Concentrations shown in the table are the highest daily maximum ozone concentration observed by any monitoring site in or downwind of the non-attainment area on the indicated day.  Thus, the highest observations noted in the table could well occur at different monitoring sites.  Each number in parentheses is the ranking of the corresponding day's meteorological ozone forming potential.

Step 3.  Estimate Rank of the Meteorological Ozone Forming Potential Associated with One Observed Exceedance per Year.

Suppose meteorological ozone forming potential has been evaluated for days spanning a 41-year period of record.  In this case, the meteorological ozone forming potential associated with days ranked at least as high as "41" (i.e., the ranked number < "41") would be expected to be exceeded < once/year.

Step 4.  Select from Among Days in the Table Generated in Step 2 Considering the Design Value and Ranking Data.

One would proceed down each column until an observed ozone concentration approaches the design value and/or a ranking equivalent to 1 expected exceedance (ExEx) per year of the estimated meteorological ozone forming potential is found.  For reasons which will be elaborated on in Section 4.3, rankings between "20" and "83" are judged to be equivalent to 1 ExEx with a 41-year period of record.  Days which are too extreme could be discarded.  For days with comparably ranked meteorological ozone forming potentials, preference is given to days having observed concentrations closest to the design value.  If the ranking scheme performs poorly or marginally, preference should be given to selecting days with observations above, but approaching, the design value.  (This assumes no other overriding issue like presence of an intensive data base or poor model performance applies--see Section 3.1 in the 1991 Guideline).

Using Table 3.2, let us illustrate the process for selecting candidate episode days.  We begin by moving down column 1 for Meteorological Regime 1.  In this example, we will assume a design value of 160 ppb.  Days C11, C21 and C41 are probably too extreme.  Such meteorological ozone forming potentials would occur with frequencies of once in 41 years, 8 years and ~3.5 years respectively.  Days C31, C51 and C61 have equivalent rankings.  Day C31 is the choice for Meteorological Regime 1, because it is closest to and slightly above the design value.  Going through a similar thought process, Day C12 is selected from Meteorological Regime 2.  Meteorological Regime 3 presents an interesting case.  Here, one day (C23) has a ranking approximately equivalent to 1 ExEx, but also has an observed daily maximum well below the design value.  The other day (C13) has a daily maximum closer to the design value, but with an episode less extreme than 1 ExEx.  In conflicts such as this, we recommend going with the day closest to the design value.  This recommendation stems from the uncertainties attendant with the rankings.  (to be discussed in Section 4.3).  Thus, we choose Day C13 for Meteorological Regime 3.  Summarizing, using our recommended guidance for episode selection, we have chosen episodes containing days C31, C12 and C13 for modeling in this example.

4.0 APPROACH 1: STATISTICAL APPROACH

The Statistical Approach includes a modeled test in which three benchmarks should be passed.  First, the number of days with predicted exceedances in defined locations should not be greater than a specified number.  The number depends on the severity of the modeled episode days.  Second, for episode days in which modeled exceedances are allowed, predicted daily maxima should not exceed a certain value.  This value depends on the severity of the selected episode as well as the shape of distributions of observed daily maxima at sites which currently just attain the NAAQS.  Third, for each day with an allowed exceedance, improvement in the number of hourly occurrences with predicted ozone greater than 124 ppb should be at least 80%.  This third requirement may be waived on any day where the model does not underpredict observed ozone.

Ranking each episode day is a critical part of the Statistical Approach.  The default procedure used to develop the rankings for each day's meteorological ozone forming potential is described more fully in Cox and Chu (1993) and in Cox and Chu (1996).  We  summarize key points of the procedure in Section 4.3.  In large modeling domains containing several Metropolitan Statistical Areas (MSA's) for which ranked data are available, which of these data to use becomes an issue.  This issue is discussed in Appendix C.

The Statistical Approach allows use of additional analyses to justify deviations from the test's benchmarks.  Weight given to these additional analyses depends on the performance of the photochemical grid model, confidence in its inputs and confidence in the procedure for ranking episodes, as well as on the strength of the additional analyses.  The three benchmarks and each of these additional analyses are discussed in Sections 4.1-4.2.

4.1 The Statistical Test

4.1.1 Benchmark 1

	Benchmark 1: For a Composite of All Episodes, The Number Of Primary Episode Days Where A Modeled Exceedance Occurs Within Any Defined Subregion Of The Model Domain Should Be < 3 Or < (N-1), Whichever Is Less.

"N" is the number of "severe" primary episode days which are modeled.  A day is considered "severe" if its associated "meteorological ozone forming potential" is expected to be exceeded fewer than two times per year (i.e., ExEx < 2.0/year).

No modeled exceedances are permitted on primary episode days with meteorological ozone forming potential which is expected to be exceeded two or more times per year (i.e.,  ExEx > 2.0/year).

Some clarifications are needed before we can illustrate use of benchmark 1 with an example.  First, what do we mean by a "defined subregion of the model domain", and why is this distinction needed?  We define a "subregion" as an area which approximates a 15 km x 15 km area as closely as permitted by the model's horizontal grid resolution.  This is illustrated in Table 4.1.

	Table 4.1 Definition of Subregions for Grids with Differing Horizontal Resolution

Grid Resolution, km

Grid Squares in Subregion

Area of Subregion, km[2]

       2

8 x 8 = 64

      256

       3

5 x 5 = 25

      225

       4

4 x 4 = 16

      256

       5

3 x 3 = 9

      225

       8

2 x 2 = 4

      256

       12

1 x 1 = 1

      144

       16

        1

      256

       18

        1

      324

We recommend that, for consistency, non-overlapping clusters of grid squares (defining the subregions) originate in the SW corner of the modeling domain.  Exceptions can be made on a case by case basis, if warranted.  The key requirement is that the subregions are defined consistently for each modeled day.  Definition of subregions by clusters of grid squares is illustrated in Figure 4.1 for a grid having 5 km horizontal resolution.  A more complete example illustrating identification of subregions is contained in Appendix A.

We believe it is necessary to define subregions within the modeling domain because of uncertainty attendant with wind fields used in photochemical grid models.  Winds affect positioning of precursor and ozone plumes.  There is no single standardized approach for specifying wind fields, and the exercise requires subjective judgment.  Thus, latitude exists for adjusting the wind field within reasonable bounds of uncertainty so as to minimize the likelihood of having more than one predicted exceedance in individual grid cells of the size used in urban applications.  For each such subregion, the maximum predicted daily maximum concentration is used in determining consistency of predictions with the benchmark.  "Maximum" refers to the highest

--------------------------------------------------------------------------------

concentration from among those grid cells that comprise the given subregion.  Defining and using subregions in this way provides a  means to remain protective of the NAAQS, given uncertainty about the wind field.

The method used to equate a ranking with an expected exceedance frequency also deserves further explanation.  Equivalency between a ranking and an ExEx is a function of the number of years for which the meteorological ozone forming potential corresponding with each day is calculated.  In the set of default information we will provide to help implement this test, we have used a 41-year period of observed meteorological conditions.  Thus, a day with a rank of "42" would be expected to have its meteorological ozone forming potential exceeded exactly once per year, on average.  Similarly, a day with a rank of "83" would be expected to be exceeded 2.0 times/year, and a day ranked "21" would be expected to occur just slightly less frequently than 0.5 times/year (i.e., once every 2 years).  If one were to use a set of regression equations applied to a 30-year period of record, rankings of "16", "31" and "61" would correspond to expected exceedance frequencies of 0.5, 1.0 and 2.0/year, respectively.  In general, the relationship between a ranking and an expected yearly frequency of exceedance (ExEx) is given by:

            ExEx = (Rank - 1)/(# of years used for ranking)

Thus, if a day were ranked 30th based on an examination of meteorological observations over a 41-year period, the day's corresponding expected exceedance frequency would be:

            ExEx = (30 - 1)/41 = 0.7 times per year.

	Examples

Problem 1.  A State models 3 episodes containing 6 primary episode days.  Modeled exceedances occur in only 1 subregion.  These occur on 3 of the days, with meteorological ozone forming potential values expected to be exceeded 0.4, 1.2, and 1.8 times per year.  The three days without modeled exceedances in this subregion have ExEx's of 1.5, 1.95 and 8.7 times per year.  Is benchmark 1 passed?

First, we see whether there are any subregions in which modeled exceedances occur on more than 3 days.  (Since the number of severe primary episode days (N) is "5", the requirement that the number of days with modeled exceedances be no more than "3" is the most restrictive requirement.)  There are no subregions having more than 3 days of modeled exceedances.

Finally, we check whether there are any days having an ExEx as frequent as 2.0 times/year or more on which a modeled exceedance occurs.  There are none. The benchmark is met.

Problem 2.  Suppose conditions are identical to those in problem 1, except now there is also a second subregion in which modeled exceedances occur on the day with 0.4 expected exceedances per year and on the day with 1.95 expected exceedances per year.  Does the area still pass the benchmark?

First, we note that there are still no subregions in which modeled exceedances occur on more than 3 days.  Note that this milestone is passed, even though there are now 4 days in which a modeled exceedance occurs somewhere in the modeling domain.

Finally, there are still no days having an expected exceedance frequency of 2.0/year or more on which there is a modeled exceedance.

Thus, the area still passes the benchmark.             4.1.2 Benchmark 2

	Benchmark 2:  Predicted Daily Maxima Corresponding With Each Allowed Modeled Exceedance May Not Be Greater Than A Concentration Derived From A Distribution Of Observed Daily Maxima At Sites Currently Just Attaining The NAAQS.

This benchmark also needs further explanation before we can illustrate its use.  We have examined the EPA AIRS data base for 1991-1993 to identify ozone monitoring sites which are currently "just attaining" the ozone NAAQS.  By "just attaining", we mean that 2-3 exceedances are observed at the site during a 3-year period, and the ozone design value is > 115 but < 124 ppb.  There are 60 sites meeting our criteria for "just attaining".  These occur in 21 States at urban, suburban and rural locations, and in areas where the predominant landuse is characterized as "commercial", "agricultural", "industrial", "residential", "forest" and "mobile".  Thus, our data include a diverse set of sites.  Despite this diversity, we find that the shape of the tail of the distribution of daily maxima is very similar at these sites.  This enables us to fit a distribution curve to a large pooled data base whose most extreme observations have an associated probability of exceedance which is very small.  Figure 4.2 shows the result of this fitting.

The ability to create the distribution of daily maxima described in the preceding paragraph is important.  We are interested in knowing what the distribution of daily maxima might look like at other sites after sufficient controls are implemented to attain the NAAQS.  In particular, we are interested in what the distribution should look like at sites which will just attain the NAAQS.  Such sites will have exceedances of 124 ppb.  We want to know whether the magnitude of a modeled exceedance is consistent with the expected shape of the distribution of daily maxima at a

--------------------------------------------------------------------------------

large number of sites which are already just attaining the NAAQS.  This can be estimated as follows:

      (1) estimate the probability that the meteorological ozone forming potential is exceeded for each day on which an exceedance is allowed;

      (2) for each such day, enter the ordinate in Figure 4.2 at the indicated probability and read off the corresponding daily maximum ozone concentration.

For example, suppose the meteorological ozone forming potential for a modeled day is ranked as the 15th highest for a 41-year period of analysis.  The probability that this potential is exceeded is:

      p = (15/41) x (1/365) = .001002

Thus, there is a 0.1% likelihood that the meteorological ozone forming potential associated with this day is exceeded.  Entering Figure 4.2, we see that a daily maximum ozone concentration of 134 ppb has an associated probability of 0.1% of being exceeded.  Putting this another way, a daily maximum concentration of 134 ppb on such an extreme day is consistent with meeting the NAAQS.

Calculation of probabilities and use of Figure 4.2 is cumbersome.  Table 4.2 may be used to test whether benchmark 2 is met.  The columns in the table correspond to the period of record used to rank the ozone forming potential for each day.  This information is provided for the convenience of those utilizing ranking procedures other than the default one reflected by the Cox/Chu statistical model.  The information is needed if a period of record differing from the default one of 41 years is used with these alternate approaches.  Those using the default rankings which we provide should use the column labeled "41" in the table.

Table 4.2.  Acceptable Upper Limits for Daily Maximum Ozone (ppb) on Very Severe Days with Allowed Modeled Exceedances

Yrs of Rec Day Rank

10

15

20

25

30

35

41

45

50

1

148

151

152

155

158

160

161

161

163

2

140

144

148

150

151

151

152

153

155

3

137

140

141

145

148

150

151

151

151

4

133

137

140

141

144

146

148

150

150

5

130

136

138

140

140

143

145

146

148

6

133

137

138

140

140

142

144

145

7

131

135

137

138

140

140

141

143

8

130

133

136

137

139

140

140

141

9

132

135

137

138

139

140

140

10

130

133

136

137

138

139

140

11

132

134

137

137

138

139

12

131

133

136

137

137

138

13

130

132

134

137

137

137

14

131

133

136

137

137

15

130

132

134

136

137

16

131

133

135

136

17

130

132

134

136

18

131

133

135

19

130

131

132

133

20

124

131

132

133

21

130

131

132

22

130

132

23

131

24

131

25

124

130

130

130

130

130

130

130

130

Table 4.2 contains several interesting features.  First, a day whose meteorological ozone forming potential is expected to be exceeded at a specified rate (e.g., 0.5 ExEx/year) has an associated daily maximum ozone concentration which remains constant, regardless of the number of years used to create the rankings.  This follows from the manner in which the ExEx rate is calculated (ExEx/yr = (rank - 1)/(# of years in data base)).  In the table, an ExEx rate of 0.5/year always corresponds to a daily maximum observation of 130 ppb.  We use this value as the maximum value permitted for an exceedance corresponding to an ExEx rate between 0.5 and 2.0/year.  For reasons identified in Section 4.3, we consider this range equivalent to an ExEx of 1.0/year, given uncertainties in the ranking procedure.  Thus, to pass benchmark 2, a modeled exceedance corresponding to a day ranked such that 0.5/year < ExEx < 2.0/year should not exceed 130 ppb.

Days ranked as severe or more severe than 0.5 ExEx/year may have modeled exceedances greater than 130 ppb.  The acceptable limits are presented in Table 4.2.  Thus, Table 4.2 should be used to establish the acceptable upper bound for a modeled exceedance on very severe episode days.

	Example

Problem 3:  Suppose the maximum modeled exceedances on the 4 days with exceedances in Problem 2 are as shown below.  Assume ozone forming potential rankings are based on a 41-year record.

Day

ExEx

Corresponding Rank

Modeled Max. Daily Maximum O3, ppb

1

0.4

16

138

2

1.2

49

130

3

1.8

74

126

4

1.95

80

128

Is benchmark 2 met?

First, we note that days 2, 3 and 4 fall in the range where 0.5 < ExEx < 2.0/year.  Benchmark 2 requires modeled daily maxima not to exceed 130 ppb for these days.  This milestone is met.

Next, we observe that day 1 is more severe.  Entering Table 4.2 in the column labeled "41", we see that the maximum allowed value for the 16th ranked day is only 133 ppb.

Thus, benchmark 2 is not met, since 138 ppb > 133 ppb.

4.1.3 Benchmark 3

Benchmark 3:  For Each Day With An Allowed Modeled Exceedance, The Number Of Daytime Hourly Exceedances Of 124 PPB Predicted Throughout The Domain Should Be Reduced By At Least 80%.

This benchmark is included to provide a safeguard against cases where photochemical grid model predictions meet EPA performance criteria but, nevertheless, tend to underpredict observed concentrations.  It could be possible in some cases like this to pass the first two benchmarks with few control measures.  The type of example we are most concerned about is one in which concentrations in the order of 150 ppb are observed on a day with high ozone forming potential, yet predictions are in the order of 130-135 ppb.  We believe a benchmark like this one is necessary to provide assurance that a control strategy passing the statistical test is likely to lead to significant improvements in air quality on occasions when observed concentrations are high.

Unlike benchmarks 1 and 2, benchmark 3 applies to predictions in individual surface grid cells rather than in subregions.  It also applies to all daytime hourly predictions (i.e., between 8 AM and 8 PM Local Standard Time (LST)) rather than just daily maxima.  The benchmark is passed if the predicted number of "grid cell-hours" with predictions > 124 ppb is reduced by at least 80% on each day for which a modeled exceedance is allowed by benchmark 1.  The calculation is made as follows:

     SUM[(# of cells)base]- SUM[(# of cells)control]  >  0.80                   SUM[(# of cells)base]                                            where each term in the above expression is summed over the 12 hours between 8 AM and 8 PM LST on each day for which a modeled exceedance is allowed.  The subscript "base" in the above expression refers to current base case conditions.  That is, if a 1991 episode were being simulated, then the required 80% improvement is relative to the 1991 model predictions used in the UAM performance evaluation.

Since benchmark 3 is included to provide protection in cases where the model underpredicts observed ozone concentrations, it may be waived for any given day if underpredictions do not occur on that day.

For each day with an allowed exceedance, we recommend comparing the maximum peak modeled and maximum peak observed concentrations throughout the domain to determine whether the model is underpredicting observed peak ozone.  If the test results do not indicate an underprediction of more than 5% for a given day, benchmark 3 need not be applied on that day.

4.1.4 Summary of the Statistical Test

The statistical test is passed if three benchmarks are passed.

      1. Limits on Number of Modeled Exceedances. No subregion in the domain may contain modeled exceedances on more than 3 or "N-1" days, whichever is less.  "N" is the number of severe days (i.e., days with ExEx < 2,0/year) which are modeled.  No exceedances are permitted on a modeled day unless it is severe enough so that its ozone forming potential would be expected to be exceeded fewer than 2 times/year.

      2. Limits on Modeled Concentrations for Allowed Exceedances. Predicted daily maximum ozone concentrations may not exceed 130 ppb on days with allowed exceedances unless the severity of the episode day is expected to be exceeded < 0.5 times per year.  For such very severe episodes daily maximum ozone predictions are limited as shown in Table 4.2.

      3. Required Minimum Level of Improvement. Reduction in the number of (surface) grid cell-hours exceeding 124 ppb during 8 AM to 8 PM local standard time should be at least 80% for each day on which a modeled exceedance is allowed.  This benchmark can be waived for a given day if the model does not underpredict observed ozone concentrations.

Appendix A contains a detailed numerical example illustrating application of the statistical test.

4.2 Weight of Evidence Determination

The statistical test described in Section 4.1 already includes substantial effort to consider corroborative evidence.  That is, the test considers severity of the modeled episodes and utilizes information developed from observed frequency distributions of monitored daily maxima for ozone. Thus, additional corroborative analyses used in weight of evidence determinations need to be compelling to permit deviations from the benchmarks in the statistical test.  The weight of evidence determination provides additional information to enable those reviewing an attainment demonstration to conclude that attainment of the NAAQS is probable even though minor exceptions to the benchmarks exist.

4.2.1 Model Performance and Results

General model performance and focused model performance may be considered in a weight of evidence determination.  If the general performance measures (see Appendix C, 1991 Guideline) indicate that model predictions of ozone are not biased low,  this supports the notion that small deviations from one or more of the benchmarks may, nevertheless, be consistent with attainment.

Focused model performance evaluation examines ozone production and the model's treatment of meteorological factors affecting ozone predictions in a small number of grid cells.  If it can be shown that a high prediction most likely reflects a model artifact rather than chemistry or physics of the atmosphere and the prediction in question affects the outcome of the test, the prediction can be disregarded.  Focused performance evaluation is described more fully in Section 5.2.

If model results indicate major improvement in predicted ozone and the episodes considered are very severe (i.e., ExEx < 0.5/year), this may also support a small deviation from a benchmark.

4.2.2 Transport

Transport of ozone and precursors into a modeling domain can be an important factor affecting ozone predictions obtained with the UAM and with other photochemical grid models.  Thus, transport affects the photochemical grid model's results.  As illustrated in Figure S.1 (executive summary), the Statistical and Deterministic Attainment Approaches are applied after model results are obtained.  That is, the attainment tests compare model results, obtained with a specified emissions scenario, transport and other assumptions, with the NAAQS.  Although it is certainly true that transport can affect the outcome of a test, it does not necessarily have to be a part of the test.

As noted in the preceding paragraph, the primary impact of transport is to affect model results, which are then compared to the NAAQS using a modeled attainment test.  There are two ways in which the modeled attainment test itself could more directly reflect transport.  First, an area-specific ranking scheme could be developed which contains variables which represent regional transport more directly than is done in the Cox/Chu default procedure which we have used.  Some suggestions on how to do this are offered in Appendix C.  Days classified as "severe" (i.e., ExEx < 2.0 days per year) using such a scheme would be allowed modeled exceedances.  A second way for considering transport in the selected attainment approach is as a part of a weight of evidence determination.  This is described further in Section 5.3 and in Appendix C.

Finally, if it is shown that overwhelming transport influences modeled exceedances in all or in a portion of the modeling domain, a policy decision could be made that the modeled attainment tests need only be applied to a portion rather than to the entire domain.  Such a decision would need to consider factors outside the scope of this guidance.

4.2.3 Other Analyses

Other analyses need to be considered in a weight of evidence determination to sway conclusions about the adequacy of a control scenario for which the benchmarks are nearly met.  These include  the following:  use of normalized trend data, use of results from observational or other models and consideration of incremental cost/benefit estimates.  These additional analyses are described in greater detail when we discuss the Deterministic Approach (see Section 5.3).  We believe weight of evidence determinations are of greater importance in this latter approach.  Factors affecting weight given to various pieces of evidence and outcomes which are consistent with deviating from one or more of the benchmarks are generally consistent with what is shown in Table 5.1.

4.3 Further Discussion of the Statistical Test

4.3.1 Derivation of the Default Ranking Procedure

The default procedure we have used to rank meteorological ozone forming potential of each day is based on a statistical model by Cox and Chu (Cox and Chu, 1993; Cox and Chu, 1996).  To develop this model, Cox/Chu considered the highest daily maximum ozone concentration observed among all monitoring sites each day in each of 32 Metropolitan Statistical Areas/Consolidated Metropolitan Statistical Areas (MSA/CMSA's) and immediate surroundings during 1983-1993.  A number of meteorological variables thought to make some contribution to observed ozone were considered for use in regression equations to explain the observed variation in the highest daily maxima.  Most important meteorological variables were daily maximum surface temperature, morning average wind speed and direction, afternoon average wind speed and direction, midday relative humidity, total opaque cloud cover, and Julian day (a surrogate for solar intensity).  When these variables plus an additional variable depicting underlying trends in ozone during 1983-93 were incorporated into a linear expression used as a scaling factor in a Weibull distribution, good agreement was found with the observed distribution of maxima during the 10-year period.  The meteorological variables alone are successful in leading to a good characterization of the variability in observed maxima.  With a few exceptions, the equations containing meteorological variables only are able to explain between 60 and 80% of the variance in observed daily maxima. Coefficients for the variables in the linear regression equation vary from city to city.

A linear equation including only meteorological variables was then used in a similar fashion to predict "meteorological ozone forming potential" for each day in the ozone season for each of the 32 MSA's/CMSA's during a 41-year period (1953-1993).  We assumed that days outside the ozone season generally have lower ozone forming potential, which is exceeded by most days within the season.  Using this procedure, it was possible to rank the ozone forming potential of each day for each MSA/CMSA during the 41-year period.

4.3.2 Sensitivity of the Ranking Procedure

We have performed some sensitivity tests to determine how rankings of days might be affected by changes in the regression equations underlying estimates of meteorological ozone forming potential.  On the basis of this, we have concluded rankings of 41 + 25 to be approximately equivalent to a 95% level of confidence,  with the distribution of error skewed such that larger uncertainty exists for larger ranked numbers (i.e., less severe days).

Thus, one finding from our sensitivity analysis of the estimated rankings is that they become more sensitive to changes in the statistical model as one moves away from the tail of the ranked distribution.  This occurs, because as one moves away from the tail, there are increasingly small differences in estimated meteorological ozone forming potential between successively ranked days.  Thus, a change in the statistical model leads to occasional large differences in a day's ranking, even though the change in its calculated meteorological ozone forming potential is minuscule.  This finding suggests that, the further one moves away from the tail of the ranked distribution, one should give increasingly greater weight to the observed ozone in selecting episodes.  For rankings above ~100 (for a 41-year record), ranked data may have broad associated uncertainty.  Because differences in calculated meteorological ozone forming potential are greater as one approaches the tail, rankings for the most extreme episodes are likely to be most stable.

We use the preceding findings as the basis for our recommendation that rankings corresponding to ExEx frequencies between 0.5 and 2.0 times per year be considered essentially equivalent.  That is, 41 + 25 is approximately equivalent to this range.

4.3.3 Performance of Ranking Procedures

We believe that for ranked meteorological ozone forming potential to be used in as quantitative a procedure as the statistical test, there should be a minimum performance requirement for the ranking procedure.  We recommend that the variation in the observed highest daily maxima which is explained by the calculated ozone forming potential values should be at least 65% (i.e., adjusted R[2] > 0.65).  This requires that the correlation between observed and predicted daily maxima be greater than 0.8.  If this criterion is not met, we recommend that the Deterministic Approach be used in the attainment demonstration.  Methods and benchmarks described for the statistical test could be used more qualitatively as part of a weight of evidence determination in the Deterministic Approach.

4.3.4 Use of Alternative Ranking Procedures

As we have previously noted, we are suggesting the Cox/Chu equations as a default procedure for deriving means for ranking the severity of episode days.  States may use alternative procedures.  Indeed, in some locations we have been unable to meet the performance criterion recommended in Section 4.3.3 using the Cox/Chu equations.  Thus, if one wishes to use the Statistical Approach, it will be necessary to develop a better ranking procedure for these areas.

To be accepted for use, an alternative ranking scheme should meet the following criteria:

1) there should be a logical physical/chemical explanation relating to ozone formation and/or transport for a variable to be included in a statistical model;

2) adding an additional independent variable to a statistical model should lead to a statistically significant improvement in the model's performance;

3) ability of a model to explain variance in the observed daily maxima should exceed that of the default procedure; and

4) the adjusted R[2] associated with a model should explain at least 65% of the variance in the observed highest daily maxima.

5.0 APPROACH 2: DETERMINISTIC APPROACH

5.1 The Deterministic Test

After a proposed strategy is modeled, the benchmark in this test is to compare the predicted daily maximum ozone concentration in each surface grid cell with 124 ppb on each of the primary episode days.  If all predicted daily maxima are < 124 ppb, the test is passed.

By "primary episode day", we mean all days except for the first day of each period selected to model.  The first day is called a "ramp-up" day, and is excluded from the test because of dependence of initialization assumptions on poorly known or absent data.  Thus, if 3 episodes were selected having 3 days, 2 days and 2 days respectively, all daily maxima predicted in surface grid cells on each of 4 (2 + 1 + 1) primary episode days should be < 124 ppb.

5.2 Exception Through Use of a Focused Model Performance Evaluation

It may happen that the benchmark in this test is very nearly passed, except for isolated modeled daily maxima which exceed 124 ppb.  In such cases, attainment can still be satisfactorily demonstrated using this approach.  First, let us define what we mean by "isolated".  This is subjective, but generally we mean no more than ~2-3 surface grid cells on any primary episode day.

If the predicted exceedances are isolated, then one needs to make the argument that each instance likely reflects inability of the model to properly characterize physical/chemical processes accompanying the modeled exceedance.  If the argument is successfully made, the modeled exceedance(s) should be disregarded.  This argument is made by reviewing available meteorological and air quality data and comparing temporal and spatial patterns with those predicted by the model during the base case and/or projection period for the incident in question.  A relatively new procedure, called "process analysis", is recommended as one means for analyzing model predictions for the purpose of these comparisons whenever feasible (Jeffries et al., 1994, Tonnesen et al., 1994, Jang et al., 1994).

The exception to the test's benchmark that predicted daily maxima be < 124 ppb should not be extended to cases in which there are numerous predicted daily maxima exceeding 124 ppb.  The 1991 Guideline provides means for rejecting episodes for which model performance is generally poor.  These performance criteria are robust, in that they look at many comparisons between observed and predicted values.  We assume that these performance tests provide protection against selecting episodes in which formation of high ozone is widely mischaracterized.  It would be quite possible however, for the model to mischaracterize ozone in a few locations and still pass these performance tests.  The exceptions to meeting the benchmark in the deterministic test are intended to provide a remedy for this concern.

5.3 Weight of Evidence Determination

If the attainment test is not passed and exceedances cannot be explained as model artifacts, the Deterministic Approach allows use of a weight of evidence determination to assess whether attainment is, nevertheless, likely.  The deterministic test is more conservative than the statistical test.  This means that control strategies leading to model results which come close to meeting the deterministic test's benchmark are more likely to result in attainment than would be true for results which almost meet the statistical test's benchmarks.  Thus, the burden of proof required from corroborative analyses is less for a weight of evidence determination to conclude that results which nearly pass the deterministic test are adequate.

A weight of evidence determination includes a subjective assessment of the confidence one has in the modeled results.  This is supplemented with a review of available corroborative information, such as air quality data.  The more extensive and credible the corroborative information, the greater influence it could have in permitting deviations from the deterministic test's benchmark.  Thus, areas having extensive, monitored information have a greater potential for justifying deviations from the benchmark.  In the following subsections, we identify factors affecting weight given to modeled results.  We then identify several corroborative analyses to consider in a weight of evidence determination.  Each of the types of analyses shown in Table 5.1 should be considered.  Arguments supporting conclusions reached in the weight of evidence determination need to be documented.  Reasons for not pursuing a particular type of evidence rigorously also need to be documented.

5.3.1 Factors Affecting Confidence in Modeled Results

Weight of evidence given to model results depends on the following factors:              -model performance;

      -confidence in the underlying data bases;

      -length of the projection period;

      -how close results come to meeting the test's benchmark.        The longer the projection period, more sparse the data base and the poorer and less comprehensive the model performance  evaluation, the lower the "weight of evidence" given to the grid modeling results relative to the other factors considered in a weight of evidence determination.  Generally, the closer results come to meeting the test's benchmark, the less compelling other evidence supporting a deviation from the benchmark needs to be.  In addition, model results showing major improvement in predicted ozone with a variety of indicators could be used to support the notion that attainment is likely.  Results from other peer reviewed grid models, which have replicated observations well, may also be used to support a weight of evidence determination.

5.3.2 Severity of Modeled Episodes

The more extreme the days selected for modeling, the greater the "weight of evidence" that a control strategy leading to model results nearly meeting the benchmark is sufficient to demonstrate attainment.  We have discussed methods for ranking episode severity at length in Sections 4.1 and 4.3.  As noted in Section 4.3, if performance of a statistical model underlying a ranking procedure is insufficient, it is not advisable to use this information quantitatively.  However, it could still be used qualitatively, in concert with other information, to support deviations from the deterministic test's benchmark.  The better the performance of the statistical model underlying the ranking and the more severely ranked a day in a selected episode, the greater the support for deviating from the benchmark.  It is not necessary to perform the analyses described in Section 4.0 in a weight of evidence determination.  Generally however, the more extensive the analysis of episode severity, the greater the weight of evidence which can be assigned.

If it is not feasible to rank meteorological ozone forming potential as discussed in Section 4.0, it is still possible to gain a qualitative sense of an episode's severity through review of air quality data.  If there are several daily maxima in the area which are greater than the area's design value, this provides qualitative information supporting a deviation from the deterministic test's benchmark.  We emphasize the importance of looking at more than a single monitoring site.  A high value at a single site could reflect an upset or unusual emission event at a source or small group of sources.  This does not necessarily imply that the accompanying meteorological conditions are severe.  A further precaution is needed in assessing severity of historical episodes on the basis of air quality observations.  Lower concentrations in recent years may, in part, reflect the results of emission reductions.  Thus, higher concentrations in earlier years do not necessarily imply unusually severe meteorology.

5.3.3 Trend Analysis

This analysis entails normalizing trend parameters (e.g, 99th percentile daily maxima) for meteorological differences observed year to year.  The model described in Cox and Chu (1993) (or a similar type model which is more skillful in explaining adjusted variance in observed daily maximum ozone concentrations) can be used to normalize observed raw trends.  We calculate normalized trends periodically for serious and above nonattainment areas with the Cox/Chu procedure. Recent normalized trends are presented in U.S.EPA (1994) and in U.S.EPA (1994a).

Figure 5.1 illustrates how the resulting information might be used in a weight of evidence determination.  Figure 5.1(a) represents an analysis performed as part of a phase II demonstration (Nichols (1995)).  The procedure calls for extrapolating the most recent 10-year normalized trend line (i.e., 1987-96) in 99th percentile daily maxima to 124 ppb.  If the 124 ppb concentration is reached prior to the statutory attainment date and relative emission reductions at least as great as those in the preceding 10 years are anticipated, one might use this as supporting justification for allowing small deviations from the deterministic test's benchmark.  The argument for permitting deviations from the benchmark would be strengthened if concurrent reductions in ambient precursor levels were observed and the regulations for further reductions were in place.  The trend analysis could be repeated for a mid-course review (Figure 5.1(b)) and again at the statutory attainment date.

The weight given to evidence produced by the trend analysis is a function of several factors.  The more air quality monitoring data available, the more confident we can be that potential exceedances of 124 ppb are being measured.  Therefore, it follows that locations having large numbers of monitoring sites which measure ozone and precursors could give greater weight to trend analysis than could other locations with less extensive data bases.  Other factors which favor giving a large weight to air quality trends include: (a) a high degree of skill (i.e., adjusted R[2]) in the ability of the statistical model (used to normalize the data) to explain observed variance in daily maxima; (b) lack of bias in the statistical model's predictions of the highest observed daily maxima; (c) a short projection period to the statutory attainment date; (d) a suitable normalized trend parameter which approaches 124 ppb; (e) presence of a pronounced normalized trend line over the preceding 10-year period, and (f) regulations in place requiring further emission reductions.  Presence of a statistically significant downward trend exceeding that expected from the photochemical grid modeling results, would support deviation from a benchmark.

--------------------------------------------------------------------------------

5.3.4 Use of Observational Models

Observational models take advantage of monitored data to draw conclusions about the relative importance of different types of VOC and/or NOx emissions as factors contributing to observed ozone.  There are at least 4 observational approaches currently under investigation within the research community: use of indicator species (Trainer, et al., 1993, Milford, et al., 1994, and Sillman, 1995), the relative incremental reactivity (RIR) approach (Cardelino and Chameides, 1995), the smog produced algorithm approach (Blanchard and Roth, 1994) and receptor models (Lewis, et al., 1993, and Henry, et al., 1994).

Currently, the potential for observational models to quantify precursor reductions needed to demonstrate attainment is not clear.  In the near term, we believe the principal value of these methods is to identify control directions (VOC, NOx, both) and source categories making major contributions to ozone formation.  Thus, their role in a weight of evidence determination is to provide means for corroborating whether a control strategy identified in a photochemical grid modeling analysis is addressing key contributors to observed high ozone.  Thus, they may be particularly valuable if used in concert with an incremental cost/benefit analysis (see Section 5.3.5).  Weight given to results from observational models depends on suitability and comprehensiveness of the available monitoring network, consistency of results with physical/chemical understanding of ozone formation and transport, and presence of several analyses which complement one another and appear to be consistent.   If these techniques confirm a strategy, the "weight of evidence" they provide supports a more aggressive approach (i.e., trying to come closer to meeting the benchmark).  If the results are contradictory, they may support a position that controlling certain emissions further in pursuit of the benchmark should be postponed.

5.3.5 Considering Incremental Costs and Benefits

When used in concert with other supporting information, we believe this is a legitimate consideration in assessing whether additional efforts are warranted to meet the deterministic test's benchmark.  By "incremental" costs/benefits, we mean the difference in costs and the difference in benefits which accompany the current strategy being considered vs. a strategy which comes closer to meeting the benchmark.  We recommend that incremental benefits be estimated by computing total dosage in the domain above 40 ppb with and without the incremental control measures in question.  Our understanding is that there is no apparent lower threshold for effects from exposure to ozone.  Our suggestion to use "40" ppb rather than "0" ppb stems from a commonly used value for continental background and from a desire to better highlight differences between strategies.  Dosage is best weighted by the spatial distribution of population.  Thus, units would look something like, "person-ppb-hours".

The incremental cost/benefit test we suggest is a qualitative rather than quantitative one.  If small incremental benefits are accompanied by large incremental costs, this supports not immediately pursuing this particular strategy to come closer to passing the benchmark. We must emphasize that, due to the qualitative nature of these kind of calculations, we are not suggesting 1:1 tradeoffs between costs and benefits.  Rather, we suggest that if the model predictions appear to be relatively unresponsive to additional controls, resulting in large incremental costs, it may be appropriate to conclude that model results are close enough to the benchmark, given other corroborative evidence.

5.3.6 Other Optional Analyses

Other types of analyses, in addition to those described, may be used to support a weight of evidence determination.  The rationale underlying use of such an analysis, results obtained and how these results support or do not support a conclusion that attainment is likely need to be documented.

5.3.7 Transport

See Section 4.2.2.

5.3.8 Summary, Weight of Evidence Determinations in the 		Deterministic Approach

Table 5.1 summarizes factors influencing "weight of evidence" determinations and their use in evaluating model results which nearly meet the benchmark(s) in the attainment test.  The middle column of the table includes factors which would increase the weight given to evidence produced by the indicated analysis.  The right-hand column describes outcomes which would contribute to evidence supporting a deviation from the modeled attainment test's benchmark(s).

Table 5.1.  Factors Affecting Weight of Evidence and Acceptance of Model Results Nearly Meeting the Attainment Test's Benchmark(s)

Type of Analysis

Factors Increasing Weight of Evidence

Factors Supporting Deviation from the Benchmark

Photochemical Grid Model

-good performance -extensive data          base -short projection        period -confidence in          inventories &        projections

-overpredictions -major improvement     in predicted AQ      using a variety      of indicators -results come very     close to meeting     the benchmark -other, peer           reviewed grid        models predict       comparable or        better               improvement in       ozone

Trend Data

-extensive            monitoring network -precursor & ozone     trends avail. -statistical model     normalizing trend    explains much        variance -little bias in        statistically        predicted highest    ozone -short projection      period -pronounced, stat.     significant          normalized trend -continued,            comparable           relative             reductions in        emissions            provided for

-pronounced            downward             normalized trend     exceeding that       anticipated with     grid model

Observational Models

-extensive            monitoring network -QA'd, self-          consistent results -plausible,           physical             explanations for     findings

-indicates sources     other than those     in modeled           strategies play      significant roles

Selected Episodes

-all met.regimes       corresponding w.     high obs. O3         considered -met.ozone             potential of         episodes exceeded    ~ 1/year

-observed O3 >>        design value -ExEx of met.ozone     potential << 1/yr

Incremental Costs/Benefits

-good documentation    for cost             estimates -lack of               alternatives for     reducing             emissions -lack of model         responsiveness       for variety of       strategies as        benchmark is         approached

-lack of model        responsiveness       accompanied by       high incremental     costs

Other (Optional) Analyses

-rationale for each    is documented

5.4 Summary of Deterministic Approach

The Deterministic Approach includes a test with a benchmark requiring that predicted daily maximum ozone in all surface grid cells on all primary episode days be < 124 ppb.  A focused model performance evaluation may be used to exclude a limited number of modeled exceedances if it is shown that these likely result from an artifact of the model.

The Deterministic Approach includes an optional weight of evidence determination which may be undertaken to demonstrate attainment in the face of model results which do not meet the test's benchmark.  The burden of proof which this determination must meet increases as photochemical grid modeling results deviate further from the benchmark.

A weight of evidence determination consists of several identified corroborative data analyses.  Results from these analyses are used qualitatively to decide whether attainment of the NAAQS is likely even though the test's benchmark is not quite met.

6.0 MULTI-STAGE NATURE OF THE ATTAINMENT DEMONSTRATION PROCESS

In Sections 1.2 and 2.0, we noted that a major purpose of our redefinition of the modeled attainment test for ozone is to take better account of uncertainty in assessing whether a control scenario will be sufficient to meet the NAAQS.  Thus far, we have addressed this goal by incorporating corroborative data either directly into a test (i.e., the statistical test) or by making use of these data in weight of evidence determinations (the Deterministic Approach and the Statistical Approach).

One of the most important causes of uncertainty however, is our inability to confidently predict what will happen 10-15 years into the future.  Uncertainty in predicting economic growth, population changes and new technology causes major uncertainty in the estimates of projected emissions this far into the future.  The assumptions made about growth can have pronounced effects on projected emissions and, therefore, on the levels of control needed to attain the NAAQS.  This arises from the compounding effect over many years.  To illustrate, the difference in emissions projected from 1990-2005 using a 3%/year growth rate as opposed to a 2%/year growth rate is 21%--a difference larger than the composite effect of several control measures.  Inability to predict where changes will occur exacerbates this uncertainty.

Because of the uncertainty inherent in long term projections, we believe that a technically viable attainment demonstration needs to contain provisions for periodic reviews of the monitoring, modeling and inventory data to assess the extent to which refinements to the originally identified attainment strategy are needed.  Such reviews could be coordinated in some manner with the Clean Air Act's required reasonable further progress assessments.  At a minimum, we require that an acceptable attainment demonstration for a "severe" or "extreme" nonattainment area contain provisions for at least one "mid-course" review between completion of the phase II analyses and the statutory attainment date, as well as a review at or shortly before the required attainment date.  We also strongly recommend that model applications in the phase II analyses make projections to the period to be considered in a mid-course review.  This will permit more meaningful comparisons between observations at the mid-course review and the modeled projections.  These comparisons, in turn, will support weight of evidence determinations performed as part of the mid-course reviews.

Thus, we require at least a 3-stage analysis:  the current (1995-97) phase II analysis; a mid-course review (circa 1999-2001) and a third review at or shortly before the statutory attainment date for "severe" areas (circa 2004-2006).  As described in Sections 4.0 and 5.0, the first stage of the analysis (i.e., the 1995-97 phase II analysis) consists of approaches which use photochemical grid modeling and a series of corroborative analyses with monitored data, statistical models and meteorological information.  As the statutory date approaches, we anticipate greater reliance on observed data will be possible.  The procedure we describe takes advantage of more reliable and extensive (e.g., PAMS) monitored air quality and meteorological data to corroborate predictions from past modeling and from improved modeling tools available in the future.  Most importantly, projections are made over progressively shorter timeframes.  Figure 6.1 presents a conceptual view of the required multi-stage attainment demonstration.

--------------------------------------------------------------------------------

7.0 SUMMARY

We have presented two approaches which are acceptable for demonstrating attainment of the NAAQS for ozone.  These are identified as a "Statistical Approach" and a "Deterministic Approach".  The approaches are designed to meet three objectives:

      1. provide reasonable assurance that the control scenario which is selected will be sufficient to attain the NAAQS by statutory dates,

      2. take account of uncertainty inherent in the data bases and in the available means for performing photochemical grid modeling, and

      3. more closely replicate the monitored attainment test and, in so doing, more closely replicate the NAAQS which permits occasional exceedances at a number of locations.

The Statistical Approach includes a statistical test and a weight of evidence determination.  Weight of evidence can be used to justify small deviations from the test's benchmarks.  Because the test already incorporates some corroborative information and is less conservative than the test in the Deterministic Approach, burden of proof needed to deviate from benchmarks is greater than for the Deterministic Approach.

The statistical test contains three benchmarks.  The first of these limits the number of days in which modeled exceedances are allowed at any given location within the model domain to three.  This limit is reduced if three or fewer "severe" days are modeled.  The second benchmark limits the magnitude of an allowed modeled exceedance.  This limit is based on severity of modeled episode days, as well as on distributions of daily maximum ozone concentrations presently observed at monitoring sites where the NAAQS is attained.  The third benchmark requires the number of occurrences in which hourly ozone predictions exceed 124 ppb to be reduced by at least 80%.  This final benchmark is included as a safeguard in the event the photochemical grid model underpredicts observed ozone.  It may be waived for any day on which no such underpredictions occur.

The statistical test depends critically on use of a procedure to rank severity of episode days.  Rankings are based on a statistical model relating highest daily maxima to meteorological variables. If this statistical model cannot explain at least 65% of the variation in an area's highest daily maximum ozone concentration, the Statistical Approach may not be used to demonstrate attainment of the NAAQS.

We have refined existing guidance on episode selection to enable consideration of the ranked severity of candidate days.  We have also recommended that greater priority be given to selecting days having observed ozone near the ozone design value.  When multiple non-attainment areas are modeled, the design value for the most severe of these governs the choice of episodes.  Weight given to the ranked values in episode selection also depends on the degree to which the statistical model underlying the rankings explains variance in the observed highest daily maximum ozone concentration.

The second acceptable approach for demonstrating attainment is called the "Deterministic Approach".  It consists of a deterministic test containing a single benchmark.  The Deterministic Approach also includes a weight of evidence determination which may be used to see whether a deviation from the test's benchmark is warranted.  The deterministic test's benchmark is met if there are no modeled exceedances of 124 ppb in any surface grid cell during any of the modeled primary episode days.  A limited number of exceptions is allowed if a focused model performance evaluation shows that the modeled exceedance(s) is likely a result of an artifact introduced by the model rather than physics and chemistry of the atmosphere.

The Deterministic Approach allows greater latitude in using weight of evidence determinations to permit deviations from its test's benchmark than is appropriate for deviations from benchmarks in the statistical test.  This stems from the more conservative nature of the deterministic test and the greater likelihood that a strategy leading to results which nearly meet the benchmark will lead to attainment of the NAAQS.  The weight of evidence determination is used to qualitatively support an argument that model results close to the benchmark demonstrate attainment.

Figure 7.1 summarizes the sequence of activities in the recommended approaches for demonstrating attainment.  We believe that the two approaches we have recommended are consistent with the NAAQS and monitored attainment test in that they both permit occasional modeled exceedances.  Consideration of episode severity in the choice of episodes is also consistent with the NAAQS.

Use of corroborative information either directly in a test or as part of a weight of evidence determination helps address uncertainty.  However, a major (perhaps the major) source of uncertainty arises from projecting future events over long periods of time.  Therefore, we have required that a technically viable attainment demonstration addressing the 2005 timeframe include provisions for at least one "mid-course" review of monitored, emission and modeling information.  To make this review more insightful, we recommend that current model demonstrations include projections and predictions for the time period envisioned for this review.

--------------------------------------------------------------------------------

8.0 REFERENCES CITED

Blanchard, C.L. and P.M. Roth, (1994), Spatial Mapping of 		Preferred Strategies for Reducing Ambient Ozone 			Concentrations Nationwide, Prepared under EPA Contract 	68D10154, Joseph Bufalini, EPA Project Officer.

Cardelino, C.A. and W.L. Chameides, (1995), "An Observation Based 	Model for Analyzing Ozone Precursor Relationships in the Urban Atmosphere", J.Air and Waste Management Association, 45, pp.161-181.

Cox, W.M. and S. Chu, (1993), "Meteorologically Adjusted Ozone 	Trends in Urban Areas:  A Probabilistic Approach", Atmospheric Environment, 27B, (4), pp.425-434.

Cox, W.M. and S. Chu, (1996), "Assessment of Interannual Ozone 	Variation in Urban Areas from a Climatological 		      	Perspective", 	Atmospheric Environment 30, pp.2615-2625.

Henry, R.C., C.W. Lewis and J.F. Collins, (1994), "Vehicle-		Related hydrocarbon Source Compositions from Ambient Data:  	The GRACE/SAFER Method", Environmental Science and 		Technology, 28, pp.823-832.

Jang, J., H.E. Jeffries, D. Byun and J.E. Pleim, (1994), 		"Sensitivity of Ozone to Model Grid Resolution:  Part I. 	Application of High-Resolution Regional Acid Deposition 	Model", Submitted for Publication in Atmospheric 			Environment.

Jeffries, H.E. and S. Tonnesen, (1994), "A Comparison of Two 	Photochemical Reaction Mechanisms Using Mass Balance and Process Analysis", Atmospheric Environment, 28, pp.2991-3003.

Lewis, C.W., T.L. Conner, R.K. Stevens, J.F. Collins and R.C. 	Henry, (1993), "Receptor Modeling of Volatile Hydrocarbons Measured in the 1990 Atlanta Ozone Precursor Study", Paper 93-TP-58.04, 86th Annual AWMA Meeting, Denver, CO., June 1993.

Milford, J.B., D. Gao, S. Sillman, P. Blossey and A.G. Russell, 	(1994), "Total Reactive Nitrogen (Noy) as an Indicator of 	the Sensitivity of Ozone to Reductions in Hydrocarbon and 	Nox Emissions", J.Geophysical Research 99D, pp.3533-3542.

Nichols, M.D., (March 2, 1995) Memorandum to Regional 			Administrators, USEPA Regions I-X, Subject:  "Ozone 		Attainment Demonstrations".

Sillman, S., (1995), "The Use of Noy, H2O2 and HNO3 as 		Empirical Indicators for Ozone-NOx-ROG Sensitivity in Urban 	Locations", J.Geophysical Research, 100, pp.14175-14183.

Tonnesen, S. and H.E. Jeffries, (1994), "Inhibition of Odd Oxygen 	Production in the Carbon Bond Four and Generic Reaction 	Set Mechanisms", Atmospheric Environment, 28, pp.1339-1349.

Trainer, M., D.D. Parrish, M.P. Buhr, R.B. Norton, F.C. 		Fehsenfeld, K.G. Anlauf, J.W. Bottenheim, Y.Z. Tang, H.A. 	Wiebe, J.M. Roberts, R.L. Tanner, L. Newman, V.C. Bowersox, 	J.F. Meagher, K.J. Olszyna, M.O. Rogers, T. Wang, H. 		Berresheim, K.L. Demerjian and U.K. Roychowdhury, (1993), 	"Correlations of Ozone with Noy in Photochemically Aged 	Air", J.Geophysical Research 98, pp.2917-2925.

U.S. EPA, (1991), Guideline for Regulatory Application of the 	Urban Airshed Model, EPA-450/4-91-013.

U.S. EPA, (1994), National Air Quality and Emissions Trends 	Report, 1993, EPA-450/R-94-026.

U.S. EPA, (1994a), Clean Air Act Ozone Design Value Study: Final 	Report, A Report to Congress, EPA-454/R-94-035.

	APPENDIX A: EXAMPLE APPLICATION

DESCRIPTION

For the purpose of this exercise we plan to demonstrate modeled attainment for a nonattainment area being simulated by a UAM domain comprised of 20 rows and 20 columns of 5Km x 5Km grid cells.  The model was run for three primary days.  Figures A.1, A.2 and A.3 depict predicted daily maximum concentrations above 124 ppb for days 1, 2, and 3, respectively.  The Cox/Chu rankings for the days are 55, 32, and 215, respectively.  Only concentrations above 124 ppb are displayed.   Grid cells are outlined with dashed lines and subregions are highlighted with bold lines.  The highest daily maximum ozone predictions within the domain are 129, 130 and 122 ppb, respectively.  The highest monitored air quality observed on those days was 145, 138 and 132 ppb.  The design value for the area is 140 ppb and the maximum ozone air quality for the last three years was 138, 130, and 126 ppb, respectively.  The model performance calculations for highest-prediction accuracy are -10%, 8%, and 3% for all three days, respectively.  For days 1 and 2 the number of daylight grid cell hours above 124 ppb are 635 and 400 for the model performance runs.  The attainment strategy reduces these numbers to 23 and 21.

APPLICATION OF DETERMINISTIC TEST

We first look at the daily maximum concentration for each grid cell on each primary day.  Notice only day 3 has no concentrations above 124 ppb.  The deterministic test's benchmark is not passed, because we have daily maximum concentrations above 124 ppb for days 1 and 2.

APPLICATION OF STATISTICAL TEST

To apply the statistical test we proceed as follows:

Benchmark 1 allows exceedances on days for which the expected exceedance frequency is less than two times per year (i.e., Cox/Chu ranking < 83 for a 41-year period of record).  Results for Day 3 are acceptable, because even though the ranking is greater than 83 (i.e., 215) there are no predicted exceedances.  The rankings for days 1 and 2 are less than 83.  Therefore, benchmark 1 allows exceedances on one of these days in each of the indicated subregions (n=2, therefore n-1=1 for any subregion within the domain).  Again, look closely at figures A.1 and A.2.  Close inspection of the subregions reveals that no subregion had  more than one day of exceedances.  In other words, across both days, the exceedances do not overlap or occur within the same subregions.  Therefore benchmark 1 is passed.

Benchmark 2 limits how high an exceedance concentration can be in order to be consistent with air quality monitored at attainment sites.  According to Table 4.2 the acceptable upper limit for days 1 and 2 is 130 ppb (rankings for both days are above 21 and 41 years of data were used, so ExEx > 0.5 times per year).  Exceedances on both days are at or below 130 ppb.  Therefore benchmark 2 is passed.

Benchmark 3 requires us to look at the model performance statistics for underpredictions on days for which exceedances are allowed.  The UAM guidance for regulatory application of UAM recommends the calculation of an unpaired highest-prediction accuracy (AU).  The AU values for days 1, 2 and 3 are -10%, 8% and 3%, respectively.  (Note: using the conventions described in the 1991 Guidance, a negative number represents an overprediction).   Benchmark 3 requires that days with allowed exceedances and an AU value greater than 5% demonstrate an 80% improvement in the areal coverage above 124 ppb for daylight hours (i.e., 8:00 AM-8:00 PM, LST).  Day 3 does not have to demonstrate 80% improvement because the AU is less than 5% and no exceedances were allowed.  Day 1 is allowed exceedances but does not have to demonstrate 80% improvement because the AU is less than 5%.  However, for day 2, exceedances are allowed and the AU model performance statistic is greater than 5%.  Therefore, we add all daylight grid cell hours above 124 ppb for the model performance run on day 2 (400 grid cell hours).  After growing emissions to the future attainment year and applying the attainment control strategy we again count the number of grid cell hours above 124 ppb (21 grid cell hours).  This constitutes a 95% improvement in the air quality above 124 ppb (((400-21)/400)=.9475).  Benchmark 3 is passed. For this exercise the statistical benchmarks are passed and the modeling satisfactorily demonstrates attainment of the ozone NAAQS.

USE OF WEIGHT OF EVIDENCE

For the above exercise the statistical benchmarks are passed.  Suppose this had not been the case and a different outcome had prevailed.  The following case is an example of how information obtained through the attainment test's benchmarks and other analysis may provide "weight of evidence" sufficient to demonstrate attainment.  Through the introduction of additional information unique to this case, we plan to illustrate a case in which modeling plus additional analysis provides sufficient evidence that attainment is probable by the statutory date, even though one or more of the statistical test's benchmarks are not passed.

Suppose the peak model prediction for day 1 was 135 ppb instead of 129 ppb.  This is 5 ppb above the limit 130 ppb and benchmark 2 is failed.  For this exercise assume benchmarks 1 and 3 are passed as previously presented.  Looking more closely at day 1 we see our model performance indicates that the model overpredicted by 10%.  There is a possibility that the future estimates are also overpredicted.  10% would imply as much as a 13 ppb (more than 5) fluctuation beyond 124 ppb.  To further support our case we calculate the areal improvement.  On day 1, 635 daylight grid cells are above 124 ppb.  The future attainment strategy reduces this number to 23 grid cells.  This represents a significant improvement in air quality (96%).  We then refer back to the air quality observed on all three of our episode days, 145, 138 and 132, respectively.  Not only is day 1's concentration the highest observed for all three episodes but it is greater than our air quality design value which is 140 ppb.  Also, our current air quality for the last three years has shown a steady improvement (138, 130, 126 ppb, respectively).  It is reasonable to believe that based on this information our control strategy will provide for attainment by the statutory attainment date.

Figure A.1:  DAY 1; Cox/Chu Ranking=55

                                      125

                                      126

                                      126

                                      127

                                      129

                                      128

                                      125

Figure A.2:  DAY 2; Cox/Chu Ranking=32

                                      125

                                      126

                                      127

                                      129

                                      130

                                      125

Figure A.3:  DAY 3; Cox/Chu Ranking=215