Air Quality Modeling Technical Support Document:
                     NJ 126 Petition of September 17, 2010

                     U.S. Environmental Protection Agency
                 Office of Air Quality Planning and Standards
                        Air Quality Assessment Division
                       Research Triangle Park, NC 27711

                               Table of Contents

I. Introduction.........................................................................................................................................3

II. EPA's Assessment of NJDEP Modeling for Finding of Significant Contribution.............................................................................................................................................4
      A. Submitted Information and Summary
      B. Model Selection
      C. Emissions and Source Characteristics
      D. Receptor and Terrain Data
      E. Meteorological Data
      F. Summary of EPA's Analysis of the NJDEP Modeling

III. EPA Modeling of Portland Emissions for 1-Hour SO2 NAAQS for Determination of Remedy..................................................................................................................................................20
      A. Emissions and Source Characteristics
      B. Meteorological Data
      C. Receptor and Terrain Data
      D. EPA Remedy Modeling Results

IV. Portland Generating Station Emission Limits..............................................................................30

V.  References .......................................................................................................................................32

Appendix A.  EPA Assessment of NJDEP CALPUFF Validation Study

Appendix B.  EPA Assessment of Site-specific Meteorological Data

      I.  Introduction

On September 17, 2010, the New Jersey Department of Environmental Protection (NJDEP) submitted to EPA a section 126 petition which asserts that Sulfur Dioxide (SO2) emissions from the Portland Generating Station (Portland Plant or PGS) in Upper Mount Bethel Township, Northampton County, Pennsylvania significantly contribute to nonattainment and/or interfere with maintenance of the 1-hour SO2 National Ambient Air Quality Standards (NAAQS) in New Jersey (NJ).  The petition included both CALPUFF and AERMOD dispersion modeling for the 1-hour SO2 NAAQS which shows violations of the NAAQS in NJ based on impacts from the Portland Plant.  NJDEP specifically petitions the USEPA to "directly regulate the Portland Plant to abate the significant contribution to nonattainment and interference with New Jersey's maintenance of, the more stringent 1-hour SO2 NAAQS."  See page 7, September 17, 2010 petition.

Dispersion modeling results submitted by NJDEP show modeled SO2 concentrations from ~7 to 17 times higher than the 1-hour SO2 NAAQS.  EPA reviewed several aspects of the NJDEP modeling to determine whether the analyses followed EPA regulations and guidance for dispersion modeling, and whether the modeling analyses provide an adequate basis for determining that SO2 emissions from the Portland Plant significantly contribute to nonattainment and/or interfere with maintenance of the 1-hour SO2 NAAQS in NJ.  Among these considerations were the choice of model(s), and the application of site-specific meteorological data that was used as input to the AERMOD model.

Based on EPA's review of the NJDEP modeling we conclude that the modeling provides an adequate basis to make a finding that emissions of sulfur dioxide (SO2) from the Portland Plant significantly contribute to nonattainment and interfere with maintenance of the 1-hour SO2 national ambient air quality standard (NAAQS) in New Jersey.  As part of the review of the NJDEP modeling, EPA concluded that it is necessary to make some technical adjustments to the modeling.  We therefore determined that it was appropriate to conduct an independent modeling  analysis to determine an appropriate remedy.

This Technical Support Document (TSD) provides the technical review and assessments completed by EPA in support of this section 126 petition.  Section II contains EPA's detailed assessment of the NJDEP modeling that served as the basis to inform the Agency's determination of significant contribution.  Section III contains details of the dispersion modeling conducted by EPA to inform the remedy for the Portland Plant, while Section IV contains details on the calculations of emissions limits based on the EPA modeling results.

      II. EPA's Assessment of NJDEP Modeling for Finding of Significant Contribution

An assessment of the appropriateness and adequacy of the modeling results submitted by NJDEP for purposes of this Section 126 petition is provided below.  This includes a summary of technical issues associated with the modeling analysis submitted by NJDEP, along with a summary of how these issues have been addressed in the modeling conducted by EPA to support our proposed remedy in response to this petition.  Note that while we believe these technical issues to be significant enough to address in relation to the remedy, they are not significant enough to alter the finding of significant contribution to nonattainment given the magnitude of the modeled violations.

   A.  Submitted Information and Summary

NJDEP submitted several analyses in support of the section 126 petition.  Among the submitted materials were a summary of the NJDEP dispersion modeling results, a modeling analysis for the 1-hour SO2 NAAQS using AERMOD, a modeling analysis for the 1-hour SO2 NAAQS using CALPUFF, and a trajectory analysis of high sulfur dioxide episodes at the ambient SO2 monitor located in Chester, NJ.  In addition, the petition references a CALPUFF model validation study which was submitted by NJDEP as part of a previous (May 12, 2010) Section 126 petition related to the Portland Plant.

NJDEP submitted two different modeling analyses of the SO2 impacts from the Portland Plant on NJ.  The first analysis (Exhibit 2 in the NJDEP petition) used the AERMOD dispersion model and the second analysis used the CALPUFF dispersion model (Exhibit 3 in the NJDEP petition).  Both models were run separately using both actual and allowable emissions rates and the CALPUFF model was also run with various meteorological input data.  Each model run showed modeled violations of the 1-hour SO2 NAAQS (annual 99 percentile at 75 parts per billion (ppb)), or about 196 micrograms per cubic meter (ug/m[3])) in NJ.  Table 1 summarizes the CALPUFF and AERMOD 1-hour SO2 NAAQS modeling results.

Table 1.  Summary of 1-hour SO2 Modeling Results Submitted by NJDEP
                                     Model
                                   Emissions
                                  Meteorology
                Maximum Modeled 1-hour Concentration (ug/m[3])
    99[th] Percentile (4[th] high)  1-hour Modeled Concentration (ug/m[3])
AERMOD
Allowable
July 1993-June 1994
                                                                         3,700
                                                                          1,402
AERMOD
Estimated Actual
July 1993-June 1994
                                                                          1,713
                                                                            467
CALPUFF
Allowable
2002 12km MM5
                                                                         15,273
                                                                          3,455
CALPUFF
Actual
2002 12km MM5
                                                                          6,740
                                                                          2,194
CALPUFF
Allowable
2003 4km MM5
                                                                         18,643
                                                                          2,468

As can be seen in Table 1, the NJDEP modeling shows modeled violations of the 1-hour SO2 NAAQS in each of the submitted modeling analyses.  The CALPUFF model concentrations tend to be significantly higher than the AERMOD concentrations and the allowable emissions results are logically higher than the results based on actual emissions.  NJDEP has shown that modeled violations, significantly in excess of the 1-hour NAAQS, associated with emissions from the Portland Plant occur in NJ when the Portland plant is operating at its allowable emissions rate.  These results indicate that a reduced emissions limit is needed at Portland in order to eliminate the modeled violations in NJ.

The petition included modeling of both allowable and actual emissions from the Portland Plant.  The allowable emissions for the Portland Plant are shown in Table 2.

Table 2.  Allowable SO2 Emissions for the Portland Plant
Portland Unit
Allowable SO2 Rate
Maximum 3-hour permit limit
                                       1
                                  5,820 lb/hr
                             8.73 tons per 3 hours
                                       2
                                  8,900 lb/hr
                            13.35 tons per 3 hours

The petition contained modeling of actual emissions based on CALPUFF for the 12km 2002 mesoscale meteorological model (MM5) based meteorology and showed results more than 10 times higher than the NAAQS.  Actual emissions were also modeled with AERMOD for the 1993 site-specific meteorology.  The AERMOD modeling with actual emissions also showed exceedances of the 1-hour SO2 NAAQS, but were about 80 percent lower than the CALPUFF results.  The 2002 CALPUFF modeling with actual emissions was based on actual SO2 emissions from continuous emissions monitoring system (CEMS) data.  The 1993-1994 actual emissions used with AERMOD were estimated based on monthly coal usage reports (CEMS data was not available for that period).

The modeling submitted by NJDEP indicates actual emissions from the Portland Plant alone cause air quality in New Jersey to exceed the 1-hour SO2 NAAQS.  The NJDEP modeling also indicates that the Portland Plant's allowable emissions cause air quality in New Jersey to exceed the 1-hour SO2 NAAQS.  Consistent with the "Guideline on Air Quality Models" published as Appendix W to 40 CFR Part 51 and with the reference in Section 126 to emissions that a source "emits or would emit", we believe that modeling based on allowable emissions is most appropriate in this case.  Therefore, the balance of the review of NJDEP's modeling and the description of EPA's remedy modeling will be limited to the model results based on allowable emissions.

Figure 1 (based on Figure 3 from NJDEP exhibit 2) shows the AERMOD predictions of the 4th high daily maximum 1-hour concentrations (99[th] percentile) based on allowable emissions.  The maximum concentration of 1,402 ug/m[3] is located on the Kittatinny Ridge on the NJ side of the Delaware Water Gap, about 7 km northwest of the Portland Plant.

Figure 1.  AERMOD predictions of the 4th high daily maximum 1-hour concentrations (99[th] percentile) based on allowable emissions.

Figure 2 (based on Figure 1 from NJDEP Exhibit 3) shows the CALPUFF predictions of the 4th high daily maximum 1 hour. concentrations (99[th] percentile) based on allowable emissions.  A review of modeling files submitted by NJDEP indicates that the maximum concentration of 3,455 ug/m[3] is located within about 100 meters of the Portland Plant at an elevation of 3 meters above stack base on the PA side of the Delaware River.

Figure 2.  CALPUFF predictions of the 4th high maximum daily 1-hour. concentrations (99[th] percentile) based on allowable emissions.

Although dispersion modeling results submitted by NJDEP show modeled SO2 concentrations from about 7 to 17 times higher than the 1-hour SO2 NAAQS, leaving little doubt that the Portland Plant contributes significantly to nonattainment of the 1-hour SO2 NAAQS in NJ, EPA reviewed several aspects of the NJDEP modeling to determine whether the analyses followed EPA regulations and guidance for dispersion modeling, and whether the NJDEP modeling analyses would provide an adequate basis for determining an appropriate remedy.  Among these considerations were the choice of model(s), and the application of site-specific meteorological data that was used as input to the AERMOD model.

   B. Model Selection

Since both the AERMOD and CALPUFF modeling conducted by NJDEP predicted 1-hour SO2 impacts that are significantly higher than the 1-hour SO2 NAAQS, the choice of model may not be critical to the finding that the Portland Plant significantly contributes to nonattainment or interferes with maintenance of the 1-hour SO2 standard in New Jersey.  However, given the significant differences in the magnitude of predicted impacts based on these two models, the level of the emission reductions necessary to eliminate the Portland Plant's contribution to nonattainment in NJ (i.e., the "remedy") would vary significantly depending upon which model result is used as the basis for such reduction.  Therefore, model selection was a key aspect of EPA's assessment of the NJDEP modeling.

According to Section 4.2.2 (b) of Appendix W, "F]or a wide range of regulatory applications in all types of terrain, AERMOD is the recommended model."  The modeling application under consideration in this Section 126 petition is generally addressed by this section of Appendix W since the transport distances of concern are less than 50 kilometers.  Although NJDEP did not explicitly question the appropriateness of AERMOD for this near-field application, they introduced the use of CALPUFF under the provisions in Section 3.2 on "Use of Alternative Models" in Appendix W, which state that there are "three separate conditions under which [an alternative] model may normally be approved for use:  (1) [i]f a demonstration can be made that the model produces concentration estimates equivalent to the estimates obtained using a preferred model; (2) if a statistical performance evaluation has been conducted using measured air quality data and the results of that evaluation indicate the alternative model performs better for the given application than a comparable model in Appendix A; or (3) if the preferred model is less appropriate for the specific application, or there is no preferred model."

NJDEP Justification for CALPUFF

Although NJDEP submitted modeling results based on both the AERMOD and CALPUFF models, NJDEP's September 17, 2010 petition cites a model validation study (submitted as Exhibit 12 with the May 12, 2010 petition) comparing the performance of the AERMOD model with CALPUFF based on the Martins Creek field study database from 1992-1993.  The Martins Creek database was included in the validation of the AERMOD model.  NJDEP concluded from this study that "CALPUFF performed better and produced predictions of greater accuracy than AERMOD."  See September 17, 2010 petition, Section IV, page 5.  The February 24, 2010 "Validation of CALPUFF in the Near-Field" submitted as Exhibit 12 to the May 12, 2010 petition indicates that "the purpose of this model validation study was to evaluate the performance of CALPUFF in this study area and determine if its use is appropriate and produces predictions of greater accuracy than the Appendix A model AERMOD.  An additional objective of the validation study was to determine whether or not CALPUFF is biased toward underestimating SO2 concentrations in this location[1]."  See Exhibit 12 to the May 12, 2010 petition, Section 2, page 2.  Footnote 1 states that "Appendix W to Part 51 CFR, Section 3.2.2e provides that an alternative refined model may be used if, among other requirements, "[a]ppropriate performance evaluations of the model have shown that the model is not biased toward underestimates;"  subsection e does not technically apply as this subsection only applies to Condition 3, see Section 3.2.2b."

From the statements summarized above, we conclude that NJDEP's use of the CALPUFF model in support of the September 17, 2010 petition is based on a claim that CALPUFF was shown to have "performed better and produced predictions of greater accuracy than AERMOD", and therefore satisfies condition (2) under Section 3.2.2b of Appendix W.  See September 17, 2010 petition, Section IV, page 5.  The criteria applied to condition (2) for use of alternative models under Section 3.2.2b are discussed in paragraph (d) of Section 3.2.2:

      d. For condition (2) in paragraph (b) of this subsection, established procedures and techniques for determining the acceptability of a model for an individual case based on superior performance should be followed, as appropriate. Preparation and implementation of an evaluation protocol which is acceptable to both control agencies and regulated industry is an important element in such an evaluation.

Condition (2) relies solely on a demonstration that the alternative model has been shown to perform better than a comparable Appendix A model, and does not entail a demonstration that the preferred model (AERMOD in this case) is inappropriate for the application.

EPA issued a memo on August 13, 2008 providing "Clarification on Regulatory Status of CALPUFF for Near-field Applications," such as the application under review here.  The August 2008 memo specifically addressed the use of CALPUFF for near-field applications under Section 7.2.8 of Appendix W on "Complex Winds" subject to the limitations and requirements for use of alternative models under condition (3) that are addressed in paragraph 3.2.2(e).  EPA issued additional guidance related to the application of CALPUFF for near-field situations in a memo dated September 26, 2008 on "Technical Issues Related to CALPUFF Near-field Applications."  The September 26, 2008 memo provides a detailed discussion on each of the main components involved in addressing the appropriateness of CALPUFF for use in near-field applications under Section 7.2.8 for complex winds, and also identified several specific technical issues and concerns regarding the limitations of the CALPUFF /CALMET modeling system to adequately simulate the 3-dimensional wind and temperature fields at a fine enough resolution to give confidence in the results.

These technical issues identified in the September 2008 memo are generally applicable to the use of CALPUFF as an alternative model in near field applications under condition (2) or (3), and inform our assessment of the appropriateness of CALPUFF for this application.  One of the issues raised in the September 2008 memo is the lack of any demonstrations of CALPUFF model performance in near-field complex terrain or other complex wind situations.  The one evaluation study often cited to support the use of CALPUFF for near-field applications is the Lovett power plant complex terrain field study (Ref.).  However, the CALPUFF evaluation results for Lovett are based on use of the CTDM surface and profile meteorological inputs and use of the Complex Terrain algorithm for Sub-Grid-scale features (CTSG) option in CALPUFF, options that essentially emulate the CTDMPLUS model and bypass the CALMET meteorological processor completely.  As a result, the published Lovett evaluation results for CALPUFF provide no information on the performance of CALMET in simulating non-steady-state winds in a near-field complex terrain setting.

The NJDEP model validation study submitted as Exhibit 12 with the May 12, 2010 petition is the first model evaluation we are aware of based on the CALPUFF model, driven by gridded CALMET wind fields, in a near-field, complex terrain setting.  By contrast, the performance of the AERMOD model for estimating impacts associated with tall stacks in complex terrain settings has been extensively evaluated and documented in peer-review journals (Perry, et al., 2005; Venky, et al., 2001), and has consistently been shown to perform better than competing models.

A final and fundamental point in relation to NJDEP's overall justification for the use of CALPUFF in this petition is that results from the model validation study are not relevant to this application of CALPUFF due to fundamental differences in the CALMET meteorological processing used in each case.  The CALMET modeling for the validation study made use of the site-specific meteorological data collected as part of the field study such that the documented CALPUFF model performance is largely dependent on the characterization of wind fields by CALMET that are informed by that site-specific data.  NJDEP used the new PROF2UP utility developed by the CALPUFF developers to facilitate a more effective use of multi-level meteorological measurements within the CALMET processor.  The PROF2UP tool has not been made publicly available by the CALPUFF developers, but it appears to have been developed to address some of the technical issues identified in EPA's September 26, 2008 memo regarding use of CALPUFF in near-field applications.  In contrast, the application of CALPUFF to estimate ambient SO2 impacts associated with Portland Plant emission in support the NJDEP petition did not use any site-specific meteorological data but relied on three different sets of MM5 prognostic meteorological data to inform the 3-dimensional wind fields generated by CALMET.  Performance of the CALPUFF model in the latter case would rely upon the ability of the MM5 prognostic model coupled with the CALMET diagnostic model to adequately simulate the wind fields in the absence of such site-specific data.  There have not been any demonstrations of the ability of CALMET with MM5 to adequately simulate the wind fields that would be relevant to this application.

Aside from the issues cited above regarding the relevance of CALPUFF validation studies to specific applications of the model, we also note that model validation is a complex process that entails several technical challenges, including uncertainties regarding the accuracy and representativeness of key input data that could affect results, as well as a wide range of statistical methods and metrics that may be applied to quantify model performance.  In some cases subtle changes to the evaluation methods can markedly affect the conclusions that might be drawn from such studies.  For these reasons, the importance of establishing a consistent set of objective procedures to evaluate the performance of dispersion models for use in regulatory modeling applications and of comparing the relative performance of competing models has long been recognized.  Section 3.2.1 of Appendix W references an EPA document (EPA, 1992) that "is available to assist in developing a consistent approach when justifying the use of other-than-preferred modeling techniques recommended in the Guideline. The procedures in this protocol provide a general framework for objective decision-making on the acceptability of an alternative model for a given regulatory application. These objective procedures may be used for conducting both the technical evaluation of the model and the field test or performance evaluation."

Although NJDEP's model validation report describes the results presented in Section 8.3 of Exhibit 12 as "Model Evaluation Results Based on EPA's AERMOD Validation Procedures," NJDEP made several changes to the model evaluation methodology as compared to the evaluation conducted by EPA in support of promulgation of AERMOD as a preferred model in Appendix W.  EPA did not have an opportunity to review or comment on these changes to the model evaluation protocol implemented in NJDEP's validation study.  EPA's evaluation of NJDEP's changes to the protocol leads us to believe that the NJDEP methods show relatively better model performance for CALPUFF compared to AERMOD, without any clear technical basis that would justify those changes.  Further details on these changes and our assessment of their impacts on the results of the validations study are provided in Appendix A to this TSD.

We also note that the spatial distribution of 1-hour SO2 impacts predicted by CALPUFF (in the petition application) is very different than the impacts predicted by AERMOD.  The CALPUFF modeling shows extremely high 1-hour SO2 concentrations very close to the Portland Plant (see Figures 1, 2, and 3 of Exhibit 3).  The highest impacts based on the 2002 CALPUFF modeling with allowable emissions of 3,455 ug/m[3] (99th percentile of daily maximum 1-hour values) occurs about 100 meters from units 1 and 2 at an elevation of only 3 meters above the stack base in Pennsylvania.  These results are physically unrealistic for buoyant plumes from tall stacks such as units 1 and 2 at the Portland Plant, raising additional concerns regarding the appropriateness of CALPUFF for this application.

Based on the issues discussed above (and the additional details contained in Appendix A to this TSD), we conclude that NJDEP has not adequately justified the use of CALPUFF in this application under either conditions (2) or (3) of Section 3.2.2b of Appendix W, and that AERMOD is the most appropriate model for this application.

   C.  Emissions and Source Characteristics

This section documents EPA's assessment of the emissions and source characteristics used by NJDEP  in the modeling submitted with the September 17, 2010 petition, and also addresses the issue of whether other nearby emissions sources and/or whether background concentrations based on representative monitoring data should be included in the modeling analysis.

As noted above, NJDEP submitted dispersion modeling results based on maximum allowable emissions as well as actual emissions.  Use of allowable emissions is consistent with guidance presented in Appendix W for modeling demonstrations for compliance with the NAAQS and we believe that allowable emissions are also appropriate for purposes of determining whether the Portland Plant significantly contributes to nonattainment or interferes with maintenance of the 1-hour SO2 NAAQS in NJ under this Section 126 petition.  The demonstration that the Portland Plant significantly contributes to nonattainment or interferes with maintenance of the NAAQS in NJ based on its actual emissions adds weight to the finding, but should not be construed as a necessary condition for this finding.

Portland Plant Emissions

The dispersion modeling submitted by NJDEP only included emissions from the coal-fired units 1 and 2 at the Portland Plant, unit 1 with a capacity of 160 MW and unit 2 with a capacity of 240 MW, which account for the overwhelming majority of SO2 emissions from the facility. There is an auxiliary boiler which burns oil and 3 small turbines (units 3, 4, and 5) which all burn oil and natural gas, and have very small SO2 emissions.

Units 1, 2, and 5 utilize continuous emissions monitoring system (CEMS).  In 2009, CEMS reported SO2 emissions combined from units 1 and 2 were 30,465 tons and emissions from unit 5 were 0.3 tons which are reported from CEMS data.  Between 2007 and 2010, units 1 and 2 operated, on average, approximately 7,000 hours per year, while unit 5 operated for less than 100 hours per year.  The auxiliary boiler, unit 3, and unit 4 do not have CEMS, but available emissions data from the 2008 National Emissions Inventory (NEI), Version 1 report the annual SO2 emissions from these sources at 0.01, 0.02, and 0.03 tons per year, respectively, for the auxiliary boiler, unit 3, and unit 4.

Units 1 and 2 at the Portland Plant are clearly the dominant sources of SO2 emissions from the facility, and the exclusion of emissions from other units at the plant is expected to have only a negligible impact on the cumulative modeled impacts from the plant.

Background SO2 Concentrations

NJDEP did not include any other nearby sources of SO2 in the dispersion modeling analysis submitted in support of this section 126 petition.  Other sources of SO2 emissions in the area include the Martins Creek facility which is located approximately 10 km to the south of the Portland Plant.  There are two units at Martins Creek, units 3 and 4, which averaged about 1,039 and 584 hours of operation respectively.  Those units each have a capacity of 850 MW and can burn either number 6 oil or natural gas.  The facility reported approximately 1,100 tons of SO2 emissions in 2009.  There are also three cement plants (Hercules, Keystone, and ESSROC) and several minor emitting units in Pennsylvania located at distances generally greater than 30 km away to the south and west of the Portland Plant.  In 2009, the Pennsylvania Department of Environmental Protection emission inventory database (PADEP eFACTS) reported SO2 emissions of 1,862 tons for Hercules, 685 tons for Keystone, and 799 tons for ESSROC, all of which are relatively low compared to the SO2 emissions from the Portland Plant.

NJDEP also did not include any background SO2 contribution based on ambient monitoring in the analysis submitted in support of this petition. No rationale is provided in the NJDEP petition to justify the exclusion of background SO2 concentration in the analysis.

Given that the modeled impacts based on Portland units 1 and 2 alone exceeded the 1-hour SO2 NAAQS by about a factor of 7 based on NJDEP's AERMOD modeling, a more complete accounting of potential background concentrations in the Portland area has no bearing on the assessment of whether Portland significantly contributes to nonattainment or interferes with maintenance of the 1-hour SO2 NAAQS in NJ.  However, the issue of background concentrations plays a more significant role in the analysis needed to support the proposed remedy in response to the petition.  These issues are discussed in more detail in Section III.

   D.  Receptor and Terrain Data

Proper treatment terrain information is important for this analysis given the potential influence of elevated and complex terrain on the modeling results.  The NJDEP analysis was based on an initial grid of coarsely spaced receptor locations across a large domain covering all potentially important impact area associated with emissions from the Portland Plant, followed by a much smaller grid of more closely spaced receptors focused on the area of expected worst-case impacts from the plant.  The initial grid, shown in Figure 2 of Exhibit 2 for the September 17, 2010 petition, included spacing of 250 meters in areas of expected high impacts with receptors spaced at 1,000 meter intervals covering the gaps between the 250 meter grids.  The initial coarse receptor grid included a total of 5,189 receptors.  The fine grid used by NJDEP in determining the controlling impact from Portland for purposes of this petition included a total of 121 receptors in a 10x10 array spaced at 100 meter intervals covering a portion of the Kittatinny Ridge on the NJ side of the Delaware Water Gap.

NJDEP applied the AERMAP terrain processor (EPA, 2004) with sixteen 7.5-minute USGS Digital Elevation Model (DEM) terrain files at 30m horizontal resolution covering most of the modeling domain and four 1-degree DEM files at 90m horizontal resolution covering the remainder of the domain.  NJDEP also used the NAD27 horizontal datum for all receptor and source coordinates.

The AERMAP terrain processor can also process terrain data based on the National Elevation Dataset (NED) format (USGS, 2002), which reflects updates to the older DEM terrain data.  Section 4.3 of the AERMOD Implementation Guide (EPA, 2009) discusses terrain elevation data sources and potential issues in light of the release of the NED data by USGS, and encourages the use of NED data for processing terrain data for use in AERMOD.  In light of this, EPA processed the terrain data for the NJDEP modeling domain using 1-second NED data (approximately 10m horizontal resolution).  The source and receptor coordinates were also converted to the NAD83 horizontal datum.  A comparison of terrain elevations and hill height scales used in the NJDEP analysis from the 100m receptor grid near the Delaware Water Gap with values based on application of AERMAP with NED data and found nearly identical results.  We conclude from this comparison that NJDEP's treatment of receptor/terrain data in their AERMOD analysis was appropriate.

   E.  Meteorological Data

Aside from emissions data, meteorological data is the other key input to dispersion models.  The AERMOD modeling conducted by NJDEP was based on one year of site-specific meteorological data collected from a 100m instrumented tower and SODAR located about 2.2km west of the Portland Plant, as shown in Figure 3, for the period July 1993 through June 1994.

Figure 3. Location of Portland Meteorological Tower

Figure 3 was included in Appendix A of Exhibit 11 to NJDEP's May 12, 2010 petition, and was taken from the document: SO2 NAAQS Compliance Modeling Protocol for GPU's Portland Generating Station, prepared for GPU Genco, prepared by ENSR Corporation, April 1999.

Section 8.3 of Appendix W provides guidance regarding meteorological data for use in dispersion modeling to demonstrate compliance with the NAAQS.  A key issue related to meteorological data is the representativeness of the data for the particular application, including spatial and temporal representativeness.  Based on a review of the data, we believe that the Portland Plant meteorological data from 1993-94 meets the basic criteria for representativeness under Section 8.3.3 of Appendix W, and therefore can be considered as site-specific data for purposes of modeling impacts from the elevated stacks for PGS units 1 and 2.  The 1993-94 data also meets the minimum criterion for length of meteorological data record of at least one year of site-specific meteorological data recommended in Section <IP_ADDRESS> of Appendix W, and it is the most representative meteorological data available to support refined dispersion modeling for the Portland Plant.  However, the difference of about 100 meters in the base elevation for the meteorological tower vs. the stack base elevation (see Figure 3) raises concerns regarding how the meteorological data were input to the AERMOD model in the NJDEP modeling anlaysis, especially given that the stack heights for units 1 and 2 are about 122 meters and that plume heights of concern for units 1 and 2 are about 200 to 400 meters above stack base.  The AERMOD modeling submitted by NJDEP used the measurement heights above local ground at the tower location for the meteorological data input to the model, effectively assuming that the measured profiles of wind, temperature and turbulence are "terrain-following."

Without adjusting for the difference in base elevation of about 100m between the meteorological data and the stacks, wind speeds are likely to be biased high and the wind directions may not be representative of plume heights.  Table 1 of NJDEP's analysis of complex winds in the region surrounding the Portland Plant, submitted as Exhibit 11 to NJDEP's May 12, 2010 petition, is shown below as Table 3 and provides an example that illustrates this point, where wind direction varies with height indicating a transition from westerly flow below about 210 meters above ground to northwesterly flow between 210 and 270 meters, and then to northeasterly flow at 300 meters and above for hour 11 on July 18, 1993.  A similar pattern occurs for hour 12 on the same day, but the transitions occur at somewhat lower heights above ground.  Accounting for the differences in base elevation between the meteorological tower and Portland Plant stacks, this transition in wind directions occurs within the range of plume heights of concern for Units 1 and 2.

Based on the concerns documented above, we have concluded that some adjustments to the measurement heights for the Portland Plant site-specific meteorological data are appropriate in order to address these issues of representativeness for this application.  However, since the maximum design value concentration in the NJDEP AERMOD modeling analysis was nearly seven times the NAAQS, we do not expect these issues regarding the meteorological data to change the overall conclusion that the Portland Plant emissions are likely to significantly contribute to nonattainment or interfere with maintenance of the 1-hour SO2.  Adjustments to the meteorological data that may play an important role in determining the remedy are discussed below in Section III, with more details provided in Appendix B to this TSD.

Table 3.  Example of profile wind direction/wind speed collected at the Portland site (hours 11 and 12, July 18, 1993) (based on Table 1 of NJDEP Exhibit 11).

   F. Summary of EPA's Analysis of the NJDEP Modeling

Use of CALPUFF

Based on a review of applicable guidance and an assessment of the CALPUFF validation study cited in NJDEP's September 17, 2011 petition, EPA concludes that NJDEP has not adequately demonstrated that application of the CALPUFF model is justified for this near-field application in support of their section 126 petition regarding the Portland Plant.  Their case for the use of CALPUFF is deficient on several key points:

   1. NJDEP's use of the CALPUFF model instead of EPA's preferred model for near-field applications, AERMOD, is based on a claim that CALPUFF was shown to have "performed better and produced predictions of greater accuracy than AERMOD" based on a single model validation study focused on the nearby Martins Creek plant.
   2. NJDEP's validation of the CALPUFF model using the Martins Creek field study data showed very similar model performance of CALPUFF vs. AERMOD based on the Q-Q plots and other components of the evaluation.  However, a close examination of key assumptions incorporated in their analysis, especially the deviation from standard practice on the number of data samples used in the calculation of RHCs and the use of predicted/observed ratios paired by rank, rather than paired in time, in the residual analysis, raises questions regarding some of the statistical model performance measures used to support their claim that CALPUFF performs better than AERMOD based on this particular field study database.
   3. NJDEP's validation of CALPUFF based on the Martins Creek data represents the only near-field complex terrain evaluation that we are aware of involving the CALPUFF modeling system, including the use of CALMET-generated wind fields, whereas AERMOD has been evaluated on at least five tall-stack/complex-terrain field studies and has shown consistently good model performance.  Therefore, even if we judge the NJDEP evaluation to be an adequate demonstration that CALPUFF performs better than AERMOD in this particular case, the weight of evidence would still favor the AERMOD model as the preferred model for this application, unless the NJDEP evaluation presented compelling evidence that CALPUFF is clearly superior to AERMOD for this application and that the proximity of the Martins Creek field study to the Portland Plant adds greater emphasis to that conclusion.
   4. Due to significant differences in the meteorological inputs to CALPUFF employed in the Martins Creek validation study, which utilized site-specific meteorological data, and the application of CALPUFF in support of this petition, which was based on MM5 meteorological inputs without any site-specific data, the Martins Creek evaluation results for CALPUFF are not relevant to its application for the Portland Plant.  The significant differences in the relative concentrations predicted by CALPUFF vs. AERMOD for the Martins Creek field study, where generally good agreement was shown, as compared to the CALPUFF vs. AERMOD results for the Portland Plant in the NJDEP September 17, 2010  petition, where CALPUFF concentrations were significantly higher than AERMOD concentrations, serve to highlight this point.

Portland Plant Emissions

EPA agrees with NJDEP's use of allowable emissions for the Portland Plant units 1 and 2 in the modeling analyses submitted for this petition.  Given the magnitude of modeled impacts in relation to the 1-hour SO2 NAAQS, the exclusion of any background SO2 concentrations due to nearby sources in the modeling or due to ambient monitored concentrations is immaterial to the conclusion regarding the Portland Plant's causing or contributing to violations of the NAAQS in NJ.

Receptor and Terrain Data

EPA believes that the approach used to define the receptor grids in NJDEP's dispersion modeling was appropriate and adequately characterized Portland's potential impacts on ambient SO2 concentrations in NJ, given the role of other model inputs in determining the peak impacts from the initial coarse grid analysis.  EPA believes that dispersion modeling analyses should be based on the more up-to-date NED elevation dataset rather than the older DEM elevation data that was used in NJDEP's modeling, and also recommends that the more standard NAD83 horizontal datum should be used as the reference for source and receptor locations, instead of the obsolescent NAD27 horizontal datum.  However, EPA confirmed through independent analyses that the minor issues regarding terrain data processing had a negligible impact on results.

Meteorological Data

While EPA accepts the use of site-specific meteorological data collected for the Portland Plant from 1993-94 as generally appropriate for dispersion modeling of emissions from the Portland Plant units 1 and 2, the 100 meter difference in base elevation between the meteorological tower and stack base raises questions regarding the representativeness of the wind data.  Based on the analysis summarized above, EPA concluded that the representativeness of the Portland Plant meteorological data could be improved by some adjustments to the measurement heights from the SODAR data and the inclusion of the σw data collected from the SODAR, which was not included in NJDEP's AERMOD modeling.

These adjustments may play an important role in determining the remedy, as explained later in section III.  However, since the maximum design value concentration in the NJDEP AERMOD modeling analysis was nearly seven times the NAAQS, we do not expect these adjustments to change the overall conclusion that the Portland Plant emissions are likely to significantly contribute to nonattainment or interfere with maintenance of the 1-hour SO2 NAAQS in NJ.

Conclusion

EPA finds that AERMOD is the appropriate model to use for this application.   NJDEP's AERMOD modeling shows maximum design value impacts from the Portland Plant, based on allowable SO2 emissions of 1402 ug/m[3] in New Jersey.  Since those concentrations are nearly seven times the 1-hour SO2 NAAQS (196 ug/m[3]), and since NJDEP's AERMOD modeling also showed significant exceedances of the 1-hour SO2 NAAQS in NJ based on an estimate of actual SO2 emissions, we conclude that the NJDEP has clearly shown that SO2 emissions from the Portland Plant cause violations of the 1-hour SO2 NAAQS in New Jersey.

      III. EPA Modeling of Portland Plant for 1-Hour SO2 NAAQS for Determination of Remedy

In the previous section we determined that the NJDEP AERMOD modeling was sufficient to make a finding that SO2 emissions from the Portland Plant significantly contribute to nonattainment or interfere with maintenance of the 1-hour SO2 NAAQS in New Jersey.  However, we noted some technical concerns with the NJDEP modeling which may affect the degree to which emissions need to be reduced to be able to meet the 1-hour SO2 NAAQS in New Jersey.  Therefore, EPA conducted an independent modeling assessment to help determine the necessary and appropriate emissions limit for Portland units 1 and 2.

This section describes the independent analyses conducted by EPA to determine an appropriate remedy to mitigate the ambient impacts from the Portland Plant to New Jersey.  The data, methods and conclusions from the study are summarized below.

   A. Emissions and Source Characteristics

This section documents the emissions and source characteristics used by EPA in the dispersion modeling conducted to determine the remedy in response to the September 17, 2010 petition.  The discussion below also addresses the issue of whether other nearby emissions sources and/or whether background concentrations based on representative monitoring data should be included in the modeling analysis.

As noted above, NJDEP submitted dispersion modeling results based on maximum allowable emissions as well as actual emissions.  It is reasonable and appropriate to model allowable emissions when evaluating whether the source "emits or would emit" any air pollutant in violation of the prohibition of section 110(a)(2)(D)(i) under a section 126 petition.  EPA interprets the term "emits or would emit" as a reference to the source's current and potential future emissions.  A determination of whether the source "emits" pollutants in violation of the prohibition of section 110(a)(2)(D)(i) could be based on modeling of actual emissions.  However, for the emissions the source "would emit" (i.e., its potential future emissions), it is appropriate to consider the level at which the source could emit given the existing constraints on its emissions  -  that is, the source's allowable emissions.

For these same reasons, EPA believes it appropriate to model allowable emissions when determining the appropriate remedy to eliminate the source's significant contribution to nonattainment and interference with maintenance.  In addition, as a practical matter, it would be difficult to determine an appropriate remedy under a section 126 petition based on actual emissions given the potential variability of actual emissions.  Because the question posed is what additional limits must be placed on the source's emissions to eliminate its significant contribution to nonattainment and interference with maintenance, it is appropriate to consider what its emissions could be in the absence of such limits.

Portland Emissions

The dispersion modeling submitted by NJDEP only included emissions from the coal-fired units 1 and 2 at the Portland Plant.  There is also an auxiliary boiler which burns oil and 3 small turbines (units 3, 4, and 5) which all burn oil and natural gas, and have very small emissions.

As documented in section II, units 1, 2, and 5 utilize continuous emissions monitoring system (CEMS).  In 2009, SO2 emissions combined from units 1 and 2 at the plant were 30,465 tons and emissions from unit 5 were 0.3 tons. The auxiliary boiler, unit 3, and unit 4 SO2 annual emissions reported in the 2008 NEI for the auxiliary boiler, unit 3, and unit 4 were 0.01, 0.02, and 0.03 tons, respectively.

Based on the emissions information summarized above, it is necessary to model emissions from units 1 and 2.  Since unit 5 has CEMS data and an easily obtainable allowable emissions rate, EPA also chose to include unit 5 emissions in our modeling analysis.  The auxiliary boiler and units 3 and 4 have very small emissions and they also do not have an easily discernable allowable SO2 emissions rate.  Therefore, EPA's modeling is based on allowable emissions from units 1, 2 and 5 at the Portland Plant.  Table 4 shows the emissions and stack parameters used in EPA's AERMOD modeling.

Table 4.  SO2 Emissions and stack parameters used in EPA's AERMOD modeling

                                    Source
                         Permitted Emission Rate (g/s)
                               Stack Height (m)
                              Stack Diameter (m)
                             Stack Temperature (K)
                             Stack Velocity (m/s)
                          Portland Plant Coal Unit 1
                                     733.3
                                    121.92
                                     2.84
                                     403.0
                                     43.3
                          Portland Plant Coal Unit 2
                                    1,121.0
                                    121.72
                                     3.79
                                     406.0
                                     36.2
                           Portland Plant Turbine 5
                                     12.0
                                     42.7
                                      6.1
                                     821.5
                                     36.6

Background SO2 Concentrations

The dispersion modeling submitted by NJDEP with the September 17, 2010, petition only included emissions from units 1 and 2 at the Portland Plant, and did not account for background concentrations of SO2 from other sources.  NJDEP did not offer any rationale regarding the exclusion of any contribution from background concentrations in the modeling.  Therefore, we address it here.

Section 8.2 of Appendix W provides guidance regarding the inclusion of background concentrations in dispersion modeling demonstrations of compliance with the NAAQS under PSD regulations.  Appendix W defines "background air quality" as including "pollutant concentrations due to: (1) Natural sources; (2) nearby sources other than the one(s) currently under consideration; and (3) unidentified sources."  See Section 8.2.1a.  EPA recently issued additional clarification regarding application of Appendix W guidance for the 1-hour NO2 NAAQS, indicating that portions of that guidance are equally applicable to the 1-hour SO2 NAAQS.  Two topics addressed in the March 1, 2011, guidance that are relevant here are the determination of background concentrations and combining modeled results with monitored background concentrations to determine cumulative impacts.  While the guidance does not explicitly address dispersion modeling analyses in the context of a section 126 petition, we believe that the guidance provides an appropriate basis for the modeling conducted for the Portland Plant in support of this action.

A review of SO2 emission sources within 50 km of the Portland Plant identified 10 sources, located mostly in Pennsylvania southwest of the Portland Plant.  Other sources of SO2 emissions in the area include the Martins Creek facility which is located approximately 10 km to the south of the Portland Plant.  There are two units at Martins Creek, units 3 and 4, which averaged about 1,039 and 584 hours of operation respectively.  Those units each have a capacity of 850 MW and can burn either number 6 oil or natural gas.  The facility reported approximately 1,100 tons of SO2 emissions in 2009.  Martins Creek SO2 emissions have dropped significantly since the coal-fired units 1 and 2 were shutdown in 2007.  There are also three cement plants (Hercules, Keystone, and ESSROC) and several minor emitting units in Pennsylvania located at distances generally greater than 30 km away to the south and west of the Portland Plant.  The location of these sources in relation to the Portland Plant is shown in Figure 4.

Figure 4. Location of background sources of SO2 within 50 km of the Portland Plant.

In 2009, the Pennsylvania Department of Environmental Protection emission inventory database (PADEP eFACTS) reported 1,862 tons for Hercules, 685 tons for Keystone, and 799 tons for ESSROC of SO2 emissions respectively, all of which are relatively low compared to the SO2 emissions from the Portland Plant.

Of the SO2 emission sources identified for possible inclusion in the modeling analysis, the Martins Creek Plant is the only source that is large enough and close enough to the Portland Plant to be considered for inclusion in the modeling analysis.  However, the SO2 emissions from the Martins Creek Plant are somewhat intermittent (as noted earlier, Martins Creek units 3 and 4 averaged about 1,039 and 584 hours of operation per year respectively).  Even more fundamentally, the purpose of this modeling is to determine the impact of the Portland Plant itself on the downwind nonattainment areas.  Any intermittent impacts from Martins Creek would be in addition to the impacts from the Portland Plant and the Portland Plant would have no obligation to remedy any violations associated solely with those emissions.  This modeling uses actual monitored background levels of SO2 such that it is reasonable to expect that the contribution of intermittent emissions from Martins Creek and other nearby sources is accounted for in EPA's analysis.  This approach is also consistent with the modeling analysis conducted by NJDEP.  Further details regarding our assessment of nearby SO2 sources are provided in the Modeling TSD.

There are currently three operating SO2 monitors within 50 km of the Portland Plant, including the Chester monitor located about 36 km southeast of the Portland Plant in Morris County, New Jersey, the Easton monitor located about 27 km southeast in Northampton County, Pennsylvania, and the Columbia Lake WMA monitor located about 2 km northeast in Warren County, New Jersey.  The Columbia monitor has only been in operation since September 23, 2010, while the Chester and Easton(2) monitors have been in operation for several years.

Of the two long term SO2 monitors, the ambient SO2 data from the Chester, New Jersey, monitor provides the most representative background concentrations for this analysis since the distribution of sources impacting the Chester monitor is more similar to the distribution of sources around the Portland Plant.  While the Easton(2), Pennsylvania, monitor is better situated to capture background concentrations upwind in relation to Portland Plant impacts in New Jersey, the Easton(2) monitor is close enough to the Lehigh Valley Cement Plants and other SO2 sources that monitored SO2 levels at Easton(2) would overestimate background concentrations applicable to this analysis.

The Columbia monitor data period is too short to serve as a source of monitored background concentrations for this application.  Given its proximity to the Portland Plant, it is likely to capture ambient SO2 impacts associated with the Portland Plant emissions under appropriate meteorological conditions.  The location of the Columbia monitor also suggests that it may provide some useful insight into background concentration levels within the area by examining the concentration distribution during periods that are not affected by emissions from the Portland Plant.

Based on an assessment of the available SO2 monitoring data, we determined that the Chester monitor is the most appropriate monitor to account for background SO2 concentrations for the Portland Plant.  Consistent with the March 1, 2011, guidance, we included monitored concentrations based on the 99[th]-percentile by season and hour-of-day from the Chester data for 2007 through 2009 (the most recent data available) to account for background concentrations.  These background SO2 concentrations by season and hour-of-day varied from 13 ug/m[3] to 60 ug/m[3].  Examination of hourly SO2 concentrations for both the Chester monitor and the available data from the Columbia monitor indicates very low concentrations (less than 3 ppb) during the majority of the hours.  However, we consider the background concentrations used in our analysis (13 ug/m[3] to 60 ug/m[3]) to be appropriate for this application given that no other emission sources were explicitly modeled.  The temporally-varying background monitored concentrations incorporated in EPA's modeling analysis are shown in Figure 5.

Figure 5. Background SO2 monitored concentrations by season and hour-of-day included in the AERMOD cumulative modeling analysis for the Portland Plant.

   B.  Meteorological Data

Aside from emissions data, meteorological data is the other key input to dispersion models.  The AERMOD modeling was based on one year of site-specific meteorological data collected for from a 100m instrumented tower and SODAR located about 2.2km west of the PGS plant, as shown in Figure 3, for the period July 1993 through June 1994.  A summary of potential technical issues with the modeling analysis submitted by NJDEP is provided above.  A more detailed discussion of these issues and how the issues have been addressed in the modeling conducted by EPA to support our response to this petition is provided in Appendix B to this TSD.  EPA's remedy modeling utilized the adjustments to the measurement heights, as well as the additional modifications to the meteorological data documented in Appendix B.

   C.  Receptor and Terrain Data

Proper treatment terrain information is important for this analysis given the potential influence of elevated and complex terrain on the modeling results.  The NJDEP analysis was based on an initial grid of coarsely spaced receptor locations across a large domain covering all potentially important impact area associated with emissions from the Portland Plant, followed by a much smaller grid of more closely spaced receptors focused on the area of expected worst-case impacts from the plant.  The initial grid included spacing of 250 meters in areas of expected high impacts with receptors spaced at 1,000 meter intervals covering the gaps between the 250m grids.  The initial coarse receptor grid included a total of 5,189 receptors.  The subsequent fine grid used by NJDEP in determining the controlling impact from Portland for purposes of this petition included a total of 121 receptors in a 10x10 array spaced at 100m intervals covering a portion of the Kittatinny Ridge on the NJ side of the Delaware Water Gap, where the highest impacts from the coarse grid run occurred.  The results of EPA's coarse grid modeling with the adjusted meteorological data showed the highest impacts closer to the Portland Plant, about 3km northeast of the plant.  Based on these coarse grid results, EPA's fine receptor grids included two 100m grids, one located near the Kittatinny Ridge similar to NJDEP's fine grid, and the other focused on the area around and toward the northeast of the plant to encompass the location of peak impacts from the coarse grid.  Figures 6 and 7 depict the location of the initial coarse grid and the final fine grid, respectively.

NJDEP applied the AERMAP terrain processor with sixteen 7.5-minute USGS Digital Elevation Model (DEM) terrain files at 30m horizontal resolution covering most of the modeling domain and four 1-degree DEM files at 90m horizontal resolution covering the remainder of the domain.  NJDEP also used the NAD27 horizontal datum for all receptor and source coordinates.

The AERMAP terrain processor can also process terrain data based on the National Elevation Dataset (NED) format (USGS, 2002), which reflects updates to the older DEM terrain data.  Section 4.3 of the AERMOD Implementation Guide (EPA, 2009) discusses terrain elevation data sources and potential issues in light of the release of the NED data by USGS, and encourages the use of NED data for processing terrain data for use in AERMOD.  In light of this, EPA processed the terrain data for the NJDEP modeling domain using 1-second NED data (approximately 10m horizontal resolution).  The source and receptor coordinates were also converted to the NAD83 horizontal datum.  A comparison of terrain elevations and hill height scales used in the NJDEP analysis from the 100m receptor grid near the Delaware Water Gap with values based on application of AERMAP with NED data and found nearly identical results.  We conclude from this comparison that NJDEP's treatment of receptor/terrain data in their AERMOD analysis was appropriate.

   D.  EPA Remedy Modeling Results

The EPA AERMOD modeling results based on the initial coarse receptor grid described above indicated a maximum 99[th]-percentile (4[th]-highest) daily maximum 1-hour SO2 concentration (including monitored background) of 841 ug/m[3] (about 321 ppb) at a receptor located about 3 kilometers north-northeast of the Portland plant (494500m E;  4531500m N).  Results from this initial analysis are shown in Figure 6.

   Figure 6.  EPA modeling results for initial coarse receptor grid.

Compared to the initial coarse grid analysis conducted by NJDEP, EPA's modeled design value is about 32% lower (compared to 1,236 ug/m[3]) and occurs at a different location within the modeling domain.  While EPA's modeling showed peak impacts much lower than NJDEP's peak design value, we note that EPA's modeled design value of 841 ug/m[3] is about 90% higher than NJDEP's modeled impact at the controlling receptor from EPA's modeling.  These differences are likely due primarily to the adjustments in the processing of meteorological data input to the model.  The adjustments to the measurement heights could result in significant differences in the transport direction for particular hours, as well as somewhat lower wind speeds.  Both of these factors could shift the modeled impact area away from the higher terrain around the Delaware Water Gap toward a different part of the domain.  The inclusion of observed σw data from the SODAR in the EPA modeling could also account for this shift in the maximum impact area from Portland.  If observed σw values are higher than the reference values used in AERMOD in the absence of observations, then modeled impacts near the Delaware Water Gap, which are associated with direct plume impaction on the complex terrain, could be significantly lower.  In contrast, larger σw values would tend to increase concentrations in the lower terrain toward the northeast by mixing the plume to the ground faster.

Based on the results from the initial coarse grid analysis, EPA developed a finer resolution receptor network which included two separate grids with 100m horizontal resolution, shown in Figure 7.

    Figure 7.  Fine-resolution (100m) receptor grids used in EPA modeling

The smaller of the two fine resolution grids covers the impact area near the Delaware Water Gap to the northwest, and is similar to NJDEP's 100m fine grid, but is extended an additional 500 meters to the north and east.  The larger fine resolution grid is focused on the area surrounding the maximum design value from the initial coarse grid, and extends about 5km north, 4 km east, 1km south and 2km west of the Portland plant.  The location of the modeled peak from the coarse grid analysis and the recently implemented Columbia Lake monitor are also displayed in Figure 7.

EPA's modeling based on the 100m fine receptor grids resulted in modeled design value (including background) of 851 ug/m[3] (about 325 ppb).  This result is only slightly higher than and near the location of the controlling coarse grid result.  Figure 8 displays the 99[th]-percentile (4[th]-highest) 1-hour SO2 concentrations (including background) based on the fine grid analysis.

Figure 8.  EPA modeling results for fine-resolution (100m) receptor grid

      IV. Portland Generating Station Emissions Limits

As detailed above (and shown in Figure 8), the modeled maximum 99[th] percentile (4[th]-highest) daily maximum 1-hour SO2 concentration (including monitored background) from the Portland Plant in New Jersey was 851.1 ug/m[3].  Table 5 shows the contribution from each of the Portland Plant units to the design value concentration.

Table 5. Contributions from Portland Plant's units 1, 2, and 5 to modeled design value

Unit 1
Unit 2
Unit 5
Background
Total
371.7 ug/m[3]
439.2 ug/m[3]
0.91 ug/m[3]
39.3 ug/m[3]
851.1 ug/m[3]

Based on this result, EPA calculated the emissions reduction needed to eliminate the Portland Plant's significant contribution to nonattainment in New Jersey.  The calculation is relatively simple in this case because emissions from the Portland Plant alone cause violations of the 1-hour SO2 NAAQS in New Jersey and background levels of SO2 are very low.  If the modeled concentration from the Portland Plant plus background is reduced to a level that is below the 1-hour SO2 NAAQS, then there will be no modeled violations of the NAAQS in New Jersey.

Based on the EPA modeling results, an 81 percent reduction in allowable SO2 emissions from Portland Plant units 1 and 2 is needed to reduce the Portland Plant contribution plus background to below the NAAQS.  The calculation is as follows:

    (Total modeled concentration)  -  (NAAQS  -  background)/(total modeled concentration).

This calculation recognizes that the assumed background concentration cannot be reduced.  The actual calculation based on Table 5 is (811.8)-(196-39.3)/811.8.  This results in a reduction of 80.7 percent, which we round to 81 percent.

In this calculation, the contribution from all modeled sources (units 1, 2, and 5) is included in the total contribution.  However, the contribution from unit 5 is only 0.1 percent of the total contribution (0.91 ug/m[3] contribution to the design value). A reduction in the unit 5 contribution would provide a negligible reduction to the modeled design value.  Therefore, it can be assumed that unit 5 emissions do not need to be reduced, and therefore can be added to the irreducible background value.  This alternative calculation gives an emissions reduction of 80.8 percent (which is essentially the same as the previous 80.7 percent calculation).  Therefore, we conclude that only emissions reductions from units 1 and 2 are needed in order to ensure that the downwind area in New Jersey will be able to attain the NAAQS and will not have maintenance problems and that a revised emissions limit is not needed for unit 5.

While a total emissions reduction of 81 percent for both units 1 and 2 eliminates all modeled violations in New Jersey, an additional question remains.  Can the emissions limit be met by over controlling one unit (by more than 81 percent) and under controlling the other unit (by less than 81 percent)?    Based on our analysis, there are many different combinations of emissions limits for units 1 and 2 that would eliminate violations of the SO2 NAAQS in New Jersey.  However, the stack parameters (exit velocity and stack diameter) of units 1 and 2 are slightly different, which causes the maximum downwind impacts from each unit to occur at slightly different locations at different times.  Therefore, the emissions limit has to be assigned to each individual unit and cannot be a combined limit.  There are many different combinations of emissions limits for units 1 and 2 that would eliminate violations of the SO2 NAAQS in New Jersey, but we are not able to examine an unlimited number of combinations.  Therefore we are proposing an emissions limit based on an 81 percent reduction in allowable emissions at both units 1 and 2.  This leads to a proposed SO2 emissions limit for unit 1 of 1105 lbs/hr (5820*0.19) and a proposed SO2 emissions limit for unit 2 of 1691 lbs/hr (8900*0.19).

As a final check on the emission limit calculations, EPA ran AERMOD again with the above emissions limits on the Portland Plant's units 1 and 2 (and current allowable emissions from unit 5).  At these proposed emissions levels, all receptors in New Jersey were below the 1-hour SO2 NAAQS.  The modeled 99[th] percentile (4[th]-highest) daily maximum 1-hour SO2 concentration was 192.2 ug/m[3] (including a background concentration of 41.9 ug/m[3]).  Figure 9 shows the results of the 81% remedy run.

Figure 9.  EPA modeling results for the fine resolution (100m) receptor grid with an 81% reduction in allowable emissions at Portland units 1 and 2

      V. References

EPA, 1992. Protocol for Determining the Best Performing Model. Publication No. EPA - 454/ R - 92 - 025. Office of Air Quality Planning & Standards, Research Triangle Park, NC. (NTIS No. PB 93 - 226082).

EPA, 2000.  Meteorological Monitoring Guidance for Regulatory Modeling Applications.  EPA 454/R-99-005.  U.S. Environmental Protection Agency, Research Triangle Park, NC (Available at: www.epa.gov/scram001/).

EPA, 2003. AERMOD: Latest Features and Evaluation Results.  EPA-454/R-03-003.  U.S. Environmental Protection Agency, Research Triangle Park, NC.

      EPA, 2004. User's Guide for the AERMOD Terrain Preprocessor (AERMAP).  EPA-454/B-03-003. U.S. Environmental Protection Agency, Research Triangle Park, NC (Available at: www.epa.gov/scram001/).

      EPA, 2008. AERSURFACE User's Guide.  EPA-454/B-08-001. U.S. Environmental Protection Agency, Research Triangle Park, NC (Available at: www.epa.gov/scram001/).

      EPA, 2009.  AERMOD Implementation Guide.  U.S. Environmental Protection Agency, Research Triangle Park, NC (Available at: www.epa.gov/scram001/)

Perry, S. G., A. J. Cimorelli, R. J. Paine, R. W. Brode, J. C. Weil, A. Venkatram, R. B. Wilson, R.F. Lee, and W. D. Peters, 2005. AERMOD: A dispersion model for industrial source applications. Part II: Model performance against seventeen field-study databases. J.Appl.Meteor., 44, 694-708.

Strimaitis, D.G., J.S. Scire and J.C. Chang, 1998. Evaluation of the CALPUFF Dispersion Model with Two Power Plant Data Sets. Tenth Joint Conference on the Application of Air Pollution Meteorology, Phoenix, Arizona. American Meteorological Society, Boston, MA. January 11 - 16, 1998.

Venkatram, A., R. W. Brode, A. J. Cimorelli, J. T. Lee, R. J. Paine, S. G. Perry, W. D. Peters, J. C. Weil, and R. B. Wilson, 2001. A complex terrain dispersion model for regulatory applications. Atmos. Environ., 35, 4211 - 4221.

                                  Appendix A

                 Assessment of NJDEP CALPUFF Validation Study

Alternative Model Justification

NJDEP's September 17, 2010 petition regarding 1-hour SO2 impacts in NJ associated with emissions from the Portland Plant cites a model validation study (submitted as Exhibit 12 with the May 12, 2010 petition) comparing the performance of the AERMOD model with CALPUFF based on the Martins Creek field study database from 1992-1993.  The Martins Creek database was included in the validation of the AERMOD model.  NJDEP concluded from this study that "CALPUFF performed better and produced predictions of greater accuracy than AERMOD."  See September 17, 2010 petition, Section IV, page 5.  The February 24, 2010 "Validation of CALPUFF in the Near-Field" submitted with the May 12, 2010 petition indicates that "the purpose of this model validation study was to evaluate the performance of CALPUFF in this study area and determine if its use is appropriate and produces predictions of greater accuracy than the Appendix A model AERMOD.  An additional objective of the validation study was to determine whether or not CALPUFF is biased toward underestimating SO2 concentrations in this location[1]."  See Exhibit 12, to the May 12, 2010 NJDEP petition Section 2, page 2.  Footnote 1 states that "Appendix W to Part 51 CFR, Section 3.2.2e provides that an alternative refined model may be used if, among other requirements, "[a]ppropriate performance evaluations of the model have shown that the model is not biased toward underestimates;"  subsection e does not technically apply as this subsection only applies to Condition 3, see Section 3.2.2b."

From the statements summarized above, we conclude that NJDEP's use of the CALPUFF model in support of the September 17, 2010 petition is based on a claim that CALPUFF was shown to have "performed better and produced predictions of greater accuracy than AERMOD", and therefore satisfies condition (2) under Section 3.2.2b of Appendix W.  See September 17, 2010 petition, Section IV, page 5.  The criteria applied to condition (2) for use of alternative models under Section 3.2.2b are discussed in paragraph (d) of Section 3.2.2:

      d. For condition (2) in paragraph (b) of this subsection, established procedures and techniques for determining the acceptability of a model for an individual case based on superior performance should be followed, as appropriate. Preparation and implementation of an evaluation protocol which is acceptable to both control agencies and regulated industry is an important element in such an evaluation.

Condition (2) relies solely on a demonstration that the alternative model has been shown to perform better than a comparable Appendix A model, and does not entail a demonstration that the preferred model (AERMOD in this case) is inappropriate for the application.

The performance of the AERMOD model for estimating impacts associated with tall stacks in complex terrain settings has been extensively evaluated and documented in peer-review journals (Perry, et al., 2005; Venky, et al., 2001), and has consistently been shown to perform better than competing models.  In contrast, the NJDEP model validation study submitted as Exhibit 12 with the May 12, 2010 petition is the first model evaluation we are aware of based on the CALPUFF model, driven by gridded CALMET wind fields, in a near-field, complex terrain setting.  The evaluation of CALPUFF against the Lovett Power Plant field study (Strimaitis, et al., 1998) did not utilize CALMET wind fields but was based on CTDMPLUS surface and profile meteorological data derived from a single meteorological tower, similar to the meteorological inputs used by AERMOD, and is therefore not relevant to NJDEP's application of CALPUFF in this case.

NJDEP's CALPUFF validation study (Exhibit 12 to the May 12, 2010 NJDEP petition) was focused on the Martins Creek power plant located about 14 kilometers south-southwest of the Portland plant, during the period from May 1, 1992 through May 19, 1993, which is one of the field study databases used in evaluating the performance of AERMOD.  While focused on impacts from Martins Creek on complex terrain located about 5 kilometer southeast of Martins Creek, emissions from the Portland Plant and two other nearby sources were also included in the evaluation database.  While the Martins Creek field study was focused on the Martins Creek plant since the ambient monitors were located on a nearby complex terrain feature within a few kilometers of Martins Creek, the general proximity to the Portland Plant and the inclusion of Portland Plant emission in the modeling could potentially give the database some additional relevance to the application under consideration here.  The emission sources included in the modeling conducted as part of the Martins Creek field study included Martins Creek units 1-4, Portland units 1 and 2, one emission unit from Hoffmann LaRoche, and WCRRF units 1 and 2.  As part of the validation of CALPUFF based on the Martins Creek field study, NJDEP modified the actual emissions for Portland unit 1 to address what are believed to be errors in the Continuous Emissions Monitor System (CEMS) data for unit 1.  These corrections increased the total emissions from unit 1 over the data period by about a factor of 2.  No inaccuracies were found in the emissions for Portland Plant unit 2.

Based on the issues and considerations discussed above, acceptance of the CALPUFF modeling submitted by NJDEP as the basis for assessing impacts of Portland Plant emissions on attainment of the 1-hour SO2 NAAQS in NJ depends on whether the validation study submitted with the May 13, 2010 petition adequately demonstrates that CALPUFF performs better than AERMOD.  Before discussing the specifics of that determination, we note again that the AERMOD model has undergone extensive peer review and model validation as the basis for its promulgation as the preferred model for "a wide range of regulatory applications in all types of terrain" (Appendix W, Section 4.4.2(b)), whereas the validation study submitted by NJDEP is the only example we are aware of involving evaluation of the CALPUFF modeling system, including the CALMET meteorological processor, for near-field complex terrain settings.  Therefore, even if we judge the NJDEP evaluation to be an adequate demonstration that CALPUFF performs better than AERMOD in this particular case, the weight of evidence would still favor the AERMOD model as the preferred model for this application, unless the NJDEP evaluation presents compelling evidence that CALPUFF is clearly superior to AERMOD for this application and that the proximity of the Martins Creek field study to the Portland plant adds greater emphasis to that conclusion.

Assessment of NJDEP Model Evaluation Methodology

Model validation is a complex process that entails several technical challenges, including uncertainties regarding the accuracy and representativeness of key input data that could affect results, as highlighted by NJDEP's review of the Portland Plant emissions data included in the original study, as well as a wide range of statistical methods and metrics that may be applied to quantify model performance.  In some cases subtle changes to the evaluation methods can markedly affect the conclusions that might be drawn from such studies.  For these reasons, the importance of establishing a consistent set of objective procedures to evaluate the performance of dispersion models for use in regulatory modeling applications and of comparing the relative performance of competing models has long been recognized.  Section 3.2.1 of Appendix W references an EPA document (EPA, 1992) that "is available to assist in developing a consistent approach when justifying the use of other-than-preferred modeling techniques recommended in the Guideline. The procedures in this protocol provide a general framework for objective decision-making on the acceptability of an alternative model for a given regulatory application. These objective procedures may be used for conducting both the technical evaluation of the model and the field test or performance evaluation."  Although NJDEP's model validation report describes the results presented in Section 8.3 of Exhibit 12 as "Model Evaluation Results Based on EPA's AERMOD Validation Procedures," NJDEP made several changes to the model evaluation methodology as compared to the evaluation conducted by EPA in support of promulgation of AERMOD as a preferred model in Appendix W.  EPA did not have an opportunity to review or comment on these changes to the model evaluation protocol implemented in NJDEP's CALPUFF validation study.

This EPA "Protocol for Determining the Best Performing Model" (Protocol) served as the primary basis for evaluating the performance of AERMOD and comparing its performance with other models using objective criteria.  The procedures recommended in this protocol were supplemented with additional qualitative and quantitative comparisons to fully inform the assessment of model performance and to shed light on factors contributing to both good and poor model performance.  The statistical comparisons of modeled to monitored concentrations recommended in the Protocol include a diagnostic component and an operational component.  The diagnostic component generally compares model performance on an hourly basis for each monitor based on subsets of meteorological conditions, providing a more rigorous test of model performance including some degree of temporal and spatial pairing of modeled and monitored concentrations.  The operational component compares model performance unpaired in time and space based on averaging periods and model metrics that are more directly relevant to a regulatory modeling application.  The operational component has typically included model-to-monitor comparisons of 3-hour averages and 24-hour averages corresponding to the previous short-term NAAQS for SO2, using the robust highest concentrations (RHC) based on an exponential tail fit to the upper end of the concentration distribution.

As stated in NJDEP's model validation report, the RHC "attempts to represent a stable estimate of the highest concentration, one that mitigates the unwanted influence of unusual events."  The RHC is calculated as follows:

where
	 N	=	number of values exceeding a threshold value, typically N = 26,
		=	average of the N-1 largest values, and
	X(N)	=	Nth largest value.

A composite performance measure (CPM) is then computed as a weighted average of the RHC comparisons for the diagnostic and operational components.  A model that performs very well on the less rigorous operational component of the evaluation but poorly on the more rigorous diagnostic component might be characterized as a model that "gets the right answer for the wrong reasons."

Three key changes implemented in the NJDEP model evaluation study relative to the methods used by EPA include: 1) the specification of the number of samples used in calculating the RHC for modeled and monitored concentrations; 2) the use of modeled and monitored concentrations paired by rank, rather than paired in time, in the residual box plots; and 3) the meteorological categories used to group model-to-monitor comparisons as part of the diagnostic component of the evaluation.  The first two issues are discussed in more detail below, including some results of reanalyzing the data from the validation study to illustrate their potential impact on NJDEP's conclusions from the study.  The potential impact of the third issue is more difficult to assess.

      Sample size for robust highest concentration (RHC) calculations

The EPA Protocol recommends using the top 26 values in the concentration distribution to calculate the RHC, whereas the NJDEP evaluation varied the number of values used to determine the RHC across the various components of the evaluation.  For example, NJDEP used the top 8 values for calculating the network-wide 3-hour RHC as part of the operational, while the top 26 values were used to calculate the network-wide 24-hour RHC.

The NJDEP model validation report (Exhibit 12 to the may 12, 2010 NJDEP petition) cites a personal communication with William Cox, one of the authors of the EPA Protocol document, as recommending that the number of samples used in calculating the RHC should be based on an evaluation of the slope of the tail distribution on an individual basis for each averaging period and receptor.  However, the NJDEP report does not cite any clear objective criteria for selecting the number of samples, and the number of samples selected by NJDEP ranged from a low of 6 to a high of 26 without any discernible pattern that would justify those selections.  While Appendix C of the NJDEP report asserts that the "a priori selection of N as 26 without examination of the data is arbitrary," this approach has been used in a number of evaluation studies in the past and provides a degree of consistency to the process.  The original reference on which the EPA Protocol is based characterizes the number of samples as being "arbitrarily chosen to be equal to 26 but may be lower in cases where there are fewer concentrations exceeding the threshold value" (Cox and Tikvart, 1990).  In this case, NJDEP selected N = 8 for the network-wide 3-hour RHC and N = 26 for the network-wide 24-hour RHC.  Since the number of samples available for the 3-hour averages is by definition much higher than the number of samples available for the 24-hour averages, the rationale provided in the Protocol for using a different value for N would not apply in this case.

The highly variable number of samples selected to compute RHCs in the NJDEP validation report represents a significant departure from standard practice, without any clear technical basis, and raises serious concerns regarding the conclusions of NJDEP's validation of CALPUFF.   For example, Figures C.1 and C.2 of the NJDEP validation report (shown below as Figures A.1 and A.2) show the ranked distribution of network-wide 3-hour and 24-hour observed concentrations, respectively, where N=8 was selected for the 3-hour RHC and N=26 was selected for the 24-hour RHC.  The overall shape of these two distributions appears to be fairly similar and there is no clear justification for such different choices in the number of samples in these two cases.

To illustrate the potential importance of this issue to the model evaluation results, Table A.1 compares modeled-to-monitored RHCs for the network-wide 3-hour and 24-hour averages based on different values of N.  The rows corresponding to the values used by NJDEP are shaded.  We note that the shaded values are the "best" ratios for each averaging period for CALPUFF (i.e., closest to 1.0), but the "worst" ratios for each averaging period for AERMOD (i.e., furthest from 1.0).  For every other case, the AERMOD ratios are closer to 1.0 than the corresponding CALPUFF ratios.  The AERMOD ratios also show less variation across the range of values for N than the CALPUFF results.  These comparisons of RHCs based on different values of N call into question the methods and conclusions of the NJDEP validation study regarding the relative performance of the CALPUFF and AERMOD models on this field study database.  However, we also point out that overall these model performance results do not suggest significant differences in performance for CALPUFF vs. AERMOD.

The Q-Q plots included in the NJDEP validation report (shown below as Figure A.3) also provide a clear visual representation of model performance based on the same data used to calculate the network-wide RHC values.  These Q-Q plots appear to corroborate the conclusion that the performance of the CALPUFF and AERMOD models on this database is similar, but that AERMOD shows slightly better overall agreement with observations.

Figure A.1.  Distribution of highest ranked 3-hour network-wide concentrations from Martins Creek validation study (based on Figure C.1 of Exhibit 12).

Figure A.2.  Distribution of highest ranked 24-hour network-wide concentrations from Martins Creek validation study (based on Figure C.2 of Exhibit 12).

Table A.1.  Comparison of RHCs based on different values of N[a]
Averaging Time; N
                                   Observed
                                  (ug/m[3])
                                    CALPUFF
                                  (ug/m[3])
                                 Ratio to Obs

                                      FB
                                    AERMOD
                                  (ug/m[3])
                                 Ratio to Obs

                                      FB
3-hr; 8
                                     659.0
                                     720.2
                                     1.09
                                     0.09
                                     570.0
                                     0.86
                                     -0.15
3-hr; 11
                                     613.3
                                     711.7
                                     1.16
                                     0.15
                                     604.2
                                     0.99
                                     -0.02
3-hr; 15
                                     587.5
                                     757.7
                                     1.29
                                     0.25
                                     605.5
                                     1.03
                                     0.03
3-hr; 26
                                     556.6
                                     785.5
                                     1.41
                                     0.34
                                     609.4
                                     1.09
                                     0.09
24-hr; 8
                                     165.6
                                     215.1
                                     1.30
                                     0.26
                                     161.4
                                     0.98
                                     -0.03
24-hr; 11
                                     162.5
                                     202.4
                                     1.24
                                     0.22
                                     169.5
                                     1.04
                                     0.04
24-hr; 15
                                     162.5
                                     193.1
                                     1.19
                                     0.17
                                     166.7
                                     1.03
                                     0.03
24-hr; 26
                                     187.0
                                     183.7
                                     0.98
                                     -0.02
                                     158.1
                                     0.85
                                     -0.17
[a] Shaded rows show values used by NJDEP.

Another issue worth noting in relation to model-to-monitor comparisons of RHCs in the NJDEP study is the inclusion of the AMS#8 monitor in the comparisons.  The EPA evaluations based on the Martins Creek field study data did not include the AMS#8 monitor as part of the model-to-monitor comparisons since it was sited and used specifically to account for background concentrations during periods when the emissions from Martins Creek would be impacting the complex terrain receptors located on Scotts Mountain about 3 to 5 km southeast of the plant.  As shown in Figure A.4 (based on Figure 1 from the NJDEP validation report), the AMS#8 monitor is almost ideally situated for that purpose.  Also, the AMS#8 monitor is about the same elevation as the Martins Creek stack tops, and about 100 to 130 meters lower than the complex terrain receptors located on Scotts Mountain.  For model evaluation field studies conducted around operating power plants, such as the Martins Creek field study, a proper accounting for background concentrations in the model-to-monitor comparisons is an important factor to consider in order to ensure confidence in the results.  A common practice, which was used in the EPA and NJDEP evaluations using Martins Creek data, is to use the monitor reporting the lowest ambient concentration on an hour-by-hour basis as the "background" monitor, and subtract its monitored concentration from the other monitored values.  Wind direction may also be a factor considered in determining an appropriate background concentration in some cases.  Section 5.2 of the NJDEP validation report indicates that background was determined on an hourly basis using the lowest concentration reported from any of the monitors.

Figure A.3.  Q-Q plots of CALPUFF and AERMOD predicted concentrations and observed concentrations (from Figure 9 of NJDEP's CALPUFF validation report, Exhibit 12 to the May 12, 2010 NJDEP petition).

The issue of including the AMS#8 monitor in the model-to-monitor comparisons is important for two reasons.  First, the highest 24-hour average monitored RHC was from AMS#8, as shown in Table 8 of the NJDEP validation report (Exhibit 12 to the May 12, 2010NJDEP petition).  Second, as indicated in Section 5.1 of the NJDEP report, emissions from the Portland Plant "were principally responsible for elevated concentrations measured" at AMS#8 that were associated with "winds blowing from the northeast quadrant."  It is clear from Figure A.4 that the other ambient SO2 monitors on Scotts Mountain are not well-situated to account for background concentrations under such meteorological conditions, raising concerns regarding the representativeness of that data for model-to-monitor comparisons.  Inclusion of the data from AMS#8 appears to be one of the main factors contributing to NJDEP's conclusion that AERMOD exhibits a bias to underpredict ambient SO2 concentrations.  This point is illustrated in Table A.2, which compares the maximum network-wide monitored RHC based on concentrations from all monitors with the maximum network-wide modeled RHC, for both 3-hour and 24-hour averages.  The data in the shaded rows comes from Table 8 of the NJDEP validation report which includes AMS#8, whereas the unshaded rows provide results without AMS#8.  Inclusion of AMS#8 did not affect the 3-hour results, where both models showed a slight overprediction bias.  However, the 24-hour results are significantly different without including AMS#8, showing modest overprediction biases without AMS#8 compared to a slight underprediction bias with AMS#8 included.  The AERMOD predicted/observed RHC ratios are closer to 1 for both the 3-hour and 24-hour results when AMS#8 is excluded from the comparisons.

Although the model evaluation methodology has historically focused on the 3-hour and 24-hour averages, consistent the short-term SO2 NAAQS prior to promulgation of the new 1-hour SO2 standard, given that this petition was submitted in relation to the 1-hour SO2 NAAQS we computed the 1-hour observed and predicted RHCs using the standard value of N=26, based on the model evaluation data provided by NJDEP.  The observed network-wide 1-hour RHC was 1,173.9 ug/m[3] compared to the AERMOD 1-hour RHC of 1,246.3 ug/m[3] (pred/obs = 1.06) and the CALPUFF 1-hour RHC of 1,303.5 ug/m[3] (pred/obs = 1.11).   These results are generally consistent with the other model-to-monitor comparisons showing both models to perform well, but with AERMOD results slightly better than CALPUFF results.

Table A.2.  Comparison of Individual Monitors Maximum RHC Compared to Observed RHC
Averaging Time; N
                                   Observed
                                  (ug/m[3])
                                    CALPUFF
                                  (ug/m[3])
                                 Ratio to Obs

                                      FB
                                    AERMOD
                                  (ug/m[3])
                                 Ratio to Obs

                                      FB
3-hr; 11
                                     505.6
                                     658.3
                                     1.30
                                     0.26
                                     616.6
                                     1.22
                                     0.20
3-hr; 11
                                     505.6
                                     658.3
                                     1.30
                                     0.26
                                     616.6
                                     1.22
                                     0.20
24-hr; 16
                                     194.6
                                     165.5
                                     0.85
                                     -0.16
                                     147.4
                                     0.76
                                     -0.28
24-hr; 16
                                     87.2
                                     165.5
                                     1.90
                                     0.62
                                     147.4
                                     1.69
                                     0.51
[a] Shaded rows show values used by NJDEP including AMS#8 and unshaded rows are without AMS#8.

Figure A.4.  Location of SO2 sources, monitors and meteorological stations used in the model validation study (based on Figure 1 from NJDEP's validation report)

      Residual Plots

Another method commonly used as part of the diagnostic component of model validation studies involves the use of residual plots, which are designed to show the distribution of model-to-monitor comparisons (represented by the ratio of predicted/observed concentration) as a function of some independent variable.  The distribution of predicted/observed ratios can be depicted graphically using a box plot (sometimes referred to as a "box and whisker" plot) that shows the median ratio (50[th]-percentile of the distribution) along with other percentiles within the distribution.  The independent variables most commonly used for dispersion model validations include source-receptor distance and meteorological parameters, such as wind speed or stability.  One of the main purposes of these residual plots is to determine whether a model has any inherent bias associated with a particular independent variable.  For example, a model might show very good agreement between modeled and monitored concentrations based on the overall distribution of impacts, but also show a significant bias vs. downwind distance, perhaps over predicting significantly close to the source and under predicting further from the source.  As such, residual plots serve as a method to diagnose model performance more rigorously than Q-Q plots or overall RHC comparisons.

The NJDEP CALPUFF validation report describes the residual box plots included in that study as representing "a method of directly comparing modeled and monitored data."  It further states that "[t]hey are generated by calculating the ratio of each model's ranked maximum prediction at any of the eight monitors for an hour to the ranked maximum measured 1-hour concentration at any of the eight monitors" (emphasis added).  The independent variable used to group the results in the NJDEP study was stability, based on the three categories of unstable, neutral, and stable.  The use of modeled vs. monitored concentrations paired by rank (similar to the data used in the RHC calculation or Q-Q plots) is not consistent with the intent of residual plots, which should be based on the distribution of predicted/observed ratios paired in time.  The Q-Q plots and RHC comparisons are intended to compare results paired by rank as part of the operational component of the evaluation.  However, those comparisons by rank are focused on the upper end of the ranked distribution since that is the only part of the distribution that is relevant from an operational perspective.  A comparison of modeled and monitored results paired by rank for the lower portion of the distribution has no relevance to evaluating a model's performance.  Therefore, the residual plots provided in Figure 9 of the NJDEP validation report (shown below as Figure A.5) provide no useful information on the relative performance of the CALPUFF and AERMOD models on this database.

Based on data provided by NJDEP, we generated residual box plots using the appropriate temporal pairing of modeled and monitored concentrations to assess what impact this would have on the comparison.  The original results presented in Figure A.5 suggest a bias for AERMOD to underpredict concentrations during unstable and neutral conditions, with better performance for stable conditions.  The CALPUFF results show unbiased performance for unstable conditions, with a trend toward under predicting for neutral and stable conditions, but the distribution crosses the ratio of 1 for all stability categories.  The residual box plots based on predicted/observed ratios paired in time are shown in Figure A.6.  These results show nearly unbiased results for AERMOD for unstable and neutral conditions, with some bias to overpredict for stable conditions (although the median ratio for stable is close to 1).  The CALPUFF results in Figure A.6 show some tendency to overpredict for all stabilities base on the mean values, although the median values are much closer to 1 than the mean values for unstable and neutral conditions.  It should also be noted that the vertical scale in Figure A.6 is much broader than Figure A.5, since the ratios paired in time exhibit a greater degree in variability than results paired by rank, as would be expected.  Figure A.7 shows the original NJDEP data paired by rank as plotted in Figure A.5 using the same vertical scale as Figure A.6 for comparison purposes.

Figure A.5.  Residual box plots for CALPUFF and AERMOD based on predicted/observed ratios paired by rank (from Figure 9 of NJDEP's CALPUFF validation report).

Figure A.6.  Residual box plots for CALPUFF and AERMOD based on predicted/observed ratios paired in time.

Figure A.7.  Residual box plots for CALPUFF and AERMOD based on predicted/observed ratios paired by rank, with same scale as Figure A.6.

Since the main rationale for use of the non-steady-state CALPUFF modeling system (including the CALMET diagnostic wind field model) over a straight-line, steady-state model such as AERMOD, is the expectation that the modeling system can more accurately account for the effect that spatially non-uniform wind fields might have on modeled concentrations, it would seem reasonable to expect that a model with those capabilities would exhibit superior performance on the basis of model-to-monitor comparisons paired in space.  To examine this issue, we compared modeled-to-monitored RHCs paired in space based on the data provided by NJDEP.  The results of these comparisons are summarized in Table A.3.  The last column of Table A.3 provides the geometric mean of the predicted/observed RHCs across the seven monitors (excluding AMS#8).  The geometric mean provides a better characterization of the mean ratio for these purposes than an arithmetic mean since over- and under-prediction biases are weighted equally in the geometric mean, e.g., the geometric mean of 0.5 and 2.0 is 1.0, whereas the arithmetic mean is 1.25, effectively placing more emphasis on the factor of 2 overprediction.  While both models show generally good overall agreement with observations, the AERMOD results show slightly better agreement with observations, paired in space.

Table A.3.  Comparison of Modeled-to-Monitored RHCs Paired in Space

                                     AMS#5
                                     AMS#7
                                     AMS#9
                                    AMS#10
                                    AMS#11
                                    AMS#12
                                    AMS#13
                                      GM
                                   Pred/Obs
Observed 3-hr
                                                                          313.6
                                                                          300.7
                                                                          352.0
                                                                          307.9
                                                                          265.1
                                                                          505.6
                                                                          432.7

AERMOD 3-hr
                                                                          314.3
                                                                          616.6
                                                                          512.3
                                                                          277.0
                                                                          452.4
                                                                          491.2
                                                                          325.6

AERMOD Pred/Obs 3-hr
                                                                           1.00
                                                                           2.05
                                                                           1.46
                                                                           0.90
                                                                           1.71
                                                                           0.97
                                                                           0.75
                                                                           1.04
CALPUFF 3-hr
                                                                          616.0
                                                                          545.4
                                                                          655.0
                                                                          527.1
                                                                          446.7
                                                                          658.3
                                                                          379.8

CALPUFF Pred/Obs 3-hr
                                                                           1.97
                                                                           1.81
                                                                           1.86
                                                                           1.71
                                                                           1.69
                                                                           1.30
                                                                           0.88
                                                                           1.39
Observed 24-hr
                                                                           62.5
                                                                           75.8
                                                                           79.9
                                                                           70.2
                                                                           87.2
                                                                           80.4
                                                                           80.0

AERMOD 24-hr
                                                                           88.1
                                                                          139.3
                                                                          147.4
                                                                           69.8
                                                                          104.7
                                                                          116.5
                                                                           74.1

AERMOD Pred/Obs 24-hr
                                                                           1.41
                                                                           1.84
                                                                           1.85
                                                                           0.99
                                                                           1.20
                                                                           1.45
                                                                           0.93
                                                                           1.12
CALPUFF 24-hr
                                                                          108.8
                                                                          130.6
                                                                          165.5
                                                                          120.3
                                                                          105.8
                                                                          113.3
                                                                           76.8

CALPUFF Pred/Obs 24-hr
                                                                           1.74
                                                                           1.72
                                                                           2.07
                                                                           1.71
                                                                           1.21
                                                                           1.41
                                                                           0.96
                                                                           1.30

	Meteorological Categories

The other notable change in the methodology implemented by NJDEP in their model evaluation study was the use of three stability categories, unstable, neutral, and stable, to define the pairings used in the diagnostic component of the evaluation.  The residual plots discussed above utilized these meteorological categories, as well as the BOOT statistical evaluation software that utilizes a bootstrap resampling technique to estimate confidence intervals on the model performance evaluation metrics.  Since the AERMOD model formulation is based on continuous functions of the Monin-Obukov lengths (L) stability parameter and does not utilize discrete stability categories, a range of Monin-Obukov lengths (L) was specified for each category based on an assumed roughness length of 0.1 meters.  The method used by EPA in its evaluations of AERMOD performance used four diagnostic categories based on wind speed and stability, i.e., low wind speed (< 4 m/s) vs. high wind speed (> 4 m/s) and stable (L > 0) vs. convective (L<0).  The latter approach for the diagnostic component is preferred over the approach used by NJDEP since it accounts for both wind speed and stability dependence on model performance, and treats the stability component in a manner that is more consistent with AERMOD model formulation.  Based on the data provided by NJDEP we were not able to ascertain what impact these diagnostic categories might have on the model performance statistics for CALPUFF and AERMOD on this database.

A final and fundamental point in relation to NJDEP's overall justification for the use of CALPUFF in this petition is that results from the model validation study are not relevant to this application of CALPUFF due to fundamental differences in the CALMET meteorological processing used in each case.  The CALMET modeling for the validation study made use of the site-specific meteorological data collected as part of the field study such that the documented CALPUFF model performance is largely dependent on the characterization of wind fields by CALMET that are informed by that site-specific data.  NJDEP used the new PROF2UP utility developed by the CALPUFF developers to facilitate a more effective use of multi-level meteorological measurements within the CALMET processor.  The PROF2UP tool has not been made publicly available by the CALPUFF developers, but it appears to have been developed to address some of the technical issues identified in EPA's September 26, 2008 memo regarding use of CALPUFF in near-field applications.  In contrast, the application of CALPUFF to estimate ambient SO2 impacts associated with Portland Plant emission in support the NJDEP petition did not use any site-specific meteorological data but relied on three different sets of MM5 prognostic meteorological data to inform the 3-dimensional wind fields generated by CALMET.  Performance of the CALPUFF model in the latter case would rely upon the ability of the MM5 prognostic model coupled with the CALMET diagnostic model to adequately simulate the wind fields in the absence of such site-specific data.  There have not been any demonstrations of the ability of CALMET with MM5 to adequately simulate the wind fields that would be relevant to this application.

The key difference between the CALMET processing used for the validation study utilizing site-specific meteorological data vs. the CALMET processing for the section 126 petition may explain why the model performance results for CALPUFF and AERMOD based on the NJDEP validation study look very similar, while the CALPUFF-modeled concentrations of Portland Plant emissions submitted in the September 17, 2010 petition are significantly different than AERMOD-modeled concentrations.  In addition to the CALPUFF model results being significantly higher than the AERMOD results, we also note that the spatial distribution of 1-hour SO2 impacts predicted by CALPUFF is very different than the distribution of impacts predicted by AERMOD.  The CALPUFF modeling shows extremely high 1-hour SO2 concentrations very close to the Portland plant (see Figures 1, 2 and 3 of Exhibit 3).  The highest 1-hour SO2 impacts based on the 2002 CALPUFF modeling occur within about 100 meters of the Portland Plant at an elevation of 3 meters above stack base on the PA side of the Delaware River.  These results are physically unrealistic for buoyant plumes from tall stacks such as units 1 and 2 at the Portland Plant, raising additional concerns regarding the appropriateness of CALPUFF for this application.

Conclusions

Based on this assessment of the CALPUFF validation study submitted by NJDEP, we conclude that NJDEP has not adequately justified the use of CALPUFF for this application under Section 3.2.2b of Appendix W, and that AERMOD is the most appropriate model for this application.

                                  Appendix B

                Assessment of Site-specific Meteorological Data

Aside from emissions data, meteorological data is the other key input to dispersion models.  The AERMOD modeling conducted by NJDEP was based on one year of site-specific meteorological data collected from a 100m instrumented tower and SODAR located about 2.2km west of the Portland Plant, as shown in Figure B.1, for the period July 1993 through June 1994.

Figure B.1.  Location of Portland Meteorological Tower

Section 8.3 of Appendix W provides guidance regarding meteorological data for use in dispersion modeling to demonstrate compliance with the NAAQS.  A key issue related to meteorological data is the representativeness of the data for the particular application, including spatial and temporal representativeness.  Based on a review of the data, we believe that the Portland Plant meteorological data from 1993-94 meets the basic criteria for representativeness under Section 8.3.3 of Appendix W, and therefore can be considered as site-specific data for purposes of modeling impacts from the elevated stacks for PGS units 1 and 2.  The 1993-94 data also meets the minimum criterion for length of meteorological data record of at least one year of site-specific meteorological data recommended in Section <IP_ADDRESS> of Appendix W.  However, the difference of about 100 meters in the base elevation for the meteorological tower vs. the stack base elevation (see Figure B.1) raises concerns regarding how the meteorological data were input to the AERMOD model in the NJDEP modeling anlaysis, especially given that the stack heights for units 1 and 2 are about 122 meters and that plume heights of concern for units 1 and 2 are about 200 to 400 meters above stack base.  The AERMOD modeling submitted by NJDEP used the measurement heights above local ground at the tower location for the meteorological data input to the model, effectively assuming that the measured profiles of wind, temperature and turbulence are "terrain-following."

Section 3.3 of EPA's meteorological monitoring guidance document provides the following discussion regarding siting of meteorological towers in complex terrain settings:

      Vertical gradients and/or discontinuities in the vertical profiles of meteorological variables are often significant in complex terrain. Consequently, measurements of the meteorological variables affecting transport and dispersion of a plume (wind direction, wind speed, and σθ) should be made at multiple levels in order to ensure that data used for modeling are representative of conditions at plume level. The ideal arrangement in complex terrain involves siting a tall tower between the source and the terrain feature of concern. The tower should be tall enough to provide measurements at plume level. Other terrain in the area should not significantly affect plume transport in a different manner than that measured by the tower.  Since there are not many situations where this ideal can be achieved, a siting decision in complex terrain will almost always be a compromise. Monitoring options in complex terrain range from a single tall tower to multiple tall towers supplemented by data from one or more remote sensing platforms. Other components of the siting decision include determining tower locations, deciding whether or not a tower should be sited on a nearby terrain feature, and determining levels (heights) for monitoring. Careful planning is essential in any siting decision. Since each complex terrain situation has unique features to consider, no specific recommendations can be given to cover all cases. However, the siting process should be essentially the same in all complex terrain situations. Recommended steps in the siting process are as follows (emphasis added):

         ::	Define the variables that are needed for a particular application.
         ::	Develop as much information as possible to define what terrain influences are likely to be important. This should include examination of topographic maps of the area with terrain above physical stack height outlined. Preliminary estimates of plume rise should be made to determine a range of expected plume heights. If any site specific meteorological data are available, they should be analyzed to see what can be learned about the specific terrain effects on air flow patterns. An evaluation by a meteorologist based on a site visit would also be desirable.
         Examine alternative measurement locations and techniques for required variables. Advantages and disadvantages of each technique/location should be considered, utilizing as a starting point the discussions presented above and elsewhere in this document.
         ::	Optimize network design by balancing advantages and disadvantages. It is particularly important in complex terrain to consider the end use of each variable separately. Guidance and concerns specific to the measurement of wind speed, wind direction, and temperature difference in complex terrain are discussed in the following sections.

Section 3.6 of the monitoring guidance states that "[s]pecific recommendations applicable to siting and exposure of meteorological instruments in complex terrain are not possible."

Given the vertical variability of wind directions in the Portland area documented in Exhibit 11 submitted with NJDEP's May 12, 2010 petition, a key component of the modeling analysis is the representativeness of the site-specific winds for transport and dispersion of the Portland emissions.  The terrain relief within about 3 kilometers of the Portland Plant ranges from about 90 meters above mean sea-level (MSL) along the Delaware River up to about 200 meters MSL, indicating that the meteorological tower is representative of exposure for the hilltops surrounding the plant.  Terrain elevations begin to increase beyond about 4 kilometers northwest of the plant, with a sudden rise in elevation to about 470 meters MSL associated with the Kittatinny Ridge that straddles the Delaware Water Gap, about 7 kilometers northwest of the plant.

Due to the terrain channeling effects that would affect wind flow along the Delaware River, the lower level winds on the Portland tower are not likely to be representative of the winds at those same levels relative to the stack base.  The local valley is relatively narrow in the vicinity of the Portland Plant with a distance of about 1 kilometer or less between the local peaks on either side of the river.

This valley channeling effect is clearly shown in the meteorological data collected near the Martins Creek plant, about 11 kilometers southwest of Portland, at a base elevation of about 100 meters MSL, as depicted in Figure 6 of Exhibit 11 to NJDEP's May 12, 2010 petition for the 10m AMS-4 wind data, shown below as Figure B.2.  The locations of the AMS-4 and other meteorological towers are shown in Figure B.3.  The AMS-4 winds show a strong bimodal distribution along a NE-SW axis, consistent with the valley orientation at that location.  The Delaware River valley is much broader at the AMS-4 location than at the Portland Plant, and is bounded by much higher terrain on Scotts Mountain toward the southeast, extending up to about 370 meters MSL, or about 300 meters above the river.  This is reflected in the fact that the 150m wind rose from the Martins Creek SODAR (Figure 7 in Exhibit 11 to NJDEP's May 12, 2010 petition, shown below as Figure B.4) still shows a pronounced NE-SW bimodal distribution indicative of valley channeling.

Figures 8 and 9 from Exhibit 11 of NJDEP's May 12, 2010 petition (shown below as Figures B.5 and B.6) compare wind roses from the 10m level of the AMS-8 tower and the 300m level from the SODAR data. The 300m SODAR data are at a level that is above any nearby terrain and are more indicative of synoptic flow patterns for the region.  The AMS-8 tower is at a base elevation of about 250 meters MSL about 5 kilometers northwest of the AMS-4/SODAR location, further from the higher terrain located on Scotts Mountain southeast of Martins Creek.  Even though the 300m SODAR level is about 150m higher than the AMS-8 10m level relative to sea level, the wind roses show a very similar pattern.  These comparisons serve to support our assessment that the lower level wind data from the Portland data are not representative of wind flow at those measurement heights at the PGS stack locations within the valley.

Figure B.2.  Wind rose for 10m AMS-4 tower located near Martins Creek Plant, taken from Exhibit 11 of NJDEP May 12, 2010 petition.

Figure B.3.  Location of SO2 sources, monitors and meteorological stations near the Portland Plant (based on Figure 1 from Exhibit 12 to NJDEP's May 12, 2010 petition).

Figure B.4.  Wind rose for 150m SODAR data located near Martins Creek Plant, taken from Exhibit 11 of NJDEP's May 12, 2010 petition.

Figure B.5.  Wind rose for 300m SODAR data located near Martins Creek Plant, taken from Exhibit 11 of NJDEP's May 12, 2010 petition.

Figure B.6.  Wind rose for 10m AMS-8 tower data located near Martins Creek Plant, taken from Exhibit 11 of NJDEP's May 12, 2010 petition.

Another comparison worth noting in NJDEP's Exhibit 11 is for wind speeds measured at the 10m level on the AMS-4 and AMS-8 towers and the wind speeds measured at comparable levels above MSL from the Martins Creek SODAR.  The average wind speed for the AMS-4 10m level located within the valley was about 2.3 m/s, compared to an average wind speed of about 3.6 m/s from the 10m level on the AMS-8 tower.  This is not surprising given the higher elevation and more open exposure relative to nearby terrain for the AMS-8 tower.  On the other hand, the average wind speed from the 150m SODAR data, at approximately the same elevation above MSL as the AMS-8 tower, was about 4.6 m/s.  This is also not surprising since the winds measured at the AMS-8 tower will be more strongly influenced by surface drag and therefore should be lower than the 150m SODAR winds which are much less influenced by surface drag.  The AMS-8 average wind speed is much closer to the 90m SODAR average speed of about 3.8 m/s.  This comparison serves to highlight our main concern regarding the meteorological data used by NJDEP in their AERMOD modeling analyses, that the Portland wind data based on measurement heights above local ground at the tower location are not representative of winds at those heights relative to stack base for Portland units 1 and 2.

Without adjusting for the difference in base elevation of about 100m between the meteorological data and the stacks, wind speeds are likely to be biased high and the wind directions may not be representative of plume heights.  Table 1 of NJDEP's analysis of complex winds in the region surrounding the Portland plant (shown below as Table B.1) shows an example illustrating this point, where wind direction varies with height indicating a transition from westerly flow below about 210 meters above ground to northwesterly flow between 210 and 270 meters, and then to northeasterly flow at 300 meters and above for hour 11 on July 18, 1993.  A similar pattern occurs for hour 12 on the same day, but the transitions occur at somewhat lower heights above ground.  Accounting for the differences in base elevation between the met tower and Portland stacks, this transition in wind directions occurs within the range of plume heights of concern for units 1 and 2.

Based on the concerns documented above, we have concluded that some adjustments to the measurement heights for the Portland site-specific meteorological data are appropriate in order to address these issues of representativeness for this application.  It is clear from some of the comparisons highlighted above that simply adding the difference in base elevation to all of the meteorological measurement heights would not be appropriate since it would ignore the effects of surface drag on the wind speeds at the lower measurement levels.  However, we believe that an adjustment based on the 100 meter height difference is reasonable and appropriate above a certain measurement level.  Given that the local terrain relief in the vicinity of the Portland Plant and meteorological tower is about 100 meters between valley floor and hilltop, and assuming that local terrain effects on flow would extend up to about 2.5 to 3 times the height of the "obstacles" based on a general analogy with building downwash influences, it is reasonable to apply the simple adjustment based on the 100m difference in base elevations to measurement heights at or above 250 meters.  It is also reasonable to assume that little or no adjustment should be applied to the lowest level winds at 10 and 30 meters due to the dominance of surface drag and other local influences.  A gradual transition in the height adjustment between these upper and lower bounds also seems reasonable.  Table B.2 summarizes the original and adjusted measurement heights for the Portland Plant meteorological data.

Table B.1.  Example of profile wind direction/wind speed collected at the Portland site (hours 11 and 12, July 18, 1993) (based on Table 1 of NJDEP Exhibit 11).

Table B.2.   Original and adjustment measurement heights for the Portland site-specific meteorological data
                              Original Height (m)
                              Adjusted Height (m)
                             Height Difference (m)
                        Average WS Difference (m/s)[a]
                                Tower or SODAR
                                                                             10
                                                                             10
                                                                              0
                                                                           0.33
                                     Tower
                                                                             30