Comment 154: Commenter 17857 recommends a subcategorization by fuel-type, along with the emission unit basis approach (instead of pollutant basis), for MACT standard setting. Such a method should address the EPA's objective for providing a meaningful standard that is also technically achievable.
Response to Comment 154: The EPA has addressed comments related to subcategorization by fuel type and establishing MACT standards on a pollutant-by-pollutant basis in response to other comments in the final rule record.
Comment 155: Commenter 18021 requests that the EPA take into account the other requirements that apply to the utility industry. These include maintaining generating capacity sufficient to meet demand through improving energy efficiency of its operating systems, which also contributes to minimizing emissions. Waste disposal practices will be impacted. Where we presently reuse materials, changes to meet new standards will make reuse impractical and incur a new cost not taken into account.
Response to Comment 155: The EPA established the MACT standards for EGUs consistent with statute. We question how we could legally consider the factors identified by the commenter in establishing standards, and the commenter has not explained their rationale for including such considerations, particularly in the MACT floor setting process. Commenter has also not explained how these factors would affect the performance of units in a manner we could quantify.
Comment 156: Commenter 18034 states that the EPA's use of multiple approaches to selecting data for the MACT floor analyses are inconsistent and arbitrary. Emission limits should not be based on single test runs. In some MACT floor analyses, the EPA uses the average of three test runs from the unit selected for the new unit MACT floor, and in other analyses, the EPA uses the lowest run of three test runs. In at least one case, the EPA MACT floor analyses spreadsheet indicates that the mean of multiple runs is used when in reality the cell formula for the mean uses the average of a single cell representing the lowest test run with no explanation given. The EPA should be using a consistent approach to selecting the data for establishing the MACT floors and provide a detailed explanation of their approach. Any deviation from the process must be justified and explained clearly so the public has the opportunity to comment. In addition, relying on isolated test runs to establish regulatory emission limits when additional data is available is questionable and technically unsound. The EPA should be making maximum use of the data available rather than arbitrarily excluding valid test data.
Response to Comment 156: The EPA has reviewed its analysis and confirmed consistency in the equations. We must establish the CAA section 112 emission limits based on the data that we have available to us; in this case, that is primarily manual testing consisting of sets of three runs. For the existing-source limits, we used the average of the three runs. For the new-source limits, to be able to be able to account for any variability, we used the individual emission test runs in developing the limits. We believe that we are using a consistent approach in our analysis of the data and making maximum use of the data available.
Comment 157: Commenter 18039 is very supportive of the draft regulations overall, but notes that some elements of the MACT assessment could be improved upon which would likely lead to more stringent limits that are both achievable and would result in greater reductions of emissions. Towards this end, Commenter encourages the EPA to carefully reanalyze its development of the MACT standards to more accurately reflect the actual observed emissions of the best performing 12 percent of facilities.
Commenter supports the EPA addressing additional HAP beyond the two pollutants (nickel and Hg) addressed in the vacated CAMR. Other metals such as arsenic, and other toxics such as HCl, are also emitted from utility boilers. The EPA's inclusion of multiple HAP in the proposed rule is appropriate as it will be more cost effective for power plants to address toxic emissions together rather than piecemeal.
Response to Comment 157: The EPA agrees that the regulation in the final rule should address all HAP from EGUs. We maintain that we have carefully considered the available data and established standards consistent with the statute.
Comment 158: Commenter 18426 has completed three 112(g) case-by-case MACT determinations for solid fuel-fired EGUs, and one major NSR determination on an existing coal-fired power plant with four solid fuel-fired EGUs. The general approach taken for the 112(g) determination was to obtain information on units permitted since 1999 using the assumption that newer units would be the best performing. A summary of the information, which includes data from permits, permit reviews, and compliance testing for comparison to the MACT emission limit standards assessed for the proposed rule, was included in the commenter's submittal. The existing units as permitted could have problems meeting the proposed standards in subpart UUUUU for HCI and Hg. For Hg, since the units do fuel switch and are designed for a higher heat input coal, Hg-specific control could be added to meet the standard.
A new unit, as designed and permitted, will not be able to meet the proposed standards in subpart UUUUU with the possible exception of Hg since the unit will likely be designed for a lower heat input coal, and organic HAP, since no standard is proposed. The standards for new units appear unachievable with the control options available now, and in the foreseeable future. And, conversely the standards for existing units appear too generous for those pollutants whose emissions can be accurately predicted if the unit is well controlled.
Response to Comment 158: We are unclear whether the commenter's point concerning the EGUs with CAA section 112(g) MACT limits is that they will be unable to meet the existing or the new-source MACT limits. Some CAA section 112(g) sources will be existing sources under the final rule and since the commenter states that the existing source standards are reasonable we assume the commenter is referring to the ability of the EGUs to comply with the new source limit. We respond to another comment in which we explain how EGUs (including those that are new sources under the final rule) with valid CAA section 112(g) limits may be provided additional time to comply with the final rule consistent with EPA regulations. Also, EPA does not agree that the new source standards in the final rule are not achievable and we have identified at least one existing source that is currently complying with all of the final new source limits.
Comment 159: Commenter 17904 states the EPA's proposed MACT floor standards are not achievable. To satisfy this requirement, the standards proposed by the EPA for existing sources must, at the very least, have been "achieved by the best performing 12 percent" of sources. The EPA has proposed limits that do not meet this standard. Additionally, the EPA's calculation of the MACT floor for Hg for lignite boilers contains errors. Finally, the EPA should properly account for variability in order to ensure sources are able to achieve the proposed standards.
Response to Comment 159: We believe that the emission limits in the final rule are achievable. The EPA believes it has adequately supported the beyond-the-floor Hg limit for low rank virgin coal units as explained in the proposed rule, supporting TSDs, and this final action. We address the compliance deadline issues elsewhere in this document. 4A02 - MACT Floor Methodology: Statistical Analysis
Commenters: 16122, 17383, 17402, 17620, 17621, 17627, 17638, 17648, 17689, 17712, 17716, 17725, 17728, 17729, 17730, 17739, 17775, 17796, 17800, 17817, 17820, 17821, 17843, 17845, 17846, 17877, 17878, 17885, 17886, 17898, 17902, 17912, 17929, 17975, 18014, 18021, 18039, 18421, 18447, 18449, 18487, 18498, 18644, 8443, 19536/19537/19538, 18023
Comment 1: Commenter 17739 states that the UPL calculation for Hg and HCl used the mean calculated from these minimum values, but used additional data from Part II and Part III to calculate the variability and, therefore, the mean and variance belong to two different and distinct data populations, and cannot be used together in the UPL calculation. The EPA should use the mean of all data and the variance of all data for the UPL calculation to be legitimately employed.
Response to Comment 1: When calculating a cut-off or prediction value to evaluate compliance of future observations using statistical approaches one must assume that future observations come from the same distribution as current and past data used in the development of the cut-off value. This same assumption was used to estimate the parameters that determine the UPL. More data usually results in better and more reliable estimates of a parameter (e.g., average or standard deviation (variability estimator)). The use of Part II and Part III data available from the top 12% performers to estimate the variability used in the UPL equation follows this approach. The Part II and Part III data used in the calculations of the UPL are data from the same sources that are in the top 12% of performers.
Comment 2: Commenter 16122 states that the EPA didn't provide in the ruling any reason for considering the 99 percent UPL for the MACT floor calculations. The commenter mentions the lack of explanation regarding selecting the significance level (why 99% UPL over the 97.5% UPL (as it did for the CAMR) or the 50% UPL). The commenter mentions that the selection of the confidence level is relevant given that an increase in the confidence level will dramatically increase the MACT floor.
Comment 3: Commenter 17620 states that the EPA does not clarify the selection of the confidence level (e.g., 99[th] against 90[th] confidence level). The commenter adds that the EPA has used the 90[th] percentile in earlier Hg rulemaking and in setting the procedure for compliance by means of fuel sampling. Lower confidence levels, the commenter adds, were also used for complying options for reduced monitoring. The commenter says that the selection of the confidence level will have an impact on public health and the environment.
Response to Comments 2 - 3: The level of confidence represents the level of protection afforded to sources whose emissions are in line with the best performers, and hence it is not arbitrary. A 99% level of confidence means that a source whose emissions are in line with the best performers has 1 chance in 100 of exceeding the floor. Several confidence limits (e.g., 99, 99.9, and 95%) have been used in past rulemakings. The use of 99.9% to estimate individual units' variability was questioned in a past rulemakings, arguing that such a high confidence level was a departure from the EPA's approach in other rulemakings, which used lower confidence levels, such as 99, 95, or 90%. It was suggested that using 99.9% overcompensates for variability.
Later rulemakings (Portland Cement, HMIWI, and Boiler MACT, except for CO) have used 99%. Based on analyses at various percentiles, the 99% values are more in line with the highest test runs for the MACT floor units than the other percentiles, seldom falling below (like the 90 and 95%) but also not substantially exceeding (like the 99.9%). This finding suggested that the 99% provides a more reasonable compensation for variability than the other percentiles, resulting in standards more representative of the level of emission reduction that sources are actually achieving on a daily basis.
In the case of the Boiler MACT limit for CO 99.9% was used. The reason is that CO emissions appear to exhibit much higher variability than emissions of other pollutants. Choosing 99.9% appeared to provide a more reasonable compensation for variability than 99%. This demonstrates that you have to always look at the actual data and the resulting limits and determine if they appear to capture variability adequately. We believe that we have done that in this case and that the 99% UPL is appropriate.
Comment 4: Commenter 16122 states that the EPA's proposed MACT floor overestimates the variability of the top performers, and that the proposed variability is really inter-source variability. Commenter states that in the preamble the EPA stated that the MACT floors were calculated considering the operational variability of the top 12% for each subcategory. Furthermore, the commenter claims that the variability considered in the equation involves more than operational variability. The commenter adds that the range of emission levels achieved by different EGUs are not similar to the day-to-day variability within an individual EGUs and for this reason the UPL overestimates the emission. The inter-variability, the commenter adds, summarizes the wide differences between the individual EGUs and not operational variability within an individual EGU.
Comment 5: Commenter 16122 states that according to the courts, the EPA can include variability when calculating MACT floors. However, the commenter adds, nowhere have courts found that EPA can account for inter-source variability other than averaging the emissions achieved by the top 12% performers. Moreover, the range of emission levels between sources with different designs, operations, and control technologies is likely much greater than the range of emission levels experienced within a source. The wide range of emission levels between different EGUs unjustifiably inflates the UPL because this range is largely caused by differences in unit design and available control technologies, not variability. Thus EPA's methodology violates the CAA by calculating a MACT floor that incorrectly accounts for differences between sources and bears no relationship to the average emission levels actually achieved by units in the top 12% performers.
Comment 6: Commenter 17402 considers that the EPA's approach is consistent with the case law interpreting section 112 and recommends that other sources of variability should be considered. Commenter recommends incorporating boiler variability and associated effects on emission controls. Boiler variability is not incorporated because test data used in the analysis was conducted at normal, full-load conditions.
Comment 7: Commenter 17402 states that the D.C. Circuit's interpretation of the CAA requires the EPA to adjust the MACT floor for variability. According to the D.C. Circuit interpretation of sections 112(d)(2) and (d)(3) the MACT floor has to be achievable by the best performing sources "under most adverse circumstances which can reasonably be expected to recur." The commenter adds that when developing MACT floors, the EPA must consider the factors that cause variability to ensure that the emissions limitations have indeed been achieved in practice by best performing units. Among the factors that may cause HAP emissions variability are operational variability of boiler and pollution control equipment, fuel variability, and test method variability. The commenter concludes that the EPA has a statutory mandate to base the MACT floor not on raw test data alone, but to take into account the reasonably anticipated worst-case emissions from the best performing sources.
Comment 8: Commenter 17402 agrees with the EPA in that variability is an element of facility performance and that it is reasonable to assess emissions of the best performers over a period of time in order to obtain a more accurate measurement of achieved emission levels. The commenter adds that HAP emissions data from best performers are based on the average of stack tests taken under uniform, steady-state and generally ideal operating conditions, however in reality this is not the case. The commenter agrees with the EPA adjusting the MACT floors for variability, because otherwise even the best performers could exceed the floor emission levels over a longer period of time when compliance is required.
Comment 9: Commenters 17739 and 18014 state that data used to establish the MACT floor only represents a snapshot in time and do not represent what a source achieves on a continuous basis. The commenters add that since the data used to calculate the UPL does not capture all operating conditions, all fuel conditions and other sources of variability, the results of the UPL are not valid because it does not ensure that the average performance of the top 12% of sources in the MACT floor would achieve the emission standards every day under all operating conditions. The commenter cites the Boiler MACT floor calculations where the EPA used a fuel variability factor.
Comment 10: Commenter 17775 states that the EPA did not make any attempt to assess the long-term variability in EGU MACT emissions. The commenter adds that the 2 or 3 days of stack sampling available were conducted under ideal plant conditions and do not reflect the long-term variability observed in the facility. The commenter adds that for coal fired EGUs, HAP emissions vary greatly because of large variations in trace element concentrations in the coal, even from coals mined from the same seam.
Comment 11: Commenter 17775 states that when the available data fails to capture long-term emissions variability attributable to varying concentrations of trace elements in coal, then the UPL would not be available to account for emissions variability. The commenter states that CEMS data from several performing units cannot comply with the MACT floor all of the time. This is the result, the commenter says, for not including variability accounting for factors such as fuel variability. The commenter criticizes the use of the UPL for new sources, when only three data points were used to calculate the MACT floor, resulting in a small variability adjustment for a single new unit compared to that used for existing units.
Comment 12: Commenter 8443 states that according with court resolutions (D.C. Circuit in National Lime Ass'n v. EPA) the standard floors must be achievable under most adverse circumstances which can reasonably be expected to occur. The commenter adds that to ensure that the emission limits is set at a level that the best performing sources will not violate, the EPA must assess the variability in the best performers and incorporate them in the calculations of the limit. For example, the commenter adds, the EPA must include the variability of control equipment as well as non-technology factors that may influence the best performing units. It is an incorrect statement that no sources among the best performers would ever exceed the floor limit.
Comment 13: Commenter 18023 states that when calculating the emission limits for the HAP the EPA uses short term stack testing data, which was collected under ideal steady-state plant operating conditions using a particular fuel, and do not account for the full range of emission control, load, and fuel conditions experience during normal conditions. The commenter adds that the EPA claims to have accounted for emissions variability when setting the HAP emission limits. The commenter cites the D.C. Circuit Sierra Club v. EPA resolution that the standard must be achievable under the most adverse conditions, and that if the standard is as stringent as the emissions control that is achieved in practice by a particular unit, then that particular unit will not violate the standard.
Comment 14: Commenter 17621 states that because the ICR did not require units to test over the full range of operating conditions, these data do not capture all range of emissions variability. According to the commenter, new measurements are needed to characterize the variability of HAP and surrogate emissions during normal plant operations. The commenter adds that other sources of variability not included are fuels burned, startup and shutdown conditions, partial load operation, and other reasonably foreseeable changes to operating conditions.
Comment 15: Commenter 18023 states that the UPL for new sources have some flaws: the use of two or three data points from a single plant and the fact that those data points do not capture the emission variability that a new plant would endure. The commenter adds that the EPA must modify the approach to make sure that the best performing units would not exceed the standard.
Comment 16: Commenter 17621 states that another issue affecting the UPL is the capture of relevant sources of emissions variability. The commenter adds that if the data from best performers do not capture temporal variability, then it will not capture the variability present in a 30-day rolling average, which is proposed by the EPA for evaluating compliance. The commenter adds that the EPA needs to recognize the full range of within-unit variability and properly incorporate it to the UPL for compliance monitoring protocols.
Comment 17: Commenter 17725 agrees with the EPA's UPL-based floor analysis approach but requests that the EPA address fuel and operating variability issues, especially when considering small datasets (e.g., new unit limit)
Comment 18: According to Commenter 17725 the UPL is good in theory because the result depends on the assumption that the best performers data capture all forms of variability including measurement variability, boiler or emission control equipment related operating variability and fuel variability.
Comment 19: According to Commenter 17621, the new unit limits adopted by the EPA do not account for emissions variability due to changes in operating conditions that can be expected over the life of a unit. New unit limits should not be based on flue gas measurements that are below the detection limit, as those values do not account for measurement imprecision.
Comment 20: According to Commenters 17725 and 18014, the UPL needs to be adjusted to account for the snapshot nature of the ICR test results used in the analysis. Commenters add that given the control conditions under which the tests were taken, these test results do not reflect operation, control and fuel variability. These test results, the commenters add, only exhibit a fraction of the potential reference measurement variability because these variability (e.g., ambient conditions, stack tester factors, etc) are fixed during each test.
Comment 21: According to Commenter 17728, the EPA variability does not fully account for the HAP emissions variability of the best performers. Stack data do not provide meaningful insight that a unit can achieve over longer periods of time, different fuels, and varying loads and operating conditions. The commenter adds that Hg limits do not properly account for the variability in Hg emissions, and that CEMS data show that even well-controlled units have large variability in Hg emissions (daily or 30-day averages), and will not consistently meet the MACT floor limit under normal operating conditions.
Comment 22: Commenter 17729 states that the MACT floors do not represent the emissions achieved by actual best performing units and that the EPA is basing the floor on pollutant standards that only few sources can achieve. The commenter adds that the EPA's method does not account for all emissions variability caused by numerous operating factors.
Comment 23: Commenter 17739 states that even though the EPA acknowledges that the floor standard should address emissions variability, it ignores a key source of variability when using the lowest measurement instead of the average of measurements for each source. The commenter adds that the EPA has addressed fuel variability in other rulemakings such as the Boiler MACT, and that this effort was not made in this rulemaking. In the PM floor, the commenter notes, the EPA did not include more data in the variability calculation as was done for Hg and HCI. The commenter further notes that the EPA said that this non-inclusion of the data implicitly ignores temporal variability in the PM floor calculation. Finally, the commenter adds that regarding filterable particulate the EPA ignored some of the data in the MACT spreadsheet and did not mention that the total particulate values represent actual simultaneous emissions for the filterable and condensable components.
Comment 24: Commenter 17775 states that the data used by the EPA, 1-hour tests of full load operations, are not representative of the emissions achieved by the best performers under all operating conditions. According to the commenter, this denotes an incorrect perception of performance by the EPA because coal-based power plants are complex systems that do not function under steady state conditions. The commenter adds that it is unreasonable to establish emission standards based on few hours and expect to achieve this value over many years of varying conditions.
Comment 25: Commenters 17817 and 17912 state that the EPA makes several incorrect assumptions that result in MACT limits that are not attainable even by the top performers each day under all operating conditions. The commenters state that three separate runs are inadequate to establish the MACT floor or to establish a permit condition, and cannot capture day to day operating variability. The commenters add that fuel variability cannot be capture with such small data, in particular if unit train fuel deliveries come from a variety of mines.
Comment 26: According to commenter 17820, after recognizing the limitations of the short term ICR test data, the EPA uses the UPL to adjust for emissions variability. The commenter commends the adjustment of the average performance but claims that this approach does not fully account for all emissions variability because the underlying data do not reflect long-term emissions variability.
Comment 27: Several commenters (17877, 17886, 17902) state that the EPA's statistical analysis is reasonable; unfortunately the EPA lacks sufficient data to perform a meaningful UPL calculation because its data do not reflect all operating conditions. The EPA computes the within-source variability using short-term test that cannot capture the level performance of the units under the most adverse conditions. Furthermore, the commenter adds, the ICR test runs were performed hours apart under identical operating conditions. Then, the commenters add that the EPA wrongly assumes a normal distribution for the data, and the data do not account for variability of control equipment and operation conditions, nor the data accounts for load variability and upper ramp up/down conditions. The commenters conclude that if the EPA fails to obtain a complete picture of operating conditions of the best performing units over a sufficiently extended period of time, it cannot hope to develop an accurate assessment of the level of performance those plants will achieve under the worst reasonably foreseeable circumstances.
Comment 28: According to commenter 17878, the EPA's UPL analysis does not account for the full range of operational, fuel, and monitoring variability. The data used to establish the MACT floors only represents a snapshot in time and does not represent what a source actually achieved on a consistent basis. A separate variability issue is associated with the EPA's treatment of units sharing a common stack. In general, the EPA treated each unit as a separate data point, even though all units sharing a common stack by definition have the same measured emission rate.
Comment 29: Commenter 17716 states that the EPA recognizes that the short term data used for SO2 standards do not capture the vast operating conditions under which a facility can operates. It also does not take into account control equipment variability and degradations. The commenter adds that this consideration is not extended to other aspects of the EGU MACT standard. Because performance tests were conducted at normal, full load conditions rather than a variety of conditions, the extent of boiler variability and its effects are ignored in the proposed standard.
Response to Comments 4 - 29: The EPA understands the need to capture all sources of variability and incorporate them in the calculations of the floor standard. It is impossible sometimes to separate the different sources of variability given the data available. However, the EPA believes that the range of sources and the available data capture the necessary variability of well performing sources used to establish the MACT floor. The EPA considers that the data used to calculate the UPL covers all mentioned sources of variability. Sources of variability can be summarized as inter-source and between-source. Examples of between-source variability included in the data are different boiler designs, operations, and control technologies. Similarly, intra-source (also referred as within-source) variability captures the day-to-day variability, a both "process" variation and measurement-error variation. The commenter suggests the consideration of other sources of variability. However, separating out the different types of variability from the available data may not be a trivial exercise, and may require a different protocol for data collection. We maintain the approach we have is reasonable in light of the available data and the commenter has not provided an alternative approach for us to consider.
The EPA disagrees with any commenter's assertions that we have violated either the statutory language or court decisions related to the variability analyses. We interpret the court's decisions as allowing us to consider both intra- and inter-source variability as long as we confine the analyses to EGUs in the MACT floor pool (e.g., top performing 12 percent). The UPL approach is a confidence interval for an average of future tests, and it is based on the average of emissions of the top 12 percent performers. Because the HAP emissions might change from time-to-time (e.g., year-to-year), a variability term is added to the average to account for this "sampling variability" so that the proposed MACT floor can be considered valid through time. Large differences between the emissions from the top performers will not only affect the variance but the average itself. As a matter of fact, the average is mostly affected by the between-source variability. For this reason, the variance includes within- and between-source variability.
The EPA considers that it has included any boiler variability contained in the available data. We believe that consideration of inter-source variability and use of the 2010 ICR Part II data address boiler and emission control technology variability. The operating conditions under which emissions measurements are taken from a source will directly affect those measurements. For example, emissions obtained from a source that is operating poorly will almost certainly be higher than when operating properly. Therefore, a source that is operating poorly will be less likely to be included in the pool of best performers than if it were operating properly. Similarly, if a source is being tested for compliance, it would be less likely to demonstrate compliance if operating poorly than if operating properly. The commenters believe that floors must be set at the average emission level achieved by the best performers when they are operating under a worst-case scenario. This approach ignores the variability in the data, and, therefore, goes against established statistical theory and practice, and it also goes against the intention of the statute.
The EPA has calculated the MACT floor using all available data. The diversity of sources and range of the data suggests that the variability sources mentioned above are being considered when calculating the MACT floors. The use of MDL data is addressed elsewhere in this document. The EPA has included a confidence level to the calculation of the standard along with a 30-day average format; in this way, the EPA is also addressing the variability noted by commenters.
Comment 30: Commenter 17648 states that the calculations of the MACT floor are based on a significant pool of sources, accounting for variability in performance of the best performers that resulted in emissions limitations providing ample degree of protection from outliers. Commenter 17648 adds that the EPA took measures to ensure data quality and took measures to resolve issues of data reported under method detection limit (MDL).
Comment 31: Commenter 17739 states that the D.C. Circuit Court has held that the standards must be achievable under the most adverse conditions which can reasonably be expected to occur. The commenter adds that the DC Circuit has concluded that the EPA may consider emissions variability in the calculation and the standard, and may set the floor at a level that best-performing sources can expect to meet every day under all operating conditions.
Response to Comments 30 - 31: The EPA appreciates the commenters' support.
Comment 32: Commenter 16122 states that U.S. Government Accountability Office (GAO), states and even EGUs have made clear that because the EPA uses long-term standards (e.g., 30-day compliance period under the EGU MACT Rule), no variability analysis is needed (reference is October 2009 GAO report entitled "Mercury Control Technologies at Coal fired Power Plants..."). Commenter adds that EGUs concluded that this long-term averaging approach takes into account dramatic swings in Hg emissions from coal and allows them to meet stringent state Hg standards. The commenter further states that if the EPA is concerned that EGUs will have difficulty meeting the proposed 30-day standard then the EPA should lengthen the compliance time period (e.g., to one year).
Comment 33: Commenter 17845 states that several institutions (e.g., GAO) and states have concluded that no variability analysis is needed because when longer-term data are used (e.g., 30-day compliance period), the average of the data takes into account any variability in Hg emissions allowing units to meet stringent state Hg standards.
Comment 34: According to commenter 17846, the use of the UPL dramatically over-estimates the emissions achieved by the top 12 percent performers. The commenter adds that the proposed MACT floor standard allows Hg emissions that are 100 times the average of the top 12 percent performers for EGUs burning non-lignite coal and 11 times the average of the top 12 percent burning lignite coal. The commenter adds that the EPA must show that its variability analysis provides an accurate picture of the relevant sources performance. The commenter adds that there is no reason for incorporating variability analyses into the EGU MACT Rule because the EPA is already using a longer-term compliance standard. The commenter adds that if the EPA is truly concerned that EGUs will have difficulty meeting the 30-day standard it is proposing, the EPA should lengthen the compliance time period (e.g., to one year).
Response to Comment 32-34: Emissions data have different sources of variability; therefore, incorporating uncertainty analysis into the MACT floor accounts for sources of variation present in the data. Because a single number (e.g., average) may not capture the variability observed under different designs, operations and control technologies, a confidence type statistic like the UPL, along with a 30-day average format, is used to ensure that different sources of variability are considered and that the proposed MACT floor remains valid under different scenarios. As mentioned elsewhere, we believe that a 30-day boiler operating day period provides flexibility sufficient for sources to operate processes and control devices to assure ongoing compliance and that the 30-day period provides for the level of environmental protection intended for this rule. The 30-day rolling average requires that the operator review and act on measurement data on at least a daily basis consistent with the enforcement and compliance provisions of the CAA (e.g., CAA section 113(d)). A rolling 12-month average would reduce that frequency to once per month. Variability  -  including that attributed to load/operational variability, fuel variability, and other sources of variability  -  has been accounted for in the setting of the proposed and final emission limits. The method used to account for this variability was described in the preamble for the proposed rule. The method is consistent with methods used in other NESHAP rules that the EPA has promulgated.
Comment 35: Commenter 17402 states that the EPA's proposed rule using a statistical formula meets legal requirements set forth in D.C. Circuit case law. The commenter states that the current formula does not contain flaws of the process presented in Sierra II, Northeast Maryland Waste Disposal, or Cement Kiln, because it is based on emissions from the best performing units, and that the EPA has provided evidence that the statistical formula estimates variability for the best performers as required by statute.
Response to Comment 35: The EPA agrees with the commenter that the approach to variability in the final rule it consistent with the statute and applicable case law.
Comment 36: Commenters 17402 and 18014 state that in the preamble the EPA stated that it used the Bhaumik and Gibbons Central Limit Theorem (CLT) approach to calculate the UPL when the data were log-normal distributed. Commenter adds that from the Agency's floor analysis spreadsheets it is not clear whether and how the EPA actually used the Bhaumik and Gibbons approach. The commenter requests further clarification in the final rule. Commenter 17402 has observed that in the case of pretroleum coke sources the EPA seemed to have used the standard UPL approach instead of the Bhaumik and Gibbons approach. The commenter requests further clarification.
Commenter 17725 states that the EPA indicated that it had used the Bhaumik and Gibbons log-normal approach for this rule when the skewness and/or kurtosis tests failed. However, the commenter adds, not evidence was found in the floor spreadsheets. The commenter requests the EPA to revise and be consistent with the description provided.
Response to Comment 36: The commenter is correct regarding the use of the standard UPL approach for most of the calculations of the MACT floors. This was possible because under the Central Limit Theorem, for moderate to large sample size (>20) the distribution of the average of emissions is approximated normal, rendering valid estimates for upper prediction limit for the average of future measurements.
Comment 37: Commenter 17620 states that the current UPL calculations use a number of discretionary choices with respect to data management. Because of the impact of these choices in the rulemaking, the commenter adds that the EPA must ensure that these choices are rational and supported by available information. The commenter adds that the UPL calculations seem to produce reasonable results in some cases, and odd results that contradict available credible information in others. The commenter urges the EPA to resolve this issue by examination and analysis of available information.
Response to Comment 37: The EPA has carefully reviewed the data management process, and has reconciled any discrepancies.
Comment 38: Commenter 17621 states that the UPL is an upper limit for future values of the same nature (coming from the same distribution) as those used in the calculation of the UPL. The Central Limit theorem (CLT) provides a setting to calculate the UPL for a dataset regardless of the distribution of the data. Note that the CLT does not change the definition of UPL, therefore the UPL can be used to evaluate if one future value or an average of future values that come from the same distribution as those used in the calculation of the UPL.
Response to Comment 38: The UPL is an upper limit for future values of the same nature (coming from the same distribution) as those used in the calculation of the UPL. The Central Limit theorem (CLT) provides a setting to calculate the UPL for a dataset regardless of the distribution of the data. Note that the CLT does not change the definition of UPL, therefore the UPL can be used to evaluate if one future value or an average of future values that come from the same distribution as those used in the calculation of the UPL falls under a certain cutoff with some level of confidence.
Comment 39: Commenters 17716 and 18014 state that for the UPL to perform as expected the data should represent the larger population. The commenters add that if the full potential variability is included in the UPL then the MACT floor is a limit that well control units will meet in most situations. If the full variability is not incorporated, then even the most controlled units will meet the UPL by chance. One of the flaws of the EPA's approach is that it assumes a normal distribution. This assumption, the commenters add , does not agree with previous documents (e.g., the Industrial Boiler (IB) MACT Rule) and the EPA's own guidance (e.g., Data Quality Assessment: Statistical Methods for Practitioners EPA QA/G-9, Report EPA/240/B-06/003 (Feb. 2006)) where log-normality was assumed. The commenters add that because the sample of best performers is not a random sample then the central limit theorem is not applicable. The commenters add that the EPA said that skewness and kurtosis tests were performed to evaluate the distribution of the data, but that revising the spreadsheets provides no indication of these tests were found. The commenters recommend to perform these tests instead of relying on the central limit theorem and to use the log-normal approach (Bhaumik and Gibbons approach for log-normal data). The commenters add that the EPA's statement that it used the Bhaumik and Gibbons approach to test compliance for log-normal could not be verified. The commenters add that they studied the distribution of 131 nickel emissions used by the EPA in its floor analysis and concluded that the data are log-normal. The commenters conclude that the EPA should keep performing the skewness and kurtosis tests, and revise its analysis to include the approach for log-normal data.
Commenter 17820 states that the EPA assumed a normal distribution of the ICR data and used an arithmetic mean instead of a geometric mean in determining emission factors (and emission rates) for units that were not sampled. This significant technical issue raises serious question about whether HAP emissions from coal-fired power plants pose risks.
Response to Comment 39: The UPL is the value below which a single test observation (or average of several test observations) from the source-to-be-tested is expected to fall with a stated level of confidence (e.g., 99 percent). By law the MACT floor limit should be based on the best performers; therefore the collection of best performers is not a representative sample of the population of sources. The calculation of the UPL does not require a random sample of elements of a population; it requires only a sample of observations. The EPA believes that all sources of variability affecting the emissions are captured in the data used to calculate the MACT floor since those data comprise a wide array of sources using different fuels. The EPA agrees with the commenter that the skewness and kurtosis testing should be performed in all excel spreadsheets for evaluating the distribution assumptions of the data. However, the EPA clarifies to the commenter that when the sample size is large, the average of the emission factors from the best performers and not the emission factors per se, follow the normal distribution. The Central Limit allows the calculation of the UPL assuming a normal distribution because the sample size used to calculate the average is large enough, this is valid regardless of the parent distribution of the data. This property of the Central Limit Theorem for moderate to large sample size allows the application of the standard UPL without the need to check for the distribution of the data.
Comment 40: Commenters 17716 and 18014 state that the assumption of the Central Limit Theorem is to calculate the UPL does not hold. Commenters provide the example of the 131 nickel emission test results which are not normally distributed. The commenters add that if the EPA had used the approach applied in the IB MACT which tested the skewness, the EPA would have concluded that the data are log-normal distributed and not normal.
Response to Comment 40: When the data used to calculate the UPL has moderate to large size (n>15) the EPA has used the Central Limit Theorem to support the use of the standard form of the UPL to calculate the MACT floor. By doing this, the EPA is not concluding that the data are normal; the conclusion is that the average of the data has a distribution which is approximately normal, allowing the use of the equation of standard UPL.
Comment 41: Commenter 17775 states that UPL calculations assume that the stack data test results are independently and randomly distributed, which is not the case for emissions data. The commenter adds that they examined long-term CEMS Hg and PM data from the best performers to evaluate if temporal correlations and infrequent operating conditions significantly affect the emissions levels. This examination showed that long-term average emissions are higher than one would expect based on stack test alone. The commenter recommends to use ratios calculated as the 99[th] percentile historic 30-day averages and the stack data to adjust the UPL for temporal correlation and infrequent events.
Response to Comment 41: The EPA understands that emissions data are usually non-normally distributed; for this reason when applying the UPL, the EPA is not relying on the normal distribution of the data, but in the distribution of the average. This is possible based on the Central Limit Theorem, which ensures that for a sample of moderate and large size, the distribution of the average tends to normal regardless of the parent distribution of the data.
Comment 42: Commenter 18023 states that an additional flaw in the EPA's approach is the assumption of the normal distribution. The commenter adds that in the Boiler MACT ruling the EPA suggested that environmental data tend to be log-normally distributed. The commenter adds that the EPA should determine the distribution of the data prior to the calculation of the UPL. The commenter adds that the assumption of normality results in underestimated variability for the top performing units. The commenter adds that they analyzed the data and concluded that the short-term stack data are not independent or normally distributed; as a result erroneously low emission limits were obtained. The selection of the data, the commenter adds, was also biased since the EPA selected the lowest value when several data points were available for a given source, resulting in low emission limits. The EPA should first determine the properties of the underlying distributions before computing UPLs.
Response to Comment 42: It is correct that environmental data tend to have a skewed distribution that follows distributions such as the log-normal or the Weibull distribution. The EPA also agrees that testing the distribution of the data is a procedure that needs to be included to validate the assumptions of the statistic approaches use in any analysis. The EPA did not assume that the data were normally distributed. The EPA used the Central Limit Theorem to justify the assumption of normality for the average of emission values for medium to large sample size. Data from different sources are assumed independent, and data from the same source but different time points will be independent because there shouldn't be correlation between data collected days or months apart. We selected the lowest emissions data for each EGU so as to be able to determine the best performing sources; additional data from EGUs among the MACT floor pool (i.e., best performing 12 percent) were then used in the variability analyses.
Comment 43: Commenter 18023 states that short-term stack data do not represent long-term emissions variability. Using CEMS HAP data, commenter created synthetic stack test data and proceed to calculate the 99[th] UPL and the 99 percentile assuming normal distribution of the data. Commenter concluded that emissions distributions have extended periods of relatively high emissions, which do not follow a normal distribution. The commenter adds that they compared the UPL with the 99[th] percentile which were actually larger than the UPL. The commenter reminds the EPA that directions provided by the D.C. Circuit requires that limits should be achievable by the very units used to develop those limits. The current data, the commenter adds, may represent the variability of the data collected and not the variability of the performing sources.
Commenter 17775 examined Hg and PM CEMS data from top performing units to determine historical long-term emission averages, and to assess whether temporal correlations and infrequent operating conditions significantly affect emission limits attainable by these units. To evaluate the performance of the UPL with the CEMS data, the commenter created synthetic stack data and proceeded to calculate synthetic-based UPL and the ratio of 99[th] percentile historical 30-day emission averages to synthetic data-based t-statistics UPLs. The commenter found that those ratios were 3.89 for the PM CEMS data which included startup and averaged 1.80 for Hg CEMS data. The commenter recommends to apply these ratios to the UPL to compensate for correlated emissions and emissions from infrequent events.
Response to Comment 43: The EPA agrees that environmental data tend to have a skewed distribution that follows distributions such as the log-normal or the Weibull distribution. The EPA did not assume that the data were normally distributed. The EPA used the Central Limit Theorem to justify the assumption of normality for the average of emission values for medium to large sample size. The UPL and the 99[th] percentile are not directly comparable in the sense that one would not expect the UPL to contain the 99[th] percentile. In most datasets, in particular in large datasets, one would expect the 99[th] percentile to be larger than the UPL. Only data from the MACT floor pool were used in the variability analyses; therefore, the data represent the variability of the best performers.
Comment 44: Commenter 17338 states that the EPA makes erroneous assumptions in unit emissions variability that led to MACT limits likely not attainable by the top performing units during every day operation and under all operating conditions. Issues mentioned by the commenter are: use of the average of three separate runs to determine MACT UPL, assume that the sample mean has a normal distribution which is not supported by the data, not incorporating efficiencies of emissions control equipment over expected and varied operation conditions. The commenter adds that the EPA must develop MACT limits that meet every day and all operating conditions of the top performing units. This, the commenter adds, will require more data and better statistical assumptions.
Comment 45: According to several commenters (17689, 17712, 17885, 18014), the EPA's assumptions about the emissions variability resulted in MACT limits likely not attainable by the top performing units during every day and under all operating conditions. Commenters add that the assumptions questioned are: the UPL is based on only three separate runs, the assumption of normality, and not accounting for the variability due to emissions control equipment over expected and varied operation conditions, whether at a constant but unusual load or under ramp up or down conditions. Commenters add that the EPA must develop MACT limits that meet "every day and all operating conditions" of the top performing units. The commenters suggest that the EPA must develop MACTs that utilize more data and better statistical assumptions.
Comment 46: Commenter 17800 states that the ICR data are based on three runs varying in time from 1 to 8 hrs. These tests are a snapshot of actual emissions and do not capture different sources of variability such as fuel and operational variability, operation of boiler and balance of plant requirements. The commenter adds that the UPL equation assumes a normal distribution of the data and this assumption cannot be validated. The commenter further adds that the EPA should retract the regulation and better substantiate the method with longer term data.
Response to Comments 44 - 46: The average of three emission test runs is typically used as the basis for determining compliance with an emission limit. The EPA considers that data utilized for calculating the UPL combines a wide range of units that incorporate all sources of variability. As noted elsewhere, we believe we have adequately incorporated variability in the analyses. The EPA did not assume that the data were normally distributed. The EPA used the Central Limit Theorem to justify the assumption of normality for the average of emission values for medium to large sample size. This assumption is valid regardless of the parent distribution of the data.
Comment 47: According to commenter 17621 the UPL does not provide an estimate of the 99[th] percentile, which commenter asserts is the main purpose of the UPL. The commenter adds that the use of the Central Limit Theorem (CLT) is inappropriate, because of the small sample size of future tests (m=1 or m=3) required to assess compliance. Because of the small sample, the commenter adds, the data are required to have a normal distribution, which is not true when dealing with emissions data. Furthermore, the commenter adds, the tests utilize to assess normality are not valid and have no enough power when the sample size is very small. According to the commenter, the UPL is irrelevant to the exceedance rate, regardless of the sample size used to calculate it.
Comment 48: According to commenter 18487, the EPA does not explain the claimed variation is captured in the 99[th] confidence UPL. The EPA also does not justify the assumption of normal distribution.
Response to Comments 47 - 48: The commenter is correct that the UPL does not provide an estimate for the 99[th] percentile because this is not the objective when using the UPL. The UPL is a confidence interval for an average of future tests, and does not try to provide an upper bound for a given percent of the sample or try to estimate the 99[th] population percentile as suggested by the commenter. One key assumption is that the future observation being predicted will come from the same distribution (e.g., will be in line with) of the best performers. The next sentences briefly describe the basis of the proposed UPL approach. A prediction interval for future value incorporates the variability around the process (e.g., fuel, controls, etc) and uncertainty about the population average. This population average is unknown. An average of future measures will also incorporate the same sources of variability. A future observation or the average of future observations can be considered as a random deviation of the average, ie. Y_future = mean + random_error. In order to produce a prediction interval for the Y_future, an estimate for Y_future is needed. The best estimate for Y_future is the average of the current values (e.g., average of the best performers). The variance of Y_future is equal to sigma^2*(1/n + 1/m) where n is the number of terms used to calculate the average, and m is the number of future observations. This information in conjunction with the Central Limit Theorem is used to calculate the upper prediction limit for the future observation or average of future observations. The above reasoning shows that the validity of the UPL depends on the sample size of the baseline sample (i.e., the number of best performers) and not on the number of future observations. The power of the test will be affected by the sample size or number of best performer data (the larger the sample size the larger the power), significance level and the assumption that the future observations will be in line with the best performers. The sample size being used to calculate the UPL is moderate to large. Finally, the exceedance rate mentioned by the commenter is related with the significance level, in this case associated with the UPL. The interpretation of the confidence level in a prediction interval is quite different from the interpretation of confidence level in a confidence interval. In a prediction interval, the confidence level represents the probability that the future observation is below the UPL, therefore it is relevant to the exceedance rate. It is granted that when the sample size is very small, then the confidence level underestimate the intended 99%.
Regarding normality distribution, see response below.
Comment 49: Commenter 17725 states that the EPA needs to address the flaws in the UPL calculation and resolve the variability issues. The first flaw mentioned by the commenter is the assumption of normality for all datasets with 15 or more tests. The commenter reminds the EPA that in its own guidance the EPA suggests that environmental data are log-normally distributed. Furthermore, the commenter adds the application of the central limit theorem is not correct given that the data are not independent. The commenter requests the EPA to include in the floor analysis spreadsheets tests for the kurtosis and skewness to better conclude about the distribution of the data. The commenter adds that in preamble the EPA said that those tests were performed but there were not found in the spreadsheets.
Comment 50: Commenters 17820 and 18014 state that the UPL approach assumes normally distributed data, and that is not the case for emissions data. The commenters add that no evidence that the ICR data were evaluated for normality. The commenters suggest that in the absence of normality, the EPA should consider other approaches such as the ones used in IB MACT ruling.
Response to Comments 49 - 50: In this rulemaking, the EPA is not assuming that environmental data are normal; by using the Central Limit Theorem, the EPA is concluding that the Mean statistic converges to a normal distribution regardless of the parent distribution or the independent of the measurements [see Fuller, W. A. Introduction to Statistical Time Series, New York, 1996. 2[nd]edition. John Wiley and Sons for an example of applications of the central limit theorem to dependent data]. When the sample size prohibits the use of the CLT, the EPA use the results from the skewness and kurtosis tests to justify the use of the log-normal based UPL [Bhaumik and Gibbons approach]. The EPA has added the skewness and kurtosis tests to the floor analysis spreadsheets. As mentioned before, the CLT can be adapted to consider not independent data. The EPA emphasizes that data from different sources are independent, although data from the same sources come from different time intervals therefore they might be considered independent.
Comment 51: Commenter 17621 states that the EPA's practice of assigning emissions from one boiler to all other boilers at the same facility when no test data were available adds a source of bias to the UPL calculations and is technically incorrect. This approach results in having multiple units with zero variability in the calculations of the UPL. The commenter adds that the facilities included in the MACT floor have up to 6 replicated emission values, increasing their impact on the calculations of the UPL.
Response to Comment 51: The EPA does not fully understand the comment. The only analysis for which emissions were assigned to un-tested units was for the nationwide emissions estimate used in the modeling analyses; the UPL analysis was not used in this instance. For the MACT floor emissions limit analyses, only emission test data were used.
Comment 52: Commenter 17621 states that the EPA wants to estimate the 99[th] percentile; for this reason, Commenter 17621 suggests that the tolerance limits will better address this need.
Response to Comment 52: The UPL is the value below which a single test observation (or average of several test observations) from the source to be tested for compliance is expected to fall with a stated level of confidence (e.g., 99%). The Upper Tolerance Limit (UTL) is the value below which a specified proportion (e.g., 99%) of the population is expected to lie with a stated level of confidence. In other words, the UTL is the value below which the population percentile corresponding to that proportion (e.g., 99[th] percentile) is expected to lie with a stated level of confidence. Operationally, the sample percentile derived from the test runs (e.g., the 99[th] sample percentile for small data sets is the largest observation) must be smaller than the UTL to be in compliance. The problem with this method is that for small data sets and proportions close to 100%, the sample percentile typically underestimates the corresponding population percentile. Therefore, the UTL is ordinarily used for large data sets (i.e., data sets much larger than those that we have for EGUs) and will not be appropriate for determining compliance from a small sample of test runs such as the size of the sample currently used to determine compliance.
Comment 53: Commenter 17621 states that in calculating the UPL the data are assumed to be independent measurements, but these data exhibit intra-unit correlations resulting from heterogeneity of the selected units. Furthermore, the UPL calculation does not account for temporal autocorrelation in the time series nature of the data.
Response to Comment 53: Values used for calculating the UPL are from different units or from the same unit but collected in different time points (even different years). In the case of values coming from different units the assumption of independence is valid. In the case of values coming from the same unit but different datasets (Part I and Part II), the time elapsed between the different measurements makes them essentially independent. Values could be considered correlated if they are consecutive values (taken minutes or hours apart).
Comment 54: Commenter 17621 states that when calculating MACT limits for existing units for some HAP, the EPA uses the lowest test series average for each EGU to calculate the pooled mean. This approach, the commenter adds, would be justified only if the same selection of lowest values from a series were used for the compliance tests. However, that is not the way the proposal is written -- it calculates the probability of a single three-run test series average exceeding the UPL.
Response to Comment 54: The EPA used the average to rank the sources and select the best performers. Then the average for the UPL is based on the data from those selected best performers.
Comment 55: Commenter 17621 states that an EGU with more test series will contribute proportionally more to the variability than an EGU with only one test series. To avoid giving more weight to some units than to other units in the data pool, it is necessary to properly apply data weighting to eliminate this bias.
Comment 56: Commenter 17739 states that the data used to calculate the variability in the UPL is not representative of the sources used to calculate the average in the UPL. The commenter adds that the EPA didn't use all the data available because some sources only had a single point, even though sources report at least six measurements. The calculation of variability, the commenter emphasizes, uses data that are not representative of the temporal or process variability for the lowest emitting units. Also, the commenter adds, the number of data points from each source used in the variability calculation ranges from 1 to 8. Fifty percent of the sources, the commenter states, have one data point although a single source accounts for 10 percent of the data used for variability. The commenter concludes that the EPA must find and correct error that may bias the results. The commenter reminds the EPA that having lots of data is not the same as having good data.
Response to Comments 55 - 56: It is true that EGUs with more data provide more information to calculate the variance used in the UPL equation, however having more data from some EGUs does not necessarily imply that bias was introduced in the calculations of the UPL. If the data available can be considered a representative sample of the expected emissions of the top performers, then it is expected that the current variance estimator would have resulted in reliable results. Having more values from different EGUs adds more information and produces a better estimate of the variance. The weighting scheme suggested by commenter will result in smaller values for the EGUs with more observations without affecting those with 1 observation, the resulting variance estimator in such a scenario would have unknown properties and may be upward or downward biased (always producing estimates below or above the true unknown population variance). The EPA has used all available data in the analyses and does not believe that any bias has resulted.
Comment 57: Commenter 17716 states that floor values cannot be based on non-quantifiable data, and asks the EPA to re-evaluate the UPL approach for small datasets and to make sure that data reflects unit performance and not uncertainty and variability, and properly reflect operating and fuel variability. The commenter stresses that these types of variability can have a big impact in the results. The commenter adds that the UPL approach creates its own outliers, particularly in the case of large datasets where large differences can be observed, in particular if some of the smallest values are near the detection limits. The commenter suggests adding a factor to ensure some degree of confidence around the results. The commenter adds that the underlying variability cannot be assessed with such limited datasets, and recommends assessing it using comparable groups of top performing sources or formulating another way to assess variability.
Response to Comment 57: The EPA considers that the UPL includes all variability captured in the data available. When available, the EPA has even included data from Part I and Part II to get a better estimate of underlying variability such as control, design, etc. When a small dataset is the only source of information available to determine the floor value, it is almost impossible to separate the different sources of variability. In the case of non-detects the EPA has applied a procedure that compensates for the non-detects as discussed elsewhere in this document. The UPL incorporates an uncertainty measure (the standard error of the estimate) in its calculation to compensate for any variability observed in the data. The UPL results are more robust if the datasets are large, and the level of confidence used in its calculation represents the level of protection afforded to sources whose emissions are in line with the best performers, furthermore, since the UPL incorporates in its calculation the standard error and a confidence level, it takes into account the variability due to the use of one single dataset, and provides assurance that 99 percent of the times the future value will be below the UPL. This is the same as saying that if the UPL is calculated 100 times using comparable best performance, 99 percent of the resulting UPLs will be larger than the average of future observations.
Comment 58: Commenters 17716 and 18014 state that the approach to select the best performers (using the lowest reported emissions value for all Part II and Part III ICR data and then ranking the sources) does not result in a random sample. This is aggravated, the commenters add, when a small number of sources are assigned unusually low emissions, resulting in the selection of those sources into the best performing group. These low emissions, the commenters say, tend to be outliers and are directly affecting the selection of the best performers and the MACT floor. The commenters recommend to average all available test data, and to use this average to characterize source performance. The commenters add that for non-Hg metals and particulate matter, the EPA used only the lowest values to assess variability. For the other floor analysis, the EPA included the results of multiple tests in the variability assessment, when available. The commenters point out the example of the HCl floor analysis and the Logan Unit 1, where the EPA selected this source as the best performing unit based on the lowest single average. The commenters suggest that a better approach would have been to take the average. Finally, the commenters suggest that in the case of multiple units with emissions below the detection limit, the EPA does not have criteria to determine the best performer.
Comment 59: Commenter 17739 states that the EPA's calculation of the UPL has a flaw in the selection of the MACT floor sources. The commenter adds that the EPA selects the lowest emission value from all emissions available for that source (Part I and Part II) and uses the lowest value to rank all the sources. The commenter adds that a better approach is to use the average. The commenter adds that a better approach would have been to take the average of the values. The commenter adds that by ranking the sources using the lowest value instead of the average, the EPA is not complying with the statue that requires the MACT floor to be based on the lowest emitting sources not the lowest value of all sources. The commenter adds that the EPA does not have data that represents operations at different loads, fuels and monitoring conditions and therefore these variability sources are not incorporated in the analysis. The commenter reminds the EPA that the EPA acknowledged that the HAP emissions would vary over time, and that not incorporating this variability could potentially result in the best performers exceeding the MACT floor. The commenter states that not having these data results in a UPL that does not account properly for the variability across time. The commenter adds that if a best performer source has more than one emission value (e.g., Part II and Part III) EPA needs to use all the data since these represent operation variability.
Comment 60: Commenter 17725 states that all available test runs should be used to assess variability and unit ranking the EPA has exasperated this phenomenon by selecting the lowest available test average to represent each source in the floor analysis for new and existing units. This practice is not statistically justifiable. By using the lowest test average, the agency is relying on a value that will tend to be an outlier. In addition to bias it introduces, the practice also affects the rank order of the units, which determines which units are included in the existing unit floor analysis and the "best performer" for each pollutant. The EPA should instead use the average of all available test data when trying to characterize a unit's performance, especially since the statute requires the limit to reflect the average performance of the sources included in the analysis.
For example, in the HCl floor analysis for existing coal-fired units, six test results were identified and used for Logan Unit 1. Logan Unit 1 was selected as the "best performing unit" based on the lowest single test average and the new unit limit was based on the results of that lowest test even though other tests for that unit were more than an order of magnitude higher. According to the commenter, this illustrates that the EPA's approach of using the lowest reported test result to represent the best performing unit does not meet the statutory requirements for identifying "achievable" emissions levels; the practice of intentionally selecting only the lowest test value suggests gerrymandering and is wholly inconsistent with the objective of determining the average performance specified in section 112 of the CAA.
Comment 61: Commenter 17725 states that the EPA should base the floor standards on quantifiable data. If a measurement is below the MDL it should be substituted with an approximation of the MDL. The commenter adds that the EPA needs to revise the data used to select the best performers because low values such as the MDL may affect the selection of the best performers. Commenter states that low values are affected by measurement uncertainty, especially those taken near the detection, so trying to quantify difference between test results for ranking purposes are futile. The commenter adds that those measurements should be steeped up by an uncertainty factor.
Commenters 17627 and 18014 recommend the EPA to conduct the UPL calculations using only data above the MDL. The commenter agrees with the EPA in that truncating the value will underestimate the UPL but insists that the UPL must be based on quantifiable data. The commenter also questions the effect of selecting the best performers. The commenter adds that this process creates outliers in the data. The commenter adds that uncertainty is playing a big role in ranking the data, and because the EPA is not controlling this uncertainty the ranking of units is based on data with lots of uncertainty because some of those measurements are near the detection limit.
Comment 62: According to commenter 17620, the EPA had used BDL and DLL flagged data to calculate the UPL. These low measurements will affect the measurement variability.
Response to Comments 58 - 62: The EPA does not believe that the statute requires or contemplates a random sample. Rather, use of data from the best performing sources is specified for developing the MACT floor limits. We did not assign "unusually low emissions" to any unit; the data available to the Agency came from emissions tests on the respective EGUs. We conducted outlier analyses and do not believe that the data in the resultant MACT floor pool data sets are outlier data. The EPA has used all available data in the analyses. For the non-mercury metal HAP and PM data at proposal, we had only one data set for each EGU because there were no total PM data available other than the Part III required testing data and no non-mercury metal HAP data. For the final rule, we are using filterable PM and there are additional data available. We discuss comments related to the minimum detection level issue elsewhere in this document.
The EPA believes that the lowest emission value available from a given source represents the best performance of the source and under this premise, it is appropriate to use the lowest emission value to rank the different facilities and to select the best performing sources. The average is affected by extreme values, both large and small, and won't represent the best performance of the source. The EPA uses all the emission data available from all best performing sources to estimate the variability in the UPL, accounting for within and between source variability.
Comment 63: Commenter 17739 states that the EPA fails to follow the same procedure employed for the HAP when calculating the MACT floor for PM. Specifically, the commenter says that the EPA did not use Part II and Part III data for the calculation of the floor for PM. The commenter states that for PM the EPA used only the minimum values in the UPL calculation. For this reason, the commenter concludes, the current floor for PM does not account for temporal variability.
Response to Comment 63: The EPA has used all available data in the analyses. For the non-mercury metal HAP and PM data at proposal, we had only one data set for each EGU because there were no total PM data available other than the Part III required testing data and no non-mercury metal HAP data. For the final rule, we are using filterable PM and there are additional data available, which we belive incorporates  temporal variability. We discuss comments related to the minimum detection level issue elsewhere in this document.
Comment 64: Commenter 17775 states that the UPL approach defines the 99[th] percentile of data if the underlying data are random in nature and representative of the objective being studied. The commenter adds that in the case of HAP emissions the EPA should have used the average of emissions from different operating conditions to calculate the UPL instead of the lowest emissions. The commenter adds that the resulting UPL is biased downward and over-sates the level of control achieved in practice by the best performing units. The commenter questions the integrity of the data, adding that those low values are prompt to having quality issues.
Response to Comment 64: To calculate the UPL the sample does not necessitate the assumption of being a random sample and representative of the population. The random constraint is usually used as a surrogate for "independent" measurements, and the independence assumption determines if in the calculation of the variance of the UPL one needs to account for correlated observations or not. The representativeness of the sample is not a requirement to calculate the UPL, what is a requirement is that the future observations that are going to be evaluated for compliance come from the same "universe" of possible emissions from which the data were taken. The EPA uses the lowest emissions to rank the sources and to select the best performers. We have discussed the use of the data elsewhere and do not believe that the low values represent outliers or have quality issues.
Comment 65: Commenter 17775 states that the EPA required Hg testing at 175 units thought to have the lowest Hg emissions based on the control technologies, and for this reason the EPA should use the average of those emissions as the MACT floor. The commenter adds that if the EPA wants to use the UPL a better approach would be to use the average to rank the best performers and use those averages to calculate the mean and variance used in the UPL. This approach results in a slightly higher MACT floor compared to that produced by the EPA. Another option suggested by the commenter is to adjust the UPL by a factor that compensates for the unaccounted variability found in the long-term Hg CEMS data that is not accounted for in the stack data.
Response to Comment 65: The EPA believes that the performance shown by the facility at the moment compliance is evaluated should be maintained on a continuing basis and not only for the compliance tests. Under this reasoning, it makes sense that the lowest emission value available from a given source represents the best performance of the source and under this premise, it is appropriate to use the lowest emission value to rank the different facilities and to select the best performing sources. The EPA has recently evaluated the association between CEMS and stack data for several sources and industries. Results led to the conclusion that both sets of data would have led to similar results for the MACT floor.
Comment 66: Commenter 18023 interprets the UPL saying that one would expect that the average performance of the top 12 percent would meet the standard 99 percent of the time, and that this is not the case. The commenter adds that it is false that the data used in the UPL calculations represent all operating conditions. The commenter adds that the resulting standard is unrealistic, and that units will not typically meet the standard. The commenter adds that because the EPA uses stack data at controlled loads on a single day, those data do not represent typical variability at all operating conditions, and the UPL approach falls short to ensure that the units that established the floor would not violate the standard.
Response to Comment 66: It is an incorrect statement that no sources among the best performers would ever exceed the floor limit at a given moment in time, which is one reason we have incorporated a 30-day averaging period to account for that potential. It is easy to demonstrate this mathematically or by counter-example. The UPL was calculated using data collected for the best performers.
Comment 67: Commenter 17620 states that in defining the test conditions that the sources must meet for compliance, the EPA directs sources to employ the 90[th] percentile "worst case" fuels, not the 99[th] percentile worst case fuels.
Response to Comment 67: The EPA has removed fuel analysis requirements from the final rule.
Comment 68: Commenter 17620 says that the EPA is not free to select the UPL probability factors without support in the administrative records for this decision. By "probability factor" the commenter seems to refer to the confidence level. The commenter is opposed to have confidence levels that could led to having all the sources passing the MACT floor. The commenter is questioning why the EPA selected 99[th] in this case, and previously selected 95 and 97.5 percent confidence levels.
Comment 69: According to commenter 17621 there is an issue with the interpretation of the UPL because some of the best performing themselves might exceed the UPL becoming non-compliant.
Response to Comment 68 - 69: It is an incorrect statement that no sources would ever exceed the floor limit at a given moment in time, which is one reason we have incorporated a 30-day averaging period to account for that potential. The level of confidence represents the level of protection afforded to sources whose emissions are in line with the best performers, and hence it is not arbitrary. A 99 percent level of confidence means that a source whose emissions are in line with the best performers has 1 chance in 100 of exceeding the floor.
Comment 70: Commenter 17620 states that the EPA's use of the "99[th] percentile UPL" results in unrealistic estimates for Hg MACT floor for conventional EGUs combusting lignite and Subcategory 1 EGUs.
Response to Comment 70: The EPA clarifies that the proposed rule is based on the UPL with 99% confidence level, and it is not based on a 99[th] percentile. We disagree that the analysis results in unrealistic estimates as claimed by the commenter.
Comment 71: Commenter 17620 suggests that in establishing the variability factors to be applied to the information in its data set, the EPA should consider: 1) public health impacts of the available options; 2) Congressional intent that some relatively large percentage of sources reduce HAP emissions as the result of MACT standards; 3) the full range of available options in performing a variability analysis; 4) the time frame of the standard and the testing data used to set the standard, compared to the data specified to determine compliance  -  variations in short term test performance will be less if EPA adopts standards averaged over longer periods of time; 5) the sensitivity of the process to "outliers;" 6) the assumptions employed in developing the set of measures by which compliance with the numerical limit will be determined; 7) the variability in test results for similar pollutants (e.g., SO2 as a surrogate for acid gas HAP and PM as a surrogate for HAP metals) exhibited by regulated sources over the years; 8) for subcategories with few units, the variability demonstrated by units in other subcategories; 9) that sources were, for the most part, under no obligation to minimize HAP emissions during testing; 10) that EPA has limited information concerning plant operating conditions during testing; 11) that, for the most recent round of testing, sources understood the purpose of the testing, and 12) that, for future testing, sources will know in advance the date of the test and can control the operating parameters of the facility based on earlier tests.
Response to Comment 71: The EPA believes that the approach taken on this final rule is consistent with that taken on other NESHAP rulemakings and the intent of the statute. Therefore, we believe we have addressed the concerns noted by the commenter.
Comment 72: Commenter 17620 states that if for a given unit one test is significantly different from the other tests available, it should be excluded from the analysis because its outlier status. Tests will be considered valid if the EPA can justify its high value; for example the EPA would need to confirm that the control devices were functioning properly during the test and that the plant operating conditions were within the reasonable worst-case operating parameters for that facility.
Response to Comment 72: It is normal practice to use the average of three emission test runs in determining compliance with an emission standard or to set emission limits. This practice allows for some measure of normal variability to be included in the results. The EPA has conducted standard statistical outlier analyses and believes that the results are consistent with the intent of the statute.
Comment 73: Commenter 17620 states that in proposal the EPA explained that some data points were removed from the analysis because they were considered outliers. The commenter states that removing data points that were two orders of magnitude greater than other data is a common practice. However, for Subcategory 1 Hg variability analysis, three tests, one for unit that satisfied the previous outlier criteria were kept in the datasets.
Comment 74: Commenter 17620 states that elimination of outliers will reduce the 99[th] UPL from 1.18 to 0.444 and the 90[th] UPL from .653 to 0.251. The commenter concludes that this demonstrates the effect of data treatment on the EPA decisions and that the method used by the EPA is overly sensitive to very high test results submitted by the industry.
Response to Comment 73 and 74: Outlier detection is a very important part of data preparation for analysis and it should be consistent across all datasets. The EPA has used standard outlier criteria in the MACT floor analyses and does not believe that the results are overly sensitive to high test data. The data sets for the final rule have been reviewed and a consistent approach has been taken with all of the data sets.
Comment 75: Commenter 17620 states that when explaining the selection of the confidence level the EPA refers to the level of protection afforded to the facilities (e.g., 1 chance out of 100 of failing the test), however, the EPA never discusses the level of protection afforded to the public.
Comment 76: Commenter 17620 recommends using the 90[th] percentile. The commenter adds that the EPA should evaluate the impact of the confidence level on public health and avoiding stack test failures at well controlled operated units.
Response to Comment 75 and 76: The EPA does not recommend the use of percentiles for calculating the floor standards. For small data sets and even a sample size smaller than 100, the sample percentile typically underestimates the corresponding population percentile, resulting in erroneous conclusions regarding the compliance of the units being evaluated. The lower the confidence level the lower the MACT floor, then the greater the potential for satisfying compliance.
Comment 77: According to commenter 17620, the EPA's claim that if the limit was based only on the average performance then the sources will exceed the limit half the time or more is true if the emissions are random in nature. The commenter adds that few state regulations result in a reduction of HAP emissions making the emissions data semi-random in nature. The commenter states that the proposed standard is not the same as that obtained using variability from sources without direct regulation. According to the commenter, if the MACT floor were based on the 50[th] percentile, then EGUs at the bottom half of the top performers would have to make some improvement to satisfy the limit, which he adds was the intention of Congress.
Response to Comment 77: The floor limit is based on the average of best performers plus variability. This average is different from, and smaller than, the average of the emissions of all sources. So, the chances of exceeding the limit increase as the sources performance deviates from the performance of the best sources. The sources utilized to calculate the floor limit constitute a subset of all available sources, with the restriction that they are the best performers. It may be possible that sources within the best performers are located in states without regulations. The variability used for calculating the UPL will depend on the magnitude of the emissions and the deviations between the inter- and intra-data sources.
Comment 78: Commenter 17620 states that he calculated the 90[th] and 99[th] UPLs for Hg content of the coal Subcategory 1. The large variability observed between the variability factors (t-statistic*Standard deviation) and the average and minimum values of the data suggests that certain data points were outliers. The commenter concludes that multipliers in excess of 10 or 20 that are used in other emissions to account for variability are unrealistic.
Response to Comment 78: The EPA agrees with the commenter regarding the effect of outliers on the statistic measures (mean and standard deviation are very sensitive to outliers) used in the calculation of the UPL. The EPA analyzed the data looking for inconsistencies before proceeding with the analysis. The EPA looked at both ends of the dataset looking for outliers (high and low values). The EPA removed data that failed the statistical outlier test.
Comment 79: Commenter 17620 notes that in the absence of controlled testing of individual units the EPA has utilized the variability between individual test runs to estimate the variability in the UPL for pollutants other than Hg. The commenter adds that following the same procedure they were able to obtain the same multipliers except for HCl.
Response to Comment 79: The EPA used the same approach for all HAP in each of the MACT floor analyses.
Comment 80: According to commenter 17620, year-to-year test results of well controlled and operated units shouldn't vary by more than an order of magnitude, especially under the compliance conditions of the proposed rule. The commenter adds that variability multipliers over 20 are unrealistic, and the EPA should use them with caution when assessing the variability of compliance units.
Response to Comment 80: The UPL utilizes emissions data from the top 12% performers. Because the top 12% might change from time-to-time (e.g., year-to-year) and it is possible that new controls and other factors add some variability to the data. Given the range of sources and controls represented in the data, it is not impossible to observe large variability in the data. The EPA had evaluated the data utilized in those calculations and decided those are valid data points obtained when the sources were under normal operation conditions. As a result, the EPA has included those data points in the calculations of the UPL.
Comment 81: According to commenter 17621, the EPA needs to revise some calculation inconsistencies. For example in Appendix A for coal mercury >= 8,300 Btu/lb the calculation of the mean used all test series, but in other tabs of the excel file only the lowest test series per unit were used. A similar inconsistency, the commenter adds, was observed with the calculation of the variance. For existing coal >= 8,300 TBtu/lb, the variance used all data (from Part II and III series), but for total PM and metals the variance was calculated using the lowest test series for each EGU.
Comment 82: Commenter 17621 noted some inconsistencies in the application of the calculation of the UPL across all parameters and EGU categories. The commenter states that in some cases, the mean emission was calculated from the lowest test series reported for each unit, although in others it was calculated from all test series reported across all of the units in the MACT pool. For total PM and metals for coal-fired EGUs, the commenter adds, the EPA did not include all Part II and III data as it was done for Hg. Instead, the commenter adds, the EPA used only the lowest test series for each EGU, which resulted in a lower MACT floor.
Response to Comments 81 - 82: Partly based on comments received, we have reviewed all of the MACT floor analyses and believe they are now consistent.
Comment 83: Commenter 17621 states that the EPA sorting of the EGUs produced inconsistent results in lb/MMBtu and lb/MWh units. According to the proposal, the commenter adds, to determine the best performers the data was sorted on the "lb/MMBtu" formatted data and the same pool of EGUs was used for the `lb/MWh' analysis. According to the commenter, it is not stated that this list was sorted by lb/MWh values before selecting the best performers for the new EGU limit. The commenter adds that this inconsistency is observed in the coal PM MACT spreadsheet, where two different sorting orders for the MACT pool results if one sorts by (lb/MMBtu) and lb/MWh.
Response to Comment 83: Partly based on comments received, we have reviewed all of the MACT floor analyses and believe they are now consistent.
Comment 84: Commenter 17621 states that by combining or pooling all data from selected best performing sources they are treated as though they comprise independent identically distributed measurements from a conceptual baseline population, ignoring in this way inter-unit heterogeneity, intra-unit and temporal correlation.
Response to Comment 84: Data utilized for calculating the UPL are from different EGUs or from the same EGU but collected in different time points (even different years). In the case of values coming from different units the assumption of independence is valid. In the case of values coming from the same unit but different datasets (Part II and Part III), the time elapsed between the different measurements makes them essentially independent. Values could be considered correlated if they are consecutive values (taken minutes or hours apart), which is not the case for the current data available. The variance used in the UPL accounts for inter-unit and intra-unit variability but it does not account for temporal correlation because it was assumed that data points from the same EGU but at different time points are independent.
Comment 85: Commenter 17621 states that baseline data (which is the data used to calculate the UPL) should be compatible with the data used for compliance. The commenter is concerned that the EPA is planning on using MACT floors based on stack data and require 30-day-averages for compliance. The commenter reminds the EPA that those two types of data (stack and 30-day averages) are not compatible.
Commenter 17975 notes that emission limits for non‐Hg metals are based on a 30-day rolling average that would include startups, shutdowns and maintenance events. Commenter 17975 goes on to state that the stack tests upon which the agency's PM standard is based do not reflect emission rates that can be achieved over a 30-day period because stack testing is usually designed to assure compliance with PM limits that must be met over a three-hour period. The commenter asserts that the Agency's 2009 "Clean Air Act National Stack Testing Guidance," which applies to both MACT and NSPS, makes clear that stack testing to determine compliance with these short term limits should be conducted under conditions that "are most likely to challenge the emission control measures of the facility with regard to meeting the applicable emission standards." The guidance further explains that stack testing should be conducted when boilers are operating at peak capacity, and using the dirtiest fuel that it can legally burn because stack testing under these conditions is designed to measure the maximum emissions that will occur within any three-hour period over the course of a year or more. The commenter believes that actual emissions averaged over 30-days during periods when units are operating at lower capacity with cleaner fuels  -  are likely to be much lower.
Comment 86: According to commenter 17638, data used by the EPA to establish floor standard does not account for emissions variability resulting from startup, shutdown, soot blowing, and malfunction. Commenter adds that the EPA needs to either propose a separate work practice standard or exempt any operational mode not included in the MACT floor for compliance with the PM limit.
Comment 87: Commenter 17716 suggests that data used in MACT floor do not account for startup and shutdown effects. According to the commenter this creates serious problems for units that are required to demonstrate continuous compliance with the PM limit using a PM CEMS since part 63 no longer includes exemptions for SSM periods. The commenter adds that even with the relatively long averaging periods (30-day rolling averages) proposed under this rule, the emissions during these non-representative periods could significantly affect the averages, especially for peaking units or other units that come on- and off-line frequently.
Comment 88: Commenter 17716 adds that it is questionable whether representative measurements could even be obtained during boiler startup/shutdown due to the dynamic flue gas conditions and technical limitations associated with the measurement technology. For instance, changes in particle size distribution or unburned fuel levels during startup affect PM CEMS measurement accuracy, particularly those using light scattering or other optically-based technologies.
Response to Comments 85 - 88: The EPA understands that stack data and 30-day averages don't seem to be compatible. However, we have used an approach consistent with other NESHAP rulemaking efforts with similar data constraints in developing the 30-day rolling average emission limits.
Comments related to startup and shutdown issues are addressed elsewhere in this document.
Comment 89: According to commenter 16721, the statement found in proposal "only 1% of monitored emissions for a compliant EGU should exceed the UPL" indicates that the 99 percentile would be a simpler estimate of the UPL, and for this reason if enough data are available, one could compare future measurements with the 99 percentile. The commenter proceeds to describe the problems of estimating the 99 percentile, and mentions that there are approaches to estimate the 99 percentiles ranging from parametric and non-parametric ones. The commenter continues describing the issues related to estimating the 99 percentile using small sample size and the uncertainty incorporated when small sample sizes are available.
Response to Comment 89: The commenter has confused the 99 confidence level UPL with the 99 percentile. Both have different interpretation and serve different purposes. The EPA intends to calculate the UPL. The UPL is the value below which a single test observation (or average of several test observations) from the source to be tested for compliance is expected to fall with a stated level of confidence (e.g., 99 percent). The 99 percentile of a sample or population is the value such that 99 percent of the measurements are no greater than this value, and 1 percent of the values are no less than this value. A data point is below or above the 99 percentile, although a data point is below or above the UPL with some confidence level.
Comment 90: According to commenter 17621, data selection affects the calculations of the UPL. An example mentioned by the commenter is the effect in the UPL of sorting the data and selecting the smallest of the measurements from a measurement run for calculating the mean of the UPL. The commenter adds that these data selection process is justified if similar data selection would be used for compliance.
Response to Comment 90: The EPA sorts the data to identify the best performers. We selected the smallest average of three runs, rather than the smallest run of the three as is implied by commenter.
Comment 91: According to commenter 17621, the EPA needs to weight the data used in the UPL equal to the inverse of the number of baseline measurements (i.e., measurements available from each best performer). This procedure, the commenter adds, will provide each unit in the pool with the same weight.
Response to Comment 91: The EPA uses one measurement from each best performer to calculate the average of the UPL, and uses all data available from Part II and III to calculate the variability estimator.
The driving force of the variability term is not the number of measurements from each best performer but the range of the measurements. Therefore, if one would like to determine a more robust and less biased estimator, one should have more measurements for those sites with larger variability or larger range. There are approaches, like the one proposed by the commenter, that result in weighted estimators. The use of the weighted estimator is to improve the estimation of the parameter. In this case the commenter would like to improve the estimator of the variance of the mean. Weighting down the measurements for those sources with more than one measurement will change the variance estimator and may introduce some bias to the UPL. An approach like this will have unknown results in the variance that would have to be investigated and, thus, was not used.
Comment 92: Commenters 17621 and 18014 state that instead of evaluating if the data are normal or lognormal distributed the EPA should model the tail of the distribution rather than the distribution of the measurements itself. The commenters add that the small sample size reduces the power of the test utilized to assess normality or log-normality, and because the distribution assumed dictates the approach to calculate the UPL, the EPA needs to use sufficient data to increase the probability of correctly detecting the distribution of the measurements.
Response to Comment 92: The EPA evaluated the distribution of the measurements from the best performers for the sole purpose of determining the best approach for calculating the UPL. The UPL equation varies if the distribution is normal or lognormal. The EPA modeled the tail of the distribution of the emissions because it used only data from the best performers. The EPA used the normal or log-normal approach because there are UPL equations developed for such distributions. There are currently several approaches that model extremes in a distribution (e.g., tail of the distributions), however there are no simpler approaches developed to calculate the UPL from such distributions. The EPA is aware that for sample sizes of 10 or lower, tests for assessing normality or log-normality will have a reduced power (probability of rejecting the null hypothesis that the data are normal or lognormal when it is false). For this reason, the EPA looks at other assessment tools such as kurtosis, skewness and histograms.
Comment 93: According to commenter 17621, the EPA appears to have used actual Part III emission test data to develop the actual annual emissions estimate used in their case study risk evaluation. The commenter adds that if the unit was not tested in the ICR, the EPA used data from a similar tested sister unit or an average for units tested in the ICR. These averages, the commenter adds, do not appear to have taken coal rank into account. The commenter adds that those data were not properly described using summary statistics. For example, average values were used when the data are skewed, and outlier analysis was not performed when extreme measurements suggest they might be outliers.
Response to Comment 93: The agency disagrees with the commenter. Please see response to comment 7 in section 1G for a discussion of the coal rank issue and response to comment 2 in section 1G for a discussion of the statistical methods used.
Comment 94: Commenter 17621 comments that because most of the data have a skewed distribution, the arithmetic average is not an appropriate statistics for estimating the bin mean value and that the geometric mean is more appropriate for developing emission factors. The commenter adds that nickel and chromium have possible outliers but those data points were left in the analysis.
Response to Comment 94: For skewed distributions, the median is a better statistic to determine central tendency. The geometric mean is equivalent to the arithmetic mean when analyzing log-transformed data. Prediction intervals calculated using the arithmetic mean after log-transforming the data result in intervals for the geometric mean. The arithmetic mean is the naïve estimator for the mean of the log-normal distribution, however, it is very inefficient when the coefficient of variation (CV=SD/average) is large and this is true even for large samples [Reference: Shen, L, Brown L. D. and Zhi, H. 2006. Efficient estimation of log-normal means with application to pharmacokinetic data. Statistics in Medicine. 25:3023-3038]. This result makes the arithmetic mean very unreliable even for large sample size, which is why we declined to use it.
Comment 95: According to Commenters 17725 and 18014, the UPL approach addresses the issue of small sample size, reflecting the greater uncertainty in the results. The commenters state that in the case of the new unit limit, the unit with lowest test results was not selected at random and its tests are not typical because they are the results of random measurements affected by calculation or reporting errors.
Response to Comment 95: The EPA understands the issues surrounding the MDL and ND measurements and applies a procedure to account for this issue. Comments related to the use of MDL data are responded to elsewhere in this document. We do not understand commenters' comment relating to the EGU serving as the basis for the new-source limit not being selected at random and, thus, its results are not typical. The basis for the new-source limit was among the data available to the Agency through the 2010 ICR; we believe that any calculation and reporting errors have been corrected.
Comment 96: Commenter 17730 states that in the preamble the EPA expressed the position to account for measurement imprecision when calculating the MACT floor. However, the commenter adds, the EPA did not undertake such adjustment because it stated "it did not know how to do it" which suggests that the UPL approach is inappropriate. The commenter recommends that the EPA consider the quality of the data used and how it impacts its ability to account for different sources of variability.
Response to Comment 96: In the proposed rule, the EPA requested comment on approaches suitable to account for measurement variability in establishing the floor emissions limit when based on measurements at or near the MDL. We received a number of comments on the issue and respond to them elsewhere in this document. Also, elsewhere in this document we address the changes made in our approach to the use of MDL data in the final rule.
Comment 97: Commenter 17739 states that when calculating the UPL for Hg and HCl, the EPA used the minimum values to calculate the average and additional data from Part II and Part III to calculate variability. The commenter states that these two different datasets represent distinct populations. The commenter states that the EPA should use all data to calculate the variance and variability of the UPL.
Response to Comment 97: The floor standard is based on the best performers' data. It is not uncommon to use different sources of data to obtain better estimates of the average and the variance. A better estimate of the latter was obtained with the incorporation of Part II and Part III data. These data contain different sources of variability mentioned by several commenters.
Comment 98: According to commenter 17796, the use of pre-regulation test data to calculate variability will likely overstate the post-regulation variability to some degree. The commenter recommends that the EPA seek appropriate sources and performance tests for this variability approach. The commenter adds that in the subcategory "Existing Sources >8300 Btu/lb," there are four tests that look like outliers and without them the UPL for Hg reduces from 1.2 lb/TBtu to 0.44 lb/TBtu.
Comment 99: Commenter 17843 states that they agree with the EPA's approach. However, the commenter adds that the EPA should use pre-regulation data with caution because these data reflect emissions variability when emissions are unconstrained resulting in an UPL that will overstate the post-regulation variability. The commenter adds that some pre-regulation sources included in EPA's analysis may not realistically reflect expect variability.
Response to Comment 98 - 99: As noted elsewhere in this document, the EPA conducted standard statistical outlier analyses and did not use data that failed. Also, as discussed elsewhere in this document, we continue to believe that use of the Part II and Part III data are appropriate.
Comment 100: Commenter 17817 recommends that the EPA review 5-year data for the top 12 performing units for both SO2 and NOX. The commenter adds that these inspections will clearly indicate the occasional absence of normal distribution in the data. The commenter states that so much the more must be expected for occasional measurement of for PM CEMs for which sufficient data are not yet available.
Response to Comment 100: The EPA understands that emissions data are usually non-normally distributed; for this reason when applying the UPL, the EPA is not relying on the normal distribution of the data, but in the distribution of the average. This is possible based on the Central Limit Theorem, which ensures that for a sample of moderate and large size, the distribution of the average tends to normal regardless of the parent distribution of the data.
Comment 101: Commenter 17817 recommends that the EPA use mean and standard deviation of the UPL using all stack test data for all units in each subcategory equipped with similar control technology. The commenter adds that the EPA has the data and just needs to make proper use of it. The commenter adds that they have prepared some analyses that can be made available to the EPA. The commenter adds that the EPA must establish MACT limits using data from the top performing units and account for all reasonable variability so that sources could be expected to perform every day and in all operating conditions.
Response to Comment 101: We discuss elsewhere in this document comments related to subcategorization by control device which appears to be what commenter is suggesting. The EPA believes it has made proper use of the data available to it and has established emission limits, and accounted for variability, based on the performance of the top performing 12 percent as required by the statute.
Comment 102: According to commenter 17820, the use of ICR data results in MACT limits that are low and overstates the level of control achieved by the best performing units. The EPA should set a limit based on filterable PM based on a calculation of variability using all test series, not just the lowest test series.
Response to Comment 102: The EPA disagrees that use of the available data results in MACT limits that are low or that overstate the level of control achieved by the best performing units. As noted elsewhere in this document, we have established a filterable PM limit in the final rule using the same approach as used for the other limits.
Comment 103: According to commenter 17820, the 99[th] percentile UPL approach is questionable in the case of new source MACT limits, because it is based on only three measurements collected under identical operating conditions. These measurements obviously cannot provide information about the distribution of the data, and as a result, the variability for new sources results in a small adjustment when compared to the variability of existing sources. The commenter questions the use of the lowest value instead of the average of each source, which according to the commenter results in an UPL biased low which overstates the level of control achieved in practice by the best performers.
Response to Comment 103: The EPA is constrained by the statute to using data available to it and basing the new-source limit on the best performing similar source. Absent other available data for the best performing unit, we used the three individual runs so as to be able to incorporate variability. As noted elsewhere, we do not believe that our UPL approach results in limits that are biased low.
Comment 104: Several commenters (17821, 17898, 17912, 18021) state that the UPL would work if the EPA uses the correct data. The current data do not capture long-term emissions variability attributable to varying concentration of trace elements in coal for example. The commenters added that they have analyzed long-term CEMS measurements for Hg and PM for some of the best performing units in some HAP groups and found that long-term average emissions are higher than those expected from stack tests alone. The commenters suggest the EPA collect more data to describe all phases of operation, in particular CEMS data. The commenters recommend that the PM standard be based on filterable PM only and not consider condensable PM. The commenters also add that the filterable PM standard should omit startup and shut down periods. Commenters 17898 and 17912 recommend the EPA use its Acid Rain database as a better means of determining what level of variability actually is associated with the most adverse conditions that reasonably recur, especially since use of the Acid Rain data will help account for startup and shutdown emissions which, as the standard is currently written, are part of the compliance demonstration. Commenter 18021 states that representative data are critical, and need to reflect the entire source category and the variability of performance within the source category when considering the need for continuous compliance and related monitoring.
Comment 105: Several commenters (17898, 19536, 19537, 19538) state that although the EPA doesn't have the same data set for consideration of Hg variability as it does for SO2, there are several units that have Hg CEMS that would have to be considered in determining the true Hg variability. The commenters add that the EPA's present method of calculating variability just based on a short stack test and an arbitrary variability factor is insufficient to properly address variability. Commenters add that if the EPA wants compliance to be determined based on CEMS data, it should then set the variability based on CEMS data rather than trying to extrapolate from single stack tests to CEMS data. In addition, by requiring use of Hg CEMS and imposing a long averaging time, EGU owners will be more vigilant about the EGU's Hg emissions on a day in-day out basis to make sure the Hg controls are being operated in a manner to ensure continuous compliance with the MACT limits.
Response to Comments 104 - 105: We believe that we have used the "correct" data. We did not have CEM data for units comprising the MACT floor pool and, thus, could not use CEM data in assessing variability. As noted elsewhere in this document, we are finalizing a filterable PM limit in the final rule. Startup and shutdown are also addressed elsewhere in this document.
Comment 106: Commenter 17843 states that although the UPL is a reasonable way to establish the MACT floor, the EPA must evaluate the process to select the best performers. The commenter adds that in the Hg MACT analysis they noted that 4 performance tests out of 80 are driving a significant portion of the "variability." Therefore, the commenter requests that the EPA review and reconsider whether these sources are appropriate for inclusion in an analysis of expected variability by post-regulated sources.
Response to Comment 106: As noted elsewhere in this document, the EPA has reviewed its analysis and conducted standard statistical outlier tests and we believe that the data in the MACT floor pools represent the best performing sources.
Comment 107: Commenters 17929 and 18644 state that the EPA needs to revise the UPL practical effect. For example, the average of top 12 percent of most coal-fired units (0.01 lb/TBtu) is transformed into an UPL of 1.0 lb/TBtu, or one-hundred times higher.
Response to Comment 107: The EPA believes its analysis, including the assessment of variability, is appropriate.
Comment 108: According to commenter 17975, the EPA needs to evaluate the data used in the UPL calculations. The UPL is sensitive to outliers and selection of the best performers. The commenter recommends that best performers selection should be done very carefully after a thorough review of all data including operating conditions during tests when the data were collected.
Response to Comment 108: The EPA has reevaluated its analyses and conducted standard statistical outlier tests and believes its analyses to be appropriate.
Comment 109: According to commenter 18014, the individual run data should be used in the floor analysis for both existing as well as new unit limits. Therefore, the UPL analysis should be conducted using individual run data so the assessment includes the run-to-run reference method variability.
Comment 110: Commenter 17725 states that individual run data should be used for each floor analysis. The spreadsheets provided in the docket reveal that the EPA did not use the data from the individual runs in the floor analysis for existing unit limits. The individual run data should be used in the floor analysis for both existing as well as new unit limits. The UPL analysis is supposed to evaluate the potential for a unit with the desired performance to pass a future 3-run compliance test. Therefore, the UPL analysis should be conducted using individual run data so the assessment includes the run-to-run reference method variability.
Response to Comment 109 - 110: The EPA disagrees that individual run data should be used for both new- and existing-source MACT floor analyses. If a company chooses to use manual testing for its compliance, it will be the average of a 3-run test that will determine compliance. Therefore, we have used this same average of 3-run tests in developing the MACT floor limits. We would use the same approach for new sources were sufficient data available for all HAP.
Comment 111: Commenter 18014 agrees on the need to use alternative data to get better estimates of variance. Commenter adds that Part III data and not Part II should be used to evaluate variability when establishing the floor for existing units when the number of units or test data is small.
 Response to Comment 111: The EPA disagrees that the Part II data should not be used to assess variability, particularly given that we only have one set of data for each of the EGUs required to test under Part III of the 2010 ICR.
Comment 112: According to commenter 18447, the EPA biased the results first by sampling a subset of power plants from which to obtain data instead of selecting all the power plants in the category. Then it selected the top 12% of plants based on the results of each individual pollutant. By skewing the results in this way, it is possible that no power plant in the country can achieve compliance for all pollutants that the EPA is attempting to regulate with this rule. The EPA should have sampled all the power plants and then selected the best performing 12% plants.
Response to Comment 112: The EPA does not believe that it has biased the results. In the Supporting Statement for the 2010 ICR, we outlined our approach to selecting the EGUs to be tested for acid gas and non-mercury metallic HAP so as to be able to use the largest number of data points in the MACT floor pool for those HAP. We do not believe that this is biasing the results. We address comments on the pollutant-by-pollutant approach elsewhere in this document.
Comment 113: According to several commenters (19536, 19537, 19538), the statute provides the EPA some discretion in defining the facility's actual performance; it does not, however, allow the Agency to use two inconsistent definitions of that term. If a plant's actual emissions are best approximated by averaging all of that plant's test data and then statistically adjusting that data to address possible variability, then the "best performing plants" are those with the lowest emissions according to that measurement. If, on the other hand, a plant's actual emissions are best approximated by its single lowest test result, then the MACT floor must be defined using the single lowest test results of the best performing plant (or plants). For example, the EPA's Hg standard for existing units burning coal of less than 8300 BTU/lb is based upon 40 "best performing" units selected on the basis of those units' lowest three-run average Hg data (in lb/MMBtu). If the EPA had used the average to determine the "best performing" units, many of the EPA's claimed "best performers" would not qualify as such. Instead, the EPA used the lowest test result to select its best performers, but included all available tests for these "best performers" and further adjusted for variability, in order to determine the floor.
Response to Comment 113: The EPA disagrees with commenter's suggested approach and believes the approach taken by the agency is appropriate and was consistently applied for all of the MACT floor analyses.
Comment 114: Commenter 17620 provides the results of a sensitivity analysis performed by the commenter to examine the impact of a range of choices available to the EPA in evaluating the Hg data for Subcategory 1 sources. It presents what the MACT floor would be, using different choices as to the "percentile" UPL to employ, as well as several different choices available to the EPA in data selection and utilization. Table 1 also sets out the "multipliers" of the arithmetic average of the best performing 12% of these sources that are associated with each of the options analyzed. These results represent but a few of a larger number of choices available to the EPA in calculating the MACT floor.
Response to Comment 114: The EPA has analyzed the results presented by the commenter and agrees with the commenter that including and excluding outliers as well as changing the confidence level of the UPL results in a wide range of UPL values. The EPA carefully analyzed the data available and concluded that all measurements were valid measurements that represent emission values obtained under regular conditions in some of the best performers. As pointed out by the commenter, the confidence level (e.g., selection of 99% confidence level instead of 80% confidence level) is another factor that determines the final value of the UPL. If the level of confidence is 99%, then future data collected for compliance from a source that performs at the level of the best performers will have 1% probability of a false positive (i.e., non-compliant) result. This provides a high level of protection to sources whose emissions are in line with the best performer(s). If the confidence level is lowered, then the probability of a Type I error (false positive) is correspondingly increased, which means that the level of protection provided to sources in line with the best performer(s) is correspondingly reduced. But on the other hand, the probability of a Type II error (i.e., false negative or compliant result when the future data come from a source with higher emissions than the top performer(s)) will be reduced, which means that higher emitting sources will be exposed more easily. The commenter's table also refers to the calculation of the uncertainty as a factor that affects the UPL. The EPA agrees that there are different options for calculating the variance, and each one is based on different assumptions. The formula to calculate the variance proposed by the commenter is based on the average of unit variances. Further study is needed to determine the properties of such uncertainty estimator (e.g., unbiased) and the effect of the sample size on it. The one selected by the EPA is the standard unbiased estimator.
Finally, the commenter proposed the average of the unit UPLs as the final UPL. Although this is an interesting approach, the purpose of the UPL is to provide a limit below which a future observation (or mean of several future observations) is expected to fall with a stated level of confidence. Source-specific UPLs will have multipliers that are determined by the number of measurements in each source, these result in larger multipliers for high confidence levels. Another aspect to consider in the proposed approach discussed by the commenter is the interpretation of the average UPL. The average UPL represents an average value below which a future observation or the mean of future observations is expected to fall. The confidence level of the average UPL would have to be investigated because it may not be equal to the individual UPL's confidence level.
Comment 115: Commenter 17620 states that, in the Hg floor for coal-fired units, the EPA initially selects the units based on the single lowest test results of the units in the subcategory and designates the units in the top 12% of this group as best performing units. It then includes other test results for those units in determining the UPL for purposes of assigning a variability factor. Because of this, the EPA's results include units with highly variable performance, and one low test result, rather than units with consistently good performance. In other MACT rulemakings, the commenter has commented that the EPA's approach is inconsistent and leads to higher MACT floors than are appropriate. The EPA should define "best performing units" after consideration of appropriate variability allowances. Available data concerning units in the top 12% that exhibit large variations in test results should be evaluated to determine the reason for the variation in performance.
Response to Comment 115: Based on an evaluation of the relevant factors and available data, the EPA determines the level of emissions control that has been achieved by the best performing sources considering these sources' operating variability. By examination of the available data, the EPA ranks the sources and selects those whose data will be used to calculate the UPL. A source that is operating poorly will be less likely to be included in the pool of best performers than if it were operating properly. We believe that use of the outlier analyses precludes the "highly variable performance" noted by commenter from biasing the results.