An extensive array of mdti train data is available describing the precision of these three mercury measurement methods. The first set of results was developed by EPNOAQPS as part of the validation of Method 29 ( Radian, 1992). Those tests, conducted by Radian C o p , were performed at a municipal waste combustor operated by American Ref Fuel in New Jersey. That data set includes 8 quad train runs with stack mercury concentration ranging from 130 to 575 pg/ dscm. The next set of data comes from the EPA efforts to validate Method IOlb ( EER,

1997). Extensive testing was conducted at a hazardous waste burning cement kiln. Testing consisted of triplicate train runs using both Methods 101 b and Method 29 ( test series 1) and quad  _ l  "

103 Pi m m r , pan 1 ' I 1 L k .

I ..... ..,., ,

2 " CI + Q= a P I 4.

. .. . .". I._ . _ .  ... I .. .

I i, I 106 c c train runs using only Method 101 b ( test series 2.). The tripiicate train tests consisted of a Method 1Olb train located in the breaching to the stack ( downstream of all air pollution controls) as well as Methods 10 1 b and 29 trains located at the same plane of the stack. These three measurements all treated as simultaneous but it is distinctly possible that the diverse locations could contribute to an increased apparent method imprecision for this data set. For the quad train tests two of the lines were dynamically spiked with Hgo and HgC12. For the ReMAP analysis, only the unspiked data fiom the quad trains have been used.

A third set of multi train data was provided by the Rig0 and Chandler tests on the MWC facility in Pittsfield, MA. These tests provide dual train measurements using EPA Method 29. A fourth set of multi train data is provided in EPA tests at the Stanislaus County MWC ( Radian, 1992).

In those tests, two Method 29 measurements were made simultaneously in the stack. From these tests, the only dual train metal data available from the EPA were for mercury.

A fifth set of data is provided by tests performed at EPA's research facilities in RTP, NC ( EPA,

1998). There are several key features to the data fiom these tests that should be pointed out as part of the R e m data analysis. First, numerous Method 29 tests were conducted as part of an effort to assess performance of multi metal continuous emission monitors. The EPA combustion facility at RTP is a pilot scale rotary kiln incinerator with a full RCRA permit. As part of the CEMS assessment, the metal content of the waste feed was varied to adjust the range of metal concentration in the stack. The physical arrangement of the stack causes the flow to travel from a mezzanine level, down m e floor, and then horizontally to a final clean up baghouse. A series of multi metals CEMS were installed in the vertical portion of this duct. Method 29 trains were located in the horizontal duct runs at the meuanine level ( before the CEMS) and at the floor Because of the wide concentration range of these paired data, these results are extremely important to the overall ReMAP effort. A carefui assessment of the data indicates that there is a distinct bias in the results. That bi only detectable because of the large number of metals being measured.

[ Material presen the Appendix discusses the data trends in some detail.] The essence of imprecision is that simultaneous measurements of the same stack gas yield different anual method data were collected simultaneously.

107 measurement results. What is unusual about the EPA pilot scale tests is that the reported concentrations for all of the metals tend to move in concert. If the mezzanine trai higher concentration for one metal ( relative t 1 sampling train), it will also indicate a higher concentration for the other metals as SME ReMAP team conducted a thorough assessment to determine the se of this bias. Unfortunately no firm conclusions were forth co e problem can be traced to the sample collection process or the h pIe as opposed to the laboratory analytical procedures. Other re also investigated.

Though there is a distinct bias, the data from these tests have been used as provided by the EPA researchers. The effect may be to slightly increase indicated standard deviation of individual method 29 data points, but the wide range of d atly improves the overall quality of the method precision estimates.

Tables 16a and 16b provide a summary of all the merc step in the analysis is to perform an outlier analysis outlined previously. The data were grouped into sets wi lOOpg/ dscm. The analysis identified four data points as having abnormally large spans. Those runs included Run Number 9 from the EPA/ and Runs 5 2,8 3 and 9 1 from the Stanislaus County tests. The span clearly out of line with the remainder o data with concentrations below 100 pg/

suspect Runs 6 3, 8 1, and 9 2 as pote program and generally accepted statistical analysis procedure all data udess there is a clear rationale for data elimination. Accordingly, only the four measurements identified as having abnormally large data spans have been eliminated f? om further analysis. Figures 35 and 36 present the data from Tables 16a and 16b showing s plots of mndarcl deviation and relative standard deviation as a function ng. It seems appare * sion data noted above. The first that test series as well as with values of RSD there is also reason to Table 16a. Methods 29 and 101 Data for Mercury 109 c 0 0 c\

0 3 0 0 r

0 0 W 0 0 rn 0 0  3 0 0 cc)

0 0 N 0 0,

0 I12 I =: 3 n . . L e w E I 0 0 0 P confidence intervals as well as the small sample bias corrected data. The t statistic for the regression is 7.45, which is well above the critical value for 67 degrees of freedom and 95% confidence. After applying a bias correction factor for the log log transformation the regression relation is found to be:

S ( Hg total, Methods 29 and 101) = 0.208 * Co. 877.

The vaIue of the power coefficient, at th fidence level is:

P = 0.877 5 0.234 or between 0.

Figure 38 presents three different forms of the p total mercury. The relative standard deviat between about 10 and 15%

above 10 pg/ dscrn. Tripli relatively narrow range of the true stack concent pg/ dscm, Method imprecision is anticipated to deviating by less than f 23.3% from the actual concentration is above 10 pgldscm, 99 out of 100 single m licate measurements 5 40.4% of the true stack concentration.

Data presented in Figure 36 are based on the assumption that standa concentration according to the regression relation shown in Figure it is possible that standard deviation could be as high as the upper c the lower confidence interval. The potential range for 99 out of 1 imprecision in the measurement method, has been calculated of true ' stack concentration. Results from those calculations are presented in Figure 39. If the precision of the mercury measurement methods is as large as the upper confidence interval, a significant variation in measurement results may be anticipated. For example, if the method standard deviation varies according to the upper confidence limit, measurement of a stack containing 500 pg/ dscm of mercury could result in a spread for 99 out of 100 data points ranging from 275 to 725 pgldscm. If the Method precision is best described by the lower confidence interval, Yo confidence level,

, t) 3 u . e t I Y 115 I16 measurement of that same stack containing 500 pg/ dscm mercury should result in a 99 out of 100 future data points falling between 428 and 572 pg/ dscm. It is noteworthy that the RSD data listed in Tables 16a and 16b for individual mercury measurements occasionally exceeded 50% arid that the firms conducting the tests were highly qualified stack festers and laboratory anafysts. This serves to underline the critical importance of precision in stack testing. From an owner's perspective, it is ow the precision of the data being gathered to determine compliance. From a regulatory development perspective, it is critically important to know the precision of data being used to establish regulations.

The best estimate of the precision of Method 29/ 101iV101B is provided by the regression analysis.

Those data were presented graphically in Figures 38 and 39. To assist the reader, Table 17 presents these data in tabular form, focusing on the range of concentrations anticipated to be of primary interest to facilities being regulated under the new and emerging US EPA emission rules. The first column lists the true concentration of total mercury in the stack ( not corrected for excess oxygen).

The second and third columns present the anticipated range for 99 out of 100 future single measurements ( at the given true stack concentration). The fourth and fifth columns provide the anticipated range for 99 out of IO0 future triplicate measurements.

117 ij L Q, v, c d C C r 118 8.0 EPA Method 29 for Multi Metals The essential elements of  PA Method 29 were presented in the previous section. As noted, the various components of the train are analyzed to determine the concentration of various metals. The method is used to determine compliance with a wide range of metals but there are very limited multi train data available to assess the precision of the method for those pollutants. For three of the key metals, cadmium, chromium, and Iead, the Agency did perform multi train, Method validation from those tests suggest that additional data may have been gathered on ocumentation of the results ( i any) were not available. Note that lead and ed pollutants under the municipal waste combustor MACT rules. Other metals rsenic, and beryllium. Other sources of multi

d by Rig0 and ChandIer as well as the EPA EM demonstration program.

ta for Antimony, Arsenic, Beryllium, Cadmium, Chromium, and Lead Tables 18 through 23 summarize the available Method 29 data for antimony, arsenic, beryllium,

, and lead respectively. In these tables, some difficulty may be experienced umber designations for data from the EPA pilot scale tests and for the EPA

able 22). Tables 18 through 23 utilize the run number designations provided in ce. The difficulty is1 that the same run numbers are repeated. Separate runs early come from different testing series but insufficient information ly describe the differences between run series or why run designations were 8.1. I Antimony taFor antimony, data is available from the Rig0 and Chandler tests and from the EPA pilot scale tests. The Rigo and Chandler tests provide dual train data in the 30 to 80 pgldscm concentration range. The EPA pilot scale tests were also dual train. They provide data in two different concentration ranges; a lower range of approximately 20 to 30 pg/ dscm and an upper range of approximately 60 to 90 pgldscm. The outlier analysis indicates that run number 2 from the 119 Table 18. Method 29 Multi Trai eviation For t Table 19. Method 29 Multi Train Data and Standard Deviation For Arsenic 121 Table 20. Method 29 Multi Train Data and S dard Deviation For Beryllium I I I I I I / Table 21. Method 29 Multi Train Data and Standard Deviation For Cadmium I23 Table 22. Method 29 Multi Train Data and Standard Deviation For Chromium 124 Table 23. Method 29 Multi Train Data and Standard Deviation For Lead 125 EPA pilot scale tests ( first of EPA pilot tests designated as run 2) has an abnormally lar, we data range. After examining several data grouping, it was concluded that this run represents a data outlier and accordingly, it was ehinated from the precision analysis. Figures 40 and 41 prdsent the antimony data in graphical form. Figure 40 is a scatter plot of the small sample standard deviation data while Figure 41 deviation. As sh s to approximately '

30%.

8.1.2 Arsenic Dora Multi train data sour standard deviation dat relative standard d on the Pittsfieid M limit for the laborat  scale tests. Those have been jnclwd Iow levels to h DataMulti train data for cadmium emissions u three sources. Tests by Rigo and Chandler provide data pg/ dscrn. The EPA pilot scale tests provide data in two ranges; a low range centered at about 20 ge of about 20 to 50 / ,

I i i j

m N +

+

+

 I .+ I * I b m I I '

I I I 7 m 0 127 t 0 128 v, cu . . a m 0 N  7 El 1 .. 1 I  t I29 0 0 0

0 o\

0 00 0

0 \ 3 0 v)

0 u

0 m 0 N 0,

0 I I 0 M CJ L 0 z i cy, 131 a 2 a CD Li + I 132 r: ti F t u pg/ dscm and an upper range that spans 40 to 80 pg/ dscm. The EPA method validation tests provide data at a very low concentration range of about 1.5 to 2.5 pg/ dscm. AI1 data in this data set pass the SPC outlier criteria and have been included in the precision analysis. Figures 46 and 47 present scatter plots of the data including the small sample bias corrected standard deviation and relative standard deviation as a function of mn average concentration. Figure 47 shows that, for concentrations above about 20 pgldscrn, the RSD covers approximately the same span as the other Method 29 metals presented above. However, the low concentration data from the Method validation tests indicate significantly higher RSD. Data in Figure 46 show that the actual standard deviation tends to increase with increasing concentration, even at the very low concentrations of the validation tests. This shows that the significant rise in RSD is a result of the denominator in the RSD calculation tending toward zero rather than rapid expansion of the standard deviation.

8.1.5 Chromium Data Three sets of multi train data are available for assessment of the precision of Method 29 for chromium. The Rig0 and Chandler tests provide data with concentration in the range of about 4 to 10 pgidscm. The EPA pilot scale tests again provide data in two ranges.

The low range results are centered at about 20 pg/ dscm while the high concentration results cover the range of about 60 to 70 pgldscm. Finally, the EPA Method validation tests are at very low cadmium concentration  ranging from about 1 to 3 pddscm. Outlier analysis for this data set indicates that two data points have abnormally large data spreads. Specifically, run number 7 from the Rigo and Chandler tests and run number 8 for the EPA method validation tests. Both of these runs are specially marked in the run number column of Table 22. The Rigo and Chandler run was eliminated from further analysis, while only the data from train C of the EPA tests were eliminated.

Note that there were several other data points indicating very large spreads but they have all been included in the analysis. Figures 48 and 49 present scatter plots of the small sample bias corrected data for standard deviation and relative standard deviation as a function of run average concentration.

8.1.6 Lead Data Three sets of multi train data are also available for Method 29 measurement of lead. The Rig0 and Chandler tests at the Pittsfield MWC provide data at lead concentrations ranging from about 400 to 1500 pgldscm. The EPA piiot scale tests provide data in two concentration ranges. The low range data cover the span of about 20 to 30 pg/ dscm . ~ ^ ? >

133 JL ; si m 3 f m I +

+ + + +

+ + +

I 134 n u bl, e.

. f""

I,. E w I I i I I B E 0 VI 0 w 0 m 0 v) 0 J ` 136 P r F L r t 11 ... F""

L r t.

IFLc" 4 I h. 2 137 whiIe the higher concentration results were obtained in the range of about 50 to 95 pddscrn. Finally the EPA Method validation test result ere obtained at concentfations in * e range of20 to 4o pg/ dscm. AI1 data in the lead data set pass the plots ofthe small sampl ias corrected stand average concentration. Note that the high co exhibit RSD values less than 13%, while th data exhibit a broader range of RSD values. from the Rig0 and Chandler 8.2 EPA Method 29 Regression Analyses After elimination of outliers, the Method 29 multi train data for antimony, arsenic, beryllium,

cadmium, chromium, and lead were analyzed with weighted regression analysis. Results from those analyses are summarized in Table 24 below.

Table 24. Results of Method 29 Regression Analysis for Various Metals There are clearly differences between the regression equations for the six metals listed above but there are also significant similarities. The variation of ndard deviation for versus concentration appears to be significantly different from the other metals. For cadmium, the maximum potential value for the p term in the p 0.61 1. For any of the other five metals, within t coefficient could fall between 0.692 and 0.929. Five of the six regression analyses produced t ction7 at the 95% con confidence bounds, t 158 I39 c c statistics that are above the critical value of that statistic. For antimony the t statistics ( at the 95%

confidence level) is slightly below the critical value and the t statistic for beryllium is only marginally above the critical vaiues. This results in wide ranges for the potential values of the power function p coefficient for both antimony and beryllium.

Recall that, with Method 29, each data pair provides data on the full range of metals. With the exception of the EPA Method validation tests, every data point, for each metal, has a companion data point for the other five metals. This is important since random error enters the measurement process through both the sampling process and the chemical analysis processes. In a multi metal procedure such as Method 29, random errors in the sampling process should be reflected in every metal being monitored. Moreover, when the value of the power coefficient is close to 1.0, random error in the sample collection process is a significant contributor to the overall Method precision.

For these reasons, there is increased reason to anticipate similarity in the various relationships between standard deviation and concentration.

Figures 52 through 57 illustrate the regression equations and the 95% confidence intervals for measurement of each of the six metals using Methud 29. Figures 58 through 63 present data on the various precision metrics, assuming that the standard deviation varies according to the regression equation. It is instructive to compare the general level for the predicted relative standard deviation and the variation of RSD for each of the six metals. Figure 58 indicates that the RSD for antimony measurement using Method 29 is basically a flat function of concentration over the entire range of available data. At a true concentration of 20 pddscm the predicted RSD is 11.8% while the predicted RSD is 9.3% at 90pg/ dscm. In the concentration range between about 20 to 100p~ ldscm,

a relatively flat RSD versus concentration trend is also observed for arsenic, beryllium, and chromium. In that range, the RSD for arsenic varies between about 15 and 16.3% ( see Figure 59).

For beryllium, RSD only varies between 17.0% and 17.8% ( see Figure 60). SlightIy greater variation is predicted for chromium but, as shown in Figure 62, the anticipated variation in RSD is only from 21.3% at 20 pg/ dscm to 18.2% at 71 pddscm.

141 142 1 143 . . d . E .  I L 9 3 = . n t 1 0 144 I45 a 0 0 0 c r: I ' l. a. & 0 cr 0 3 .

I47 I 0 E . I 0 o\

0 00 0 P 5 VJ 9 a M I 1 v3 L i 0 QI 0 01 0 b 0 W 0 Wl 0 d 0 m 0 N 0 I 0 I 149 0 v)

c J . , I s 0, 0 151 s 0 3 3 z 0 rc 0 \ 3 0 In 0 Q

0 m 0 r 4 2 0 E 0 m = \ M a I I m u L c .. I 2 ~

. I  4 k F;;; 1 0 i

k:. l s s 0 0 d m ( u s 0 0 0 rn .

0 0 d 1 0 0 c\ 1

0 0 0 . 1 0 0 W 0 0 Q 0 0 d 0 0 c\ 1 0 _. t anticipated when concentration drops below 20 p, a/ dscm.

As noted earlier, the data in Figures 58 through 63 are based on the assumption that standard deviation varies according to the re ssion equations. However, within g5% confidence bounds, it is distinctly possible that standard ation be greater Or lines. Figures 64 through 69 present the anticipated ranges for 99 out of 100 future a function of the true stack concentration, assu the upper 95% confidence limit, according to 95% confidence limit. These figures illustrate the critical importance overall precision assessment. As indicated by results presented in Fisures 5 deviation of Method 29 antimony measurements varies according to the out of 100 future measurements ( at a stack concentration within 23% ofthe true stack that future data might have a spread as large as + 5 1.9% a ng that that standard deviation varies according to e regression The significance of the unce more dramatic for Method 29 measurements of be Method 29 beryllium measurements are made at true stack concentrations of 80 pg/ dscm, available data indicate that 99 out of 100 future measurements Wil concentration ( 45 to 115 pg/ dscm). Also, 99 out of 100 triplicate measurements are expected to fall within k 25.3% of the true stack concentration ( when true concentration equals 80 pddscm).

However, at the 95% confidence level, it is only possible to conclude that the future single measurements will fall within f: 97.5% of the true concentration. This large spread between our best estimate and the 95% confidence limits is a direct result of the limited amount of multi train data for beryllium using Method 29. The potential range for fu er metals is standard deviation concentration is even As indicated in Figures 6o and 66y when within 43 8% Of the true illustrated on the respective figures. W E . I 5 155 0 0

0 o\

0 00 0 b 0 \ 3 0 v,

0 d 0 m 0 P 4 0 i i"" PP f .). 5 157 M E 3 a f 0 U rn . A ki

p\ 1 L,

E L, .1 0 m 0 ri 0 I 159 160 i_

Earlier discussion of the results presented in Table 24 noted a similarity in the regression analysis for five of the six metals, including antimony, arsenic, beryllium, chromium. and lead. It was pointed out that Method 29 collects a sample that is subsequently analyzed for all of the target metals.

Random errors in the sample collection and sample recovery operations are likely to contribute similar random error to each of the measured metal concentrations. Finally, it was noted that random error in the sample collection and recovery process tend to drive the value of the power function coefficient toward 1.0 while random error in the analytical analysis are more typically characterized by constant standard deviation ( p equals zero in the regression equation). In Table 24,

note that with the exception of cadmium, the regression analysis for five metals indicates that the p .

coefficient is between 0.703 and 1.039 sugsesting that random error in sampling and recovery are major contributors to overall megsurement imprecision.

The general similarity in these resression results further suggests that for these five metaIs, it may be appropriate to assess a composite precision estimate for Method 29. To perform that assessment, the multi train data presenred in Tables 18, 19, 20, 22, and 23 were combined into a single data set.

With the exception of the multi train data for tead, the majority of the individual data sets were from tests where the metal concentrations ranged from single digit to less than 100 pg/ dscm. Further the majority of the lead data are from this same concentration range. To form a composite data set for regression analysis, the Rig0 and Chandler data were eliminated from the lead data yielding data for all five metals in the same concentration range. The data were appropriately weighted for the number of degrees of freedom and then subjected to a regression analysis. The resuItant regression equation has 158 degrees of freedom and produced a t statistic equal to 10.66  well above the critical t statistic. The analysis suggests that the ' relationship between S and concentration is described by the equation:

The value of the p coefficient ( 0.821) is easily within the 95% confidence ranges forp determined by regression analysis for each of the five individual metals ( see Tabte 24). Figure 70 is a plot of the regression equations for the composite data set including the 95% confidence intervals on that 161 regression. Figure 71 presents a comparison of the re, oression equation for each of the five individual metals and the regression equation from the composite analysis,

It is important to reiterate that only a limited body of data is available to assess the precision of Method 29. Moreover, one of the critically important data sets, the EPA pilot scale tests, contains a known bias, Inchdins these biased results in the analysis clearly results in a slight over estimate of the standard deviation at any given stack concentration ( partico However, if those data were eliminated from the assessment, the result would be an even larger aver estimate of the Method's imprecision. As a result of the limited quantity of data and the known biases in some of those data, the precision of Method 29 for metals can only be generally estimated.

The analysis of the composite data set provides a basis for the overall assessment of the method's precision. Figure 72 presents various precision metrics determined from the regression analysis of the composite d t. Based on the available data, it appears that Method 29 provides an RSD between 13 and when the metal loading is between 20 arid about IOOug/ dscin. At ineta1 concentrations b about 10 pg/ dscm, the imprecision of the method appears to increase asymptotically. Relative to future measurements, if the precision of Method 29 conforms to the composite analysis and if the metals concentration is greater than 20 pg/ dscrn, 99 out of 100 single measurements should deviate from the true concentration by no more than 45%. Similarly, in the same rage, 99 out of IO0 triplicate measurements should deviate from the true concentration by less than 26%. Unfortunately, the available data do not support a more definite assessment.

Tabie 25 provides a summary of the anticipated range of measurement results for application of Method 29 for determination of the concentration of antimony, arsenic, beryllium, chromium, and lead. Data in this table are derived from analysis of the composite data set and cover the concentration range of 4 to 100 pg/ dscm. Note that the data in the table do not include correction for oxygen content in the stack.

163 , 6 l e 0'

rl P 2 E f ED 0 0 c r  5:

a 3 + M c 164 i;

P I65 I Table 25. Range of Anticipated Future Metals Data Antimony, Arsenic, Beryllium, Chromium, or L I I66 9.0 Other Measurement Methods.

1 An attempt was made to gather multi train data for a variety of additional EPA measurement methods. No multi train data or method validation test repons were uncovered for EPA methods 0030 or 001 1 for volatile and semi volatile organic emissions. The same can be said for EPA Method 23a, which is a special procedure for measuring dioxin and furan emissions. In the absence of multi train measurement data, it is not possible to fully assess the precision of these methods.

167 This page Intentionally left blank.

168 "~

10. Conclusions The ReMAP study has assembled a database containins all known multi train data sets using various EPA measurement Methods. Data in that database have been subjected to a detailed analysis to assess the precision of the Methods as a function of stack concentration. The scope of the available data is Cxtremely limited, especially considering the importance of results from application of the Methods. Certain of the Methods, especia Methods 5 and 5i have significant databases and the precision of those methods is relatively well established. Using relative standard deviation as the precision metric, the precision of Method 5 is between 5 and 1 I% when applied to stacks with a broad range of concentrations. If applied with attention to detail, this method is capable of providing reasonably precise resuits, even at stack particulate concentrations as low as 15 mg/ dscrn. Whether an RSD of 5 to 1 I% is sufficiently precise is likely to be application specific.

Methqd 5i was specifically developed to provide precise measurement resuIts for particulate matter concentrations below 50 mgdscrn. Based on available data, this precision of this Method has no statistically significant variation with stack concentration. Pooled anaiysis of the data indicates that the Method ( when applied at concentrations between about 5 and 50 mg/ dscm) has a characteristic standard deviation of 1.43 mg/ dscm.

Method 23 for dioxin and furan emissions is a critically important method for current EPA emission regulations and for public perception of risk associated with emissions from combustion facilities.

There is only a very small database for assessment of the precision of this Method. Based on available data, the anticipated RSD for measurements of the total mass of tetra through octa chlorinated dioxin and, furan is estimated to be between 6.3 % and 20% for stack concentrations in the range of 2 to 27 ng/ dscm ( higher RSD at lower concentration). Recall that the stated range of stack concentrations are given on an " as measured basis" and therefore do not include excess air diIution correction factors ( e. g., correction to 7% 02). The anticipated range for 99% of future individual measurements is t 2.57 times the standard deviation.

Method 23 is also used to determine dioxin and furan emissions calculated as ITEQ. Analysis of available data for emissions expressed in this manner did not yield an accep~ able regression 169 expression. That is, the t statistic for the regression was less than the critical t statistic at the 95%

confidence kvel. This implies that stron sociated with application of Toxic Equivalence Factors may be masking k weighting factors, variation between simultaneous significantly amplified relative to differe data and when used to dete variation of Method performed to determine a characteristic Stan the pooled standard concentration range 0.02 to 0.9 ng ITE confidence limit, the possible range of Method 23 data point indicates stack concentration that data point ( at the 95% confidence level) could be as low  Moreover, using the limited quantity of currently available data and at the 95% confidence level, it is not possible to determine compliance with a PCDDPCDF emission limit below 0.095 ng ITEQ/ dscrn. on the avaiIabie ote that, at the 95%

Method 26 for hydrochloric acid measurements was found to be as precise as any of the manual measurement methods. Typically, the RSD for this and lo% RSD does increase at very low HCI concentrations but the regression analysis suggests that RSD should only increase to 15.9% at concentrations as low as 1 mg/ dscrn.

Mercury emission measurements are also regulations. A relatively large array of mult analysis. The analysis of results found that the and 101 had minimal variation with ppldscm the measurement method RSD varied from 9.6 to 12.4 percent. A significant portion of the overall database is for measurements at relatively low mercury concentrations. Forty out of 73 mercury data points had average concentrations below were at average stack concentration less than 30 pgld data was available for the ReMAP ercury measurements by Methods 29 As concentration drops from SO to 5 pg/ dscm, the ReMAP analysis shows that RSD is anticipated to increase from 12.4 to I 5.4%.

170    Method 29 is also used for measurement of other metals Precision analysis was completed for six other metals including antimony, arsenic, beryllium, cadmium, chromium, and lead. The analysis shows that five of the six metals all behave similarly with respect to measurement method precision.

Data for all metals except cadmium exhibit a standard deviation versus concentration relation where the power function coefficient ( p) has a value of approximately 0.82. Composite analysis of data for this group of metals suggests that Method 29 provides an RSD on the order of I3 to 18% \ vhen the metal loading is between 20 and about IOOpg/ dscm. Data for lead is available at much higher concentration and the method RSD for lead appears to asymptote between 5 and 10%. At metal concentrations below about 20 pg/ dscm, the imprecision of the Method appears to increase asymptotically. For cadmium, based on the available data standard deviation of the Method has a different relation with concentration. The indicated value of the p coeflicient is approximately 0.45 suggesting that for the available cadmium data, random error ( or differences) in the chemical analysis was a significant contributor to the overall imprecision of the Method.

One additional conclusion from the ReMAP Phase 1 project is that there is a pressing need for additional multi train data to refine the precision estimates of the  PA Reference Methods. If such experimental programs are to be conducted, significant attention should be given to the appropriate range of stack concentrations. Results presented in this report can help to guide the test planning efforts. 1. nn A C 9. EPA Reference Method 23  Determination of Polychlorinated Dibenzo p Dioxins and Polychlorinated Dibenzofurans from Stationary Sources. 56 FR 5758, February 13, 1991 ( with several revisions).

10. Validation of Emission Test Method for PCDDs and PCDfs. Prepared by Midwest Research Institute, EPA Contract No. 68 02 4395, Work Assignment 23 for  PA EMSL. February 24, 1989.

Rig0 & Rig0 Associates and A. J. Chandler and Associates under the Direction of ASME. June 1 1. Retrofit of Waste to Energy Facilities Equipped with Electrostatic Precipitators. Prepared by 1997.

Aggregate Kiln. Prepared by Energy and Environmental Research Corporation, Contract No. 68 D2 0164 for the  PA Office of Solid Waste. October 10, 1997. 12. Dioxins/ Furans, HC1, C12 and Related Testing at a Hazardous Waste Bumin, ( J Light Weight v 13. EPA Reference Method 23  Determination of Polychlorinated Dibenzo p Dioxins and Polychlorinated Dibenzofurans from Stationary Sources. 56 FR 5758, February 13, 199 I .

14. OMSS Field Test Report on Carbon Injection for Mercury Control Completed at the Ogden Martin of Stanislaus, Inc. Prepared by Radian Corporation, Conrract No. 68 D i 00 IO for the EPA Office of Research and Development. September 1992.

15. EPA Reference Method 29  Determination of Metals Emissions from Stationary Sources. 61 FRl8262, April 25, 1996.

16.  PA Reference Method 1 0 1 a  Determination of Mercury Emissions from Sewage Sludge Incinerators. 47 FR 24703, June 8, 1982.

17. Proposed Draft EPA Method 10 1 b  Determination of Mercury Species from Stationary Sources.

18. Validation of Draft Method 29 at a Municipal Waste Combustor. Prepared by Radian Corporation, Contract No. 68 D9 0054 for the  PA Emission Measurements Branch. September 30, 1992.

19. Mercury CEMS Demonstration at the HoInam, Inc. Hazardous Waste Burning Cement Kiln in Holly Hill, SC. Energy and Environmental Research Corporation for EPA Ofice of Solid Waste and Emergency Response. October 1997. ( also see 62 FR 67788).

20. Internal EPA Study at the Pilot Scale Rotary Kiln Incinerator Located in Research Triangle Park, NC. SarnpIing and Analysis performed by  PA.

21. Emissions of Metals and Organics from Municipal Wastewater Sludge Incinerators. Prepared by Entropy and DEECO, Contract No. 68 CO 0027, for EPA Risk Reduction Engineering Laboratory.

\ I y c ,

c  i Appendix Statistical Analysis Procedures for the ReMAP Program Prepared by:

Charlie Hendrix Statis tical Consult ant r I," u Procedures for Analyzing Simuitaneousfy Sampled Concentration Data to Determine Measurement Precision ( Random Error)

This appendix is concerned with the statistical methods used in the ... b 1 analysis of simultaneousiy sampled data.

Confidence Interval on a Mean  # ¶ W i  ab+ Suppose w e have collected the following replicate data from a process operating under fixed conditions.

103.2 107.9 101.6 109.1 105.3 The average of these is 105.42; the standard deviation is 3.132 with 4 degrees of freedom ( df). If these are representative data from this process, then the 95% confidence interval on the true mean p is 105.42 of freedom ( df). t = 2.776. The 95% interval on p is 105.42 3.89 or 101.53 to 109.31. " 95% of the intervals calculated in this manner will encompass the true mean, p" From this we infer that " the probability is 95% that the interval 101.53 to 109.31 has captured the true mean, p" t* 3.132/& where t is the reference value of t with 4 degrees cerns about whether the data came from a non normal little bearing on this calculation under most Confidence Interval on a S tandard Deviation The following replicate data also came from a process operating under a fixed set of conditions.

1.03 1.24 0.91 1.36 0.97 1.22 The standard deviation of these data is S = 0.177 with 5 degrees of freedom ( df). If these are representative data from this process, then the 95% confidence interval on the true standard deviation a is calculated in the following manner. Go to Tabie 1; ent with df = 5; find the factors 0.624 and 2.453 under the heading " For 95% onf. Int." Multiply 0.177 by each of these factors to obtain 0.110 and 0.434. df 1 2 3 4 5 6 7 8 . 9 10 12 15 20 25 30 50 100 200 500 paae 2 For 95% Conf. I n t .

PF il`

0.566 3.727 0.599 2.875 0.624 2.453 0 .644 2.202 0 661 0.675 I. 826 1.755 0.717 1.651 0.73 1.548 0.765 1.444 0.785 1.380 1.337 1.243 0 * 942 1.066 " 95% of the intervals calculated in this ma standard deviation 0.'' that the interval 0.110 to 0.434 has captured the true standard deviation sigma ( o)". This assumes that the data c underlying " true standard deviation o" wh accuracy of confidence intervals on CT is affe original data. er encompass the true e from a process with an From this we infer that " the pro biiity ' is 95y0 Table 1 is derived from the Confidence Interval on Siama  M u l t i d e Measures of Variation Suppose we have data from six tests. Each pair of data was obtained by drawing simultaneous samples and analyzing those samples for the concentration of a pollutant.  Time Data S t a n d a r d Deviation. S df 1 1 1 1 1 1 2: 30 PM 27.2 30.0 1.980 5: 50 PM 20.9 27.2 4.455 6: 25 PM 28.7 33.1 ' 3.111 7: 15 PM 25.5 24.2 0.919 4: 15 PM 19.1 23.7 3.253 5: 05 PM 28.3 26.8 1.061 *"^ . . .). , . , . s I /. < _ paae 3 We are concerned with the inherent variation due to the sampling and analysis procedures ( measurement precision), measured as the variation within tests. The standard deviation of each pair of samples is shown.

Our objectives are to ( 1) estimate the standard deviation a due to sampling and asurement precision) and ( 2) to calculate a 95% confidence int estimate of 0.

cal to average the standard deviations and report that as t h e estimate of 0. Unfortunately that average will be a of ci. This bias can be substantial. A more accurate estimate of CT is found by sed estimate iations. Pooling requires squaring the individual to obtain variances; weight averaging the variances to obtain the pooled variance; then take the square root to find the pooled standard devia ance is weighted by the number of degrees of freedom assoc variance. The number of degrees of freedom andard deviati is the sum of the degrees of ndard deviati Pooled Variance = ( Sum of df* Variance)/( Surn of df)

80) z + 1"( 3.253)* + ... + 1*( 0.919)*]/[ 1 + 1 + 1 + 1 + 1 + 11 = 45.99816 = 7.666 Pooled S = 67.666 = 2.77 with 6 df.

Pooled S is an estimate of G. We can calculate the 95% confidence interval on ( r by entering Table 1 with 6 df to find the factors 0.644 and 2.202, Multipl 2.77 by each of these factors to find 1.78 and 6.10. The probability is 95% th'at this interval has captured the true value of 0.

Can we calculate a confidence interval on CJ by treating the individual , 3.253, ... a 0.919) as " data"; calculate those " data" and then calculate confidence limits data" as if we were calculating the confidence The answer is a qualified " yes", but only after taking that have not been addressed at this point. d for the analysis of the R MAP data. Although the involves fitting the data to a mode by least squares and nderlying principle is founded on weighted ~

 ~     _  111

  __  _

paae 4 The ReMAP Procedu re The structure of the ReMAP data precludes direct pooling to estimate CJ h e primary data is re was designed t of deviations in e fact that G varies nonlinear least square method. ition to obtaining Tens of thousands of synthetic d were examined. The sirnutat scale of C ; sometimes Monte Carlo simulation estimates of G and also show that the con approximately as stated. in this manner. Cases of N = 12 pairs of data; N = 24 airs of data; and other cases r t vals on ( sigma) are This will be discussed later.

d PI i i ,   n 1, The ReMAP Model S = kCP Out of all the possible models that could be used to associate S with Cy why was S = kCp chosen? Is this mogel adequate for all of the pollutants? These questions ... and others concerning this model form,.. are discussed in more detail in Fittina Sta ndard De viation vs. Concentration Data to Alternative Models.

In choosing a model form, one of the first requirements is that the variation in the residuals ( residual = difference between observed and e constant along the scale of concentration. This is s variance". Simple graphics depicting S as a function requirement is not met in the ReMAP data ... at least in which those data span realistic ranges on the scale of C. Before fitting the data to a model we need to stabilize the variance.

. When the data are presented in Ln tn coordinates, statistical tests confirm that the variance is has been done by linearizing S = kCP to mogeneous along the scale of concentration.

Typical values of p range from 0 5 to 0.9. The exceptions were pollutants in which the concentration spanned narrow ra ges and the confidence intervals on p were wide. Since values of p have been associated with sampiing variation as opposed to variation due to chemical analysis, the power law model has merit aside from its statistical properties.

A fundamental difficulty with the ReMAP data lies in the fact that these data are poorly arranged along the scale of concentration. Rather than being concentrated at two . or three positions on that scale, the data are often ttered; sometimes concentrated near the centroid and sparse at the seldom focused near t h e ends of the concentration scale. Furthermore, data collected from one source are sometimes positioned near one level of concentration and data from other sources are positioned at other levels. in order to obtain precise estimates of p, we need data that spans decades ( factors of 10) on the scale of concentration; in reality there are instances in which the data bareiy spans one decade. These factors virtually preclude building models more complex than the form S = kCp. Where there were questions about this, the adequacy of this model form was testing by t h e usual stati tical procedures; in every instance it was found that a more complex mode could not be justified. ,

This is not to claim that a power law model will be adequate for ail The choice of model form is presently limited by the factors future data. noted above. paae 6 7 The Relationshi  mi n r d D v i i n A standard de sigma. Sigma is the When we estimate ( 3 truth" ( accuracy, unbia truth" ( precise; small If we calculate S from a using other data fro e interval arou If we collect two simultaneous samples, the standard deviation of that sample must be rnuitipli 1.253 to make it an unbiased estimate of u. With larger amounts o in the sample, the bias is s is a brief table of bias cur w 2 3 4 5 10 Bias Correction Factor 1.253 1.128 1.085 1 Remember each value of S estimate of a. r estimate of 6; Averaging values this average will be rn will still not be accur r"

r k c If we intend must multiply each before averaging. The bias relafed to the number of sample standard deviation standard deviation; it is rela individual sa mpie s ta nda i d de via tio n s tirnate of G, the as correction factor t the average sed to to get the ~~ An Examole Table 2 shows data in which the standard deviation varies with t h e average. Our purpose is to find an empirical relationship between S and the sample averages.' For N = 2, unbiased S = 1.253' s. 3%

" w,

' F & I

klrd 1 I Lr* r Table 2 Samo le Data Ava. c 1 893.7 1080.2 986.95 2 2240.4 2127.0 2183.70 3 2070.3 2219.7 2145.00 4 529.4 553.3 541.35 5 2351.1 2652.2 2501.65 6 358.9 342.7 350.80 1316.20 2434.95 2479.60 1434.80 743.6 655.9 699.75 1 3 960.3 1099.1 1030.00 1 5 247.2 297.6 272.40 16 1866.4 2247.1 2056.75 12 1356.8 1392.8 1374. ao 1 4 1949.0 1803.1 1876.05 S a n b  iS5 d S 131.88 565.25 a o . 19 Z30.48 105.64 122.37 16.90 51.18 212.91 255.78 1 1 . 4 6 14  3 6 2 0 7 . 6 1 250.14 161.43 202.27 13.15 16.48 26.87 33.67 62.01 7 7 . 7 0 25.46 3 1 . 9 0 98.57 123.55. 103.17 129.27 35.64 4 4 . 6 6 269.20 337.31 Fig. 1 shows the relationship between S and Avg. C. Fig. 2 presents the same data in Log Log coordinates. The nature of this relationship is more visible in Fig. 2 than in Fig. 1. These data will be fit to the rnqdel p after linearizing to Ln( S) = Ln( k) + p* tn( C). This is equivalent to working in Log Log coordinates as in Fig. 2.

By fitting the data in Table 2 to Ln( S) = Ln( k) + p* Ln( C) we not only obtain estimates of k and p but also certain statistics that tell us how accurateiy this model will predict G. A least squares analysis of these data is shown in Table 3. . . . . .

i a 2 . i i 3 E paae 8 0 0 0 0 0 0 rn ( u O 0 m 0 0 0 0 0 0 m 0 0 cu rn 0 m ( v P 7 0 0 0 rc) c3 0 w B L c [ ,

r l El ! j iJ P L i ' c5 Y p"

cn P 2 9 LY LJ w Table 3 RIT 0.05 & 0.01 = 2.17 & 2.98 VARIABLES SH OF COEFF T

0 Intercept 1 P o w e r C o e f f i c i e n t p = 0 0.34030 2.15 RESSUMSQ STDDEV OF RES DF 12.16245 ..

ics I n Ln U n i t s In O r i a i n a l Metr OBSVD PRED RESID STD RES O b s e r v e d P r e d i c t e d D i f f . 68.69 96.56 1 5.107 4.230 0.878 0.94 165  2 5 2 4.610 4.810  0.200  0 . 2 1 100.48 122.73  22.25 3 4.886 4.797 0.089 0.10 132.37 121.14 11.23 4 3.053 3 . 7 9 1  0.738  0.79 21.18 44.29  23.11 5 5.586 4.909 0.677 0 . 7 3 266.78 135.55 131.23 6 2.664 3.474  0,809  0.87 14.36 32.25  17.89 7 5.561 4.440 1.121 1.20 8 5.310 4.890 0.420 0 . 4 5 9 2.802 4.903  2.101  2.25"" 16.48 134.67  118.19 10 3.517 4.503  0.986  1.06 33.67 90.29  56.62 11 4.353 3.978 0.375 0.40 77.70 1 2 3.463 4.472  1.009  1.08 3 1 * 90 13 4.816 4 . 2 6 1 0.556 1 2 3 . 5 1 1 4 4.862 4.699 0.163 129.27 1 0 9 . 8 4 19.43 1 6 5.821 4,766 1.055 1.13 337.31 117.47 219.84 15 3.799 3.289 0.510 0 . 5 5 44.66 26.81 17. as RES SUMSQ FROM REGRESSION = 12.16245 RES SUMSQ DIRECT = 12.16245 A v e r a g e Observed = 122.3331 A v e r a g e P r e d i c t e d = 89.5749 B i a s C o r r e c t i o n Factor = 122.333 89.5749 = 1.3657 Ln( S) =  0.8093 + 0.731* Ln( C)

0.731 s = 0.445. c ln Table 3 the observed and predicted values of S and their residuals ( differences between observed and predicted) are shown in Ln units, then in their original metrics. Sample calculation:

From Table 2, first row: C = 986.95 Spred = E~ p( U. 8093)* C*~ 3~ = 0.445' Co 737 = 68.69 ( Table 3)

The average of the observed values is 122.3331; t h e average of the predicted values is 89.5749. The ratio of the average observed to average predicted is 122.3331/ 89.5749 = 1.3657. This offset or bias was caused by linearizing d fitting in terms of Ln( S) followed by conversion back to the original u The unbiased model for predicting CJ from C is:

0.731 0.731 Est G = 1.3657* 0.445* 6 Or Est 0 = 0.608. C where Est CY is an unbiased c 1 10 100 1000 BY compare for other estimate of sigma.

Fst ( r 0.608 60.8%* F( sD 3 . 2 7 3 2 . 7 0 * 17.6% 9.5% 17.62 94  8 2 k * extrapolation fitting all of the pollutant data to consistent models we can model coefficients for one pollutant against model coefficients pollutants. paae 12 vow Good is This Model?

The answer to t with the model. Let` model building process. on what we intend to do T CRIT 0.05 & 0.01 = 2.17 & 2 . 9 8 VARIABLES F COEFF T 0 Intercept 1 Power Coefficient . p = 0.73084 . 0.34030 2.15 . .

RESSUMSQ STDDEV OF RES DF R SQ 1 2 . I6245 0.93207 1 4 0.2478 The model coefficients are  0.8093 and 0.731. SE OF COEFF is a measure of how well we have estimated the coefficient, p. t = 0.73087/ 0.34031 = 2.15. A t ratio larger abou 2 0 implies we have detected a relationship between 0; and C. A t ratio 2 ( approx) means that CT is not a constant, but is associated with C. substantially larger than 2.0 are ecessary to a coefficient. The reference value t at the w ~ 5 is actually 2.1 5 . See the line: T CRIT 0.05 & 0 .  1 7 Ed 2  9 8 . So our observed value of t ( 2.15 as compared to the reference value 2.17) is " right on the edge". However, t ratios tely estimate a model 5% probability level)

The 95% confidence interval on the coefficient p is:

0.731 2.17* 0.340 or from 0.00 to 1.47 which suggests that the true coefficient eo d be as small as " zero" ( implying no association between C and 0) r as large as 1.47. It would be misleading to report p = 0.731 without the uncertainty in this estimate. If we compare a power coefficie r one pollutant to that of another, we must recognize that when t or P is modest, th confidence interval on that value of p m R SQ ( R squared) = 0.2478 means this model explains 4778% of the variation in t h e data. STDDEV OF` RES is t h e standard deviations of residuals. This is t h e standard deviation of the differences between observed and predicted values in Ln( S) units. This is not an estimate of 0, STDDEV OF RES is a measure of variation among values of Ln( S). It is also an estimate of the standard deviation that we would find if we could run replicate tests under a fixed set of conditions and report the variation among values of Ln( S). In this context " the data" is in terms of Ln( S). paae 13 A logical extension to this would be to inquire " how good is a prediction made from this model?" This suggests we will calcuiate the unbiased estimate of C ( Est 0) for a fixed value of C.... then calculate a confidence interval on that predicted value. Before doing this, here are two values that we will need. One of these is the average of Ln( C) and the other is average of the L ( s). Avg( LnC) 7,1116. Avg( LnS) = 4,3881.

The confidence interval on Est CT is best found by re stating the model in a different format:

Original Format: Ln( Est CT) =  0.8093 + 0.731* Ln( C)

New Format: Ln( Est 0) = Avg( LnS) + 0.731*[ Ln( C)  Avg( LnC)] New Format: Ln( Est o) = 4.3881 + 0.731*[ Ln( C)  7.11161 The variance of a prediction of Ln( o) is:

Var( Lno) = [ StdDev of Res12/ N + [ SE( Coeff) 12*[ Ln( C)  7.1 11 6j2 Eq. 1 Table vslue of t I T CRIT 0.05 & 0.01 = 2.17 I & 2.98 VARIIBLES COEFFICIENTS SE OF COEFF T RATIO I 0 Intercept  0,80 932 RESSUMSQ STDDEV OF RES DF R SQ 12.16245 0.93207 14 0.2478 1 Power Coefficient p = 0.73084 0.34030 2.15 I I I

t ratio calculated r N = 1 6 f r o m data L* i Var( Lno) = [ 0.9321 ]*/ I6 + [ 0.3403]**[ Ln( C)  7.1 1 16J2 Var( Lncr) = 0.0543 + O. l158*[ Ln( C)  7.111612 P LA SE( Lno) = dVar( Lno) predicted value of Lno, also called the standard error of This is the standard deviation of the prediction. See note at the bottom of page 14. _. "~ paae 14 Here is t h e sequence for calculating a 95% confidence interval on CT:

Step 1: Calculate the predicted valu either of:

Ln( Est CT) =  0.

Ln( Est a) = 4.3881 +. 0.731*[ Ln( C)  7.1 1161 SE( Est 0) = dVar( Est 0) See footnote.

Step 3: The 95% confidence interval on Ln( Est 0) is then Ln( Est G) t* SE( Est ) where t is the table or reference value of t at the 0.05 level of significance. t = 2.17 in this instance. r L'

Step 4: Revert to the orig I units and the unbiased estimates:

Take antiin of Est CT Take antiLn of the lower bound Take antiLn of the upper bound Multiply Est 0, the lower bound, and the upper bound by the bias correction factor 1.3657 Est G and t h e 95% confide e On are shown in 4

The expression Standard Error ( Symbol: SE) means the Standard deviation Of a StatiStiC.

Literally, the standard deviation of a value calculated from data. This is Stan designed to keep issues about " t ~ e standard deviation" ( calculated from " the from issues about the standard deviation of other quantities calculated from data. 4 LOWlS U? p? Z A V P T ; I ~ ~ C LiFir t s t CT Limbic 10 0 . c93 3.254 118.073 20 0.249 5 . 4 2 5 118.174 50 0.946 10.600 118.756 100 2.583 17.594 119.830 500 24.797 57.058 131.283 1000 55.878 94.704 160.5C9 zoao 84.433 151.189 292.637 200 6.979 29.203 122.537 5000 96.804 307.125 974.353 A graphical analysis of this table is presented in Fig. 3.

Minimiz, ina t h e Width of the Confidence Interval Equation 1 ( repeated here) determines the width of the confidence interval ' on the estimated value of sigma, Est G.

Var( Est a> = [ StdDev of ResI2/ N + [ SE( Coeff)]**[ Ln( C)  AvgLn( C)]' Eq. 1 Whereas Est CT is a measure of variation due to sampling and analyzing, the StdDev of Res ( standard deviation of residuals) is a measure of the variation among individual values of LnfS). Reducing the variation in sampling and analysis methods would reduce t h e magnitude of probably reduce StdDev of Res as well. A reduction in StdOev reduce the width of the confidence intervals.

N is the " number of tests" ( number of duplicates, triplicates, quads, etc.); literally t h e number of rows in the data table from which we fit S vs C. Increasing N will decrease [ StdDev of Res]*/ N and will therefore decrease the width of the confidence interval.

SE( Coeff) is strongly influenced by the span of t h e data along the scale of C, the concentration of a pollutant. Varying C over a wide range will decrease SE( Coeff) and will therefore decrease the width of the confidence interval. concentrated at a " low value of C" and one half of the data are concentrated at a " high value of C", then SEfCoeff) will be minimized. SE( Coeff) is also reduced by increasing N and by reducing StdDev of Res. Furthermore, if about one haif of the data are [ Ln( C)  AvgLn( C) J2 is a function of where the prediction is being g the scale of C. When a prediction is made at the centroid of tn( C) = AvgLn( C) and this term goes to zero. When this is so, then Var( Est 0) = [ StdOev of Res]*/ N. This means that the width of the confidence interval will be minimized when predictions are made at the centroid of Ln( C). .

The best way to visualize this is with an example in which data are concentrated at two points on the scale of C. The data in Table 5 are simulated data from the same source as the foregoing example.

Data Sample 1 243.3 2 5 3 . 6 2 237.3 233.7 3 151.1 143.5 4 2p0.4 216.6 5 1 7 3 . 1 207.6 6 141.8 159.0 7 233.2 208.4 8 139.9 153.6 9 1733.4 1828.4 10 2022.6 1754.9 11 2127.9 2014.9 12 2064.0 2047.6 13 2016.6 1934.1 1 4 1s 1 6 2003.9 1808.6 Ava. C s 147.36 5.37 248.45 7.28 235.50 2.55 233.50 23.90 190.35 24.40 150.40 12.16 220.80 17.54 146.75 9.69 67.18 1780.90 1888.75 189.29 2071.40 79.90 2055.80 11.60 1975.35 58.34 1822 .) 2 0 77.78 2326.75 233.98 1906.25 138.10 gzbiased s 9.12 3.20 6.73 29.95 30.57 15.24 21.98 12.14 84.18 237.18 100.11 14.53 73.10 97.46 293.18 173.04 I is, Table 6 is t h e feast squares analysis of the data in Table 5. The relationship between S and C is shown in Figures 4 and 5.

i x I I L Se are equivalent to the lower and upper 95% confidence ! inten/ af on the slope in Ln Ln coordinates; i. e., on the model coefficient p. Recall that the , L lower limit on p was 0.00; hence " A" is horizontal. This may aid in seeing how the imprecision in estimating the slope impacts the confidence interval on predicted values of sigma. This imprecision in estimating the slope is reported as SE of Coeff. pz"* i paae 18 . . . . i

f? ~ i ~  ~

~. ~ ~  __

paae 20 T a b l e 6 T a b l e = slue of t I T CRIT 0.05 & 0.01 = 2.17 & 2.98 VAR IA9 LES COEFFICIENTS SE OF COE? F T RATIO 0 Intercep I 1 Power Co 0  18436 4.73 RESSUMsQ STDDEV OF RES I . " 10.51455 I fficient p =

. .

I t  r a t i o calculated StdDev of Res N = 16 from data I n Ln U n i t s I n Oriainal Metrics OBSVD PRSD RESID STD RES O b s e r v e d Predicted D i f f .

1 2.210 2.791  0.580  0.67 9.1 16.29  7.17 2 1.163 2.744  1.581  1.82 3.20 15.55  12.35 3 1.907 2.335  0.429  0.49 6.73 10.33  3.60 15.43 14  52 4 3.400 2.737 0.663 0.76 29.95 5 3.420 2.559 0.861 0.99 30.57 12.92 1 7 . 6 5 6 2.724 2.353 0.371 0.4 15.24 io. 52 4  72 7 3.090 2.688 0.402 0  4 21.98 ' 14.70 7.28 8 2.497 2.332 0.164 0.19 12.14 10.30 1.84 9 4.433 4.506  0.0t3  0.08 84.18 90.60  6.42 10 5.469 4.558 0.911 1.05 237.18 95.36 141.82 If 4.606 4.638  0.032  0.04 100.11 103.34  3.23 12 2.676 4.631  1.955  2.26** 14.53 102.66  88.13 73.10 99.16  26.06 13 4.292 4.597  0.305  0.35 14 4.579 4.526 0.053 0.06 92.42 5.04 15 5.681 4.739 0.941 1.09 114.36 178.82 16 5.154 4.566 0.588  68 173.04 96.13 76.91 RES SUMSQ FROM REGXESSION = 10.51455 RES SUMSQ DIRECT = 10.51456 Average O b s e r v e d = 75.1069 A v e r a g e P r e d i c t e d = 56.2545 B i a s Correction Factor = 75.1069/ 56.2545 = 1.3351 0.871 Est c = 1.3351' 0.1335* C t r t 0.871 Est CJi = 0.178' C The 95% confidence interval on p is:

an improvement over 0.00 to 1.47 in the previous example,

New Format: Ln( Est G) = Savg + 0.8712*[ Ln( C)  Cavg] New Format: Ln( Est 0) = 3.58126 + 0.8712+[ Ln( C)  6.422821 The variance of a prediction is ( Eq. 1 is repeated here):

Var( Est 0) = [ StdDev of Res]'/ N + [ SE( Coeff) J2"[ Ln( C)  Avg of Ln( C) l2 Eq. 1 t 0) = [ 0.86S63J2/ 16 + [ 0.18436j2'[ Ln( C)  6.422821'

Var( Est 0 ) = 0.04694 + 0.0344*[ Ln( C)  6.42282J2 Note that lSE( Coeff)]* = 0.0344 compared to 0.1158 in the previous exampfe. By concentrating data at the extremes of the experimental space ~

the width of the confidence interval has been reduced significantly. Est oand the 95% con ence on are shown in Tab, e 7.

Table 7 Lower Averace c . L i m i t 10 0.236 20 0.564 50 1.765 100 4.125 500 24.812 1000 43.978 5000 113.262 200 9.375 2000 68.641  1: 324 2.422 5.381 9.843 18.004 40.000 73. 1 6 7 133.836 297.342 Upper Limit 7.421 10.400 16.402 23.484 34.577 64.235 121.723 260.95: 780.600 A graphical analysis of Table 7 is presented in Figure 6 . ..

e . I v3 O I I I 0 0 paae 22 The simulated data used in these examples ( Tables 2 and 5) came from this model: 0 = 0.28* C0* 8. The value of C was varied randomly in Table 2 and varie domly around two points on the scale of C in Table 5. calculated from C; from a Source with mean C and n 0, two random normal numbers were generated; those numbers are the data. This process was repeated 16 times. When C = 100 the true RSD = 11 . WO; when C = 1000 the true RSD = 7.0%. These levels of ical of those encountered in the analysis of t h e actual The model derived from data in the first simulation ( Tabfe 2) is Est a = 0.608* C0* 73' The 95% confidence interval on p is 0.00 to 1.47.

The model derived from data in the second simulation ( Table 5) is Est 0 = 0.178* C0 871 The 95% confidence interval on p is 0.47 to 1.27.

With this in mind, here are some important conclusions.

Even when simultaneously sampled data come from a " perfect" situ ation such that:

e> the underlying model is exact and the data are contaminated only by random variation C> there are no concerns about sample contamination, selective or biased sampling of particles, or other " special causesy'

finding a relationship between Est CT and the average concentration C yields model coefficients and predictions that are subject to statist ica 1 u nc e rt ai n t y .

The evidence of this uncertainly is illustrated by the fact that two sets of data from a " perfect situation" produce models that have different coefficients and therefore different estimates of Est CF as a function of the average concentration C.

. SE( Coeff) is a major contributor to wide confidence intervals Although not discussed in detail here, SE( Coeff) is on Est 0. strongly influenced by ( 1) the allocation of experimental points along the scale of C; ( 2) the amount of data: and ( 3) the inherent variation in t h e data. Weiqhted Rearession Although the majorit samples with octets. Reca When those values are derived fro should be weighted in acc information in each value As a rule, t h e qua 21 there were insta is Q = Sum of W, t( S,  pred Si) 2 where Si is an observed standard deviation; pred Si is the predicted is the weight assigned to the i th inversely proportional to the varia approximation, Minimize Q = Sum of ( 1/ Var Si)'( Si  pred Si) 2 The variance of Si is ( an approximation) proportional to O be assigned to each servations used to a as The eight assigned to each of that standard deviation; and Wi weights should be 1/[ 2( N  I)]. Thus the re1 is 2( N  1) where N is th given value of S. This is observed valu freedom associated with that value of S. If N = 2 the weight is 1. If N = 3 the weight is 2, etc.

unweighted regre )  But the for weighted regression are s the confidence intervals requires For these reasons we reco regression be used for this Of to just the number of degrees of The calculations for weighted regression to those for In particular, calculating matrix Operations e designed for weighted Errors. Outliers, and Mavericks paae 25 Questionable data is given a diversity of names, such as mavericks fliers, outliers, sports, and blunders. These aberrations can be caused by contamination of samples, switched or mislabeled samplss, faulty equipment or reagents, key entry errors, and a host of other events. Such data is a source of frustration, and often causes pointless discussions and wasted effort. Some statistical criteria are available to assist in learning whether or not the largest or smallest observation is significantly far removed from the main body of the data. 1 In this section we will address this matter from two perspectives. The first of these is an omnibus examination of all of the data in one pass. The other is more focused, and addresses only one triplicate or quad.

Table 8 was derived from Table 5. The data were broken into two categories... low C and high C... with the expectation that variation will be constant ( or virtually constant) at low C and that variation will be constant at high C. Of course we expect that variation will change between low C and high C, which was the point to breaking the table into these two categories.

Ranae Ava, C 248.45 10.3 1 243.3 253.6 3.6 2 237.3 233.7 235.50 147.30 7.6 3 151.1 143.5 233.50 33.8 4 250.4 216.6 5 173.1 207.6 150.40 17.2 6 141.8. 159.0 220.80 24.8 7 233.2 208.4 146.75 13.7 8 139.9 153.6 Data SamDle 190.35 34.5 Avg Range = 18.19 9 1733.4 1828.4 1780.90 95.0 10 2022.6 1754.9 1888.75 267.7 11 2127.9 2014.9 2071.40 113.0 12 2064.0 2047.6 2055.80 16.4 13 2016.6. 1934.1 1975.35 82. i 15 2161.3 2492.2 2326.75 330.9 16 2003.9 1808.6 1906.25 195.3 14 1877.2 1757.2 1822.20 110.0 Avg 3an'ge = 151.35 The range is shown for each simultaneous sample. The average range is reported for each of the two groups.

In Statistical Process Co whether any of these ranges a quads we only test for abnor small ranges is not meaningful. determine whether any of the ra s customary to test . . . . . . . , . . . .

SamDle Size, n 2 4

2 3 . 2 6 7 3 2 . 5 7 5 4 2  2 8 2 5 2 . 1 1 5 Step 1: The data used to calculate ranges were duplicates. The sample size is n = 2.

Step 2: Multiply the average range by 0, for each category. 3.267 x 18.19 = 59.4 3. 151.35 = 494* 5 Step 3: Compare t dividual ranges ag these limits. When C is ranges larger than are suspect. When C is questionable. t ges larger than 494.4 are  All of the ranges passed the test. re is no evidence for any of the data.

WC t t Now let's consider how t data was not concentrated at t derived from Table 2. Table 10 is ran ed to a in ca'e Of c. Ta and then broken into three cat ego ries . r t  * F k Fm  247.2 297.6 358.9 342.7 743.6 655.9 893.7 1080.2 529.4 553.3 960.3 1099.7 7 1 4 6 3 . 0 1169.4 12 1356.8 1392.8 10 1415.8 1453.8 1 4 1949.0 1803.1 ! 1 6 1865.4 2247.1 3 2070.3 2219.7 r 2 2240.4 2127.0 Lw 8 2 5 4 9 . 1 2320.8 9 2488.9 2470.3 5 2 3 5 1 . 1 2652.2 L ,

The rules are applied as before. i I L c t 9vrJ. c 3. a ?.? a 350.80 15.2 272.40 5  3 . 4 541.35 23.9 699.75 87.7 985.95 185.5 72.94 = k v g Range 1030.00 1316.20 1374.80 1434.80 1 8 7 5 . 0 5 130 139.4 293.6 35.3 3 8 . 0 145.9 58 = &.. g R a n g e 2056.75 380.7 2145.00 149.4 2183.70 113.4 2434.95 228.3 2479.50 18.6 2501.65 301.1 238.22 = Avg R a n g e Step 1: The data used to calculate ranges were duplicates. The sample size is n = 2. D, = 3.267 Step 2: Multiply the average range by 0, for each category. 3.267 x 72.94 = 238.3 3.267 x 130.58 = 426.6 3.267 x 238.22 = 778.3 Step 3: Compare the individual ranges against these limits category by category as before.

F n i any of the data. . AI1 of the ranges passed the test. There is no evidence for excluding L as an exam n using this If there are gaps in the ranke categories at those points. There i of data will be in each category. ctice we should average C, then break the data into ement that t h e same amount .

The extension of this t quads only requires using . . other values of D4. n If we examine only one set or triplicates or quads, then Dixon's r procedure is appropriate.'

Consider the following sirnultaneo ly data:

22.3 29.4 49.1 28.2 Step 1: Rank the data from smallest to largest:

= ( 49 1  29 4M49.1  23 8) 23.8 28.2 29.4 49.1 Step 2: Calculat r = 0.779 Table 11 P o 95 P o . 99 n 3 0.941 0.988 4 0.765 0.889 5 0 I 642 0.780 Step 3: The sample size n = 4.

Step 4: The calculated value r than Po. 95 ( POag5 = 0.765). The evidence suggests that the remainder Of Ihe data*

If the calculated value of r exceeds the table value at Po. 99 then the evidence is even stronger.

_.

' Dixon, W. J. and Massey, F. J.; Introduction to StatisticaI Analysis, 3rd Ed.; McGraw Hill; 1969. pages 328  330.  p t h F 1 f L .

Dixon's r is of the form: r = ( Distance between the largest and its nearest neighbor)/( Full Range of the data.) This can be arranged to inquire about the status of a number that is unusually low when compared to the remainder of the data.

Examole: 121 179 185 193 r = ( 179  121)/( 193  121) = 0.805 This is farger than the table value of r ( 0.765) with n = 4. So the 121 is inconsistent with the remainder of the data.

Dixon's r cannot be used with n = 2 data. It assumes the data came from a normal distribution. This is a reasonable assumption with simultaneously sampled concentration data.

See Table 2 test 9 and Fig. 2. S = 16/ 48 seems to be unusually low. This is also appears as a I small samples ( espe standard deviation to of duplicates s may even be " zero" occasionally. So even though these may seem unusual, they are not. / Iy duplicates and triads) is is very low. In the for the paae 30 A power law model was used throughout this report because the data will not support more than two There may be instances in which we will need to entertain other model forms. For example, please draw curve to expr relationship between S and C in Fig. 7. efore proceedi The relationship between S and coordinates. It is reasonable to con model, perhaps S = a i kCP. In S wifl approach a when C = 0. This suggests the does not seem to be linear in LkLn Before we begin setting up the tools to estimat a, k, and p in this extended model, we should work through the 1. Test the data to deter ther the appearance of curvature ( in Ln Ln coordinates) is " real" o hance variation data. If the perceived curvatu to random va the data ... and not a syst then attempting to fit thes will be misleading and disapp transforming to t n  L n coordinates, of course. he test for curvature after 2. The most direct way to test for curvature is to fit the data to the usual power faw model ... Ln( S) = Ln( k)  i pLn( C) ... then ask whether there is evidence of " lack of fit". This is easily done by testing to see whether the data will support adding associated with b is larger than the appropriate reference value of the t ratio ( as a rule, larger than 21, then there is evidence of curvature; the simple power law model is not adequate. If the t ratio associated with b is notably smaller than 2, there is no fir evidence of curvature; the power law model is adequate; attempting to f t h e data to a more . complex model to account for curvature is pointless and misleading. b[ Ln( C) I2 to the model. If the t ratio 3. The purpose of ( 2) above is not to build a completed model. The purpose is to test for the presence of curvature beyond that which is accommodated by the power law model. If curvature is detected ( t ratio for b is larger than, say, 2), then we may be justified in considering an alternative model. S = a + kCp would be a candidate. F"

L F !

U 0 0 P 0 0 T O T 0 0 0 0 0 r M p paae 32 This process was followed to make that judgement about these data.

AS A S 248.5 7.28 2375.0 58.34 235.5 2.55 1622.0 77.78 147.3 5  37 234.0 233.5 23.9 24.4 150.4 12.16 .  220.8 . 17.54 146.7 9.69 1281.0 67.18 2189.0 189.29 2071.0 7 9 . 9 2056.0 11.6 1906.0 138.1 18.6 6.48 26. 1.19 . 23. 5.18 32.4 12.06 34.6 3.12 39.6 12.82 45.7 2.96 4 8 . 9 7 . 1 0 In Fig. 8 t h e power law mod explained 66.5% of the variation in Ln( S). In Fig. 9 the model was extended by adding a as suggested on page 30. This extended model expla variation in Ln( S). coefficient; this increas in R sq does not is important. The t ra ratic term is an indication of whether that term is a w to the model. adratic coefficient 69.2% of the R sq will always increas when we add another model rove that the quadratic term The t ratio for t h e quadratic coefficient is on 3 6 ; far below the reference value of 2.08 ( with we do not have sufficient evi power law model S = kCP. What have come from a straight line rela This does not mean we have " prove relationship prevails in Ln Ln coordin evidence in favor of a more complex justification for adding a quadratic term to little justification for pursuing an alternative model such as S = a + kCP.

It is possible t h a t with additional data we may learn that curvature is actually present, and that an extended model form was justified. attempting to build a model more complex than S = kCp is not appropriate with the existing data. ( say, S = a + kCp).. then the coefficients in that model will be poorly estimated; the confidence intervals on those coefficients will be extremely wide; and we have gained nothing more than t h e satisfaction of explaining a little more of t h e variation in t h e data. But If we pursue this ... if we build an extended model Figure 8 page 33 smm. 2 Dstenninant = 1.0000 mi?. 1 GOING IN VX? IABUS comrcms SE OF am T IIATIO T CFUT 0.05 & 0.01 = 2.07 & 2.80 0 Intercept 2.77468 1 average conc 0 . 68217 0.10316 6.61 ESSUMSQ S? pDEvOFRES DF R SQ 16.02539 0.85348 22 0.6653 < 66.5%

In In Units In Original Units OBSVD PREl RESID STD RES Observed Pr& i. ct& Diff 1.985 2.826  0.841  0.99 7.28 16.88  9.60 0.936 2.789  1.853  2.17 1.681 2.469  0.789  0.92 5.37 11.82  6 45 3.174 2.784 0.390 0.46 23.90 16.18 7.72 3.195 2.644 0.550 0.64 24.40 14.08 10 . 32 2.498 2.484 0.015 0.02 12.16 If. 98 0.18 2.864 2.746 0.119 0.14 17.54 15.57 1.97 2.271 2.467  0.196  0.23 9.69 11.78  2 . 09 4.207 3.945 0.263 0.31 67.18 51 . 67 15 . 51 5.243 4.310 0.933 1.09 189.29 74.47 114.82 4.381 4.273 0.108 0.13 79.90 71.71 8 19 2.451 4.268  1.817  2.13 11.60 71.35  59.7 4.066 4.366  0.300  0.35 58.34 78.73  20.3 4.354 4.106 0.248 0.29 77.78 60.70 17.08 5.4 4.352 1.104 1.29 234.00 77.62 156.38 16 0.712 0.83 138.10 67.76 70.34 1.869 1.058 0.811 0.95 0.174 1.294  1.120  1.31 1 645 1.312 0.333 0.39 2 40 1.436 1.054 1.23 1.138 1.481  0.343  0.40 2,551 1.573 0.978 1.15 1.085 1.686  0.601  0.70 1.960 1.717 0.243 0.28 2.55 16.27  13.72**

s SQSQ FEicM REQESSION = 16.02539 RES rage &. served = 42.0829 Average Predi Figure 9 page 34 s w m . 3 Determinant = 0.9 WRLABLES S SE OF EEET T EAT10 = 2* 08 6r 2* 82 0 Intercept 1 average conc 6.39 2 quadratic 0.08073 1.36 <

14.12537 0.83738 21 0.6924 69.2%  . .  66.5% . = 2.7% _ . RESSUJSQ STDDEVOFWS DF R SQ Note Ln( S) = 2.461 + 0.657*( Ln( C)  5.44) + O. ll*( Ln( C)  5.44) sq 1 2 3 4 5 6 I 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 In Ln U n i t s In Original Units OBSVD PPED RESID STDRES Observed Predicted D i f f 1.985 2.511  0.526  0.63 7.28 12.32  5.04 1.681 2.  0.508  0.61 5.37 8.93  3.56 3.174 2.470 0. 23.90 11.82 12 . 08 24.40 10.38 14.02 12.16 9.03 3.13 2.864 2.433 0 17.54 11.. 40 6.14 2.271 2.187 0 9.69 8.91 0  78 4.207 3.912 0.295 0.35 67 . 18 50.01 17.17 5.243 4,498 0.745 0. 189.29 89.83 99.46 4.381 4.434  79.90 84.31  4 . 41 2.451 4.426  11.60 83.61  72.01** ~

4.066 4.593  58.34 98.76  40 . 42 4.354 4.162 77* 78 64.23 13.55 5.455 4.568 0.887 1.06 234.00 96.38 137.62 4.928 4.341 0. 0.70 138.10 76.75 61.35 1.869 1.503 0. 6.48 4.50 1.98 1.645 1.557 0. 5.18 4.75 0.43 2.490 1,595 0.895 1.07 12 . 06 4.93 7.13 1.138 1.610  0.472  0.56 3.12 5.00  1.88 2.551 1.645 0.906 1.08 12.82 5.18 7.64 1.085 1.692  0.607  0.72 2.96 5.43  2.47 1.960 1.706 0 5 51 1.59 0.936 2.476  1.539  1.84 2.55 11.89  9 . 34 0.174 1.552  1. 1.19 4.72  3 . 53 EES SmSQ rn REGRESSION Average Cbserved = 42,0829 D I E C T = 14.72535 Average Predicted = 32.0234 lxLaa5 The foregoing data can be fit directly to a model of the form S = a + kCP using t h e following technique. Set p to a constant ( say, 0.6) and fit S = a + kC0.6. Since this model is linear in the coefficients a and k, ordinary least squares methods can be used to estimate those coefficients. Repeat this process for p = 0.8, 1.0, etc. When this is done:

0 a k Resi& zl Sum S a 0.6  10.27 1.24 40.52 0.8  1.61 0.25 39.96 1.0 2.95 0.053 39.66 1.2 5.60 0.012 39 50 1.4 7.26 0.0024 39.43 1.6 8.39 0.00052 39.41 The coefficient a is an estimate of ( r when C = 0. Negative values of a imply that CT becomes negative as the average concentration approaches zero. So p c 1 is certainly not acceptable.

The residual SUM of squares ... a direct measure of how well a model explains the variation in S, is insensitive to the least squares combinations of a, k, and p. The estimates o f all three coefficients are highly correlated. minimizes the r er. This m very wide. This will happen when the data is not cap There is no unique combination of coefficients ` that ual sum of squares; one combination is as good as any that the confidence intervals on the model coefficients of accurately able situation estimating the coefficients in the residual s u m of squares I{ have a " sharp" well defined minimum.

The underlying chemistry or physics may suggest that G converges to a iimit greater than zero; Fig. 7 suggests this. What harm is done by fitting the S vs. C data to a model ... viz., S = a + kC p... that supports this theory about the lower bound on o? That depends on t h e definition of harm. The prior analysis showed there is insufficient evidence to support " curvature" in Ln Ln coordinates. If an alternative model is used to claim that t h e data follows a certain theory, when in fact the model coefficient that wouJd support that theory is not supported by the data, then that is a poorly founded claim. The resolution to this is to g. et data . that wilt rr properly test theory; in this instance, iet data at very low levels of C. In any event, confidence intervals on a, k, and p should be repofied. ,

h np paae 36 The Performance of t h e ReMAP Process The ReMAP process i ing relationships between estimates of sigm degrees of freedo derived from small samples process can be found in text statistical procedure. It is ther the ReMAP. process perform Monte Carlo simulati ReMAP process. By perfu intervals on sigma encu verify that statements interval has encompassed sigma" are accurate.

The simulations are comprised of the following s I . Generate data that lese bles actual data. The model used to models cited in the were clustered near .2* Co. 8 This resembles simulated data tions they were scattered d we mean the data and in another band to generate the random normal deviates 3. Steps 1  2. were repeated ( say) 20 times to generate 20 sets of simultaneous samples, each comprised of a pair ( N = 2) of 4. The sample average and sample standard deviation S were calculated for each pair of data. of S was m by the small sample correction factor 1.253 to obtain un estimates of sigma. L paae 37 of the form Ln( S) = Ln( K) + p'Ln( C). This equation was used to predict Ln( S) for each of the 20 combinations. 5. The ( sample average, S) combinations were used to build a model  I

 r b. 2 Estimates of 1 Ln( S) were converted to estimates of S by Exp( Ln( S)).

L A a.

6. As in the ReMAP process, the ratio of the average of the observed vatues of S to the average of the predicted values of S was used to calculate the bias correction factor, BCF.

7. 95% confidence intervals were calculated in terms of Ln( Sj and 8. The predicted values of Ln( S) and the intervals OR Ln( S) were '

r

kJ F=

i c i L intervals on Ln{ S). irrm I ienr to estimates of sigma and the confidence intervals on ugh Exp( Ln( S)). The completed prediction equation is of / bR14 bur t h e form Est Sigma = BCF'kCP.

$""

9. This equation ... the line of regression ... was traversed in small steps along t h e entire length of the line ( from C = 500 to C = 5000. The upper and lower confidence limits were calculated The frequency with which those intervals the true values of Sigma ( the true values of sigma m Sigma = 0.2' co.*) was recorded. L P r kn n 8. This entire process from Step 1 through Step 9 was repeated The foregoi imulates getting 10,000 complete sets of ReMAP data; fitting those vto the model Ln( S) = Ln( K) + p* Ln( C); applying the L+ appropriate bias correction factors; and ( because we know the true value of Sigma in these simulations) observing bow frequently the population of confidence intervals actually encompass the true values of sigma. ..  .  ... """" 10,000 times; in some instances 30,000 times.

L UIIY The simulations also provided information about biases in I estimating sigma usi the ReMAP process. The bias... 100'( Est Sigma  S a)/ Sigma.., ranged from  0.1% at low levels of C ( hence low values of sigma) to + 1.1% at high levels of C ( high levels of Y" S sigma). This means that hen the estimated value of sigma is, for example, 10, the true of sigma could be as low as 9.89. Using the * same examp vel of concentration, the true value of sigma l__ y could be 10. P L e This bias is trivial when compared to other considerations such as the the confidence intervals on sigma.

I *_ Y ~

~~ ~  ~ C~

oaae38 To verify the software of this software was written generating data from the Y = 60 + 0.3' X was use fitting the data to the m to a model of t h e form Y =

? he usual 95% confi value of Y in the mann d for these simulations, an identical copy h the following changes. Instead of ... these data were fit the 95% confiden designed to perform; an for simulating the ReMA 3 that purpose, between 97% a encompassed the true values o " 95% confidence intervals" act the frequency of capturing data; simulations were run z I; u 97% of the intervals calculat the frequency of capture is prese ReMAP process.

Simulations were run to understand t r, The magnitude of this increase Lui i 1 compared to 95%) depends up0 When we have two simultaneous sarn 97%  98% intervals. Whe " 95% intervals" are actual1 produce a distribution of S that is hig will produce a distribution of S that i cause of intervals t h a t are somewhat broader t h a n expected. paae 39 No real harm has been done by this small deviation from expected. it just means that we should remember that what we are calling 95% intervals are closer to 97%  98% intervals. Our probabifity of capturing sigma with these intervals is a little higher than anticipated,

In the foregoing discussion we have described the performance of the ReMAP process in terms of individual confidence intervals. These intervals are calculated and presented throughout this report. This means that when we estimate a confidence interval at a point along the scale of C, we can declare that " 95% of the intervals calculated in this manner value of sigma. ( We know it's really 97%  98%, but is discussion we'll skip that detail.) This also means n the scale of C) " the probability that this confidence sigma is 95%". S there is a probability that sigma is a litt larger than the calcu ller than the calcufated lower limit. In ity of " about 1% in each tail." limit, or a littl reduces to a There is another concept that should be considered. This is the concept of a confidence interval on a line as a whole. These intervals are concerned with probability that a confidence interval captures the true values of sigma at every point along a line, ( over t h e range of the tated as " the probability is X% that this confidence the true values of sigma at all points alo e intervals that f capture sigma. In this n is directed to the frequency at which sig d of the line to the other, rather than at i Thus the frequ captured", not in terms of individual scale of c u m on. This is also th of capturing is in terms of " lines ts captured. Capturing the entire line ( the line is Sigma = 0.2` Co. 8) as opposed line is a rather stringent requirement.

In this report the cited confidence intervals intervals, or points on the line, not whole line statement " 95% of the intervals calculated in sigma" means 95% of the individual intervals. interpretation of confidence intervals. to individual points on the are in terms of individual intervals. Thus the this manner will encompass This is the usual For more information concerning whole line confidence intervals, see Natrella, M. G.; Experimental Statistics; National Bureau of Standards Handbook 91; August 7963; pages 5 15 and 5 16., paae 40 Simulations with the model Y = 60 + 0.3' X ( with the usual confidence interval reporting 95% ind confidence intervals our 95% confiden s of the mean of Y at .

every point along t h e entir at when we are equivalent to 86% obability is 86% that Simulations of the ReM with the model ) show that the fr wal is 91%  92%. Th ct that individual ot 95%. Thus in the y is about 91%  92% t true values of sigma ai the line. This c Remember, in the context of a whoie line " failure to capture the indeed. The P nfidence interval, a would be counted as a t al confidence interv uch that the tr wals. This is ' The precise outcomes from these simulations are dependent on several factors: 7 1. The constants k and p in the underlying model Sig i 1  2. The amount of data in the simultaneous samples. 3. The number of samples us to build the models. 4. The points at which predic n t h e scale of C.

6 ! LJ L This proprietary simulation software can be modified to study specific situations in more detail. paae 41 What If There Is No Relationshb Betwe en Est cs a nd Co ncen tration ?

If the t ratio associated with p is trivial ( notably less than 2) then we have failed to detect a relationship between Est CY and C. This dues not mean there is no relationship; it only means that whatever relationship there may be, it was not detected in this set of data.

Our ability to detect such a relationship is influenced by the range over which the concentration w varied. If the data span a narrow range ( viz., only about one decade, or a factor of 10) the dispersion in the values of S may be too small to detect the actual change in 0.

If the data are badly distributed along the scale of C... for instance, concentrated near the centroid instead of near the ends of the scale ... our ability to establish the relationship between 0 and C may be degraded. Thus our inability to establish a relationship may simply be due to data that are poorly distributed on the scale of concentration.

The inherent variation among individual values of S, cannot be Attempting to establish a overcome unless we have sufficient data. relationship between G and C with insufficient data spread over narrow ranges virtualty insures that the t ratio for p will be low; so low, in fact, that we may not detect the presence of a relationship much less establish the nature of it.

Any of the foregoing factors, or combinations of those factors, can be responsible for a failure to detect a relationship between cr and C. It. cannot be overemphasized that a " failure to detect" does not imply there is no relationship. data spread over a wide range of concentrations. Rather, it implies that we did not have sufficient If the t ratio for p is low ( notably smaller than 2) then we may decide to take the following position. Since we have not established a hip, then we may deciare that G is a constant. This is equivalent ring that the individual vafues of S effectively came from one ommon source whose true standard deviation is G, Under this practice we coutd ... if we elect to do so .... simply pool the individual values of S and use that as the estimate of G. When doing this, it is appropriate to use the factors in Table 1 to calculate confidence bounds an 0. L paae 42 The ITEQ data ( Table 12 of the main body of this report) will be used to illustrate this practice. The reported as 0.0457, 0.001 2, ... of these provides a 1 df estirn sum and divide by 2 Variance = 0.00071 2. deviation; pooled S = 1.40 ( interpolated in 0.0207 and 0.0374; equivalent to drawi limits are re1 uncertainly in estimating the slope, 1 i Pooling is not the same as averaging standard deviations  The small sample bias correction factor is not needed when pooling if the number of pooled degrees of freedom is greater than 10. Llau. 3 Recommendations Although the variation in simultaneocsly sampled data causes uncertainly in the relationship between G and C, the width of the confidence intervals on G can be minimized by acquiring data at well chosen points along the scale of C. In planning for future data ... anticipating a model of the form Est ci = kCP ... about one half of the data " high" level of each pollutant. t h e t ratio on the power coefficient, and minimize the width of the confidence intervals on ( r for a given amount of data. Id be collected at a " low" level and one half should be collected at a This will minimize SE( coeff), maximize if there is interest in pursuing alternative models, then the For every coefficient in the model used to following rules apply. associate 0 with C, data should be Concentrated at a point along the scale of C. This means that for a model with two estimated coefficients ( viz., Est CT = kCP) the experimental data should be concentrated at two points' on the scale of C, as in Table 5 and Fig. 5. For a model of the form Est CT = a + kCP the data should be concentrated at three distinct points' on the scale of C. If the data are poorly dispersed, as in Table 2 and Fig. 2 then the SE( Coeffs) will be large and the confidence intervals on estimates of Est 0 may be wide. Attempting to estimate three model coefficients from poorly dispersed data can only lead to confusion.

There is no justification for using models more complex than Est ci = kCp with the ReMAP data at this time.

Within this report there is sufficient information to allow u s to estimate the amount of data needed and the best positioning of that data on the scale of C so as to reduce the confidence intervals on 0 to pre specified widths. AIthoug h these calculations could be done analytically, it is better to do them with Monte Carlo simulations because of the complexity introduced by the bias corrections. Moreover, with simulations we can quickly explore " what if cases" before investing in additional data, A further advantage of Monte Carlo is that it makes the underlying models ( and the assumptions) completely visible and unambiguous as compared to analytical methods that require in depth knowledge of statistical methods.

* Data should be concentrated at additional points on the scale of C in order to test for lack of fit.


[Federal Register Volume 82, Number 103 (Wednesday, May 31, 2017)]
[Proposed Rules]
[Pages 24924-24925]
[FR Doc No: 2017-10926]

-----------------------------------------------------------------------

ENVIRONMENTAL PROTECTION AGENCY

40 CFR Part 52 and 81

[EPA-R05-OAR-2016-0044; FRL-9962-71-Region 5]

Air Plan Approval; Michigan; Redesignation of the Belding Area in
Ionia County to Attainment of the 2008 Lead Standard

AGENCY: Environmental Protection Agency (EPA).

ACTION: Proposed rule.

-----------------------------------------------------------------------

SUMMARY: The Environmental Protection Agency (EPA) is proposing to
approve the State of Michigan's request to revise the designation of
the Belding nonattainment area to attainment of the 2008 National
Ambient Air Quality Standards (NAAQS) for lead. EPA is also proposing
to approve the related elements of emissions inventories and a
maintenance plan. EPA is proposing to approve reasonably available
control measure/reasonably available control technology measures and a
comprehensive emissions inventory as meeting the Clean Air Act (CAA)
requirements. EPA proposes taking these actions in accordance with the
CAA and EPA's implementation regulations regarding the 2008 lead NAAQS.

DATES: Comments must be received on or before June 30, 2017.

ADDRESSES: Submit your comments, identified by Docket ID No. EPA-R05-
OAR-2016-0044 at http://www.regulations.gov, or via email to
<EMAIL_ADDRESS>For comments submitted at Regulations.gov,
follow the online instructions for submitting comments. Once submitted,
comments cannot be edited or removed from Regulations.gov. For either
manner of submission, EPA may publish any comment received to its
public docket. Do not submit electronically any information you
consider to be Confidential Business Information (CBI) or other
information whose disclosure is restricted by statute. Multimedia
submissions (audio, video, etc.) must be accompanied by a written
comment. The written comment is considered the official comment and
should include discussion of all points you wish to make. EPA will
generally not consider comments or comment contents located outside of
the primary submission (i.e. on the web, cloud, or other file sharing
system). For additional submission methods, please contact the person
identified in the ``For Further Information Contact'' section. For the

full EPA public comment policy, information about CBI or multimedia
submissions, and general guidance on making effective comments, please
visit http://www2.epa.gov/dockets/commenting-epa-dockets.

FOR FURTHER INFORMATION CONTACT: Matt Rau, Environmental Engineer,
Control Strategies Section, Air Programs Branch (AR-18J), Environmental
Protection Agency, Region 5, 77 West Jackson Boulevard, Chicago,
Illinois 60604,<PHONE_NUMBER><EMAIL_ADDRESS>
SUPPLEMENTARY INFORMATION: In the Final Rules section of this Federal
Register, EPA is approving the State's SIP submittal as a direct final
rule without prior proposal because the Agency views this as a
noncontroversial submittal and anticipates no adverse comments. A
detailed rationale for the approval is set forth in the direct final
rule. If no adverse comments are received in response to this rule, no
further activity is contemplated. If EPA receives adverse comments, the
direct final rule will be withdrawn and all public comments received
will be addressed in a subsequent final rule based on this proposed
rule. EPA will not institute a second comment period. Any parties
interested in commenting on this action should do so at this time.
Please note that if EPA receives adverse comment on an amendment,
paragraph, or section of this rule and if that provision may be severed
from the remainder of the rule, EPA may adopt as final those provisions
of the rule that are not the subject of an adverse comment. For
additional information, see the direct final rule which is located in
the Rules section of this Federal Register.

    Dated: May 4, 2017.
Robert A. Kaplan,
Acting Regional Administrator, Region 5.
[FR Doc. 2017-10926 Filed 5-30-17; 8:45 am]
 BILLING CODE 6560-50-P