1 1 U. S. ENVIRONMENTAL PROTECTION AGENCY 2 3 4 26TH ANNUAL CONFERENCE ON ANALYSIS OF POLLUTANTS IN THE 5 ENVIRONMENT 6 7 C O N F E R E N C E 8 9 10 May 14, 2003 11 12 13 CHICAGO, ILLINOIS 14 15 16 17 18 19 20 21 22 23 24 25 2 1 U. S. ENVIRONMENTAL PROTECTION AGENCY 2 PUBLIC MEETING 3 CHICAGO HOLIDAY INN MART PLAZA 4 May 1, 2003 5 MR. TELLIARD: This 6 public meeting is being sponsored by 7 the Office of Water, Office of Science 8 and Technology. The notice on the 9 door... I don't know if most of you saw 10 that... said please check all sharp 11 objects and firearms. I hope you did 12 that.

13 I am going to spend about ten 14 minutes kind of giving an overview of 15 what we did, why we did it, and where 16 we are at, and then we will start a 17 list of speakers who want to come up.

18 Most of these, from what I can tell,

19 are basically to congratulate the 20 Agency on their tremendous effort at 21 getting this out and how pleased they 22 all are with it. There are a few 23 dissenters whose names I won't 24 mention.

25 304( h), under the Clean Water 3 1 Act, 304( h) says that the Agency will 2 promulgate, pontificate, and implement 3 methods for the measurement of 4 compliance, and in so doing, we have,

5 under this, specified methods as 6 published in the Code of Federal 7 Regulations under Part 136. 8 It is important to understand 9 the things we are talking about today 10 apply only to Part 136 or the 11 wastewater methods. They do not apply 12 to Safe Drinking Water methods as 13 published under 141. So, we are only 14 talking about wastewater methods in 15 Part 136 today. So, the comments need 16 to kind of be directed that way.

17 In June, 1999, the Agency 18 published Method 1631B, the analysis 19 of mercury at trace level, and a 20 number of regulated industries felt 21 that the method was too tight, and 22 they didn't like a number of things in 23 it, they wanted some things added, and 24 after very cordial negotiations, we 25 came down and resolved most of the 4 1 methods issues, and one of the issues 2 that came out of the settlement 3 agreement was an agreement to review 4 the quantitation limits and detection 5 limits as specified in Part 136.

6 We had actually started working 7 on this a little earlier, but some 8 people felt it was moving at the speed 9 of molasses and wanted to speed it up.

10 Now, you all know detection 11 limits. We have some 50 different 12 ways of expressing detection and 13 quantitation. There is the MDL, there 14 is the IDE, the IQE, the CAQ, the 15 PQL, the political quantitation level,

16 all of which have been out there 17 floating around by various agencies 18 and organizations. Well, finally, we 19 are going to try to bring this to 20 conclusion by moving ahead and 21 resolving some of the applications as 22 it fits into the overall program.

23 The settlement agreement says 24 that EPA must assess existing Agency 25 and alternative procedures for 5 1 determining detection and quantitation 2 limits under the Clean Water Act. At 3 a minimum, we had to evaluate the 4 method detection limit procedure as in 5 Part 136, Appendix B, and the minimum 6 level for quantification included in 7 Method 1631. EPA also must submit its 8 assessment for formal peer review 9 which we have done, and we had it 10 peer reviewed by two chemists and two 11 statisticians. This peer review was 12 completed in August, 2002.

13 We have had three Federal 14 Register notices regarding this 15 particular document, the one notifying 16 the world of this meeting today as 17 well as the publication on March 12th 18 of the document describing the EPA's 19 assessment, and also the notice 20 concerning changes to 40 CFR Part 136,

21 as reflected in the document.

22 General terms, method detection 23 limit basically represents the lowest 24 concentration that allows for a 25 difference between the sample that 6 1 contains a substance and one that does 2 not. That is basically the one I got 3 out of the 8th grade science book.

4 Quantification limits are basically 5 used, in our case, in the ML,

6 represents the lowest concentration 7 that can be measured with some degree 8 of confidence and can actually tell 9 how much is there, and, of course, the 10 big glitch here is the question of 11 degree of confidence.

12 When we looked at the various 13 approaches, there are 50 different 14 terms that we find floating around for 15 describing detection and quantitation,

16 and there is a general lack of 17 consensus among various Government 18 organizations, FDA, NASA, us, the Army 19 Corps of Engineers, all of which have 20 their own little ways. Of course,

21 then, the consensus standard 22 organizations, ASTM, AOAC, Standard 23 Methods all have their approaches 24 also.

25 So, establishing detection and 7 1 quantitation limits is not what you 2 would exactly call an exact science,

3 and this has continued in the latest 4 articles in a number of journals which 5 continue to debate on large is small 6 and how small is large, which was the 7 latest article I read.

8 Most of the concepts are broken 9 into two groups, one, basically, that 10 the error is effectively constant at 11 the low concentration range, and 12 limits are based on errors observed in 13 replicate measurements made at a 14 single low concentration, and that is 15 basically what Curry has done, ACS,

16 ISO, IUPAC, and EPA. The other 17 approach is basically that the error 18 varies as a function of concentration,

19 and limits are based on the error 20 observed at replicate measurements 21 gathered across the range of detection 22 and quantitation, and examples like 23 this are USAFAMA and ASTM in their 24 procedures.

25 Now, one of the reasons we 8 1 would like to come to some sort of 2 consensus... and the consensus does not 3 mean that there is a single answer.

4 It means that we agree that things 5 within a certain range are acceptable 6 to the Agency simply so that when we 7 get down to the question of comparing 8 data, there is some reference between 9 how we got A and B and C. There is 10 an effort and continues to be an 11 effort so that as we spend $ 1 trillion 12 for this set of data that some other 13 agency or some other organization can 14 use that data. So, therefore, we 15 would like to have data of known 16 quality.

17 Historical use of the MDL and 18 ML, it is basically used to verify 19 laboratory performance and this takes 20 many forms. Some are contract 21 requirements, some are State QA 22 requirements, and, basically, the 23 laboratories are required to 24 demonstrate their ability to measure 25 at the detection limit and also to 9 1 measure at the level of quantitation.

2 Many laboratories use the MDL 3 procedures as a means of demonstrating 4 and advertising their ability. We can 5 measure 0.02 fentagrams. Who cares?

6 But that is part of the thing.

7 Demonstrate the acceptability of 8 laboratories as modification, and EPA 9 uses the MDLs published in approved 10 test methods as a standard, but it 11 does allow flexibility across the area 12 if people want to use another 13 approach.

14 Now, on some of these, for 15 example, in our contracts, we require 16 that the contract laboratory 17 demonstrate that they can reach the 18 MDL, and, actually, in our contracts,

19 we require that they do it once, and 20 then, from then on, they have to,

21 quote, calibrate and demonstrate at 22 the ML but not at the MDL.

23 Although it is not our intent,

24 when we started this thing, to use the 25 MDL or the ML as a regulatory limit, 10 1 we have, in fact, published a number 2 of different articles, statements,

3 guidance, that said we don't think 4 that the MDL should be a regulatory 5 number. That does not mean it has 6 not shown up in permits or in rules 7 across the country, but that was our 8 recommendation.

9 And the other concern is what 10 do you report, reporting all those 11 values found above the ML or above the 12 MDL, reporting only those values found 13 above the MDL but flagging those that 14 were less than the ML but above the 15 MDL, reporting all the values found 16 but flagging those that were found 17 below MDL, and all of the above. So,

18 you have various levels of reporting 19 and various formats for reporting.

20 In the assessment, we had a 21 study plan which basically reflected 22 the agreement that was signed. This 23 draft was put together by December,

24 2001 and subject to peer review and 25 finalized in April. The information 11 1 used in the assessment was to look at 2 basically three different literature 3 researches that we did, one in ' 97,

4 one in ' 98, and one in 2001.

5 The goal was to identify the 6 concepts, procedures, and issues that 7 we were going to consider in the 8 assessment, and more than 100 9 documents describing detection and 10 quantitation concepts and issues were 11 identified.

12 The actual data sets that were 13 used in the study were, one, the EPA 14 detection and quantitation limit study 15 which was affectionate referred to as 16 the 6000 data set, and, basically,

17 that was 16 different con... 10 EPA 18 method at 16 different concentration 19 run with 7 replicates.

20 The second study was the GC/ MS 21 threshold study which, again, looked 22 at the performance of ability to 23 measure and measure the effect of 24 having a background correction on or 25 off on the GC/ MS and its impact on 12 1 the data.

2 The American Automobile 3 Manufacturers study looking at 4 estimating the alternate minimum level,

5 and that involved nine laboratories 6 using Method 2000.7, 245.2.

7 The data base developed for 8 Method 1631 and 1638 interlab studies,

9 and that was used to, again, calculate 10 and compare the various procedures.

11 Concepts and procedures 12 considered. EPA considered detection 13 and quantitation terms, concerns,

14 concepts, or procedures that were 15 advanced or published in the 16 literature and by various EPA offices,

17 other government agencies, non

18 government agencies, and others. Many 19 of the concepts considered were not 20 fully evaluated, because they were not 21 fully supported by corresponding 22 detection and a definition of 23 detection procedure; did not reflect 24 the entire measurement process, that 25 is to say, extraction, concentration, 13 1 and the instrumentation; were not 2 uniquely designed for a single 3 program; were no longer advanced by 4 the organization or group, for 5 example, the alternate minimum level 6 had fallen by the wayside, so it was 7 not considered.

8 EPA evaluated four sets of 9 concepts that were widely referenced 10 and generally reflected the diversity 11 of the concepts advanced to date,

12 Method Detection Limit, the MDL; the 13 ML; the Interlaboratory Detection 14 Estimate, the IDE; the Interlaboratory 15 Quantitation Estimate, the IQE by 16 ASTM; the Limit of Detection and Limit 17 of Quantitation by ACS; the Critical 18 Value and Minimum Detectable Value 19 which is the one put forward by IUPAC 20 and ISO. So, these were the 21 procedures and concepts that were 22 evaluated in the study.

23 In addition, EPA considered six 24 specific issues that were raised by 25 the litigants in the settlement 14 1 agreement, and these were the criteria 2 for the selection of appropriate use 3 of statistical models, the methodology 4 for parametric estimates; statistical 5 tolerance and prediction; and the 6 criteria for design of detection and 7 quantitation studies; including the 8 selection of the concentration level;

9 interlaboratory variability; and 10 incorporation of elements of 11 probability design. These were added 12 as, we felt, reflected in the 13 settlement agreement.

14 Additional issues that were 15 also considered as a result of the 16 peer review and other comments were 17 the matrix effects, minimumization of 18 false negatives and false positives,

19 cost and ease of implementation, how 20 well detection and quantitation limits 21 published in the method reflect 22 individual laboratory capability. So,

23 these were added to the previous list.

24 EPA developed six criteria 25 based on the considerations of the 15 1 issues. One, the criteria should be 2 scientifically valid, can be tested, 3 has been peer reviewed, has a known 4 error rate, is supported by a well 5 defined procedure, and has gained 6 widespread acceptance. Two, should 7 address demonstrated expectations in 8 the laboratory in method performance,

9 including routine variability. Should 10 be supported by a practical and 11 affordable procedure that a single lab 12 can use to evaluate method 13 performance. Detection concepts should 14 provide a 99 percent confidence that 15 the substance will be detected.

16 Quantification concepts should identify 17 the concentration that gives a 18 recognizable signal and is consistent 19 with the capabilities of the method 20 when performed by an experienced staff 21 in a well operated laboratory.

22 TRACK 02: ... come up to this 23 podium and to give comment on the 24 proposed regulatory changes, and this 25 is an opportunity for us to take 16 1 comment. We are not going to debate,

2 argue, fight, or carry on over the 3 comments. These are comments that 4 people feel represent their 5 organization or their personal 6 opinions, and they will be taken that 7 way.

8 So, with that... the other thing 9 is I should point out for the 10 speakers, please speak into the 11 microphone. We are recording this,

12 and after it is done being recorded,

13 we will have a stenographer type it 14 up, and we will include this in the 15 court record. So, your comments are 16 formal comments, and they will be 17 going into the record. So, please 18 speak clearly and into the microphone.

19 SPEAKER: And our first 20 speaker this morning is Richard 21 Burrows, Severn Trent Laboratories.

22 MR. BURROWS: I would 23 like to thank Bill for the opportunity 24 he has given us to speak here on a 25 subject that is of profound interest 17 1 to the environmental lab testing 2 community. I am actually speaking on 3 behalf of the American Council of 4 Independent Labs.

5 This presentation, we are going 6 to cover a number of issues. I am 7 going to start off with a discussion 8 of whether we think the MDL procedure 9 works or not. You can probably guess 10 that we think it does have a few 11 failings, so we are going to describe 12 what those are, talk about why the MDL 13 procedure sometimes fails, what 14 alternatives there are, and how all of 15 this eventually relates to the ML or 16 the quantitation limit.

17 I am going to focus on 18 demonstration of the detection limit 19 for routine analysis at an 20 environmental testing lab. That is 21 probably 99 percent of the MDL studies 22 that are actually done. However, the 23 procedures that we are describing 24 could also be used for secondary 25 detection limit for a new method at 18 1 Part 136 if they were given a bit of 2 a multi lab flavor, and we will 3 continue to develop our ideas for how 4 that could be done.

5 The MDL is really a critical 6 concept to environmental testing labs,

7 because it is used not just for tests 8 performed under Part 136 but for all 9 environmental analysis programs,

10 drinking water, wastewater, solid 11 waste. Even if the particular agency 12 doesn't agree with the MDL, we still 13 have to do the MDL, because it is 14 typically mandated at the State level.

15 If the MDL revisions are 16 promulgated, we expect the revised 17 regulations will become an immediate 18 requirement for most of our work.

19 When we take a look at what 20 really controls what your detection 21 limit is, it seemed to us that there 22 were two fundamentally different types 23 of tests. One type of test, it is 24 really the variation of the blank that 25 controls the detection limit, and that 19 1 would be for a test such as 2 inductively coupled plasma optical 3 emissions spectroscopy where you always 4 get a result. It may be a negative 5 result, but you always get a numerical 6 result in any test for a blank or for 7 a sample.

8 On the other hand, there are 9 some tests where, most of the time,

10 the result that you get for the blank 11 is simply ND. These are tests where 12 the results are filtered through a 13 threshold or through requirements to 14 detect a group of peaks in the same 15 retention time window, qualifier ions 16 may be within the same... within a 17 given ion ratio, and in this case, it 18 is really not so much the blank but 19 more the ability to qualitatively 20 identify an analyte when it is present 21 that controls the detection limit.

22 It is a policy decision that 23 has been made that the MDL is going 24 to be an estimate of the measured 25 concentration at which there is 99 20 1 percent confidence that a given 2 analyte is present in a given sample 3 matrix. We think that maybe it would 4 be a better policy decision to define 5 the MDL as a concentration where there 6 is actually 99 percent confidence of 7 actually detecting a given analyte,

8 and I will come back to that.

9 For all this last 20 years or 10 so that we have been doing MDLs, we 11 have been operating with this theory 12 or hypothesis that this level at which 13 you do have confidence distinguishing 14 the measured result from the blank is 15 determined by multiplying the standard 16 deviation of seven or more spiked 17 replicate analyses by the Student t 18 value. That is the theory, and the 19 next normal step for a theory is to 20 test it.

21 Now, if you take a look at the 22 EPA's technical support document which 23 is a valuable read, I would say, for 24 anybody who hasn't seen it and is 25 available in the EPA's docket, they 21 1 identified 161 relevant publications, 2 but as far as we could tell, none of 3 these included a really convincing 4 test or demonstration that the MDL 5 procedure really worked, in other 6 words, that it did identify the 7 concentration where, for subsequent 8 routine analysis, you could distinguish 9 a measured result from a blank. So, we 10 thought we would try and test it.

11 Bill mentioned the Episode 6000 12 data set that EPA had generated.

13 Here, for a number of different 14 analytical techniques, there were seven 15 or eight replicates performed at a 16 whole number of different 17 concentrations, very useful data set.

18 I will pick one example out of here,

19 Method 200.8 which is ICP/ MS analysis 20 for chromium, and these are the 21 results that were obtained for this 22 data set:

23 The EPA calculates their MDL as 24 0.07 g/ l, and from that, calculated 25 an ML of 0.2. If we take a look at 22 1 this data, we can see that the 2 recovery at low values and including 3 at the MDL is extremely high and that 4 the mean value that is obtained in 5 this first group of spiking 6 concentrations is all about the same. 7 In other words, what is going on here 8 is that there is some kind of bias 9 that is causing an excessive positive 10 result relative to the spiking level.

11 It could be contamination in 12 the blank. It could be some kind of 13 calibration effect. We don't really 14 know what it is, but what this bias 15 results in is the fact that, really,

16 if you have a true concentration at 17 0.07 g/ l, you don't have any 18 confidence at all of distinguishing 19 this result from a blank, and looking 20 at this data, we think the MDL really,

21 given this data set, ought to be about 22 1 or 15 times higher than the MDL 23 that is calculated using the EPA's 24 procedure.

25 Why is that happening? Well, 23 1 the MDL makes some assumptions. One 2 assumption that it makes is that when 3 you take a look at the distribution of 4 results from a group of MDL 5 replicates, the distribution of the 6 blank is going to be the same and 7 that the distribution of the blank is 8 going to be centered around zero, and 9 from that, you calculate the MDL here 10 with a 99 confidence level that you 11 are going to be able to distinguish a 12 result from the blank.

13 So, what happens if the blanks 14 actually have positive bias? What 15 happens is that the MDL stays exactly 16 where it is, because what you are 17 calculating it from, the distribution 18 of the MDL replicates, stays the same.

19 MDL stays just where it is. But now,

20 if you look at the distribution of the 21 blanks, it may actually encompass or 22 even be higher than the MDL value.

23 So, as a result, now, if we 24 look at our blank results, this shaded 25 area where we get results above the 24 1 MDL, these are all our false 2 positives. MDL really ought to be 3 there.

4 So, if we now think about well,

5 how could we incorporate this blank 6 bias into the MDL equation such that 7 we get a more sensible result for the 8 MDL, one way of doing it is to 9 include the mean of the blanks in the 10 calculation, and here we have the mean 11 of the blanks plus the standard 12 deviation times Student t. So, I 13 think that gives us a bit of an 14 improvement.

15 The other variable in that 16 equation is the standard deviation.

17 So, another critical point is, is the 18 estimate of the true population 19 standard deviation good? And there is 20 some reason to believe that it might 21 be.

22 For several years now, the USGS 23 has been touting what they call the 24 long term MDL. EPA does mention this 25 in their technical support document,

25 1 and they say that they will go on to 2 discuss the issues raised by the long 3 term MDL, but it is not really very 4 clearly discussed in that technical 5 support document, unfortunately. So,

6 the USGS long term MDL involves a 7 large number of replicates, an 8 extended period of time, multiple 9 instruments, operators, calibrations,

10 and sample preparation events.

11 We could imagine that the 12 standard deviation generated from that 13 would be higher than a very short term 14 set of measurements. The ASTM IQE 15 makes some similar specifications that 16 it is critical that ordinary within

17 lab variation must be allowed to 18 affect the measurement process, as 19 happens in routine measurement, and 20 that, ideally, each measurement used 21 in the calculation of the detection 22 limit would involve a different 23 analyst, a different day, random 24 orders, maximum number of qualified 25 analysts possible, maximum number of 26 1 qualified measurement systems.

2 This is in contrast to the MDL 3 where the assumption is made, perhaps 4 incorrectly sometimes, that the idea 5 is that all the replicates should be 6 done in a short period of time, and 7 this first statement here is from the 8 technical support document. It says 9 when the MDL procedure is followed as 10 intended, in other words, the MDL is 11 determined by an experienced analyst 12 on each device or instrument, so,

13 individual device. And the second 14 statement comes from the actual 15 Federal Register notice. It tells you 16 to prepare a sufficient quantity of 17 matrix to provide samples for a 18 minimum of seven analyses. That 19 certainly implies that you are going 20 to do them all at the same time.

21 So, the assumption is made that 22 the Office of Water procedure 23 specifies that the replicates should 24 be done on one day by one analyst,

25 one prep event, one calibration.

27 1 Well, does that matter? We 2 thought we would try and test that. 3 So, we gathered a group of data. We 4 have an MDL for this set of data 5 which is a typical 40 CFR MDL, single 6 instrument/ single analyst MDL. We 7 compared this to the standard 8 deviation that we calculated from 200 9 routine method blanks, and we also 10 compared it to 19 method blanks that 11 were performed in a single batch on a 12 single day.

13 One example I pulled out of 14 this data set, the lead, the MDL was 15 1.63 g/ l. The long term blank 16 standard deviation was 2.3, and the 17 short term blank standard deviation was 18 0.4. In other words, the long term 19 standard deviation was 5.5 times 20 bigger than the short term standard 21 deviation. When we use our equation 22 using the mean plus Student's t times 23 the standard deviation of long term 24 blanks, we get a result of 5.5. So,

25 we get a result here that is quite 28 1 different from the MDL that was 2 calculated by the short term procedure.

3 Well, let's graphically take a 4 look at what is going on here. Here,

5 we have got our MDL replicates again, 6 relatively small standard deviation.

7 We calculate the MDL from these. It 8 is down here, and when we actually 9 look at our distribution of routine 10 blanks, the distribution looks like 11 this. It is a much wider 12 distribution. The MDL, we think,

13 ought to be out here, and now, this 14 shaded area here represents the false 15 positives.

16 What does it mean in the real 17 world? Well, we take a look at of 18 those 270 blank results we had for 19 lead. 10.7 percent of them were 20 greater than the MDL. If we used our 21 equation of Student's t times the 22 standard deviation long term blank plus 23 the mean of the blanks, 2.6 were 24 greater than the MDL, and we also 25 calculated using three times the 29 1 standard deviation which has been 2 touted in some publications, and now 3 we had 0.4 percent of the results were 4 greater than the MDL.

5 Expanding that to a bigger data 6 set, we took six labs' worth of data 7 for 22 elements and did the same 8 thing, and overall, we had 14.2 9 percent of the blank results were 10 greater than the MDL. Of course, this 11 means that you are going to get... so,

12 these are false positives, and this is 13 going to be happening in the samples 14 as well as in the blanks. So, 14.2 15 percent sounds pretty bad, and 16 remember that this 14 percent is for 17 each individual element. So, that is 18 an average of three false positives 19 per analysis.

20 When we used our equation of 21 the long term standard deviation of 22 the blanks plus the mean, we had 1.8 23 percent of the results were greater 24 than the... TRACK 02 ENDS.

25 TRACK 03... than the short term 30 1 standard deviations.

2 So, again, to look at this 3 graphically, we have got our MDL 4 replicates here, relatively small 5 standard deviation. Our short term 6 blanks also have small standard 7 deviation. The MDL, using the Office 8 of Water procedures calculated out 9 here, and here is our distribution of 10 the long term blanks and where we 11 think the MDL should be.

12 I want to define a couple of 13 terms here. The LC and LD are terms 14 that were originally defined by Curry 15 back many years ago before I even 16 started to think about detection 17 limits or even environmental chemistry,

18 but they are internationally accepted 19 terms now for the critical level and 20 the detection limit or internationally 21 accepted maybe excepting the United 22 States.

23 The critical level is set at 24 the standard deviation of blank times 25 a constant related to the confidence 31 1 level desired, and the LC is exactly 2 what the current MDL procedure 3 targets. LD described the level at 4 which you have got detection 5 capability, and LD would be the level 6 at which you have got 95 or 99 7 percent confidence of actually 8 detecting a true value that was in a 9 sample.

10 We can derive both LC and LD 11 from these long term blank 12 measurements, making the assumption of 13 constant variance across this range.

14 LD is simply two times higher than LC.

15 Let's take a look at what this 16 looks like in a distribution. If we 17 have a true concentration at LD, then 18 our distribution is comfortably 19 different from the distribution of the 20 blanks. In other words, if we have a 21 true concentration at LD, then our 22 result is something that we are going 23 to have confidence that we can 24 distinguish from a blank, so we can 25 say it is a true result.

32 1 If we look at the distribution 2 of a true value at LC, then here, if 3 the value is... 50 percent of the time,

4 you are going to get a result that is 5 greater than our distribution of the 6 blanks. So, at LC, only 50 percent 7 of the time are we going to be able 8 to say that we have a true detection.

9 We would recommend using LD as 10 the detection limit or the MDL rather 11 than LC but reporting data down as low 12 as LC. This means now that the 13 detection limit would become what most 14 people actually think of the detection 15 limit as, the lowest limit of reliable 16 detection. It is quite difficult to 17 explain to people that your method is 18 not necessarily able to detect 19 something at the method detection 20 limit. That just gets people very 21 confused.

22 EPA covers this concept a 23 little bit in the technical support 24 document. The one objection that is 25 raised is that if LD is used as a 33 1 censoring limit, then 50 percent of 2 the results of the true value of LD 3 would be censored, and that is why it 4 is important that while we would 5 report LD as our detection limit, we 6 would continue to report results down 7 as low as LC so that we will get the 8 best of both worlds, essentially. We 9 wouldn't be censoring results that we 10 know are really true. Yet, we would 11 be informing the data user what the 12 true detection capability of our 13 method was so that a correct decision 14 could be made about whether this 15 method was suitable for the data use 16 or not.

17 Unfortunately, things are never 18 quite that simple, and as I mentioned 19 early on, some methods, there is a 20 lack of instrument response at 21 the... in the blanks as a result of 22 threshold or method required analyte 23 identification criteria. Therefore,

24 for these methods, there is no way you 25 can use the standard deviation of the 34 1 blanks to evaluate the MDL.

2 Looking at this graphically 3 again here, distribution of the blanks 4 is everything is as zero. Everything 5 is non detect for the blanks. But if 6 we have our set of spikes at LD, the 7 detection limit, then we would have a 8 distribution as shown here. And we 9 would calculate an MDL from this 10 distribution of blanks at a level that 11 there may indeed be very little 12 confidence that you can detect 13 anything.

14 We went back to EPA's Episode 15 6000 data set and had a look at 16 Method 524.2, a GC/ MS method, and we 17 see here that 72 out of the 81 18 analytes have 7 replicates that are 19 all non detect at a concentration at 20 or above the MDL, and some of the 21 analytes have 7 replicates that are 22 all non detect at more than 3.18 times 23 the MDL, in other words, all non

24 detects at a level which would 25 normally be described as the 35 1 quantitation limit. So, obviously,

2 that is not making a lot of sense 3 here.

4 I think that, for these tests 5 where there is filtering the MDL as it 6 is currently determined is a pretty 7 meaningless number. It is not the 8 level at which you have got confidence 9 in distinguishing something from a 10 blank. It is not the level at which 11 you have got confidence that you can 12 actually detect something. It really 13 doesn't appear to be a number that 14 really means anything.

15 So, how do we discover what the 16 detection limit is for these types of 17 tests that have got thresholds and 18 filtering? We believe a statistical 19 approach is going to be extremely 20 difficult or is not going to work well 21 because of the very varied 22 requirements for analyte detection.

23 First of all, you have got to 24 detect a chromatographic peak. Then,

25 you have got to consider you might 36 1 need to have co elution of qualifier 2 ions. You might need to meet certain 3 ion ratios. You might have to do 4 pattern recognition. A human might 5 have to do pattern recognition of 6 maybe an aroclor pattern or a mass 7 spectrum, and to do a statistical 8 analysis of this becomes almost 9 impossible.

10 So, we considered an empirical 11 approach, and this type of empirical 12 approach is included in the EPA's 13 proposed MDL procedure as a technique 14 for making an initial estimate of the 15 MDL. We think that the initial 16 estimate does a far better job than 17 the statistical calculation that would 18 follow it.

19 So, essentially, you simply 20 spike at increasingly lower 21 concentrations until a detection 22 limit... until the detection fails,

23 until you fail to detect the compound.

24 Very important here that routine 25 conditions for detection are used.

37 1 You shouldn't just being trying to 2 pull something out of the noise that 3 you wouldn't try to pull out of the 4 noise in a regular sample.

5 Once you had determined the 6 lowest limit that you could truly 7 detect something, you would repeat 8 this spike of this concentration in 9 several batches to verify and then 10 continue those verifications on a 11 quarterly basis.

12 Again, we don't think that the 13 data should be censored at this lowest 14 limit of detection that we have 15 determined. Otherwise, we would throw 16 away at least 50 percent, even 17 assuming 100 percent recovery, or 18 maybe even more than 50 percent if we 19 don't have 100 percent recovery of our 20 data if we censored at LD.

21 For these tests, again, because 22 we don't have any standard deviation 23 of the blank, LC is a difficult number 24 to really properly determine or even 25 define, but we think right now a 38 1 reasonable lowest limit to report 2 consistent with the current procedures 3 would be to calculate LC the way that 4 the current MDL is calculated, and we 5 are open for other suggestions for the 6 lowest limit to report the censoring 7 limit.

8 So, again, now to go on and 9 discuss how this all relates to the 10 minimum level or the quantitation 11 limit, EPA provides three varied 12 definitions of the minimum level. One 13 is the lowest level at which the 14 entire signal gives a 15 recognizable... entire system gives a 16 recognizable signal. Another 17 definition is the lowest acceptable 18 calibration point, and a third 19 definition is 10 times the standard 20 deviation of the MDL replicates. We 21 think these three definitions are kind 22 of difficult, because they would all 23 tend to lead you to a different 24 number, and they are supposed to be 25 defining the same thing. 39 1 We also think that if the MDL 2 were targeted at LD and it were 3 determined with full consideration of 4 the routine variability and blank 5 bias, then two or three times the MDL 6 would be a reasonable quantitation 7 estimate. Again, if a quantitation 8 limit is going to be set for a 9 nationwide method to be proposed at 10 Part 136, then a multi lab study 11 should be performed to define the 12 precision and accuracy at the ML.

13 We don't think that ten times 14 the standard deviation should be used,

15 because that could be below the level 16 at which there is confidence 17 distinguishing a result from a blank,

18 as we have seen in some of the EPA 19 6000 data. We shouldn't use a 20 calibration point alone, although a 21 calibration point should be included 22 at the ML, and that is because of 23 weaknesses in the criteria for an 24 acceptable calibration which is 25 typically based on the correlation 40 1 coefficient. Because of weaknesses in 2 correlation coefficient concept, you 3 can essentially have your low 4 calibration point wandering around all 5 over the place and still get a very 6 good correlation coefficient. You can 7 certainly get .999 correlation 8 coefficient even if you get no result 9 at all for your low calibration point.

10 So, to summarize our position,

11 we believe that routine lab 12 variability must be included in any 13 detection limit procedure. A sample 14 coming into a lab may be analyzed 15 randomly on different qualified 16 instruments by different analysts,

17 different batches, and different 18 calibrations. If you don't include 19 those sources of variability in your 20 detection limit estimate, then there 21 is no hope of getting an accurate 22 assessment of the detection limit.

23 We think the detection limit 24 should target Curry's limit of 25 detection, but we would avoid 41 1 excessive false negatives at a true 2 concentration of LD by reporting 3 results down to the calculated value 4 for LC, the critical value. And we 5 believe that there is not a single 6 procedure that will provide a good 7 detection limit estimate for both 8 unfiltered tests such as ICP and 9 highly filtered tests such as GC/ MS,

10 but we are proposing two 11 straightforward methods for determining 12 detection limits for all types of 13 tests.

14 Further information is available 15 on our website that we have 16 established where you can also visit 17 and make comments that would be 18 directed back to the ACIL Technical 19 Committee on our procedure. That is 20 at acil mdl. org, or you can call or 21 email me directly.

22 Thank you.

23 MR. TELLIARD: Thank 24 you, Richard. Our next speaker is 25 Nancy Grams from Advanced Earth 42 1 Technologies. 2 MS. GRAMS: I will go 3 ahead and start. I am here today 4 representing the task group of ASTM.

5 It develops... can you hear me?

6 SPEAKER: We can't hear 7 you.

8 MS. GRAMS: All right.

9 There is a handout that is going 10 around. Naomi passed a few of them 11 out. Unfortunately, I didn't make 12 enough copies. I didn't expect to see 13 so many faces here today of so many 14 people so interested in the MDL.

15 Hopefully, most of you can share them.

16 Many of you who are members of my 17 task group have seen these materials 18 and don't really need a copy.

19 Additionally, my comments today,

20 my written comments as well as my oral 21 comments, will be a part of the 22 record, making them a part of the 23 record, and those materials will be in 24 the written materials. So, if you 25 don't get them today, you can go to 43 1 the docket and find them.

2 Also, ACIL has made an offer to 3 allow people to post information to 4 their website relevant to these 5 comments, and I will try to make sure 6 that the materials get posted.

7 So, to repeat, I am here today 8 representing the task group that 9 developed the IDE and the IQE.

10 In the Office of Water's MDL 11 assessment effort, of all the myriad 12 of detection and quantitation terms 13 that Bill mentioned in his opening 14 remarks, the IDE and the IQE were two 15 concepts that were carried through the 16 entire assessment process and were 17 critically evaluated at the very end 18 against the MDL and the ML using the 19 six criteria that Bill outlined 20 earlier.

21 I fear that for many of the 22 people here today who are not regular 23 participants in ASTM, you never heard 24 of the IDE and the IQE before, so 25 what I wanted to do today was to try 44 1 to acquaint you folks both with ASTM,

2 with the history of the standards, the 3 IDE and the IQE, and to introduce you 4 just a little bit to the standards.

5 I am probably not going to have a 6 lot of time to do that, but I did put 7 some materials in the written comments 8 that give you a little bit more depth 9 about the IDE and the IQE in terms of 10 the technical content, and I put in 11 some references to some documents that 12 I think are also very valuable in 13 understanding the technical side of 14 these issues.

15 So, after that, I guess my 16 other purpose here today is to try to 17 persuade you of the merits of the IDE 18 and the IQE and of the merits of 19 EPA's Office of Water adopting these 20 procedures.

21 I just want to mention in 22 passing the Technology Transfer Act in 23 case many of you have not heard of 24 this. This Act requires Federal 25 agencies such as the EPA to first 45 1 consider the use of public consensus 2 processes and standards before 3 unilaterally, and at taxpayer expense,

4 going it alone. There are many 5 examples of Government agencies working 6 with ASTM, in particular, to develop 7 standards and the agencies adopting 8 them.

9 And I guess one of the first 10 things... the first point I would like 11 to make is that I think that we all 12 should consider the Technology Transfer 13 Act as a positive thing for us as 14 taxpayers, and I believe the ASTM 15 process has a number of merits that 16 unilateral development by the EPA does 17 not have for standards, and I just 18 want to start by giving you some 19 background to give you my reasoning 20 for this.

21 The American Society for 22 Testing Materials has recently been 23 renamed ASTM International, and this 24 organization has been around a long 25 time. There are 30,000 volunteers. I 46 1 think a lot of people don't recognize 2 how big an organization and how 3 important an organization ASTM is.

4 It has 130 technical 5 committees. We have face to face 6 meetings. Unlike a lot of standards 7 development organizations, we actually 8 get together, and we fight it out.

9 Don't we, Bill? There is an extensive 10 tiered balloting system so that there 11 is really protection in the balloting 12 system, and there is strict 13 adjudication of negative votes. The 14 negative voter in ASTM has a very 15 strong position.

16 And, as I said, one of the 17 merits of the ASTM standards is that 18 they meet the criteria for use under 19 the Technology Transfer Act.

20 I want to talk just a little 21 bit about the committee itself. There 22 are a lot of different committees, but 23 the ASTM committee that I participate 24 in primarily is D19, and this 25 committee talks about water. We do 47 1 everything about water. We measure 2 water, we measure the stuff in water,

3 we do all sorts of stuff with water.

4 This committee was established in 5 1932, and I want you to think about 6 this for a minute. This committee is 7 70 years old. Long before anybody was 8 even thinking about an EPA, this 9 committee was developing water 10 standards.

11 We have 300 current standards,

12 and one of the things about ASTM is 13 that all of these standards have to be 14 reassessed at least every five years.

15 So, any standards that are outdated 16 are removed if they are not being used 17 anymore, and we update the information 18 that is in them so they are not 19 stagnant.

20 We have 400 active members on 21 the committee, and in ACM, you have to 22 be active. If you don't vote, you 23 are thrown out.

24 The committee is diverse in 25 representation. Agencies are 48 1 represented there. For many, many 2 years, EMSL Cincinnati, when they were 3 doing methods development for EPA, was 4 at the committee, and more recently,

5 Bill Telliard's group at the Office of 6 Water has been participating as has 7 Bill himself.

8 There are many different 9 persons from industry and consultants 10 to industry who participate in the 11 committee, and this includes the 12 litigants in the matter of the MDL.

13 We also have good representation by 14 instrument manufacturers and by 15 laboratories. So, we are really 16 diverse and representative in the 17 environmental industry.

18 I just want to point out that 19 ASTM D19, both within ASTM and within 20 our industry, was the leader in 21 requiring interlaboratory method 22 validation. Now, there is 23 interlaboratory, and there is 24 intralaboratory, and it is going to be 25 very important for everyone today to 49 1 keep those two concepts separate.

2 This is among laboratories.

3 And it was also very early on 4 in providing precision and bias 5 statements within all of its methods. 6 If you go to the ASTM methods under 7 D19, you will see that every method 8 there has a precision and bias 9 statement. Many of them are in 10 individual matrices of water such as 11 groundwater and waste water, et 12 cetera, et cetera.

13 We have a standard that we use 14 in order to determine the precision 15 and bias and to make these precision 16 and bias statements. This standard 17 was first approved in 1969, and I want 18 you to think about this as well.

19 Maybe EPA was a twinkle in Uncle Sam's 20 eye at the time.

21 The committee recognized the 22 importance and the responsibility, as 23 method developers themselves, for 24 validating the methods they put out 25 there. The standard D2777 has often 50 1 been revised. It is revised to, you 2 know, record the accumulated wisdom,

3 and it is revised based on the state 4 of the science. There are many things 5 that have changed over the years,

6 statistics, environmental chemistry,

7 lots of changes.

8 It specifies that for every 9 method that our committee produces,

10 testing method, we will do an 11 interlaboratory method validation 12 study. This is what is commonly 13 called in the committee a round robin 14 study. It is multi lab, multi

15 concentration.

16 D2777 defines the working range 17 of the method. In our methods at our 18 committee, when you write the method,

19 you have to state what the working 20 range of the method is, and you cannot 21 extrapolate outside of your 22 interlaboratory study.

23 Precision and bias statements 24 are generally equations that are 25 developed by matrix within the water 51 1 matrices. The studies, the analyses 2 of the study data, and the precision 3 and bias statements are all reviewed 4 and must be approved within the 5 committee, and the information is also 6 used to establish ongoing CQ 7 parameters.

8 What the committee was really 9 thinking about in doing and generating 10 these IDE and IQE standards was to 11 extend the range of the validation,

12 and I think you need to think about 13 this for a minute, because it may not 14 be totally intuitive. This industry 15 is going to be moving toward 16 measurement uncertainty and reporting 17 measurement uncertainty. All the IDE 18 and IQE do for our committee is extend 19 from the general working range to 20 probe down and try to figure out, on 21 an interlaboratory basis for a method,

22 where the limits of detection and 23 where the limits of quantitation 24 really are.

25 In discussing the IDE and IQE, 52 1 I think it is important for me to 2 talk about the history at D19 of 3 detection and quantitation, about the 4 task group, and the participation of 5 EPA and the litigants. In the late 6 1980s, committee D19 attempted to 7 develop a single standard, one 8 standard, to address all the issues of 9 detection and quantitation, both single 10 lab and multiple lab, intralab and 11 interlab.

12 After many drafts and rounds of 13 balloting, the committee was totally 14 frustrated. We couldn't get anywhere.

15 In 1990, I volunteered to head 16 an effort to try again. I set out to 17 bring interested parties together and 18 to foster communication and to clarify 19 both the science and the issues.

20 Instead of just trying to draft a 21 standard which hadn't worked before...

22 BEGIN DISC 2, TRACK 23 01... groups. We do meet twice yearly.

24 As I said, ASTM does go face to face.

25 We had joint meetings both with 53 1 the quality group which is the group 2 in which I participate, that 3 subcommittee, and organic and inorganic 4 subcommittees. We had 40 plus members 5 who attended each and every meeting.

6 At many of our meetings, we had 80

7 plus people.

8 My task group consists of 200 9 people. By the time, which was ten 10 years, we got the IDE and the IQE 11 standards done, we had invested 12 thousands of hours, and these are all 13 highly qualified professionals who had 14 been involved. It is an enormous 15 investment both by agencies and by 16 private individuals and corporations.

17 Now, this is the next point I 18 would like to make. It is extremely 19 unlikely that, in our lifetimes, this 20 level of effort will ever be repeated 21 again.

22 And what are our work products?

23 As I mentioned before, rather than 24 just writing a standard, I think our 25 most important work product was 54 1 communication, and it continues to be.

2 In our meetings, we have people from 3 USGS, we have people from the Office 4 of Water, we have people from other 5 Federal agencies, we have States, and 6 we have people from the industry side,

7 and we all get together, and we 8 communicate.

9 My task group did another thing 10 that was quite different. We did a 11 lot of education. We had a number of 12 tutorials, because we found that 13 people really didn't understand some 14 of the underlying concepts associated 15 with detection and quantitation and 16 the needs, especially statistical 17 needs, in developing an appropriate 18 procedure.

19 We also developed software.

20 You know, one of the criticisms of the 21 more complex procedures is that people 22 can't do them themselves, and even I 23 can use the software. It is very 24 simple.

25 I just want to note for anybody 55 1 who is seriously interested in 2 detection or quantitation and/ or coming 3 to better understand the ASTM, in 4 June, we are meeting in Louisville,

5 and the task group meets on Tuesday,

6 and then we have a half day seminar on 7 Wednesday where we are really going to 8 go into a lot more depth, and Bill 9 and his folks at the Office of Water 10 are going to be there, and we are 11 going to have a good time.

12 So, now I need to look at how 13 much time I have, and I am running a 14 little behind, so I am just going to 15 kind of rush ahead here.

16 As I mentioned before, ASTM has 17 this D2777 standard, and we actually 18 piggybacked the IDE and the IQE on it,

19 and I kind of mentioned that before.

20 Again, as I said, in the written 21 materials, there is a lot more depth 22 of information on what the interlab 23 detection estimate is, so I am just 24 going to run through this.

25 It is based on Curry's theory.

56 1 Rather than using a single 2 concentration, though, it uses a 3 multiple concentration study design.

4 We determine the critical level and a 5 detection level which is the IDE. We 6 regress the measured concentration 7 versus true concentration. That is 8 what is called the calibration design.

9 We use true interlaboratory data. We 10 don't just pool single operator data.

11 And we use weighted least squares to 12 take care of the difference in the 13 magnitudes, the concentration issues.

14 Here is one of the things we 15 had a lot of discussion on and a 16 tutorial on, this thing called 17 confidence bounds. You know, I 18 haven't used the word confidence 19 limit. Modeled around the regression.

20 So, you are just kind of figuring out,

21 you know, where most of your data is 22 going to be, whether it is, you know,

23 the 99 percent control limit or the 24 95, and we use both, one to control 25 false positives, the other, negatives.

57 1 We use statistical tolerance 2 intervals, and in my next slide, I am 3 going to start talking about this a 4 little bit. There are other 5 intervals, and EPA has used confidence 6 intervals. In the standard in the 7 software, we have used a tiered 8 approach to modeling standard 9 deviation. If I have time, I will go 10 into that just a little bit.

11 Those of you who have been 12 doing MDLs have thought you have been 13 doing them to say something about the 14 next or any future detection decision 15 you may make, but when you are doing 16 an MDL, that is not actually what you 17 are doing. When you are using a 18 confidence interval, you are developing 19 an understanding of the specified 20 confidence you have in the mean of the 21 population. So, if you went back and,

22 in every sample you ever analyzed, you 23 analyzed it seven times and took the 24 mean, then your MDL might mean 25 something.

58 1 Another interval that is 2 frequently used is the prediction 3 interval, and this is getting there,

4 because it says with a specified 5 degree of confidence, the next 6 measurement that you will measure will 7 be within a calculated interval, and 8 many people use the prediction 9 interval.

10 We use the tolerance interval,

11 and, you know, there was a big debate,

12 and we finally voted on it, and we 13 decided to use the tolerance interval.

14 It was sort of the most scientifically 15 sound approach from the majority of 16 the task group's point of view. We 17 used it to quantify the uncertainty in 18 a proportion of future measurements. 19 The interlaboratory quantitation 20 estimate was our second standard.

21 This was finished in 2000. Same 22 calibration regression, same modeling,

23 and I will just talk about the 24 modeling for a second here.

25 Our standard... and we debated 59 1 this at length as well... requires that 2 you go from a very simple model for 3 standard deviation to a more complex 4 on. So, we start with assuming that 5 the standard deviation is constant,

6 then we assume the standard deviation 7 is linear, then we assume that it is 8 hybrid which is the same as the Rocke 9 and Lorizano model, for any of you who 10 are interested, and only then, if none 11 of those fit in ascending order, then 12 we allow the use of exponential. And 13 these are all in the software. The 14 software does them all.

15 Based on a quantification 16 defined as known and control 17 precision, this is another one you 18 have heard today in the prior talks,

19 that you can take your detection limit 20 and multiply it by a factor and get 21 your quantitation limit. We spent 22 many an hour discussing this in our 23 task group, and the basic consensus 24 was that quantitation is a totally 25 different concept, and while you may 60 1 get by with making some simplifying 2 assumptions to use a factor, that it 3 is really not the scientific way to go 4 about it, and the scientific approach 5 to quantitation is that you must 6 control precision.

7 In the references I give you,

8 David Coleman has an article, and it 9 relates detection and quantitation to 10 significant digits and to relative 11 percent difference. I strongly 12 recommend that anybody who is 13 seriously interested in this topic 14 read David's article.

15 We use 10 percent RSD as our 16 default in our software, because that 17 is pretty well known throughout the 18 sciences, but the software allows 19 other DQOs.

20 I will just summarize what I 21 have said so far. The Office of 22 Water's extensive evaluation identified 23 ASTM IDE and IQE as the only fully 24 developed, strongly founded procedures 25 worthy of detailed assessment. I have 61 1 sought today to give you an 2 understanding of the stature of ASTM 3 and of D19 and of the level playing 4 field of consensus established at 5 ASTM, and I have tried to give you 6 some understanding of the standards 7 that have been developed in our task 8 group and to fill you in on the 9 history of the development of these 10 standards along with the participation 11 of EPA, others, and the litigants in 12 our efforts.

13 I have tried to portray the 14 level of effort that went into the 15 development of these standards, the 16 significance of these standards in 17 that it is very unlikely that a group 18 will be brought together to do this 19 ever again. I have tried to show you 20 that it was developed by a large,

21 diverse group of credentialed 22 professionals with a long term vision 23 for our committee and with a 24 commitment to high quality water 25 testing validation.

62 1 Our committee is working to 2 utilize the IDE and the IQE and the 3 software itself, and we are committed 4 to improving these standards and our 5 software over time. I also hoped to 6 show today that the IDE and the IQE 7 are the only viable candidates under 8 the Technology Transfer Act for EPA's 9 Office of Water adoption.

10 Most importantly, I have tried 11 to highlight the actual issues that 12 are here in the litigation today,

13 interlab/ intralab, intralab/ interlab,

14 because it is the litigants in this 15 matter who are concerned about 16 potential interlaboratory uses of the 17 MDL procedure. Most everybody else 18 who is going to comment on this is 19 going to be interested in 20 intralaboratory.

21 The Office of Water has to deal 22 with both realities and has proposed 23 to use a single procedure for both.

24 Under the Technology Transfer Act, the 25 office could, instead, choose to adopt 63 1 the IDE and the IQE specifically and 2 just for the interlaboratory purpose.

3 This way, they would issue high 4 quality validation of methods, they 5 would satisfy the litigants in the 6 matter, they would produce detection 7 and quantitation estimates with a sound 8 technical basis, and it would allow 9 EPA to harness the ASTM D19 process,

10 its resources, and its volunteer 11 laboratories. It would also facilitate 12 maintenance and update of the 13 procedures, as ASTM does update their 14 procedures which the CFR discourages.

15 I believe it is extremely 16 important that when you, you yourselves 17 here today, develop your comments that 18 you recognize and differentiate and 19 critically evaluate the interlaboratory 20 uses of the MDL. Should the MDL 21 procedure, as it is now written, be 22 used by the Office of Water to 23 establish minimum sensitivity 24 at an interlaboratory level when it 25 approves methods, or should the Office 64 1 of Water use the Technology Transfer 2 Act and the IDE and the IQE?

3 Is the new MDL procedure 4 detailed enough? Does it describe 5 exactly how EPA will do its national 6 level studies to determine the 7 sensitivity of methods at a national 8 level? This is not the 9 intralaboratory stuff that Richard was 10 talking about. It is the 1 percent of 11 MDL studies that are ever done which 12 are the published MDLs that are 13 published with the methods.

14 Is the MDL procedure the right 15 balance of science and defensibility 16 against practicality? Is the proposed 17 procedure detailed enough to ensure 18 controlled use? Will the MDL 19 procedure, using the new MDL procedure,

20 be a good estimate? Is it really a 21 good enough estimate?

22 Will it be a sound basis... and 23 here is the real crux... for setting 24 compliance limits where water quality 25 based effluent limits are lower than 65 1 the capabilities of the method? 2 In closing and as you might 3 guess, I would like to challenge EPA 4 to consider the IDE and the IQE, and I 5 would like to challenge them as to 6 their assumptions that were made about 7 their appropriate criteria. In the 8 materials I passed out, I have 9 provided... and also in the written 10 docket will be... an alternative 11 assessment. I used their criteria. I 12 assess their criteria, and I assess 13 the IDE, the IQE, and I assess the MDL 14 against those criteria.

15 As time is limited, I wanted to 16 just focus real quickly on two 17 criteria. The Office of Water, in 18 criteria 3 and 6, makes the assumption 19 that it is critically important that 20 one have only one procedure, one MDL 21 procedure. And as I said, we divided 22 it into four when the majority of the 23 task group made their decisions about 24 how to deal with this issue.

25 I think these assumptions that 66 1 they made, 3 and 6, are inappropriate.

2 I think that whether or not you need 3 one procedure or whether you need more 4 than one procedure should have been 5 assessed itself.

6 And, at the very end of the 7 materials I passed out, I have 8 provided an alternative list of 9 criteria that I think should have been 10 assessed, and I ask you to look at 11 that list and to look at the varied 12 uses of the MDL, including the 13 interlaboratory uses, and, for 14 yourself, make a determination about 15 whether or not one MDL procedure is 16 adequate.

17 Thank you very much for your 18 consideration, and I want to take the 19 time to thank all the various members 20 of my task group for the hours and 21 hours of efforts they have put in over 22 the years.

23 MR. TELLIARD: Our next 24 speaker is Naomi Goodman from the 25 Electric Power Research Institute. 67 1 MS. GOODMAN: Nancy said 2 a lot of what I had intended to say,

3 so I will spare you.

4 I represent EPRI, and just to 5 give you a little history, EPRI began 6 considering issues of detection and 7 quantitation in, as far as I can 8 determine, about 1984. That is 19 9 years. In that time, they have 10 conducted at least five or six major 11 round robin studies to evaluate low

12 level detection and quantitation of 13 various analytical methods.

14 They have been involved in 15 efforts to develop alternative 16 detection and quantitation criteria,

17 starting with the alternate minimum 18 level, continuing with the involvement 19 in the IDE and IQE development 20 process. I am currently a member of 21 Nancy Grams' task group, and I have 22 been for the last three or four years.

23 So, I don't want to take any 24 more of your time, tack that on top of 25 those 19 years, to restate things that 68 1 have been said many times. The 2 deficiencies of the MDL and ML have 3 been covered by the previous speakers,

4 and I am sure the following speakers 5 will also address them.

6 I just want to make a couple of 7 comments about EPRI's perspective on 8 these issues.

9 My primary concern about the 10 current proposed procedure is that it 11 does not specify the methods that are 12 going to be used for interlaboratory 13 validation nor say what is going to be 14 done with the data that are produced 15 as a result of that study. Although 16 the Office of Water has recognized 17 that method development on a nationwide 18 level does require multi laboratory 19 validation, they do not go into any 20 details.

21 In previous interlab studies,

22 the procedure had been to take the 23 maximum MDL produced by any of the 24 participating laboratories that passed 25 project criteria and to use that as 69 1 the MDL upon which the ML would then 2 be based. This was the procedure that 3 was used in the 1631 study and the 4 draft 1638 study which EPRI 5 participated in. Financially, we 6 collaborated with EPA, and we also did 7 our own independent data analysis.

8 Since that subset of 9 laboratories is fairly random, you try 10 to pick laboratories that are well 11 performing, but another set of 12 laboratories would produce an entirely 13 different set of MDLs, that maximum 14 MDL value is totally arbitrary and 15 would be different if you picked 16 different laboratories. This is a 17 very weak effort on the part of the 18 Office of Water to ensure that 19 laboratories do not fall out of 20 compliance merely because they are on 21 the wrong side of the distribution 22 curve. There has to be some more 23 scientific way to go about it.

24 However, my primary concern, as 25 Nancy Grams has already addressed, is 70 1 the use of the MDL and ML for setting 2 prescriptive criteria that can be 3 applied, rightly or wrongly, by 4 jurisdictions to ensure compliance with 5 the permits and other regulatory 6 limits. ASTM developed the IDE and 7 IQE to address this particular usage 8 and no other, and in their technical 9 assessment document, EPA has made some 10 attempts to cram the standards intro 11 an application for which they were 12 never intended which is single lab 13 application.

14 In their criteria number 3, the 15 Office of Water states, the approach 16 should be supported by a practical and 17 affordable procedure that a single 18 laboratory can use to evaluate method 19 performance. This is a limitation 20 that the Office of Water set on 21 itself. It is not required by the 22 settlement agreement.

23 My viewpoint is that detection 24 and quantitation criteria that are used 25 on a nationwide basis must be... excuse 71 1 me... that are used on a single lab 2 basis must be obtainable at a low cost 3 in instrument time and chemist time.

4 However, this is not necessarily the 5 case for nationwide usage.

6 A full range validation study 7 to support a procedure such as the 8 IDE/ IQE is affordable for a procedure 9 that will only be completed once for a 10 single method or method revision. We 11 don't expect that to happen very 12 often, and we expect it to be paid 13 for, hopefully, by EPA in collaboration 14 with interested parties who may include 15 the laboratory community, may include 16 industry groups, may include vendors.

17 There has certainly been no lack of 18 parties in the past who have been 19 willing to step up and support these 20 kinds of studies where their own 21 interests are involved.

22 EPRI collaborated with the 23 Office of Water on two such studies,

24 on Method 1631 and the draft Method 25 1638 which was nine metals by ICP/ MS.

72 1 We funded addition of blank pairs and 2 low level concentration pairs that 3 would allow calculation of an IDE or 4 IQE, and we conducted an independent 5 analysis of the data.

6 Our findings were that these 7 data produce reasonable estimates of 8 the interlab detection and quantitation 9 limits. Unlike the proposed MDL/ ML 10 procedure, the ASTM approach can be 11 used to determine when a method is not 12 capable of achieving data quality 13 objectives.

14 In the 1638 study, we 15 determined the overall interlab 16 variability at the ML determined by 17 the Office of Water for reagent water.

18 Out of nine metals, only one had an 19 RSD of less than 10 percent. The 20 other eight had RSDs ranging from 19 21 to 90 percent.

22 For single lab variability, if 23 you want to look at that, five of the 24 nine elements had RSDs greater than 20 25 percent at the Office of Water's ML. 73 1 I suggest to the Office of 2 Water they should broaden their 3 assessment of detection and 4 quantitation criteria to allow 5 alternative procedures such as the 6 IDE/ IQE. Although the proposed 7 Appendix B offers the possibility of 8 using such procedures, it then 9 nullifies the concession by stating,

10 providing the resulting detection 11 limits meets the sensitivity needs for 12 the specific application, that is to 13 say, if a method is sensitive enough 14 that the quantitation limit is well 15 below the regulatory limit, it is 16 acceptable to consider all sources of 17 variability, but it is precisely those 18 parameters with detection and 19 quantitation limits close to regulatory 20 criteria that have the greatest need 21 to consider interlab variability.

22 Thank you.

23 MR. TELLIARD: Thanks,

24 Naomi. Our next speaker is Larry 25 LaFleur. 74 1 MR. CHRISTMAN:

2 Substituting for Larry LaFleur...

3 MR. TELLIARD: We 4 switched?

5 MR. CHRISTMAN: We 6 switched again.

7 MR. TELLIARD: Okay, Jim 8 Christman.

9 MR. CHRISTMAN: Thanks,

10 Bill.

11 Good morning. I am going to 12 make just a few brief remarks on 13 behalf of the Inter Industry Analytical 14 Group or IIAG, as we call it, which is 15 a group that, in turn, represents the 16 views of many NPDES permittees who 17 depend on these analytical procedures 18 to prove compliance with their permit 19 limits.

20 As you know, as you well know,

21 the Office of Water's proposals for 22 23 detection and quantitation limits were 24 published in March of this year in the 25 Federal Register. The comment period 75 1 is now open, and comments are due July 2 10th. The IIAG feels like this is a 3 really important rulemaking for some 4 reasons that I will go into directly.

5 Way back when the world were 6 young, we were all young and foolish 7 and before we grew embittered by the 8 world and had our dreams dashed 9 against the rocks of experience, when 10 the first fish with lungs were 11 crawling out of the primordial 12 ooze... this was back in March of 13 1994... there was a draft guidance 14 document from EPA, never made final,

15 discussing use of detection and 16 quantitation limits in the permitting 17 process. That was, I guess,

18 controversial, but some of the ideas 19 in that paper are still showing up or 20 reflected, I think, in the March 21 Federal Register notice.

22 After that guidance document in 23 March of ' 94 came out, the IIAG formed 24 itself. Even back then, the folks in 25 this group recognized that EPA was 76 1 going to keep setting water quality 2 criteria lower and lower and lower.

3 Mercury limits, for example,

4 are very low, based on wildlife 5 criteria. At the same time, we knew 6 the Agency would be needing analytical 7 methods that would be capable of 8 measuring down at these lower and 9 lower levels, and the IIAG, as a 10 group, felt that there is an 11 obligation here to make sure that 12 these new methods are as reliable as 13 they can be made.

14 So, the IIAG tried to work 15 along with EPA through the years, and 16 while this work was going on, EPA came 17 out with Method 1631 for mercury and 18 came out with 1668 for PCBs which have 19 very low levels of detection and 20 quantitation. And as you have heard 21 several times, there was some 22 litigation on that and a settlement 23 agreement, and in the agreement, EPA 24 undertook to do a broad reassessment 25 of detection and quantitation. It 77 1 arose out of the mercury litigation,

2 but it wasn't just directed at 3 mercury. It was a more general view 4 of the subject as a whole.

5 The IIAG continued, you know,

6 working with EPA. We were very 7 hopeful that that reassessment would 8 produce concepts of quantitation and 9 detection that would allow us to agree 10 on what they should be, that would use 11 sound science, and, most of all, would 12 avoid legal disputes in the future,

13 because like everybody else, we want 14 to avoid litigation as much as we can,

15 and we want to avoid disputes. So, we 16 had hoped... we were very hopeful about 17 this reassessment and the rulemaking 18 that has come out of it.

19 Now, when we looked at the 20 proposal in the Federal Register, on 21 the other hand, we were concerned that 22 maybe this was a continuation of the 23 old way of doing things with minor 24 modifications that don't solve the 25 problems that we are concerned about. 78 1 One of the concerns that the 2 permittees have, the regulated 3 community, is that... remember that that 4 March, ' 94 guidance document suggested 5 that when a permit limit is set below 6 the level that the method can reliably 7 measure, you can use the ML as a 8 measure of quantitation limit and as a 9 compliance limit. So, it is this 10 unique feature of the ML which can be 11 used as a compliance limit when permit 12 limits are set very low that is of 13 much concern to the members of the 14 IIAG, and it is one of the reasons we 15 are very interested and very concerned 16 about this rulemaking.

17 The problem is if the ML is not 18 got right at the front end with good 19 science, the ML is set too low, then 20 there may be violations of permits 21 that are not true violations but 22 merely the result of variability 23 or noise. And everybody wants to 24 avoid permit violations for all kinds 25 of reasons. There are penalties 79 1 attached, but, you know, it is hard on 2 the reputations of the companies that 3 have the permits, and many companies 4 have imposed upon themselves zero 5 violations policies, and if you get a 6 violation, even if it is a supposed 7 violation or an apparent violations, it 8 violates these policies and causes lots 9 of trouble.

10 So, we want to get the 11 detection and quantitation levels right 12 right at the front.

13 We think for that reason alone,

14 this rulemaking should be important not 15 only to the people who have permits 16 but to the States, the regulators who 17 write permits, the regulators who 18 enforce permits, and everybody who has 19 those sorts of interests in this 20 subject.

21 Well, moreover, we think there 22 is a legal principle, and I will just 23 touch briefly on it. The legal 24 process goes into the due process of 25 law which is really just fairness. It 80 1 is written into the Federal 2 Constitution and probably into the 3 constitution of every State in the 4 country, and the way the courts have 5 expressed this legal principle is every 6 citizen and every NPDES permitted is 7 entitled to fair notice, fair notice 8 of the conduct which will make him 9 break the law, conduct which the 10 Government deems to be illegal.

11 A further principle is that if 12 you are being penalized for illegal 13 conduct in an area where you can't 14 measure it reliably, that probably 15 violates this fundamental principle.

16 One court in the District of Columbia 17 said, in a very short passage, that 18 well, it may be okay to set permit 19 limits below where you can reliably 20 measure them, but you may not be able 21 to... may not have the legal power to 22 enforce them down at those levels, and 23 these legal principles are one reason 24 that we see the importance of this 25 rulemaking, because... END OF TRACK 01 81 1 BEGIN DISC 2, TRACK 02:

2 ... proposed MDL and ML procedure.

3 They are three in number. There are 4 others, but let me just give you three 5 important ones.

6 One, which has been mentioned 7 several times, is the use of 8 interlaboratory variability. We think 9 incorporating interlaboratory 10 variability is very important, because 11 many regulatory decisions using data 12 rely on data from many labs. So, the 13 variability between and among different 14 labs is going to be very important.

15 Second is a concern for 16 accuracy. If the quantitation level 17 and, therefore, the compliance level in 18 some permits is based on mere 19 repetition of measurements and how 20 close together they are rather than,

21 in addition, how close they are to a 22 true number, a concern for accuracy,

23 then there will be a problem and 24 vulnerability to legal challenge there.

25 Finally, we feel like the MDL 82 1 is being asked to carry too much 2 weight on its shoulders. The MDL is 3 used for a variety of ways in the 4 regulatory process, as you know. It 5 is used as a figure of merit when a 6 method is published or promulgated,

7 used as a QA/ QC tool for individual 8 laboratories, used as a basis for the 9 ML, as a basis for the compliance 10 level for some permits, and in 11 addition to all of those uses that the 12 MDL has, EPA has had to try to come 13 up with a procedure that is relatively 14 cheap and simple so that the labs can 15 use it without undue difficulty.

16 Well, those are good, you know,

17 good motives, but we kind of feel that 18 the MDL just can't bear all that 19 burden by itself, and it would be 20 better to split up some of the 21 functions that the MDL is expected to 22 serve and pick a tool that is more 23 precisely tailored using sound science 24 for the individual uses of it.

25 To that end... and I now come to 83 1 my close... the IIAG, although we really 2 like the ASTM International proposals 3 you just heard about, we also came up 4 with a sort of another alternative 5 which we proposed to EPA and they were 6 kind enough to mention in the Federal 7 Register notice and put in the docket.

8 So, you can get a copy of the IIAG 9 proposal which is two parts either 10 from me if you will see me after the 11 meeting, or I am sure you can get it 12 from EPA as well, and EPA has invited 13 comment on it. We hope you will look 14 at that and, if you like it, comment 15 to EPA before July or by July 10th.

16 The two part IIAG alternate 17 proposal which you are going to hear 18 more about from Larry LaFleur in a few 19 minutes has a sensitivity test and a 20 full range of validation. Again, we 21 split up the functions.

22 The sensitivity test is 23 designed to be used by labs, for 24 example, as a proficiency test upon 25 startup. It does not rely on 84 1 interlaboratory data, and the labs can 2 do it themselves, and it is fairly 3 simple and easy to do.

4 The second part of the proposal 5 is the full range validation which 6 does collect interlaboratory data, but 7 the individual labs would not have to 8 bear that burden. We envision that 9 the interlaboratory data would be 10 collected by the Government, by method 11 developers, or maybe even by groups of 12 volunteers who have an interest in 13 getting a method approved. 14 So, you can look at the 15 detailed... the writings we have put 16 into the docket, and in order to give 17 you a few more... a little bit more 18 information about this IIAG proposal, I 19 would like to present Larry LaFleur 20 who is with the National Council for 21 Air and Steam Improvement, and he is 22 going to tell you about both parts of 23 the IIAG proposal, both the sensitivity 24 test and the full range validation.

25 Larry?

85 1 MR. LAFLEUR: Thanks,

2 Jim.

3 As Jim said, I am going to try 4 and provide you a little more 5 clarification as to some of the 6 details in the IIAG sensitivity test 7 and then go on and talk about the full 8 range validation.

9 What we did was we started out 10 looking at some of the multiple uses 11 the EPA has for the MDL, and,

12 specifically, we were looking at their 13 QA/ QC uses. Those we identified were 14 initial proficiency demonstrations,

15 basically, a test that is required if 16 an analyst makes a significant change 17 in the method, and the third one we 18 found was, you know, documentation for 19 method modifications that are allowed 20 under the flexibility in Part 136 21 methods.

22 So, we asked ourselves, if we 23 are just focusing on this one end use,

24 what is the MDL? From the analyst's 25 point of view, it is really just low

86 1 level precision, and that is all we 2 are really doing in the MDL study from 3 the context of QA/ QC, challenging the 4 lab's low level precision. So, if you 5 focus on just that one end use, you 6 will get a better idea of what we are 7 shooting for when we talk about the 8 sensitivity test. It is targeted just 9 for the initial startup criteria, just 10 for, you know, documentation of changes 11 from changes in equipment, different 12 analysts, as EPA currently interprets 13 that in their methods, and then,

14 finally, as a verification of 15 sensitivity when method modifications 16 are implemented.

17 What we are not trying to do is 18 add a new element or a new layer of 19 QA/ QC. We are trying to just identify 20 the current uses the EPA has 21 for the MDL in the QA/ QC context and 22 see if there is a more efficient way 23 to do it. So, this is not a new 24 requirement. We don't propose this 25 for continued use. It is a substitute 87 1 for the use of the MDL.

2 One easy way to think of it 3 conceptually is it is really very 4 similar to the initial precision and 5 accuracy test except we are using it 6 to test low level precision. So, the 7 spike level will be very low, but we 8 are not imposing a recovery criterion.

9 The MDL doesn't do that. We are not 10 adding to that test. So, it is really 11 just a test of low level precision 12 similar to your initial precision and 13 accuracy test.

14 So, how would this work? Well,

15 when EPA is developing a method, they 16 would select the spiking levels and 17 the number of replicates. If they 18 modeled the sensitivity test after the 19 IPR test, they would use four 20 replicates. We wouldn't envision them 21 using any more than seven, because 22 that is what they are currently using 23 in the context of the MDL.

24 The idea is to keep this simple 25 and functional and focused on the 88 1 objective. So, we would think that 2 the spike level would be just a simple 3 dilution of the lowest calibration 4 standard. That might be half, or it 5 might be a third, but the idea is to 6 keep it simple. We don't want to 7 challenge the lab to make out 8 detailed, complicated cocktails in 9 order to preform this test.

10 During method development and 11 validation, EPA would generate QC 12 performance specifications for the 13 method. It seems like the logical 14 thing to do would be use the model 15 they use now for developing the 16 precision specifications in the IPR.

17 They have a certain experimental 18 procedure and statistical procedures 19 that are outlined in their guidelines 20 for alternative test procedures at 21 tire 3, national approval.

22 Once they have done all that 23 work, then we would envision the 24 sensitivity test being incorporated 25 into the analytical protocol. The 89 1 spike levels would be designated. The 2 number of replicates would be 3 designated. The QC spec would be 4 designated, and that would be first 5 proposed and then, finally, promulgated 6 as part of the method.

7 How would it work from the 8 laboratory's perspective? Well, it is 9 fairly simple. Basically, the lab 10 would prepare the required number of 11 replicates, and this would be spiked 12 reagent water. They would spike it at 13 the level designated in the method.

14 They would analyze the samples through 15 the entire analytical protocol so it 16 is similar in that it tests the entire 17 analytical procedure and not just an 18 element of calibration. They would 19 just simply calculate the standard 20 deviation, and in this sense, compare 21 it against the QC specification. They 22 wouldn't multiply it by the Student's 23 t value to calculate an MDL. They 24 would just compare it against the QC 25 spec. 90 1 If you are below that QC spec,

2 you have passed, and you move on and 3 start running samples.

4 What are the advantages? Well,

5 the primary advantage is in order to 6 meet the spiking requirements in the 7 current CFR Part 136 Appendix B 8 procedure, for a multi analyte method,

9 many laboratories are required to 10 perform three, even four sets of seven 11 replicates. So, you are talking 21 or 12 28 analyses. That seems like an awful 13 lot of work just to demonstrate a 14 certain level of proficiency at low 15 levels.

16 So, the primary advantage of 17 the sensitivity test is it sets a 18 certain number... and, again, we think 19 that is between 4 and 7... number of 20 replicates at a prescribed level.

21 Yet, at the same time, it will provide 22 a credible challenge of the 23 laboratory's performance at low 24 concentrations.

25 Other advantages we think would 91 1 be that all labs would be performing 2 the tests at the same level.

3 Currently, each lab picks their own 4 MDL spike level, so you really can't 5 compare one MDL in one lab versus 6 another, because you don't know what 7 the spike levels were.

8 There would be a very specific 9 pass fail criterion which is explicitly 10 stated in the method. Thus, all the 11 labs would be evaluated against that 12 same criterion.

13 Again, since we focused 14 specifically on the QA/ QC aspects of 15 the use of the MDL, it will be a far 16 more efficient test for the 17 laboratories.

18 So, in summary, by focusing on 19 very specific objectives, we can 20 develop a much more efficient tool 21 than the current use of the MDL. Yet,

22 at the same time, we will still be 23 providing a credible challenge of 24 laboratories' performance at low 25 concentrations. The test is as simple 92 1 as we can possibly make it, consistent 2 with the goal of providing a 3 meaningful challenge to the labs.

4 Now, there has been some 5 confusion about what we intended, so I 6 want to tell you what we don't think 7 the sensitivity test would be. It is 8 not a detection limit. It is really 9 just a QA/ QC tool that would be used 10 in very limited aspects for proficiency 11 testing.

12 It is not a substitute for a 13 reporting threshold. You would have 14 to look elsewhere to find that.

15 Again, there are different objectives 16 and goals and quality objectives that 17 you would want to establish for that 18 kind of a reporting threshold.

19 Finally, there was no intent 20 here for this to be utilized as a 21 quantification level.

22 Again, the second part, this 23 presentation was actually prepared by 24 John Phillips. He was unable to 25 attend today, so he asked me if I 93 1 would try to work my way through it,

2 and I will do my best. This covers 3 the second part of the IIAG proposal,

4 that of the full range validation.

5 The basic assumptions I think 6 you have heard a little bit of 7 earlier. Basically, the IIAG believes 8 that any data used for regulatory 9 compliance purposes must fall within 10 the methods quantification range.

11 Secondly, we believe quantification is 12 defined by both acceptable precision 13 and bias.

14 So, when we look at the MDL 15 when it is derived as a multiple of 16 the ML, we don't believe it is a 17 quantification level. First of all,

18 it does not have known precision.

19 Secondly, it doesn't even consider 20 accuracy or bias.

21 As we envision it, the full 22 range validation would be an 23 interlaboratory study using a minimum 24 of seven laboratories. Nancy mentioned 25 one of many standardized protocols for 94 1 conducting these kinds of studies, the 2 D2777 process, but there are others.

3 It would develop precision and bias 4 through the entire working range of 5 the method. The data could be used to 6 establish method performance criteria.

7 It could also be used, if it was a 8 correctly defined study, to establish 9 the LC and LD that Richard spoke about 10 earlier as well as LQ, and it has some 11 utility in the context for reporting 12 measurement uncertainties.

13 We believe that the full range 14 validation should be used for every 15 method used for regulatory compliance 16 purposes or any new use of a method 17 for regulatory compliance. This would 18 include, of course, in the context of 19 this meeting, the methods that are 20 promulgated at 40 CFR 136.

21 At a minimum, we would like to 22 see it applied to all newly 23 promulgated methods.

24 Now, we see that the full range 25 validation would only be carried out 95 1 by a method developer, by EPA, or by 2 consortia of laboratories and/ or 3 industries, and I will give you an 4 example of the latter a little later 5 in the talk.

6 The key thing here is that any 7 individual laboratory would not be 8 expected to repeat this work. This is 9 really just for method development and 10 promulgation.

11 Why do we think we need 12 interlaboratory data? We think we 13 need to characterize the performance of 14 the method under the same conditions 15 that it will be used and practiced for 16 regulatory compliance testing purposes,

17 and that includes accounting for all 18 sources of variability. If you 19 include 20 interlaboratory data, you really 21 capture, in some respects, all the 22 other sources of variability which 23 would include batch to batch,

24 instrument to instrument, analyst to 25 analyst. So, it is sort of an all

96 1 encompassing way to account for 2 uncertainty and variability.

3 Well, we are all aware of the 4 various different sources of 5 variability and error that are in 6 methods, calibration, cross sample 7 contamination, carryover, individual 8 interpretations of how the method ought 9 to be performed, differences in 10 techniques or skills by a given 11 analyst. Variation in reagents is 12 becoming more important, because 13 reagents contamination is starting to 14 become an issue when we go to very,

15 very low trace levels. Finally, of 16 course, the accuracy of the standards 17 that we use for calibration.

18 Another reason why full range 19 validation is important is there is a 20 variety of different uses that States,

21 permittees, and the Office of Water 22 use Part 136 methods for. These 23 include effluent guideline 24 developments, NPDES permit application 25 procedures that the States might use 97 1 then for reasonable potential 2 determinations, receiving water 3 surveys, TMDLs, and, of course, NPDES 4 permit compliance testing.

5 We feel that many of these 6 different uses have different 7 measurement quality objectives, so each 8 of them should be assessed... or the 9 method should be assessed against those 10 measurement quality objectives.

11 To illustrate this, we are 12 going to use a few examples and then 13 sort of walk through how the full 14 range validation would work.

15 The Office of Water's 16 definition for quantification, for 17 compliance purposes, is essentially 18 based upon a goal of 10 percent RSD,

19 but they have no criteria at present 20 for accuracy. So, the purpose of 21 this... we are assuming that the Agency 22 would establish a recovery requirement,

23 say, in the range of 50 to 100 percent 24 which most of the EPA methods are 25 capable of producing. 98 1 For our second example, let's 2 assume that a State wants to survey a 3 river looking for hot spots, and they 4 want to use State certified labs for 5 this. They might even write this in 6 as a special permit condition for a 7 permitted. They may define their 8 measurement quality objective as plus 9 or minus 30 percent RSD with a 10 recovery of 70 to 130 percent, and in 11 this case, they want to control both 12 false positives but also have no more 13 than 5 percent false negatives.

14 Our third example is a client 15 who may want to submit samples to any 16 one of three or more qualified labs to 17 perform some sort of in process 18 technology evaluation, process 19 optimization, perhaps in preparation to 20 coming into compliance with their NPDES 21 permit. Here, the engineers think 22 they might need 25 percent RSD,

23 accuracy of 80 to 120 percent, and,

24 again, the same 1 percent false 25 positives and 5 percent false negative 99 1 rate. 2 So, how can EPA, the State, or 3 the lab or the permittee decide 4 whether a method will be suitable for 5 meeting these measurement quality 6 objectives? What we would like to 7 show you is how we think the full 8 range method validation would work to 9 accomplish that end.

10 I am going to use that using 11 some data that was generated in an 12 interlaboratory validation study on 13 PCBs using Method 608 using capillary 14 electron capture analysis. This work 15 was undertaken by the Michigan 16 Manufacturers Association which is, I 17 think, a good illustration of how we 18 envision interested parties willing to 19 undertake this kind of work and then 20 provide the kind of interlaboratory 21 data necessary.

22 The study included 11 23 laboratories, 16 different 24 concentrations, looked for aroclor 1016 25 and 1260. They conducted the study 100 1 over a 20 week period to account for 2 routine measurement variability, and 3 overall, they generated 19,800 data 4 points.

5 Now, this is a plot of the 6 actual concentrations reported by the 7 laboratories versus the trues 8 concentrations. This gives you an 9 idea of the number of different spike 10 levels that were used, the spacing of 11 those spike levels, and we have got a 12 regression analysis using weighted 13 least squares and ordinary least 14 squares.

15 You can't really see the 16 details in this, but if you plot 17 recovery versus true concentration, you 18 can see that the study revealed that 19 there was a severe bias of the 20 recovery in the low concentration 21 range, and at the high range, they 22 observed a low bias.

23 Well, in evaluating the three 24 examples we have, we can look at these 25 data to see where a method would give 101 1 us acceptable recoveries. In the 2 first example, we assumed that EPA 3 would use a 50 to 150 percent recovery 4 criterion. That criterion could be 5 met using these data or should be met 6 using this method from 0.02 ppb up to 7 1.2 ppb, in this case, the highest 8 level studied in the experimental 9 design.

10 In our second example, the 11 State river survey, we said we 12 presumed that they would want 70 to 13 130 percent. Those criteria could be 14 met between 0.3 ppb and 0.35 ppb.

15 The third in process survey,

16 the criteria could be met between 0.04 17 and 0.18. And you can go on. I 18 mean, this is just an illustration of 19 how this kind of data can be used to 20 examine different measurement quality 21 objectives in the context of recovery.

22 Next, we look at precision.

23 Here is the raw data plotted,

24 interlaboratory standard deviation 25 versus true concentration. Again, I 102 1 am going to look at a different plot 2 that is a little easier to see. This 3 is relative standard deviation versus 4 true concentration. We stated our 5 criterion on the basis of relative 6 standard deviation.

7 They looked at four different 8 models, the models that Nancy mentioned 9 earlier that are in the 10 IQE... IDE... excuse me... and in this 11 case, they found the hybrid model and 12 the SL model... I can't remember which 13 one that is... but they found those two 14 matched the data best.

15 So, again, going to our 16 examples, the first one, EPA's goal of 17 10 percent RSD for their methods for 18 NPDES permit, we find that the method 19 never did achieve, in the entire range 20 of the method, a 10 percent RSD.

21 If you look at the 22 State's... our example 2, a State river 23 survey, the measurement quality 24 objective was 30 percent. That can be 25 achieved at any concentration above 103 1 0.08. 2 The third example was the in

3 process survey. We said plus or minus 4 25 percent RSD, and that can be 5 achieved at any concentration above 6 0.1.

7 So, now we have examined using 8 the full range validation recovery and 9 precision. The next is the false 10 positive/ false negative rate. This 11 particular study was, in fact, designed 12 using the ASTM IDE procedure. So, we 13 can calculate an LD for a 1 percent 14 false positive rate and a 5 percent 15 false negative rate, and this study 16 showed that that could be achieved at 17 levels above 0.082.

18 So, in conclusion, what we have 19 tried to do is illustrate how full 20 range validation can be used to 21 provide the necessary information for 22 all data users of Part 136 methods for 23 a variety of the different uses to 24 which those methods are put to meet 25 their measurement objectives. In the 104 1 first example, we found that the 2 methodology would not be suitable for 3 regulatory compliance testing, because 4 it was never able to achieve the 5 target 10 percent standard deviation.

6 In the second example, we found 7 the working range in the low end was 8 bounded at 0.082 by control of false 9 positives and false negatives, while 10 the upper end was the 0.35 was 11 controlled by acceptable recovery.

12 In our third example, we showed 13 that the low end of the workable range 14 for that purpose was 0.1 ppb driven by 15 the lowest level at which you could 16 achieve acceptable precision up to the 17 0.18 ppb upper limit which was again 18 controlled by acceptable recovery.

19 So, in general, that is how we 20 envision a full range validation study 21 being conducted and how it might be 22 used.

23 Thank you, Bill.

24 MR. TELLIARD: Thank 25 you. Our next speaker is Ed Askew,

105 1 Davenport Water Pollution Control 2 Laboratory.

3 MR. ASKEW: Good 4 morning. I will make it brief, short,

5 and to the point, and you all can go 6 to lunch or whatever you want to do,

7 Bill.

8 I am here... basically, I run a 9 wastewater treatment laboratory. I am 10 also a pretreatment coordinator for the 11 City of Davenport. In that city of 12 125,000 people, we have over 60 13 categorical and significant industries,

14 including the largest aluminum rolling 15 mill in the world, discharging to our 16 POTW.

17 I am also the Part 4000 18 coordinator for Standard Methods for 19 the examination of water and 20 wastewater, the oldest and most senior 21 voluntary consensus body for water and 22 wastewater. I also contribute... I 23 mean, I vote, read and vote, for D19 24 ASTM.

25 But what I am here for today is 106 1 specifically... and I will put my hat 2 on for this... the hat I wear all the 3 time is I am a chemist, and I work 4 the bench. I manage the laboratory.

5 I apply the analytical results to 6 enforcement under the pretreatment 7 program. So, this MDL document is of 8 importance to me and, actually, to my 9 people in the laboratory.

10 To start it out, I am in 11 support of the proposed rule. There 12 can be modifications to it. There can 13 be improvements to it, but I think I 14 will be key on that is I am in 15 support of it.

16 I have two procedural concerns 17 with this. Now, I am going to use a 18 term called classical analysis. As 19 Part 4000 coordinator, we do 20 distillations, colorimetric analysis.

21 Thank God I don't do BOD. I do it on 22 a regular basis, but I am not on that 23 committee. So, that is classical,

24 titrametric, gravimetric, colorimetric.

25 Instrumentation, if I use that, 107 1 I am applying that to GC, GC/ mass 2 spec, HPCL, and such.

3 So, first, I have two 4 procedural concerns, and they are very,

5 very small. The first is there is in 6 the technical support document for the 7 assessment of detection and 8 quantitation approaches which I read 9 3.3.1, sources of variability, page 10 3.19, quote, however, it has been 11 EPA's experience that concern over 12 matrix effects may be somewhat 13 overblown.

14 This statement which may be 15 true for instrumental analysis such as 16 GC and GC/ mass spec is not equally 17 true for classical analysis such as 18 ammonia, cyanide, phenols, oil and 19 grease, et cetera when applied to 20 direct industrial discharges to the 21 collection system of a POTW, something 22 that has not really been pointed out 23 here, but if an industry meets or is 24 below the categorical limits or the 25 local limits, they do not have to 108 1 retreat their discharge before it is 2 discharged to the POTW, publicly owned 3 treatment words, basically, the 4 municipal or the districts that collect 5 this wastewater and treat it.

6 Therefore, matrix interferences 7 are key when you apply 40 CFR 136 8 methods and the method detection limit 9 to those. So, that is something I am 10 just pointing out, that this needs to 11 be added in, considered, whatever.

12 And a case in point near and 13 dear to my heart... and I am sure Bill 14 is going to just cringe... is the 15 abysmal failure of the oil and grease 16 1664 method standard of stearic 17 acid/ hexadecane to work in industrial 18 wastewater from industries that have 19 high concentrations of surfactants such 20 as commercial laundries that wash rags 21 used in... those little red rags you 22 see that the mechanics wipe their 23 hands off with and such. We have one 24 industry or one industrial laundry in 25 our POTW collection system that bulk 109 1 washes those rags. Trying to get 1664 2 with stearic acid and hexadecane to 3 work in that just doesn't work. There 4 are just too many surfactants in 5 there.

6 Also, we have industries that 7 produce a lot of fatty acids, grease,

8 trying to get hexadecane and stearic 9 acid to work in that. So, you know,

10 that is the type of example where a 11 matrix needs to be considered in 12 method detection limits.

13 The other thing is the overall 14 method detection limits for 15 instrumental analysis are more clearly 16 defined by the analytical process than 17 those for classical analysis. This is 18 clearly illustrated by the comparison 19 of gas chromatography to a gravimetric 20 analysis such as total suspended solids 21 or residue non filterable.

22 A gas chromatography can have 23 increasing sample volume or mass loaded 24 on the column. So, you can increase 25 the sample volume. You can use a 110 1 Goerstel, a concentrator, purge trap, 2 something like that. You are limited 3 by the diameter of the column, the 4 thickness of the film on the 5 capillary, the stationary phase, and 6 by, basically, detector overload seen 7 on some, say, mass selective detectors 8 and mass spectrometer. You can only 9 put so much in it, and after that 10 point, it doesn't matter if you can 11 concentrate it more than that. Your 12 detector can only take so much.

13 But let's take total suspended 14 solids. According to Standard Method 15 2540D, 20th edition, you must be able 16 to recover at least 2.5 mg of solid.

17 It also mentions later on in the 18 method that you must filter up to... it 19 is allowing you up to 1 liter to 20 filter. In some cases, you can't get 21 2.5 mg out of a liter. So, what has 22 happened in a lot of State, they have 23 put the emphasis on mass collected and 24 not the amount of volume filtered.

25 So, they allow you to filter more than 111 1 a liter. Collect 2 liters, collect 3 2 liters, collect 10 liters, we still 3 want 2.5 mg.

4 So, basically, for the 5 classical analytical methods, as you 6 develop the method detection limits,

7 part of the process will be 8 determining what are you going to 9 measure. Volume? Mass? Unlike your 10 GC mass specs where you have an 11 electrical signal, in some cases, you 12 are actually talking about unique 13 physical measurements.

14 And let's not even bring in 15 temperature, pH, BOD. All of those 16 are applied to every NPDES permit 17 holder.

18 So, like anything else, if you 19 complain, you also have to have some 20 solutions. So, here is my solution to 21 it or proposed corrective action.

22 I think the EPA, along with the 23 voluntary consensus standard bodies 24 responsible for the analytical methods 25 in 40 CFR 136... and we are talking 112 1 about ASTM, Standard Methods, USGS,

2 AOAC... should develop a guidance 3 manual. Now, it has been pointed out 4 to me everybody is short of staff.

5 But for the voluntary consensus boards,

6 they already have people in place that 7 have helped develop these methods. I 8 know for Standard Methods, there are 9 joint task groups that actually develop 10 these methods. Those would be key 11 people to be brought into a guidance 12 manual development, and it 13 specifically, then, can outline how do 14 you develop the method detection limit 15 for this method.

16 Now, we have already seen a lot 17 of work done by ASTM that should not 18 be put to waste. I think in a 19 consensus building, we can develop a 20 guidance manual that can address these.

21 So, my suggestion is,

22 basically, completion of this meeting 23 today and in the process during the 24 public comment, the EPA should contact 25 the voluntary consensus standards 113 1 bodies such as Standard Methods, ASTM, 2 USGS, AOAC, et cetera... that includes 3 the Lab Practices Committee for the 4 Water Environment Federation and any 5 other voluntary group... and start 6 developing the guidance manual to 7 actually implement whether you call it 8 method detection limit, quantitation 9 limit, anything else, to provide the 10 working manual for the laboratorians or 11 the chemists, the analysts, to actually 12 put some of this into place.

13 So, in conclusion, the need of 14 the proposed rule establishing 15 procedures for detection and 16 quantitation is manifest by the spate 17 of lawsuits by special interest groups 18 bent on preventing the further 19 implementation of the Clean Water Act.

20 I clearly believe this. I see this 21 every day. Not to knock anybody in 22 here, but lawyers and business leaders 23 consistently use the lack of consensus 24 of the scientific community on 25 detection limits to negate good 114 1 analytical, scientific results. I have 2 seen this on a daily basis.

3 Though there will never be a 4 unified field theory... nods to Albert 5 Einstein... for detection limits, never 6 will be, the proposed EPA rule is an 7 excellent starting point, and the 8 authorship of the guidance manual will 9 only help clarify the majority of 10 concerns from the scientific community.

11 Thank you.

12 MR. TELLIARD: Thank 13 you.

14 We have on our calendar here a 15 one hour break where you can have your 16 caviar and champagne for lunch. For 17 those of you who can't do that, there 18 is a restaurant upstairs, there is a 19 deli downstairs, and a McDonald's 20 across the street. We didn't want to 21 give you more than an hour, because we 22 thought you might enjoy yourselves.

23 So, we will see you back here 24 at 1: 00 o'clock.

25 I would like to thank this 115 1 morning's speakers very much. Thank 2 you.

3 ( WHEREUPON, a luncheon recess was 4 taken.)

5 MR. TELLIARD: The first 6 speaker this afternoon is Patty Lee 7 from the Hampton Roads Sanitary 8 District.

9 MS. LEE: Thank you all 10 very much for coming back after lunch.

11 I know that could have been a 12 challenge.

13 Jim Pletl and myself are going 14 to be commenting on our issues that we 15 have with the proposed MDL procedure,

16 and we would like to thank EPA for 17 opening this forum up for us to give 18 our comments. We are also 19 representing, in addition to HRSD, we 20 are representing the Virginia 21 Association of Municipal Wastewater 22 Agencies, and we are also going to be 23 providing comments to AMSA on detection 24 and quantitation.

25 Our general comments... we would 116 1 like to start with that and then 2 address each one of the six criteria 3 that EPA used for the development of 4 the technical support document. First 5 of all, we would like to say that we 6 agree with ACIL's proposal. We 7 support it in concept. There are a 8 few things that we might would like to 9 have changed or whatnot, but we will 10 provide that in the comments to EPA 11 later on.

12 The ACIL proposal is user 13 friendly and has minimal costs 14 associated with it, because you are 15 basically using QA/ QC criteria that you 16 already have with each method batch or 17 each analytical batch anyway.

18 Therefore, it makes it very affordable 19 and something that your bench analysts 20 can use without a lot of problems.

21 ACIL's approach, as we heard 22 earlier, does take into account false 23 negatives, false positives, and, most 24 importantly, it does correct for bias,

25 because that is definitely a problem 117 1 with some of the methods that we have.

2 Like several other people in 3 the room, I am also a member of the 4 ASTM committee that developed the IDE 5 and the IQE, and we looked at that 6 concept as well, and although we feel 7 that it has merit, it probably is more 8 appropriate. The most appropriate use 9 would be for interlaboratory data.

10 You have to have a lot of data points,

11 and the data points need to be in a 12 specific range. Then, you need 13 someone that has a good deal of 14 statistical knowledge to actually set 15 up and run the study.

16 So, because it is so onerous 17 and it is so large, we feel that 18 although it is very well suited for 19 interlaboratory MDL development such as 20 that associated with method validation,

21 we feel that it is a little too 22 onerous for a single laboratory to 23 have to do that.

24 So far as the first criterion 25 that EPA used which was that detection 118 1 and quantitation approaches should be 2 scientifically valid, our first comment 3 is they must be scientifically valid.

4 People are signing off on DMRs, so we 5 have to have good science behind these 6 limits that we are getting.

7 EPA's proposal is also still 8 concentration dependent in that it is 9 based on the EDL. If you select the 10 wrong concentration, you could end up 11 with falsely low or falsely high MDLs.

12 It still does not really encompass 13 anything for bias.

14 Because of this, because it is 15 so concentration dependent, and because 16 it doesn't correct for bias, we feel 17 that we really can't meet the 18 condition of 99 percent confidence in 19 the analyte being present. Therefore,

20 we feel that it fails condition 3 of 21 criterion 1.

22 Criterion 2 states that the 23 approach should address demonstrated 24 expectations of the laboratory and 25 method performance, including routine 119 1 variability. Although the ML proposal 2 or concept that they have on the books 3 now does address, you know, rounding 4 and things like that, it really 5 doesn't address are we really at, can 6 we really even see ten standard 7 deviations of the MDL, because that is 8 currently what EPA's ML is, that is,

9 ten standard deviations of the MDL.

10 It works really good for some methods,

11 but there are a lot of methods out 12 there where ten standard deviations is 13 just inadequate, and there is no 14 guidance on what to do if that happens 15 to be the case.

16 Another issue is that the long

17 term variability is not addressed by 18 EPA's procedure. Standard deviation,

19 over time, does increase, and that 20 will ultimately affect the laboratory's 21 performance of this.

22 Again, the ML is based on a 23 multiplication factor, and it really is 24 not suitable for one size fits all.

25 Really, we need some guidance on if 120 1 they decide to stay with this concept, 2 what do we do when ten standard 3 deviations is just simply inadequate?

4 We also feel that the values,

5 the MDL and ML values, that you obtain 6 should be something that can be 7 routinely achieved by most competent 8 laboratories, and the value shouldn't 9 change. Based on what happens now is 10 you develop an MDL one day, and it is 11 basically good for that one day. It 12 may only be good for one calibration 13 in some cases. So, we really need 14 something when we get a number it is a 15 firm number, not something that is 16 floating.

17 For criterion 3, the approach 18 should be supported by a practical and 19 affordable procedure that a single 20 laboratory can use to evaluate method 21 performance. We agree with this.

22 Bench analysts need to be able to 23 understand and to use these types of 24 concepts.

25 ACIL's concept really is cost 121 1 effective, and it is easy to use,

2 because you are already using data 3 points that you are going to generate 4 in your analytical batches anyway. 5 Another problem with evaluating 6 method performance is the lack of any 7 mention of a guidance on how to 8 determine significant figures in the 9 MDL and the ML. We are currently 10 seeing regulatory requirements that 11 specify the number of significant 12 figures that must be reported, and, in 13 a lot of cases, it exceeds the 14 capability of the method.

15 So, that is a problem when we 16 are reporting data that is not 17 certain, and we have people having to 18 sign off on DMRs. So, it is very 19 important that we have certain data,

20 we have an approach or a guide on how 21 to determine the significant figures 22 and how to, you know, appropriately 23 round the data, because we really need 24 consistency in this area for data sets 25 to be comparable.

122 1 Criterion 4 states the 2 detection level approach should 3 identify the signal or estimated 4 concentration at which there is 99 5 percent confidence that the substance 6 is actually present when the analytical 7 method is performed by experienced 8 staff in a well operated laboratory.

9 Because the EPA's MDL procedure is 10 concentration dependent, it does not 11 correct for bias, and that, in a lot 12 of cases, could make it absolutely 13 impossible to consistently recognize 14 the analyte as being present at a 99 15 percent level of confidence. So, we 16 really, again, we need a number that 17 is going to be 18 firm.

19 Criterion 5 states that the 20 quantitation limit approach should 21 identify the concentration that gives a 22 recognizable signal that is consistent 23 with the capabilities of the method 24 when the method is performed by 25 experienced staff in a well operated 123 1 laboratory. Because the ML is a 2 straight multiplication factor of your 3 MDL and you are trying to achieve ten 4 standard deviations of your MDL, you 5 can't always produce a recognizable 6 signal at ten standard deviations of 7 your ML, and there is no guidance on 8 how to handle this situation if your 9 method or your capabilities simply 10 can't meet that. 11 So, we would like to see 12 something in the regulations stating 13 that this is how it works.

14 Significant figures, again, we have to 15 report self to fulfill regulatory 16 requirements that really does exceed 17 the capabilities of the method.

18 Unfortunately, that is another 19 instance of the regulations ruling the 20 science instead of the other way 21 around. We really need to base this 22 stuff on good science and have the 23 regulations follow the good science.

24 The rounding rules that were in 25 the ML procedure don't always work,

124 1 because, in some instances, when you 2 have to round down, you are violating 3 some of the basic precepts of the 4 quantitation level. What is happening 5 is, when you are rounding down, you 6 are going into an area of higher 7 uncertainty, because you are 8 approaching your MDL.

9 Then, that ultimately will mean 10 that you can't always and consistently 11 meet a 99 percent confidence level 12 that your analyte is there, let alone 13 quantify it, and this is a real issue, 14 because we need to have data that the 15 laboratory and the data user feel 16 confident in.

17 It is very difficult, as a lab 18 person, to know I am reporting more 19 significant figures that I really 20 should be, than I really can see, but 21 we have the regulatory requirements,

22 and it is also very difficult when you 23 stop and think maybe I really don't 24 have a good MDL. Maybe it truly was 25 a snapshot of the one day that I did 125 1 it, and because of the way the ML is 2 structured, if your MDL is flawed,

3 then your ML is going to be flawed as 4 well.

5 So, what we would like to see 6 is that to come up with a procedure in 7 which the data user and the laboratory 8 have a great deal of confidence in the 9 MDL and ML, because from a regulatory 10 perspective, it is very important when 11 you sign off on your DMR that you feel 12 really good and you have very sound 13 data.

14 With that, I would like to turn 15 this over to my colleague, Jim Pletl,

16 for further comments. 17 MR. TELLIARD: Thanks,

18 Patty.

19 MR. PLETL: Can 20 everybody hear me okay? Patty just 21 went through five of the six criteria 22 that EPA used in trying to select an 23 approach for calculating MDLs and MLs,

24 and I am going to focus on the sixth 25 criterion which is really the 126 1 regulatory implications of the 2 proposal. To refresh your memory,

3 criterion number 6 is the applicability 4 of the approach to decisions under the 5 Clean Water Act and support for State 6 and local implementation of 7 measurements.

8 EPA states that the MDL/ ML 9 approach proposed has been successfully 10 applied and successfully used across 11 the country, but EPA did not define 12 the term successful in its support 13 document. It appears that their 14 definition is based on the fact that 15 this approach has been used in various 16 regulatory programs over a period of 17 time.

18 Success is usually based on 19 attainment of a goal or goals rather 20 than how long or how hard you have 21 been trying to meet that goal. In 22 this particular case, a successful 23 approach would meet the needs of all 24 of its users while minimizing decision 25 error rates, but it is not evident 127 1 that that evaluation has actually taken 2 place.

3 For example, successful 4 application of an MDL approach might 5 include MDLs and MLs where the limits 6 are low enough to ensure protection of 7 the environment but to allow the 8 reporting of data with adequate 9 certainty that the data generated was 10 actually reliable.

11 So, what I am going to do is I 12 am going to go over a number of issues 13 that were either touched on in the 14 technical support document that relate 15 to regulatory issues and talk a little 16 bit about those implications and why 17 we believe, at least, that we 18 really don't have evidence or, at 19 least, obvious evidence that the 20 approach that we are going to be using 21 or the one we have been using really 22 characterizes success. 23 EPA has chosen not to include 24 establishment of compliance thresholds 25 using MDLs or MLs as a criterion for 128 1 selecting an approach for developing 2 detection and quantitation limits, but 3 this may result in selecting an 4 approach that does not meet the needs 5 of all of its users. Data collected 6 to meet NPDES permits are often 7 interpreted relative to detection and 8 quantitation limits, to determine 9 reasonable potential to exceed water 10 quality standards, limit magnitude, how 11 stringent your limit actually is, and 12 whether you can comply with that 13 limit.

14 The proposed detection and 15 quantitation limits which do not 16 reasonably address all factors 17 influencing them will lead to either 18 false positive or false negative errors 19 in their procedures, and the procedures 20 I am referring to are these RP, the 21 reasonable potential, and the limit 22 calculations. Therefore, the certainty 23 and reliability of detection or 24 quantitation limits will directly 25 determine whether the discharger 129 1 receives a legally enforceable permit 2 limit, the magnitude of that limit,

3 and whether a discharger can comply 4 with that limit.

5 It is imperative that the 6 selected approach provide detection and 7 quantitation limits that minimize the 8 error rates of these procedures. Our 9 review indicates that the proposal made 10 by EPA does not adequately address all 11 those factors affecting the reliability 12 of the detection and quantitation 13 limits and that the uncertainties of 14 regulatory decisions using the EPA 15 approach are and will be unacceptable.

16 So, in our view, we have two 17 choices. We can either eliminate the 18 use of the MDL and ML as a regulatory 19 decision point and use it only as a 20 measure of lab performance, or we can 21 continue to use MDLs in regulatory 22 decisions, but we have to adjust the 23 way we calculate them to provide the 24 certainty required to make regulatory 25 decisions. It is one or the other.

130 1 EPA makes quite clear in the 2 support document that it is focused on 3 driving MDLs lower to make sure that 4 we protect the environment, and,

5 apparently, from the way it is 6 currently calculated, they believe that 7 the approach adequately addresses false 8 positives when they use the 99 percent 9 confidence level. However, there is 10 no quantitative evaluation showing that 11 the uncertainty associated with false 12 negatives is balanced by that for 13 false positives.

14 In fact, the decision error 15 rates, both analytically and in the 16 regulatory program, should have been 17 used as a criterion for selecting an 18 approach. This evaluation is necessary 19 to conclude that the proposed approach 20 is defensible and will truly lead to a 21 successful program.

22 Another issue I want to talk 23 about is censoring which was briefly 24 addressed in the support document.

25 The proposal does not address how data 131 1 should be censored relative to the 2 detection and quantitation limits.

3 Yet, as I just talked about, these 4 limits have a great deal of impact on 5 permit issues.

6 Although it does not believe,

7 EPA does not believe, that censoring 8 is an issue when selecting an MDL or 9 ML approach, EPA states in its support 10 document that it is acceptable to use 11 the worst case approach of reporting 12 data less than MDL or ML as equal to 13 these values. So, even though they 14 don't think censoring should be 15 addressed, they voice their opinion as 16 to how it should be addressed.

17 The detection and quantitation 18 limits themselves and how data is 19 interpreted relative to these limits 20 are critical components of the NPDES 21 program. Without instruction in the 22 regulation or guidance on how to 23 interpret data less than detection and 24 quantitation, there will be and 25 currently is great variability among 132 1 the States and EPA regions on whether 2 a permittee should get a limit, the 3 magnitude of that limit, and compliance 4 with the limit.

5 EPA acknowledges this openly in 6 their support document. EPA also 7 states that censoring is an issue no 8 matter which approach you choose. We 9 agree and believe that this is 10 justification in itself that censoring 11 should be addressed in the regulation.

12 Given the current proposal,

13 dischargers characterized by the same 14 data will be held to very different 15 NPDES requirements, depending on their 16 location in the country rather than 17 the quality of that discharge. This 18 approach does not adequately balance 19 the risk of decision errors and 20 unacceptably displaces regulatory 21 decisions from the sample quality which 22 is where it should be to the 23 geographic location of the lab or the 24 regulatory agency that is making the 25 decisions.

133 1 Additionally, lack of regulation 2 on this topic will lead to data sets 3 that cannot be pooled between 4 regulatory agencies. If you have 5 different censoring approaches, you are 6 going to have a mishmash of all kinds 7 of data out there, and this is going 8 to hinder any efforts... we heard Jim 9 Hanlon talk earlier in the week about 10 watershed management... this certainly 11 is going to hinder that effort.

12 Patty talked a little bit about 13 the issue of DMR certification. If 14 permittees are unsure as to whether 15 data were truly detectable or 16 quantifiable, based on the approach 17 used to calculate these reporting 18 limits, or how data is censored, how 19 can permittees certify the data as 20 true and accurate? Once data is 21 signed, it is no longer open to 22 interpretation or challenge by a 23 permitted. Therefore, it is critical 24 that permittees be very sure that data 25 deemed detectable of quantifiable 134 1 actually meet these definitions with a 2 known degree of uncertainty.

3 EPA has stated in the WET 4 methods rule... and WET stands for Whole 5 Effluent Toxicity, and these were 6 methods that were recently finalized 7 back in November of 2002... that the 8 detection limit concept does not apply 9 to WET methods, and such limits cannot 10 be calculated for WET methods.

11 WET methods have analytical 12 limitations just like chemistry. EPA 13 has attempted to address these 14 limitations using its PMSD approach,

15 and those of you who were here earlier 16 in the week heard EPA talk about that.

17 Although the intent of this approach 18 is to address, in part, false 19 positives, it does not provide the 20 protection against false positives that 21 is necessary in a regulatory program.

22 AMSA, Association of 23 Metropolitan Sewage Agencies, has met 24 with EPA and provided information 25 allowing the development of WET 135 1 censoring points comparable to EPA's 2 MDL, but it is too early to say where 3 that is going to go. The regulation 4 must provide an approach for 5 calculating detection limits for the 6 WET methods if it continues to hold 7 that the WET methods must be used to 8 assess compliance with water quality 9 standards and form the basis for 10 legally enforceable NPDES limits.

11 EPA plans on developing MDLs 12 and MLs for all new methods that it 13 develops, but it also needs to think 14 about the ones that it has already 15 promulgated and to go back and make 16 sure that we have those spaces filled 17 in.

18 In its proposal, EPA has not 19 included a procedure for calculating 20 MDLs and MLs for methods rather than 21 MDLs and MLs for laboratories. EPA 22 states that labs must achieve 23 acceptable levels of sensitivity, but 24 it does not provide a procedure for 25 defining what is acceptable.

136 1 Standards must be set so that 2 using environmental data to implement 3 regulatory programs like the NPDES 4 permit program will know the 5 limitations of those methods and 6 accommodate those limitations. Failure 7 to do this leaves regulators and 8 permittees not knowing whether specific 9 methods are capable of reliably 10 supporting various regulatory decisions 11 and actions. This regulation must 12 include methods for calculating MDLs 13 and MLs which most competent labs can 14 meet before it is finalized and 15 promulgated.

16 Methods developed by consensus 17 based organizations must also provide 18 the data that will meet this new 19 requirement before EPA can adopt them 20 for Clean Water Act use. Clearly,

21 there will always be inconsistency in 22 lab performance and data uncertainty 23 without such a standard.

24 There are some regulatory 25 programs other than those for the 137 1 Clean Water Act that reference the MDL 2 procedure proposed by EPA. Yet, this 3 current rule, as I understand it, is 4 restricted to Clean Water Act programs.

5 For consistency in decision making 6 across programs, it would seem 7 desirable to have one approach for all 8 programs.

9 It is also desirable for labs 10 conducting analyses for multiple 11 programs to use the same procedure for 12 each program. Otherwise, you have 13 multiple efforts going on within a 14 single laboratory.

15 EPA does not provide sufficient 16 reasons for why standardized procedures 17 for all programs using the MDL and ML 18 approach is not being proposed, and we 19 do not see a scientific or technical 20 reason why standardization across 21 programs cannot occur. We ask that 22 this issue be revisited and reasons be 23 better communicated.

24 Finally, EPA states in its 25 support document that all new methods 138 1 developed by EPA for promulgation in 2 40 CFR should use standardized 3 procedures for calculating MDLs and 4 MLs, but it will not support this 5 standard for other organizations 6 developing methods. EPA provides four 7 reasons for this.

8 First, EPA states that no 9 single method will meet the needs of 10 all organizations and applications.

11 Yet, it is not clear that an 12 identification and an evaluation of 13 these needs has been conducted.

14 We would hold that not only the 15 needs of organizations developing the 16 method should be addressed but all 17 users of data, including the 18 permittees.

19 The Office of Water is 20 concerned that a new approach would 21 expend resources in other programs in 22 order to use the new mandatory 23 approach. This may be true, but if 24 the approach is better technically, and 25 it meets all the users' needs, 139 1 basically, we are inferring that we 2 shouldn't go there even if we come up 3 with a better method.

4 The third, EPA believes that 5 fewer methods would be promulgated 6 under 40 CFR if other organizations do 7 not adopt the new mandatory approach 8 to calculating MDLs and MLs. Well,

9 quite frankly, promulgation of new 10 methods is currently at a standstill,

11 and it can't get much worse.

12 Finally, EPA holds that the new 13 approach could be burdensome, but it 14 didn't acknowledge when it did its 15 analysis that this should be precluded 16 by its criterion number 3 which says 17 that it has to be practical and 18 affordable.

19 So, EPA's reasons for not 20 requiring one approach for calculating 21 MDLs and MLs is not persuasive, and 22 failure to do so will simply 23 perpetuate inconsistency in the 24 reliability of the data. In summary,

25 we believe that use of the MDL and ML 140 1 concepts in regulatory programs demands 2 that the promulgated approach 3 adequately address the issues 4 associated with implementation of these 5 concepts in these programs.

6 Thanks.

7 MR. TELLIARD: Thanks,

8 Jim. Appreciate it.

9 I have an announcement. We are 10 asking if anyone has picked up a 11 leather daytimer belonging to Barbara 12 Burn... pardon? Beard. That is how 13 you spell Beard? Okay. And if so,

14 would you please return it to Jen who 15 is at the registration table? So, we 16 are looking for a leather daytimer.

17 Thank you.

18 Our next speaker is Scott 19 Houston.

20 MR. HOATSON: Hoatson.

21 MR. TELLIARD: Hoatson.

22 MR. HOATSON: Hello.

23 From our standpoint... I am from Delmar 24 Analytical, and I have been in the 25 commercial lab business for a lot of 141 1 years, going on 25, I think, atthis 2 point in time.

3 A lot of the MDL issues that we 4 have or that I see are not necessarily 5 from a procedure itself but on how it 6 is used, and if we don't change how it 7 is used, then we need to change the 8 procedure. So, right now, up here, if 9 data reported below the level of 10 calibration wasn't used for compliance 11 purposes, we probably wouldn't even be 12 here, because this is where the 13 biggest argument is.