Technical Support Document for the Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Not Intended for Circulation Do Not Cite, Quote, or Distribute August 2, 2002 Draft Document for Peer Review  Do Not Circulate  August 2002 i Acknowledgments This document was prepared by Maria Gomez Taylor, Henry D. Kahn, William A. Telliard, Khouane Ditthavong, and Charles E. White, of the Engineering and Analysis Division in EPA's Office of Water. Harry McCarty and Lynn Riddick with DynCorp I& ET and Dale Rushneck with Interface, Inc. provided assistance under EPA Contract No. 68 C 01 091. Sidina Dedah and Kathleen Stralka with Science Applications International Corporation provided assistance under EPA Contract No. 68 C 99 233.

Questions or comments about general aspects of this assessment should be addressed to:

William A. Telliard USEPA ( 4303T) 1200 Pennsylvania Avenue, NW Washington, DC 20460 telliard. william@ epa. gov Questions or comments about statistical issues related to this assessment should be addressed to:

Henry Kahn USEPA ( 4303T) 1200 Pennsylvania Avenue, NW Washington, DC 20460 kahn. henry@ epa. gov Draft Document for Peer Review  Do Not Circulate  August 2002 ii Draft Document for Peer Review  Do Not Circulate  August 2002 iii Table of Contents Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  1 1.2 Clause 6 Settlement Agreement Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  1 1.2.1 Clause 6a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  1 1.2.2 Clause 6B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  2 1.2.3 Clause 6d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  2 1.2.4 Clause 6e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  2 1.2.5 Clause 6F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  2 1.3 EPA's Approach to Conducting this Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  3 1.3.1 Study Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  4 1.3.2 Material and Data used in the Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  4 <IP_ADDRESS> EPA's ICP/ MS Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  5 <IP_ADDRESS> EPA's Episode 6000 Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  5 <IP_ADDRESS> EPA's Episode 6184 Study ( the " GC/ MS Threshold Study") . . . . . . . 1  6 <IP_ADDRESS> AAMA Metals Study of Methods 200.7 and 245.2 . . . . . . . . . . . . . . . 1  7 <IP_ADDRESS> Method 1638 Interlaboratory Validation Study . . . . . . . . . . . . . . . . . . 1  7 1.4 Terminology used in this Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  7 Chapter 2 Overview and History of Detection and Quantitation Limit Concepts . . . . . . . . . . . . . . . . . 2  1 2.1 Currie's Call for Standardization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  1 2.2 Development of the MDL and ML as Practical Embodiments of Currie's Proposal . . . . . 2  3 2.2.1 Method Detection Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  3 2.2.2 Minimum Level of Quantitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  4 2.3 Concepts Advanced by Other Organizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  5 2.3.1 EPA Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  5 2.3.2 Industry supported Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  7 2.3.3 Concepts Advocated by the Laboratory Community and Voluntary Consensus Standards Bodies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  8 Chapter 3 Issues Pertaining to Detection and Quantitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  1 3.1 Analytical Chemistry Concepts Pertaining to Detection and Quantitation . . . . . . . . . . . . 3  2 3.1.1 Blank versus Zero Concentration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  2 3.1.2 Lack of Instrument Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  2 3.1.3 Matrix Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  3 3.1.4 Measurement Quality over the Life of a Method . . . . . . . . . . . . . . . . . . . . . . . . . . 3  5 3.2 CWA Regulatory Issues Affecting Detection and Quantitation . . . . . . . . . . . . . . . . . . . . . 3  5 3.2.1 Detection and Quantitation Limit Applications Under CWA . . . . . . . . . . . . . . . . . . 3  5 <IP_ADDRESS> Method Development and Promulgation . . . . . . . . . . . . . . . . . . . . . . . 3  5 <IP_ADDRESS> Method Performance Verification at a Laboratory . . . . . . . . . . . . . . . . 3  7 <IP_ADDRESS> National Pollutant Discharge Elimination System . . . . . . . . . . . . . . . . 3  7 <IP_ADDRESS> Non Regulatory Studies and Monitoring . . . . . . . . . . . . . . . . . . . . . . . 3  9 Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 iv 3.2.2 Descriptive versus Prescriptive Uses of Lower Limits to Measurement . . . . . . . . . 3  9 3.2.3 Compliance Evaluation Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  10 3.2.4 Accepting the Procedures of Voluntary Consensus Standards Bodies . . . . . . . . . . 3  11 3.2.5 National versus Local Standards for Measurement . . . . . . . . . . . . . . . . . . . . . . . . 3  12 3.2.6 Cost and Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  12 <IP_ADDRESS> Implementation of a Detection/ Quantitation Limit Procedure by a Method Developer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  13 <IP_ADDRESS> Implementation of a Detection/ Quantitation Limit Procedure by a Laboratory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  13 3.2.7 Use of a pair of related detection and quantitation procedures in all Clean Water Act applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  13 3.3 Statistical Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  14 3.3.1 Sources of Variability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  14 3.3.2 Censoring Measurement Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  14 3.3.3 Outliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  17 3.3.4 Criteria for the Selection and Appropriate Use of Statistical Models . . . . . . . . . . 3  18 <IP_ADDRESS> Short History of Modeling Measurement Results . . . . . . . . . . . . . . . . 3  18 <IP_ADDRESS> Criteria for Selecting Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  20 <IP_ADDRESS> Current Practices with Available Data . . . . . . . . . . . . . . . . . . . . . . . . 3  21 3.3.5 Methodology for Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  22 3.3.6 False Positives and False Negatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  22 3.3.7 Statistical Prediction and Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  24 <IP_ADDRESS> Prediction Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  24 <IP_ADDRESS> Tolerance Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  25 Use of Tolerance and Prediction in Setting Detection and Quantitation Levels . . 3  25 3.3.8 Design of Detection and Quantitation Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  25 <IP_ADDRESS> Spike Concentrations and Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . 3  26 <IP_ADDRESS> Probability Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  26 <IP_ADDRESS> Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  26 Chapter 4 Evaluation Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4  1 4.1 Criterion 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4  1 4.2 Criterion 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4  1 4.3 Criterion 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4  2 4.4 Criterion 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4  3 4.5 Criterion 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4  4 4.6 Criterion 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4  5 Chapter 5 Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  1 5.1 Detection Limit Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  1 5.1.1 Evaluation of the MDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  1 <IP_ADDRESS> Description of the MDL Concept and Procedure . . . . . . . . . . . . . . . . . . . . 5  1 <IP_ADDRESS> Assessment of the MDL Against the Evaluation Criteria . . . . . . . . . . . . . . 5  3 5.1.2 Evaluation of the ASTM International Interlaboratory Detection Estimate ( IDE) . 5  6 <IP_ADDRESS> Description of the IDE Concept and Procedure . . . . . . . . . . . . . . . . . . 5  6 <IP_ADDRESS> Assessment of the IDE Against the Evaluation Criteria . . . . . . . . . . . . 5  7 5.1.3 Evaluation of the ACS Limit of Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  10 <IP_ADDRESS> Description of the ACS LOD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  10 <IP_ADDRESS> Assessment of the LOD Against the Evaluation Criteria . . . . . . . . . . 5  11 5.1.4 Evaluation of the IUPAC/ ISO Critical Value ( CRV) . . . . . . . . . . . . . . . . . . . . . . . 5  12 Table of Contents Draft Document for Peer Review  Do Not Circulate  August 2002 v <IP_ADDRESS> Description of the ISO/ IUPAC Critical Value ( CRV) Concept and Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  12 <IP_ADDRESS> Assessment of the CRV Against the Evaluation Criteria . . . . . . . . . . 5  13 5.1.5 Evaluation of the IUPAC/ ISO Detection Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  14 <IP_ADDRESS> Description of the IUPAC/ ISO Detection Limit Procedure . . . . . . . . 5  14 <IP_ADDRESS> Assessment of the ISO/ IUPAC MDV Against the Evaluation Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  15 5.2 Quantitation Limit Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  16 5.2.1 Assessment of the EPA Minimum level of Quantitation ( ML) . . . . . . . . . . . . . . . 5  16 <IP_ADDRESS> Description of the ML Concept and Procedures . . . . . . . . . . . . . . . . 5  16 <IP_ADDRESS> Assessment of the ML against the Evaluation Criteria . . . . . . . . . . . . 5  17 5.2.2 Assessment of the IQE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  19 <IP_ADDRESS> Description of the IQE Concept and Procedure . . . . . . . . . . . . . . . . . 5  19 <IP_ADDRESS> Assessment of the IQE Against the Evaluation Criteria . . . . . . . . . . . 5  20 5.2.3 Assessment of the ACS Limit of Quantitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  23 <IP_ADDRESS> Description of the ACS LOQ Concept and Procedure . . . . . . . . . . . . 5  23 <IP_ADDRESS> Assessment of the ACS LOQ Against the Evaluation Criteria . . . . . . 5  24 5.2.4 Assessment of the IUPAC/ ISO Limit of Quantitation . . . . . . . . . . . . . . . . . . . . . . 5  25 <IP_ADDRESS> Description of the ISO/ IUPAC LOQ Concept . . . . . . . . . . . . . . . . . . 5  25 <IP_ADDRESS> Assessment of the IUPAC/ ISO LOQ Against the Evaluation Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  25 Chapter 6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6  1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . R  1 Appendix A Literature Search Regarding Detection and Quantitation Limit Concepts . . . . . . . . . . . . A 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A 1 How the search was conducted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A 1 On line citation index search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A 1 General on line literature search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A 1 How the results are presented . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A 2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A 3 Appendix B is not done yet, but it will go here when it is. It will be sent to the reviewers shortly. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B 1 Appendix C is not done yet, but it will go here when it is. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C 1 Appendix D Draft Revised MDL Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D 1 Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 vi List of Figures Figure 2 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  1 Figure 2 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  2 Figure 3 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  15 Figure 3 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  16 Figure 3 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  17 Figure 3 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3  19 Draft Document for Peer Review  Do Not Circulate  August 2002 1 1 Chapter 1 Introduction 1.1 Background On June 8, 1999 ( 64 FR 30417), EPA promulgated ( i. e., published in a final rule) Method 1631B: Mercury in Water by Oxidation, Purge and Trap, and Cold Vapor Atomic Fluorescence Spectrometry for use in Clean Water Act programs. The method was developed specifically to measure mercury at ambient water quality criteria levels and includes a method detection limit ( MDL) of 0.2 ng/ L ( ppt).

Following promulgation, a lawsuit was filed challenging EPA on the validity of the method. The basis of the challenge included several specific aspects of Method 1631 as well as the general procedures used to establish the MDL and minimum level of quantitation ( ML) published in the method although the promulgated MDL of 0.2ng/ L and ML of 0.5 ng/ L were not affected by the settlement.

In order to settle the lawsuit, EPA entered into a settlement agreement with the Alliance of Automobile Manufacturers, Inc., the Chemical Manufacturers Association, and the Utility Water Act Group ( collectively the " Petitioners") and the American Forest and Paper Association (" Intervenor") on October 19, 2000 ( the " Settlement Agreement"). Under the terms of the Settlement Agreement, Clause 6 EPA agreed to perform an assessment of detection and quantitation limit concepts. The complete text of Clause 6 is provided in Exhibit 1 1. A summary of Clause 6 is provided in Section 1.2. This is followed by a description of EPA's approach to the assessment, including the material and data evaluated ( Section 1.3).

1.2 Clause 6 Settlement Agreement Requirements Clause 6 of the Settlement Agreement is titled Reassessment of Method Detection Limit and Minimum Level Procedures. The clause consists of five subclauses, a  b and d  f. ( There is no subclause c).

1.2.1 Clause 6a Clause 6a broadly defines the scope of the assessment and provides a schedule for completing the initial phase. Specifically, Clause 6a requires EPA to:

° Sign and forward to the Office of Federal Register ( OFR) a notice inviting public comment on a reassessment of existing EPA procedures for determining the detection and quantitation limits of contaminants in aqueous samples. ° Forward the notice to OFR on or before February 28, 2003. ° Provide a period of at least 120 days for public comment on the notice. ° At a minimum, include the MDL procedure published at 40 CFR part 136 and the ML procedure described in Section 17.8 of Method 1613B in the reassessment of detection and quantitation limits.

Clause 6a also provides EPA with the option of:

° Inviting comment on one or more alternative procedures for determining and describing test sensitivity, and ° Proposing modifications to the existing procedures. Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 1 2 1.2.2 Clause 6B Clause 6b requires EPA to subject its reassessment to a formal peer review and describes requirements associated with this peer review. Specifically, Clause 6b requires EPA to:

° Submit the reassessment of existing procedures ( including any proposed modifications thereof) and any evaluation of alternatives for peer review. Peer reviewers are defined as experts in the field of analytical chemistry and the statistical aspects of analytical data interpretation. ° Conduct the peer review in accordance with EPA's current peer review policies ° Prepare a charge to the peer review panel that requests the peer reviewers to consider: < Criteria for selection and appropriate use of statistical models < Methodology for parameter estimation < Statistical tolerance and prediction < Criteria for design of detection and quantification studies, including selection of concentration levels (" spiking levels") < Interlaboratory variability, and < Incorporation of elements of probability design.

1.2.3 Clause 6d Clause 6d requires EPA to provide the Petitioners and Intervenor ( the " litigants") with an opportunity for review of the Agency's assessment concurrent with the Clause 6b peer review.

1.2.4 Clause 6e Clause 6e requires EPA to provide the litigants with:

° An opportunity to meet periodically ( i. e., every six months) to discuss the Agency's progress during development of the assessment. ° A plan for performing the assessment on or before the second of these meetings and, ° Copies of relevant documents, where appropriate, in advance of these meetings 1.2.5 Clause 6F Clause 6f establishes a schedule and requirements concerning final action on the assessment. Specifically:

° On or before September 30, 2004, EPA is to sign and forward to the OFR a notice taking final action on the assessment, and ° Coincident with publication of this notice, EPA is provide the litigants with an opportunity to meet and discuss the implications of the final notice and/ or the need for any subsequent EPA action in light of the final notice. Chapter 1 Draft Document for Peer Review  Do Not Circulate  August 2002 1 3 Exhibit 1 1. Full Text of Clause 6 of the Settlement Agreement 6. Reassessment of Method Detection Limit and Minimum Level Procedures a. On or before February 28, 2003, EPA shall sign and forward to the Office of the Federal Register for prompt publication a notice inviting public comment on a reassessment of the existing Agency procedures for determination of sensitivity of analytic test methods for aqueous samples, specifically, EPA procedures for determining the detection limits and levels of quantitation of contaminants in aqueous samples, including, at a minimum, the " Definition and Procedure for Determination of the Method Detection Limit" published at 40 C. F. R. Part 136, Appendix B, as well as the " minimum level" procedures, which is described in section 17.8 of Method 1631B. The notice shall invite comment on EPA's evaluation of one or more alternative procedures for determining and describing test sensitivity. The notice also may propose modifications to the existing procedures. The notice shall invite public comment for a period of no less than one hundred twenty ( 120) days.

b. Prior to publishing the notice inviting public comment on EPA procedures for determining test sensitivity, EPA shall submit its reassessment of existing procedures ( including any proposed modifications thereof) and its evaluation of alternatives for peer review by experts in the field of analytical chemistry and the statistical aspects of analytical data interpretation. In its charge to the peer review panel, EPA shall request that the peer review consider: criteria for selection and appropriate use of statistical models; methodology for parameter estimation; statistical tolerance and prediction; criteria for design of detection and quantification studies, including selection of concentration levels (" spiking levels"); interlaboratory variability; and incorporation of elements of probability design. EPA ( or its authorized representative) shall conduct the peer review in accordance with EPA's current peer review policies in the January 1998 Science Policy Council Handbook ( EPA 100 B 98 00), including any subsequently developed EPA peer review documents that may revise or amend that Handbook.

[ c. Note  there is no clause " 6. c" in the settlement agreement]

d. During the peer review period, EPA shall also provide an opportunity for concurrent review and comment by the Petitioners and Intervenor.

e. In the development of the reassessment/ assessment of alternatives, EPA shall provide the Petitioners and Intervenor with a periodic opportunity to meet ( i. e., every six ( 6) months) on the Agency's progress. EPA shall prepare and present the Petitioners and Intervenor with the Agency's " plan" for conducting the reassessment/ assessment of alternatives on or before the second such periodic meeting. Where appropriate, EPA shall provide the Petitioners and Intervenor with copies of relevant documents in advance of such meetings.

f. On or before September 30, 2004, EPA shall sign and forward to the Office of the Federal Register for prompt publication a notice taking final action on the notice described in subparagraph 6. a. Coincident with publication of the notice of final action, EPA shall provide Petitioners and Intervenor an opportunity to meet to discuss the implications of the final notice and/ or the need for any subsequent EPA action in light of the final notice.

1.3 EPA's Approach to Conducting this Assessment This draft document details the Agency's assessment of detection and quantitation limits. This assessment is being conducted in accordance with a plan described in Section 1.3.1 and is based, in part, on an assessment of the data described in Section 1.3.2. Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 1 4 1.3.1 Study Plan EPA developed a technical approach for 1) conducting the assessment, and 2) complying with all applicable requirements of the Settlement Agreement. The approach was documented in a draft study plan that has since formed the general framework for the assessment described in this TSD. EPA also conducted a literature search to identify and review issues and concepts that should be considered when developing the plan. A summary of this literature review is provided in Appendix A to this TSD.

Although the Settlement Agreement did not require EPA seek formal peer review on its draft plan, the Agency chose to do so anyway. The peer review was initiated in December 2001, conducted in accordance with EPA's current peer review policies, and performed by two statisticians and two chemists. EPA reviewed comments and recommendations offered by these reviewers, and where appropriate, revised the plan to reflect their comments. EPA also reviewed, and where appropriate, revised the plan to reflect comments provided by the litigants following their concurrent review.

The study plan described roles and responsibilities for implementing the plan, provided a background discussion of detection and quantitation limit concepts, including the MDL and ML, and outlined a series of 11 events associated with the Agency's assessment of detection and quantitation limit concepts. The relationship between those planned events and this TSD is summarized in Exhibit 1 2.

1.3.2 Material and Data used in the Assessment In order to perform the assessment described in this document, EPA sought to collect documentation describing existing detection and quantitation limit concepts and procedures and data that could be used to evaluate these procedures.

Documentation concerning the existing concepts and procedures was obtained by performing a literature search as described in Appendix A to this TSD, and where necessary, by purchasing copies of concepts or procedures from the organizations that published them.

In performing this assessment, EPA hoped to identify a large body of data containing results generated at, below, and above the region of interest. EPA found, however, that few data sets existed that met these requirements. To date, EPA has been able to identify only five data sets that were of use in fully evaluating variability in the range of analytical detection and quantitation. Three of the five were developed by EPA for the express purpose of studying the relationship between measurement variation and concentration across a wide variety of measurement techniques and analytes. EPA refers to these data sets as " EPA's ICP/ MS data," " the Episode 6000 study," and " the Episode 6184 study." In all three cases, replicate measurement results from each combination of analyte and measurement technique were produced by a single laboratory over a wide range and large number of concentrations. The fourth data set was developed by the American Automobile Manufacturer's Association ( AAMA) for the purpose of estimating one particular kind of quantitation value. That quantitation value is called an Alternative Minimum Level ( see Gibbons et al., 1997). In the AAMA study, replicate results were measured at a limited number of concentrations by multiple laboratories using EPA Method 245.2 ( cold vapor atomic absorption, or CVAA) for mercury and EPA Method 200.7 ( inductively coupled plasma/ atomic emission spectroscopy, or ICP/ AES) for twelve other metals. The fifth data set was jointly gathered by EPA and the Electric Power Research Institute ( EPRI) to support an interlaboratory validation of EPA Method 1638.

The studies from which these five data sets were obtained are summarized in sections <IP_ADDRESS>  <IP_ADDRESS> below. Additional information about these studies can be found in Appendix B to this TSD. Chapter 1 Draft Document for Peer Review  Do Not Circulate  August 2002 1 5 <IP_ADDRESS> EPA's ICP/ MS Study The objective of this study was to characterize variability in EPA's draft Method 1638 for nine metals by inductively coupled plasma with mass spectroscopy ( ICP/ MS). The nine metals were silver, cadmium, copper, nickel, lead, antimony, selenium, thallium, and zinc. The ICP/ MS instrument used in this study averages triplicate scans to produce a single measurement of each element at each concentration. Such averaging is typical of ICP/ MS design and use.

In preparation for the study, the ICP/ MS was calibrated using triplicate scans averaged to produce a single measurement of 100, 1,000, 5,000, 10,000, and 25,000 nanograms per liter ( ng/ L) for each element. Originally, the instrument was calibrated using unweighted least squares estimates under the assumption of linearity. Subsequently, the analytical results were adjusted with weighted least squares estimates. Weighted least squares estimates are based on the assumption that variability increases with increasing analyte concentration.

Although the instrumentation has the capability to provide intensity results for each of the three scans at each concentration, averaging the three scans to produce a single measurement is the normal operating mode, and the average was used to produce the measurements in this study. Draft Method 1638 specifies the use of average response factors rather than least squares estimation of linear calibration curves, although it does allow for the use of such procedures.

All nine metals were spiked into water to produce solutions at concentrations of: 0, 10, 20, 50, 100, 200, 500, 1,000, 2,000, 5,000, 10,000, and 25,000 nanograms per liter ( ng/ L). Each solution was subsequently divided into seven aliquots, and each aliquot was analyzed beginning with the blank ( zero concentration) and followed by analyses from the highest to the lowest concentration.

Results at multiple mass to charge ratios, or m/ zs, were reported for each metal, although draft Method 1638 specifies only one m/ z for eight of the nine metals. For lead, m/ zs 206, 207, and 208 are specified. This study of variation only used the data associated with m/ zs that are specified in draft Method 1638.

<IP_ADDRESS> EPA's Episode 6000 Study The Episode 6000 study was designed to characterize the variability of results from 1/ 10th of the method detection limit to concentrations into the normal quantification range for a variety of measurement techniques. The analytes and analytical techniques studied were:

° Total suspended solids ( TSS) by gravimetry ° Metals by graphite furnace atomic absorption spectroscopy ( GFAA) ° Metals by inductively coupled plasma atomic emission spectrometry ( ICP/ AES) ° Hardness by ethylene diamine tetraacetic acid ( EDTA) titration ° Phosphorus by colorimetry ° Ammonia by ion selective electrode ° Volatile organic compounds in water by purge and trap capillary column gas chromatography with a photoionization detector ( GC/ PID) and electrolytic conductivity detector ( GC/ ELCD) in series ° Volatile organic compounds by gas chromatography with a mass spectrometer ( GC/ MS) ° Available cyanide by flow injection/ ligand exchange/ amperometric detection ° Metals by inductively coupled plasma spectrometry with a mass spectrometer ( ICP/ MS) Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 1 6 A total of five laboratories were employed for these analyses; each parameter and method combination was tested by one of these laboratories. A method detection limit ( MDL) study was conducted for each combination of analyte and analytical technique.

After determining the MDL, each laboratory analyzed seven replicates at a series of concentrations created at 100, 50, 20, 10, 7.5, 5.0, 3.5, 2.0, 1.5, 1.0, 0.75, 0.50, 0.35, 0.20, 0.15, and 0.10 times the MDL determined above. In a few instances ( described below), laboratories analyzed more than seven replicates.

A variant of the iterative procedure for detemining the method detection limit ( 40 CFR part 136 Appendix B) was used for organics. Methods for organics normally list many ( 15 to 100) analytes, and the response for each analyte is different. Therefore, to determine an MDL for each analyte, the concentration of the spike must be inversely proportional to the response. Making a spiking solution with 15 to 100 different concentrations is cumbersome and error prone. A more straightforward approach, and one used in this study at EPA's suggestion, was to run seven replicates at decreasing concentrations until signal extinction, then pick off the concentration( s) appropriate for the MDL. In some cases the laboratories picked off the concentrations, in others cases, EPA did. This approach was generally applied for organics analysis, however laboratories also had the option of using some combination of the monotonically decreasing concentrations described above and a few selected concentrations to achieve the desired spiking levels.

Spiked solutions were analyzed in order from the highest concentration to the lowest. This approach was used to ( 1) minimize carry over effects and ( 2) prevent the collection of some costly, but less than informative, data.

Carry over can occur when a high concentration sample is followed by a low concentration sample. Carry over is usually less than one percent but can be a few percent in some methods. Therefore, if a sample containing a pollutant at 100 times the MDL is followed by a sample containing that pollutant at only 0.1 times the MDL, the sample at 0.1 times the MDL could be compromised by carry over. Running the samples in decreasing order of the chosen spike levels is not expected to compromise successively lower measurements.

For methods that do not produce a signal for a blank, the signal will disappear somewhere below the MDL, i. e., a zero will be reported. Laboratories were told that when three nondetects ( out of seven measurements) were reported, it was not necessary to move to the next lower concentration, because it was not reasonable to have laboratories measure seven zeros, move to a lower level, measure seven zeros, etc.

<IP_ADDRESS> EPA's Episode 6184 Study ( the " GC/ MS Threshold Study")

Data from this study also were collected to characterize the variability of results from concentrations at, and sometimes lower than, the detection limit to spike concentrations well into the normal quantification range of the method. Specifically, the laboratory generated data for 82 semivolatile organic compounds by EPA Method 1625C ( semivolatile organic compounds by GC/ MS). MDLs were not determined for these compounds. Instead, a series of samples consisting of 17 concentration levels were prepared and analyzed as follows: 50.0, 20.0, 10.0, 7.50, 5.00, 3.50, 2.00, 1.50, 1.00, 0.75, 0.50, 0.35, 0.20, 0.15, 0.10, 0.075 and 0.050 ng/: L ( or : g/ mL). Each concentration level was analyzed in triplicate with the mass spectrometer thresholds set to 0, and again in triplicate with the mass spectrometer thresholds set to 1000, a level typical of that used in routine environmental analyses. As with the Episode 6000 data set, samples were analyzed in order from the highest to the lowest concentration, and measurements were made as often as possible using the same calibration. Chapter 1 Draft Document for Peer Review  Do Not Circulate  August 2002 1 7 <IP_ADDRESS> AAMA Metals Study of Methods 200.7 and 245.2 The American Automobile Manufacturer's Association conducted an interlaboratory study of EPA Method 200.7 ( metals by ICP/ AES) and Method 245.2 ( mercury by CVAA). Nine laboratories participated in the study, and each reported data for the following 13 metals: aluminum, arsenic, cadmium, chromium, copper, lead, manganese, mercury, molybdenum, nickel, selenium, silver and zinc. Study samples were analyzed by EPA 200.7 for 12 of the metals; mercury was determined by EPA 245.2.

The nine laboratories were randomized prior to the start of the study. Five matrix types ( including reagent water) were selected, including four that were representative of the automotive industry. Each matrix was spiked at five concentrations in a predetermined concentration range. Matrix A ( reagent water) was analyzed in all nine laboratories, and three laboratories analyzed each of the other four matrices. All analyses were repeated weekly over a five week period. As a result, a total of 6825 observations were obtained, which includes 2925 observations for matrix A ( 9 labs * 13 metals * 5 spike concentrations * 5 weeks) and 975 ( 3 labs * 13 metals * 5 spike concentrations * 5 weeks) for each of the other four matrices ( 6825 = 2925 + [ 975 * 4]). There were two missing values for chromium in matrix A from laboratories 1 and 9.

Starting from a blank, or unspiked sample, all target analytes were spiked at four concentrations to yield a total of five concentrations per matrix. Concentrations ranged from 0.01 to 10 : g/ L for mercury and selenium on the low end, and from 2.0 and 1000 : g/ L for mercury and selenium on the high end. In addition, the concentrations were matrix dependent. The same concentration ranges for each metal by matrix combination were used for all five weeks of the study.

<IP_ADDRESS> Method 1638 Interlaboratory Validation Study The Method 1638 interlaboratory validation study was conducted by EPA to evaluate performance of the method and to gather data that would allow revision of existing performance specifications, including detection and quantitation limits. To accommodate stakeholder interests and expand the scope of the study, the Electric Power Research Institute funded the distribution of additional samples to study laboratories.

A total of eight laboratories ( and a referee laboratory) participated in the study. The study was designed so that each participating laboratory would analyze sample pairs of each matrix of interest at concentrations that would span the analytical range of the method. Each laboratory was provided with 11 sample pairs ( a total of 22 blind samples). These included 1 filtered effluent pair, 1 unfiltered effluent pair, 4 filtered freshwater pairs, and 5 spiked reagent water pairs. All eight laboratories received and analyzed the same sample pairs ( a total of 176 analyses). To measure the recovery and precision of the analytical system, and to monitor matrix interferences, the laboratories were instructed to analyze matrix spike and matrix spiked duplicate samples on specified field samples in each filtered and unfiltered matrix. Laboratories were further instructed to spike aliquots of the designated MS/ MSD samples at 1 5 times the background concentration of the analytes determined by analysis of an unspiked aliquot of the sample. The laboratories also were instructed to perform all other QC tests described in the method, including the analysis of blanks.

1.4 Terminology used in this Document We use the term " quantitation" in this document because of its common usage among analytical chemists, even though we recognize that the term " quantification" ( i. e., the act of quantifying) is more appropriate. Also, when referring to detection and quantitation, we use the words " approach" or " concept" to refer, generically, to the procedures used to establish detection and quantitation limits or the theories on which those procedures are based. We use the word " limit" rather than " level" to indicate that the detection Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 1 8 Exhibit 1 2. Relationship of TSD to Planned Approach for Assessment of Detection and Quantitation Limit Concepts Event 1, Develop a detailed plan for responding to Clause 6: This event was completed in April 2002 when the draft plan was revised to reflect peer review and litigant comments.

Event 2, Identify and explore issues to be considered: The Settlement Agreement identified six specific issues that should be considered during the assessment of detection and quantitation limit concepts and subjected to formal peer review. During development of its technical approach, EPA identified a number of other issues that should be considered during assessment of detection and quantitation limits. EPA listed and described each of these issues in the study plan and noted that identification of issues is likely to be a dynamic process, in that as a suite of issues is identified and discussed, other issues may surface. Finally, EPA stated its intent to prepare an " issue paper" that fully explains and discusses each of the identified issues. Chapter 3 of this TSD serves the function of the " issue" paper described in the plan.

Event 3, Develop Criteria against which Concepts can be Evaluated: After fully considering all relevant issues, EPA planned to develop a suite of criteria that could be used to evaluate the suitability of various detection and quantitation procedures for use in CWA programs. Chapter 4 of this TSD provides and describes the criteria selected by EPA after its consideration of all pertinent issues.

Event 4, Evaluate Existing Procedures for Establishing Detection and Quantitation Levels: EPA planned to evaluate existing detection and quantitation limit concepts used or advanced 1) by voluntary consensus standards bodies ( VCSBs), 2) in the published literature, 3) EPA. As per the terms of the settlement agreement, the MDL and ML were explicitly targeted for inclusion. EPA committed to evaluating concepts published by ASTM International and ISO and to consider concepts and procedures offered by other organizations such as the American Chemical Society ( ACS) and the International Union of Pure and Applied Chemistry ( IUPAC), as well as other concepts that have been adopted by EPA for use in other programs or that were identified during EPA's review of the published literature. Chapter 2 describes the existing concepts that EPA evaluated in this assessment. Where appropriate, these concepts also are discussed in context to the issues that are identified and discussed in Chapter 3. Chapter 5 presents the results of EPA's assessment of each concept against the evaluation criteria established in Chapter 4.

Event 5, Development and Evaluation of Alternative Procedures: EPA planned to develop and evaluate alternative procedures only if the Agency's assessment of existing procedures suggested that modifications or alternatives to the existing procedures is needed. EPA noted that its primary objective in developing such alternatives ( or modifications) would be to address deficiencies noted in Event 4 and improve the performance of the procedures that best meet the criteria established in Event 3.

Event 6, Peer Review of the Agency's Assessment: Initiated August 2002. EPA will summarize and formally respond to peer review comments received on the assessment.

Events 7  11, Activities to be taken followed Peer Review. After considering peer review comments, EPA will finalize its strategy regarding the FR notice to be proposed per the terms of Settlement Agreement Clause 6a and take the actions necessary to ensure publication of that notice. and quantitation concepts are directed at the lowest concentration or amount at which an analyte is determined to be present ( detection) or may be measured ( quantitation). In choosing the word ` limit' we do not mean to imply any sense of permanence. We recognize that measurement capabilities improve over time, and that detection or quantitation ` limits' established today may be superceded by future developments in analytical chemistry. Draft Document for Peer Review  Do Not Circulate  August 2002 2 1 Figure 2 1 Chapter 2 Overview and History of Detection and Quantitation Limit Concepts It is not possible to measure the concentration of a substance in water all the way down to zero. As an analogy, consider the following example: imagine measuring an object less than 16th of an inch in length with a rule marked in 1/ 16th inch increments. How well can the length of the object be measured using only the ruler? Similar issues arise as chemists try to measure ever smaller concentrations of substances in water. In response to the challenges associated with measuring low concentrations, chemists have defined numerical values that provide points of reference for reporting and using measurement results. These values are usually referred to as detection and quantitation limits. This chapter provides an overview of detection and quantitation concepts and procedures in analytical chemistry and their use in Clean Water Act applications.

2.1 Currie's Call for Standardization Since 1968, most of the literature regarding detection and quantitation has referenced the work of Dr. Lloyd Currie, recently retired from the National Institutes of Science and Technology ( NIST, formerly the National Bureau of Standards). In 1968, Currie published a paper in which he reviewed the then current state of the art, presented a three tiered concept for detection and quantitation, and demonstrated his concept with operational equations for a single laboratory. In his paper, Currie reviewed eight existing definitions for the concept of detection, and reported that when these eight operational definitions were applied to the same data, they resulted in numerical values that differed by nearly three orders of magnitude. These results made it impossible to compare the detection capabilities of measurement methods using available publications. Dr. Currie proposed standardizing on theoretical definitions that he called the critical value, the detection limit, and the determination limit. ( In 1995, writing on behalf of International Union of Pure and Applied Chemistry [ IUPAC], Dr. Currie used the term " quantification limit" instead of his original term " determination limit." Substantial agreement with the International Standards Organization [ ISO] on the meaning and language of detection and quantification was achieved later, although some " subtle differences in perspective" remain Currie[ 2000]). His purpose for these definitions was to create a system in which the standard documentation of any measurement method would include a statement of capabilities that were directly comparable to any other method for measuring the same substance.

Currie used terms from statistical decision theory as the basis for his three tiered system. In Currie ( 1968) and Currie ( 1995) he defined the critical value as the measured value at which there is a small chance that the concentration in the sample is zero. Consequently, any measured result greater than or equal to the critical value is considered evidence that the sample contains the substance of interest. Currie was careful to emphasize that the decision as to whether the substance has been detected is made by comparing the measurement result to the critical value. Figure 2 1 shows a critical value selected such that measurements greater than the critical value have less than a 1% chance of being associated with a sample that does not contain the substance of interest. The area under the curve to the right of the critical value is the probability that a measured value will exceed the critical value. The area under the curve to the left of the critical value is the probability ( much greater) of observing a value that is less than the critical value. Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 2 2 Figure 2 2 Currie ( 1968 and 1995) used the term detection limit to refer to a true concentration that has a high probability of generating measured values greater than the critical value. That is, measurements on samples that contain concentrations equal to the detection limit have a high probability of exceeding the critical value and are, therefore, unlikely to result in a decision that the substance is not detected in the sample. In Currie's concept, the critical value and the detection limit are related and functionally dependent, but it is clear that the detection decision is made on the basis of comparing sample by sample measurements to the critical value. While Currie's terminology is consistent with standard statistical decision theory, it is in all likelihood responsible for a great deal of confusion among chemists and others who may associate the term ` limit' with some sort of decision point. Currie ( 1995) states: " The single, most important application of the detection limit is for planning. It allows one to judge whether the CMP [ Chemical Measurement Process] under consideration is adequate for the detection requirements." Figure 2 2 shows a detection limit selected such that 99% of the measurements on a sample containing this concentration are expected to be above the critical value. The bellshaped curve centered at the detection limit illustrates how likely various measurement responses are when the concentration of the substance in a sample is equal to the detection limit. That is, the figure shows the probability density of values measured in a sample with a true concentration equal to the detection limit. The area under the curve to the left of the critical value is equal to 1% of the total area, while the area to the right is equal to 99%.

Currie ( 1968, 1995) defined the determination limit, later renamed the quantification limit, as ( quoting Currie, 1995) " performance characteristics that mark the ability of a CMP to adequately ` quantify' an analyte." Quantification limits " serve as benchmarks that indicate whether the CMP can adequately meet the measurement needs. The ability to quantify is generally expressed in terms of the signal or analyte ( true) value that will produce estimates having a specified relative standard deviation ( RSD) commonly 10%." This translates into a quantification limit equal to a multiplier of 10 times the standard deviation ( a measure of measurement variability) at the limit. The multiplier of 10 ( equal to the inverse of the RSD) is arbitrary, but has been used widely. IUPAC selected 10 as a " default value" ( Currie, 1995), implying other values are possible. In papers published in 1980 and 1983, the American Chemical Society's Committee on Environmental Analysis also recommended the use of a multiplier of 10 for determining quantitation limits ( see McDougal, et al. Analytical Chemistry, Vol. 53, p. 2242, 1980 and Keith, et al., Vol. 55, pp. 2210 2218, 1983). Measured concentrations greater than the quantitation limit are considered to be reliable by chemists, although from a statistical perspective, any measured value along with knowledge of the precision of the measurement, is useful.

Currie's goal of having measurement method developers publish directly comparable descriptions of detection and quantitation capability remains elusive more than thirty years after publication of his first paper on this topic. Even if Currie's three tiered concept were used, the treatment of related issues causes difficulty in comparing methods. Some of these issues include between laboratory variation, selection of appropriate statistical models, design of the detection and quantitation capability study, and statistical prediction and tolerance. These and other issues are discussed in Chapter 3 of this document. Chapter 2 Draft Document for Peer Review  Do Not Circulate  August 2002 2 3 2.2 Development of the MDL and ML as Practical Embodiments of Currie's Proposal In 1981, staff at EPA's Environmental Monitoring and Support Laboratory in Cincinnati, Ohio, published a procedure for determining what they referred to as a method detection limit ( MDL). The MDL functions as a practical, general purpose version of Currie's critical value. The MDL was subsequently promulgated for use in CWA programs on October 26, 1984 ( 49 FR 43234) at 40 CFR 136, Appendix B. Prior to formal development of the MDL, the EPA Office of Water had included the term " minimum level" ( ML) or " minimum level of quantitation" in some methods for analysis of organic pollutants. These methods were proposed on December 3, 1979 and subsequently promulgated on October 26, 1984, along with the MDL. Additional information about the MDL and ML is provided below in Sections 2.2.1 and 2.2.2.

2.2.1 Method Detection Limit Conscious of the definitions provided by Currie and others, Glaser et al. ( 1981) stated " The fundamental difference between our approach to detection limit and former efforts is the emphasis on the operational characteristics of the definition. [ The] MDL is considered operationally meaningful only when the method is truly in the detection mode, i. e., [ the] analyte ( the substance of interest) must be present." Expanding on this reasoning, Glaser et al. ( 1981) developed MDL estimates for methods that produce a result of zero for blanks, such as EPA Methods 624 and 625, for determination of organic pollutants by gas chromatography/ mass spectrometry ( GC/ MS). Blank variability exists, whether or not it can be detected by measurement processes. Failure to detect this variability may be attributed to insufficient sensitivity of the measurement process or, as is the case with some measurement processes, thresholds that are built into equipment which censor measurements below certain levels.) Currie's critical value is dependent on the ability to estimate measurement variability of zero concentration samples. In cases where the substance is not detected in direct measurements on blanks, an alternative approach to estimating blank variability must be used. Another option is to estimate measurement variability at concentrations that represent the lowest possible levels where a signal can be detected. This is the basic approach of the MDL which provides a general purpose, straightforward operational procedure for estimating a quantity analogous to the Currie critical value when measurement processes applied to blank samples do not produce detectable signals. More complex statistical procedures for estimating blank variability are possible and may be preferable from a rigorous statistical perspective, but the MDL has been found to be satisfactory by chemists in a wide range of applications.

In 1984, the MDL became a regulatory requirement for certain wastewater discharge permits authorized under the Clean Water Act. To determine the MDL, at least seven replicate samples with a concentration of the pollutant of interest near the estimated detection capabilities of the method are analyzed. The standard deviation among the replicate measurements is determined and multiplied by the t distribution for n 1 degrees of freedom ( in the case of 7 replicates, the multiplier is 3.143, which is the value for 6 degrees of freedom).

Both the MDL concept and the specific definition at part 136 have been used within EPA by the Office of Ground Water and Drinking Water ( OGWDW), the Office of Solid Waste ( OSW), the Office of Emergency and Remedial Response ( OERR), and others. The MDL also has been used outside of EPA in Standard Methods for the Examination of Water and Wastewater, published by the American Public Health Association ( APHA), the American Water Works Association ( AWWA), and the Water Environment Federation ( WEF), in methods published by the American Society for Testing and Materials ( ASTM), and elsewhere. Assessment of Detection and Quantitation Concepts 1The refined definition of the ML first appeared in EPA's 1994 draft National Guidance for the Permitting, Monitoring, and Enforcement of Water Quality based Effluent Limitations Set Below Analytical Detection/ Quantitation Levels The draft guidance was never finalized. However, the refined definition of the ML has remained in use.

Draft Document for Peer Review  Do Not Circulate  August 2002 2 4 Despite such widespread use, some members of the regulated industry and others have claimed that the MDL is a less than ideal concept for detection. Specifically, critics have faulted the MDL because:

° There are some inconsistencies between the definition and the procedure ° It does not account explicitly for false negatives ° It does not account for bias ° A prediction or tolerance interval adjustment is not provided ° It does not account for interlaboratory variability These issues are discussed later in this document.

2.2.2 Minimum Level of Quantitation The minimum level of quantitation ( ML) was originally proposed on December 5, 1979 ( 44 FR 69463) in footnotes to Table 2 of EPA Method 624 and to Tables 4 and 5 of EPA Method 625. The ML was defined as the " level at which the entire analytical system must give recognizable mass spectra and acceptable calibration points" ( in the footnote to Table 2 in Method 624) and as the " level at which the entire analytical system must give mass spectral confirmation" ( in the footnotes to Tables 4 and 5 in EPA Method 625).

Between 1980 and 1984, EPA also developed Methods 1624 and 1625, and promulgated these methods along with the final versions of EPA Methods 624 and 625 on October 26, 1984 ( 49 FR 43234). The definitions of the ML in the promulgated versions of EPA Methods 1624 and 1625 were the " level at which the analytical system shall give recognizable mass spectra ( background corrected) and acceptable calibration points" ( in footnote 2 to Table 2 in Method 1624) and as the " level at which the entire GC/ MS system must give recognizable mass spectra ( background corrected) and acceptable calibration points" ( in footnotes 2 to Tables 3 and 4 in Method 1625).

As EPA developed additional methods over the next decade, the definition of the ML was generalized to " the lowest level at which the entire analytical system must give a recognizable signal and acceptable calibration point." ( See, e. g., Section 24.2 of EPA Method 1613 at 40 CFR 136, Appendix A.) In generating actual numerical values for MLs, the lowest calibration point was estimated from method development studies and included in the methods, although a specific calculation algorithm was not used. Laboratories were required to calibrate their analytical systems with a multi point calibration ( i. e., calibrate using different concentrations over the range of the instrument) that included a standard at the lowest calibration point listed in the method ( i. e., the ML).

In response to comments on the ML by the regulated industry and some state regulatory agencies, EPA refined the definition of the ML in 1994 as 10 times the standard deviation used to determine the MDL. 1 Because the MDL is commonly determined as 3.14 times the standard deviation of seven replicate measurements, the ML was commonly calculated as 3.18 times the MDL. ( The figure of 3.18 was derived by dividing 10 by 3.14; if more than 7 replicates were used to determine the MDL, both the MDL and the ML multipliers are adjusted accordingly based on values from the t distribution.) This calculation makes the ML analogous to Currie's quantification limit and the American Chemical Society's limit of quantitation ( LOQ), Chapter 2 Draft Document for Peer Review  Do Not Circulate  August 2002 2 5 which is defined as ten times the standard deviation of replicate or low concentration measurements ( McDougal, et al. ( 1980) and Keith, et al. ( 1983)).

To simplify implementation of the ML, the definition was expanded to state that the calculated ML is rounded to the number nearest to ( 1, 2, or 5) times 10n, where n is an integer. The reason for this simplification is that calibration of an analytical system at some exact number ( e. g., 6.27) is difficult and prone to error, whereas rounding to the number nearest to ( 1, 2, or 5) x 10n provides a practicable value. The most recent definition of the ML is " the lowest level at which the entire analytical system must give a recognizable signal and acceptable calibration point for the analyte. It is equivalent to the concentration of the lowest calibration standard, assuming that all method specified sample weights, volumes, and cleanup procedures have been employed. The ML is calculated by multiplying the MDL by 3.18 and rounding the result to the number nearest to ( 1, 2, or 5) x 10n, where n is an integer," and this definition was contained in the version of EPA Method 1631 that was promulgated on June 8, 1999 ( 64 FR 30417) ( see Section 17.8 of EPA Method 1631).

The ML will generally be somewhat lower than Currie's quantification limit even when similar sample sizes and estimation procedures are used. This is because the standard deviation used to calculate the ML will generally be smaller than the standard deviation at the lowest concentration at which the relative standard deviation is 10%. This is due to the fact that, in almost all cases, standard deviation is nondecreasing with increasing concentration, and generally tends to increase as concentration increases.

Although the ML has been used successfully in EPA methods for more than 20 years, some members of the regulated industry and others have claimed that the ML is less than an ideal concept for quantitation because it:

C Does not allow for interlaboratory variability, and C Is based on a multiple of the standard deviation rather than a fitted model These concerns are discussed later in this document.

2.3 Concepts Advanced by Other Organizations To expand somewhat on Currie ( 1968), standardizing on operational definitions of detection and quantitation would benefit society by making it easier to compare and select measurement methods based on low level measurement capability and requirements in particular applications. Unfortunately, in spite of agreement on general principles and definitions advanced by Currie and his supporters, consensus on procedures that would result in comparable detection and quantitation estimates has been elusive. Sections 2.3.1  2.3.3, which are by no means an exhaustive list of the various concepts advanced to date, highlight concepts that have been most widely advanced for environmental applications.

2.3.1 EPA Concepts Over the years, a number of detection and quantitation limit concepts have been developed, suggested, or used by EPA among the various organizations charged with responding to differing program mandates. In part, this situation reflects actual differences in the mandates, and in part, it reflects the fact that no concept advanced to date has emerged as a clear ` winner' that meets all needs for all people. Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 2 6 Concepts that have been used or suggested by EPA include the:

° MDL and ML ( described in Sections 2.2.1 and 2.2.2) ° Instrument detection limit ( IDL) ° Practical quantitation limit ( PQL) ° Contract required detection limit ( CRDL) and contract required quantitation limit ( CRQL)

Instrument Detection Limit: EPA methods for analysis of metals have historically included an instrument detection limit, or IDL. Functionally, the IDL is similar to the MDL except that the IDL includes temporal variability ( it is determined on 3 non consecutive days) and does not include all sample processing steps ( the IDL characterizes the detection capabilities of the instrument as opposed to the method). Because IDLs do not reflect the entire measurement process and, for the most part, have been used only for measurement of metals, EPA did not consider the IDL as a potential alternate to the MDL when conducting the assessment described in this TSD.

Practical Quantitation Limit: The practical quantitation limit, or PQL, was established in the 1980s by EPA's drinking water program as the lowest concentration at which reliable measurements can be made. The PQL is defined as the lowest concentration of an analyte that can be reliably measured within specified limits of precision and accuracy during routine laboratory operation conditions ( 52 FR 25699, July 8, 1987). The PQL is a means of integrating information on the performance of approved analytical methods into the development of a drinking water regulation. The PQL incorporates the following:

° Quantitation, ° Precision and bias, ° Normal operations of a laboratory, and ° The fundamental need to have a sufficient number of laboratories available to conduct compliance monitoring analyses.

EPA uses two main approaches to determine a PQL for an analyte under the Safe Drinking Water Act ( SDWA). One approach is to use the data from Water Supply ( WS) studies ( e. g., laboratory performance evaluation studies conducted by the Agency as part of the certification process for drinking water laboratories). The PQL is established at the concentration at which at least 75% of the laboratories in the study, or the subset representing EPA Regional laboratories and state laboratories, obtain results within some predetermined percentage of the true value of the test samples ( e. g., ± 30%). This approach is used in most cases when sufficient WS data are available to calculate a PQL. The WS data approach was used to determine the PQLs for Phase V inorganic chemicals such as antimony, beryllium, cyanide, nickel and thallium ( July 17, 1992; 57 FR 31800), as well as many other contaminants regulated under the SDWA.

In the absence of WS data, the second approach that EPA uses is the multiplier method. In this approach, the PQL is calculated by multiplying the EPA derived MDL by a factor between 5 and 10. The exact multiplier varies and sometimes depends on the degree of concern about the specific contaminant ( i. e., based on a human health risk assessment for consumption of drinking water).

Application of the PQL has been traditionally limited to drinking water. Furthermore, the PQL may not be related to the lowest quantitation limit because 1) the PQL is associated with the analyte and may be determined irrespective of a specific analytical method, 2) the performance evaluation ( PE) samples from which it is derived contain pollutant concentrations that may be well above the true limit of quantitation, 3) the multiplier used to calculate a PQL when PE data are not available is somewhat dependent on concerns about risks from human exposure to contaminants in drinking water, and 4) the resulting PQLs may be too high other for regulatory purposes have been perceived as too high for regulatory purposes. In addition, Chapter 2 Draft Document for Peer Review  Do Not Circulate  August 2002 2 7 because EPA has privatized the performance evaluation program for drinking water laboratory certification, it is not yet clear that appropriate data will be available in the future. Based on these facts, EPA did not conduct an assessment of the PQL for CWA applications.

In the late 1980s, EPA's Office of Solid Waste ( OSW) adopted a different version of the PQL as a quantitation limit. No procedure for establishing the limits was given; instead values were extrapolated from CLP CRQLs ( see below). Since 1994, OSW has actively removed the term " PQL" from its revised methods, replacing it with the Estimated Quantitation Limit ( EQL). The term PQL and the original numerical values remain in a few older OSW guidance documents.

Estimated Quantitation Limit: EPA's Office of Solid Waste has defined the EQL as the lowest concentration that can be reliably achieved within specified limits of precision and accuracy during routine laboratory operating conditions ( see SW 846, Chapter 1). The EQL is generally 5 to 10 times the 40 CFR 136, appendix B MDL but may be chosen to simplify data reporting. For many analytes, the EQL analyte concentration is selected as the lowest non zero standard in the calibration. Sample EQLs are highly matrix dependent. As noted in most newer SW 846 methods, the EQLs are provided for guidance and may not always be achievable. Because the EQL is not rigorously defined and is guidance, because the EQL may be based on the MDL, and because the EQL can be the lowest calibration point and would, therefore, overlap the ML, EPA did not consider the EQL further in its assessment of detection/ quantitation concepts.

Contract Required Detection and Quantitation Limits: EPA's Superfund program has adopted the use of contractually required limits that are based on consensus among analytical chemists about levels that can realistically be achieved in commercial laboratories using a contractually specified method. Laboratories that participate in the Superfund Contract Laboratory Program ( CLP) are required to demonstrate that they can achieve the specified CRDLs and CRQLs. The CRQLs are based on the concentration of the lowest non zero calibration standard, in a fashion analogous to the original derivation of the ML. Because few CWA applications involve the use of EPA contract laboratories, EPA did not consider the CRDL or the CRQL as viable alternatives to the MDL and ML when conducting the assessment described in this document.

2.3.2 Industry supported Concepts The regulated industry has demonstrated an interest in detection limit concepts since EPA first promulgated the MDL and ML for use in CWA programs in 1984 ( 49 FR 43234). As part of that rule, EPA promulgated Methods 601 through 613, 624, 625, 1624, and 1625 for organic compounds at 40 CFR 136, Appendix A and EPA Method 200.7 for metals by inductively coupled plasma spectrometry ( ICP) at 40 CFR 136, Appendix C. EPA also promulgated the MDL procedure at 40 CFR 136, Appendix B. The Virginia Electric Power Company ( VEPCO) brought suit against EPA for use of the MDL in the promulgated methods. In a settlement, EPA agreed that the MDL would be applicable to the 600 series organic methods only; i. e., it would not be applicable to EPA Method 200.7. The settlement agreement did not preclude future use of the MDL by EPA or the right of VEPCO to bring suit in such future use.

After the VEPCO settlement, the regulated industry, mainly through efforts of the Electric Power Research Institute ( EPRI), remained involved in detection and quantitation approaches to be used under EPA's CWA programs. The first concepts that industry advanced were the compliance monitoring detection level ( CMDL) and compliance monitoring quantitation level ( CMQL) ( Maddalone, et al., 1993). The CMDL/ CMQL were variants of EPA's MDL/ ML that attempted to adjust for interlaboratory variability.

The regulated community continued its efforts to develop alternate detection and quantitation concepts with development of the alternate minimum level ( AML) in the mid 1990s ( Gibbons, 1997). The Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 2 8 AML is based on statistical modeling of standard deviation versus concentration, which requires large amounts of data.

Most recently, the regulated industry has funded development of the interlaboratory detection estimate ( IDE) and interlaboratory quantitation estimate ( IQE). The IDE/ IQE have been balloted and approved by ASTM's Committee D 19 for water as standard practices D 6091 and D 6512, respectively. These concepts take into account all possible sources of variability to arrive at detection and quantitation limits that are higher on average than the limits produced by other concepts. Because the industry community has shifted support from the CMDL/ CMQL and the AML to the IDE and IQE, and because EPA is not aware of other organizations that currently advocate the earlier proposals, EPA did not consider industry concepts other than the IDE/ IQE in its assessment of possible alternatives to the MDL and ML.

As with all other concepts advocated to date, the IDE and IQE have fallen short of being ideal concepts for detection and quantitation for all organizations and applications. To date, EPA is not aware of a demonstrated implementation of the IDE or IQE in the development of an analytical method. Specific concerns that have been raised about the IDE and IQE are that:

° They contain an inappropriate allowance for false negatives, ° The IDE and IQE are based on prediction and/ or tolerance intervals that are unsuitable, ° The IQE requires a large amount of costly data in order to be able to model variability versus concentration, including data generated in multiple laboratories, and ° The complex statistical procedures involved in calculating an IDE and IQE are an unreasonable burden on the analytical chemists that typically develop, modify, and use methods.

2.3.3 Concepts Advocated by the Laboratory Community and Voluntary Consensus Standards Bodies In 1980 ( McDougal et al., 1980) and 1983 ( Keith et al., 1983), the American Chemical Society's Committee on Environmental Improvement ( CEI) advanced concepts for the Limit of Detection ( LOD) and Limit of Quantitation ( LOQ). The ACS LOD is defined as the lowest concentration level that can be determined to be statistically different from a blank. The recommended value for the LOD is three times the standard deviation of replicate measurements of a blank or low level sample. The LOD is roughly equivalent to the MDL in numerical terms and conceptually equivalent to Currie's critical value.

The ACS LOQ is defined as the level above which quantitative results may be obtained with a specified degree of confidence. The recommended value for the LOQ is 10 times the standard deviation of replicate blank or low level measurements. Because the LOD and LOQ are still used by the analytical community, they have been included in EPA's reassessment of detection and quantitation concepts.

In the mid 1980s, the ACS CEI introduced the concept of the Reliable Detection Limit ( RDL) and the Reliable Quantitation Limit ( RQL). The RDL and RQL were attempts at simplification of the LOD and LOQ. Both the RDL and the RQL involved applying a multiplier to the standard deviation derived from replicate measurements of a low level sample. Neither concept received acceptance by the analytical community. Because the RDL and RQL are no longer being advanced by ACS, they were not considered for evaluation in EPA's assessment of detection and quantitation limit concepts.

In 1999 ( Currie, 1999a, 1999b), IUPAC and ISO reached substantial agreement on the terminology and concepts documented by Currie ( 1995), although " subtle differences in perspective" of the organizations remain Currie ( 2000). IUPAC and ISO have not, to date, published methods that include limits reflecting Chapter 2 Draft Document for Peer Review  Do Not Circulate  August 2002 2 9 these standards. Similarly, although ASTM adopted the IDE in 1997 and the IQE in 2000, ASTM has not included any IDE or IQE values in methods approved through the ASTM ballot process. On the other hand, ISO and ASTM have published methods that employ the MDL. Because IUPAC and ISO have approved the critical value, detection limit, and quantification limit, and because ASTM has approved through ballot the IDE and IQE, EPA has included these concepts in its assessment of detection and quantitation limit concepts. Draft Document for Peer Review  Do Not Circulate  August 2002 3 1 Chapter 3 Issues Pertaining to Detection and Quantitation As part of the Settlement Agreement concerning the Agency's reassessment of detection and quantitation limit concepts, the Agency agreed to consider several specific issues pertaining to these concepts. These issues included:

° Criteria for selection and appropriate use of statistical models; ° Methodology for parameter estimation ° Statistical tolerance and prediction ° Criteria for design of detection and quanitification studies, including selection of concentration levels (" spiking levels") ° Interlaboratory variability, and ° Incorporation of elements of probability design.

In developing its plan for conducting this assessment, EPA identified a number of other issues that should be considered. These issues include:

° Concepts of the lower limit of measurement ° Concepts in relation to EPA Office of Water applications, including  method performance verification at a laboratory,  method development and promulgation,  National Pollutant Discharge Elimination System applications,  non regulatory studies and monitoring,  descriptive versus prescriptive uses of lower limits to measurement, and  use of a pair of related detection and quantitation procedures in all OW applications ° Censoring of measurement results ° Sources of Variability ( including, but not limited to interlaboratory variability) ° False positives and false negatives ° Measurement quality over the life of a method ° Matrix effects ° Background contamination ° Outliers ° Instrument Non Response ° Accepting the procedures of voluntary consensus standards bodies ( VCSBs) ° National versus local standards for measurement, ° Ease of use ( i. e., ability of study managers, bench chemists, and statisticians to do what is required by a detection or quantitation limit procedure) ° Cost to implement the procedures ° Laboratory specific applications Concepts concerning the lower levels of measurement were discussed in Chapter 2. For clarity and brevity, EPA has organized the remaining criteria into three subsections that follow. Section 3.1 discusses the issues that are primarily driven by analytical chemistry concerns, Section 3.2 discusses the issues that are primarily driven by CWA regulatory requirements, and Section 3.3 discusses issues that are primarily driven by statistical concerns. Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 3 2 3.1 Analytical Chemistry Concepts Pertaining to Detection and Quantitation 3.1.1 Blank versus Zero Concentration Analytical chemists rarely, if ever, say that a sample contains zero concentration of a substance of interest. Even when the sample is created in a laboratory for the purpose of containing as little substance of interest as possible ( a blank), analytical chemists recognize the possible contribution of the blank to the final measurement result. The ability of a laboratory to reduce the concentration of a substance in the blank is often the limiting factor in attempts to make measurements at ever lower levels.

A classic example of the potential problem is illustrated by the seminal works of Patterson in the late 1960s and 1970s ( e. g., Patterson and Settle, 1976). Patterson demonstrated that the majority of concentrations of lead reported in the literature for such diverse matrices as urban dust, open ocean waters, and biological tissues were in error by several orders of magnitude. The source of the " gross positive errors" was contamination introduced during sample collection, handing, and analysis. Interlaboratory studies of the day designed to determine consensus values for reference materials were, in fact, determining the consensus values for background contamination across laboratories. Patterson recognized the value in running blank samples ( samples thought not to contain the substance of interest) to demonstrate that the sample collection, handling, and analysis processes were not introducing contamination. Patterson subsequently developed the techniques for " evaluating and controlling the extent and sources of industrial lead contamination introduced during sample collecting, handling, and analysis" that form the basis of the " clean techniques" used for metals analysis today, and that are incorporated in EPA Method 1631 among others.

The most common analytes for which contamination problems are encountered in environmental measurements are metals, primarily zinc because of its ubiquity in the environment. On the other hand, it is rare to find contamination in the measurement of organic compounds, except for methylene chloride, acetone, and a few other volatile organic compounds used as solvents in analytical laboratories. Therefore, for determination of metals, a blank is usually included or compensated in the calibration whereas, for organics, except for the solvents, the concentration in the blank is assumed to be zero and there is no compensation of the calibration.

Measurement methods designed to determine substances at very low concentrations may include requirements for the preparation and analysis of a variety of blanks that are designed to identify the extent and the sources of contamination. Analysts understand that " blank" does not mean zero concentration, but that through careful control and evaluation, it is possible make measurements for which the blank contribution is sufficiently small to be considered negligible.

Useful detection and quantitation limit concepts must address the potential contribution of the blank, through both the design of the study that generates the detection and quantitation limit estimates and the evaluation of the study results.

3.1.2 Lack of Instrument Response Instruments do not always produce a result from an appropriately prepared sample. Sometimes this is attributable to uncontrollable instrument limitations, sometimes it is attributable to controllable instrument settings ( thresholds) established by the manufacturer or the laboratory, and sometimes it occurs randomly. As an example, gas chromatograph/ mass spectrometer ( GC/ MS) instruments often contain thresholds below which no instrument signal is reported. With no instrument signal reported, no measurement result can be reported, and the instrument will report zero in such cases to indicate the lack of a signal. To understand the Chapter 3 Draft Document for Peer Review  Do Not Circulate  August 2002 3 3 concept of an instrument threshold it may be helpful to think of background static heard on a citizen band ( CB) radio or a walkie talkie. The static is present, but it has no meaning. Turning the " squelch" knob to the point at which the static is filtered out also may make it impossible to hear the caller. In the context of detection, increasing the instrument threshold may cause the instrument to miss the substance of interest at low levels.

In 1997, EPA conducted study of 82 semivolatile ( acid and base/ neutral) organic compounds measured by EPA Method 1625 in order to observe the performance of a GC/ MS instrument both with and without thresholds. In the study, solutions at up to 17 concentration levels were analyzed with the thresholds on and with the thresholds off. Samples were analyzed at decreasing concentrations, including a blank, with triplicate determinations at each concentration. For measurement results obtained with the thresholds turned on, all of the measurements made on the blank were reported as zero. This was not a surprising result, given the purpose of the instrument thresholds. For measurements obtained without thresholds, 27 of 230 measurements on the blank ( 11%) were reported as 0.000 ng/ L and no measurement results were reported lower. The fact of instrument non response at low concentrations has both direct and indirect impacts on estimating detection and quantitation levels.

There are two direct impacts of non response at low concentrations. One is that Currie's critical value ( see Chapter 2) is either problematic to estimate or it does not exist. The other is that EPA's method detection limit exists but is problematic to estimate.

If the assumptions required for Currie's critical value were true, then approximately 50% of the measurement results should have been negative and their average would have been approximately zero. Without measurement results below the expected average, the average will always be a little high and getting a good estimate will be difficult or impossible. If the measurement method cuts off at some value above zero then Currie's critical value and detection limit do not technically exist.

With instrument non response at low concentrations, the estimation of EPA'S MDL becomes problematic. In order to meet the requirements of the MDL definition, it is necessary to find the concentration at which the measurement method ceases to generate measurement results, and many laboratories have run repeat measurements in order to find this concentration. This problem manifested itself in EPA's variability versus concentration ( Episode 6000) studies. The 40 CFR 136, Appendix B procedure suggests iteration until the measured MDL is within a factor of 5 of the spike level. For the Episode 6000 studies, EPA instructed laboratories to use a factor of 3 instead of 5 in an attempt to more narrowly define the lowest spike level at which measurements could be made. Several laboratories asked for relief from this requirement and EPA relented after learning of the difficulties in attempting to achieve the factor of 3. A conclusion that can be drawn from this outcome is that detection limits are somewhat variable and not easy to define. Further details are in the results of the studies given in Chapter 4.

In summary, both the Currie system and the EPA system have theoretical problems with addressing instrument non response. Any operational system for detection or quantitation will need to take this issue into account.

3.1.3 Matrix Effects " Sample matrix" is a term used to describe all of the substances, other than the pollutant( s) of interest, present in an environmental sample. In the case of a wastewater sample, this would include the water itself, as well as any other dissolved or suspended materials. For any given measurement, some of the substances may interfere with the measurement, while others may be substances that have no effect on the measurement. Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 3 4 Interferences in the sample may act either positively ( i. e., increasing the measured results), negatively ( i. e., decreasing the measured results), or even preventing the measurement from being conducted.

" Matrix effect" is a term used to describe a situation in which a substance or combination of substances in the sample ( other than the substance( s) of interest) influence the results of the measurement or cause the results to fail performance criteria. Positive interferences may inflate the results for the substance or make it difficult to distinguish one substance from another. However, unless the positive bias is consistent and predictable, the measurement result may be unreliable. Negative interferences may suppress the results for the substance to the point that they cannot be distinguished from background instrument noise.

For these reasons, many believe that detection and quantitation limits should be determined in " real world" matrices rather than in reference matrices intended to simulate method performance in a particular matrix type. Problems with such an approach, however, are that:

° Many real world matrices contain the target pollutant at levels well above the detection or quantitation limit, making it impossible to characterize what can and cannot be detected at low levels. Diluting the sample to dilute the target pollutant concentration is an option. However, this also has the potential to dilute any interferences that might be present, thereby defeating the purpose of using the real world matrix. ° It is not possible to anticipate and obtain samples of every possible matrix on which a method might be used when the method is being developed and detection/ quantitation limits are being established. ° Use of a reference matrix to establish detection and quantitation limits allows the results to be reproduced ( i. e., confirmed) by an independent party; such confirmations will not possible with many real world matrices that may be subject to seasonal, diurnal, or other types of variability. ° The cost of determining detection and quantitation limits in every possible matrix would be prohibitive.

Given these difficulties, EPA believes that reference matrices should be used to establish method detection and quantitation limits, but that the procedures for defining these limits should allow for evaluation of data collected in particular matrices of concern. EPA also believes that such matrix specific determinations should only be used when all efforts to resolve the matrix interferences have been exhausted.

In some cases, finding a matrix effect indicates that the analyst should select a more appropriate method. For example, a colorimetric method for the measurement of sulfide may be a poor choice for the analysis of a sample that is very cloudy or darkly colored. In other cases, characteristics of the sample such as its pH may destroy the substance of interest, effectively preventing analysis for that substance.

Nearly all of the newer analytical methods approved at 40 CFR 136 describe the preparation and analysis of quality control samples that are designed to indicate the presence of matrix effects ( e. g., matrix spike and/ or matrix spike duplicate samples). Many of these methods also contain techniques for addressing matrix effects. Further, EPA has developed guidance documents that amplify the discussions in those methods ( e. g., Guidance on Evaluation, Resolution, and Documentation of Analytical Problems Associated with Compliance Monitoring, June 1993, EPA 821 B 93 001). For the determination of mercury by EPA Method 1631 that is the subject of the Settlement Agreement, additional guidance on resolving matrix interferences to achieve specified detection and quantitation levels is provided in EPA's Guidance for Implementation and Use of EPA Method 1631 for the Determination of Low Level Mercury ( March 2001, EPA 821 R 01 023). Following the techniques in the methods and guidance will usually reduce adverse effects of the sample matrix on detection/ quantitation limits and measurement results. Chapter 3 Draft Document for Peer Review  Do Not Circulate  August 2002 3 5 3.1.4 Measurement Quality over the Life of a Method ° We have all heard the expression " Practice makes perfect." It applies to the quality of measurements made with a given method over time. We can demonstrate it using simple techniques like laboratory control charts. The improvements are a result of experience, as well as improvements in equipment over time.

EPA expects changes in performance when new staff are trained. For this reason, many EPA methods specify that " start up tests" be repeated each time new staff arrive. It is not unusual to see slight increases in measurement variability as new staff are trained. However, when new staff become as good as the existing staff, control charts will show it.

As with most other areas of technology, measurement instruments are constantly improving. Instrument companies and laboratories are increasing data processing power, speed of analysis, and the reduction of chemical or electronic " noise." Any of these instrument improvements are expected to improve the measurement method in determining the concentrations of environmental pollutants. This process can be illustrated for a variety of EPA methods. A case in point is EPA Method 1613 for determination of polychlorinated dibenzo p dioxins and polychlorinated dibenzofurans. Development of this method began in 1988. At the time, high resolution mass spectrometer systems that were commercially available were able to achieve a detection limit of approximately 4 pg/ L and an ML of 10 pg/ L. By the time that EPA proposed the method in 1991, the Canadian government published its own version that went down to a quantitation limit 5 pg/ L. By the time EPA officially published Method 1613 in 1997, many laboratories performing the analysis had replaced or supplemented their old instruments with newer models. As a result, many laboratories performing analyses using Method 1613 routinely measure sample results at levels 10 times lower than those analyzed routinely only 10 years earlier.

Given that measurement capabilities improve over time, EPA believes that a detection and quantitation limit concept should be supported by procedures that will allow individual laboratories and other organizations to affordably characterize such improvements.

3.2 CWA Regulatory Issues Affecting Detection and Quantitation Section 3.2.1 below provides a brief overview and a discussion of Clean Water Act activities that involve chemical measurements and are, therefore, directly impacted by detection and quantitation limit concepts. Specific issues that must be considered in the context of these CWA applications and EPA's regulatory obligations are discussed in Sections 3.2.2  3.2.6.

3.2.1 Detection and Quantitation Limit Applications Under CWA The Clean Water Act directs EPA, states, and local governments to conduct a variety of data gathering, permitting, and compliance monitoring, and enforcement activities. Many of these activities depend directly on environmental measurements, and therefore, are affected by detection and quantitation limit concepts as discussed in the subsections that follow.

<IP_ADDRESS> Method Development and Promulgation Section 304( h) of the Clean Water Act requires EPA to promulgate analytical methods that can be used to support data gathering requirements under the Act. These methods are promulgated at 40 CFR part 136, and include methods developed by EPA as well as those developed by other organizations, such as Standard Methods, AOAC International, ASTM International, the U. S. Geological Survey, instrument Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 3 6 vendors, and others. Upon request by a laboratory, permittee, instrument manufacturer, or other interested party, EPA also is required to consider alternate testing procedures ( ATPs). If EPA deems these ATPs are acceptable for nationwide use, they too, are published at 40 CFR part 136. A primary objective in promulgating methods developed by EPA and by other organizations is to provide the regulatory community, permittees, and laboratories with as many options as possible so that they may choose the method that yields the best performance at the lowest cost for the application.

In recent years, EPA has developed methods for promulgation at 40 CFR 136 only when no other methods are available that meet the immediate or anticipated regulatory need. The National Technology Transfer and Advancement Act of 1995 ( NTTAA) requires government agencies to use methods published by voluntary consensus standards bodies ( VCSBs), such as Standard Methods and ASTM International, when VCSB methods are available. EPA accepts that these methods have been through a sufficient level of testing, peer review, and scientific acceptance to warrant proposal if they meet EPA's regulatory needs. When an individual laboratory, permittee, or other organization submits a request for approval of an alternate test procedure, however, EPA requires that the procedure be subjected to a level of testing that demonstrates that the method provides sensitivity, accuracy, and other measures of performance comparable to an approved method.

The lack of widespread consensus on detection limits has obvious impacts on EPA's responsibility to promulgate methods under CWA. Nearly all organizations that develop methods uses a different concept, and many organizations have changed concepts over the years. The result is that a number of different concepts for detection and quantitation are embodied in the methods approved at 40 CFR part 136. The vast majority of the approved methods include the MDL, which as noted in Section 2.2.1, has been used by several EPA Offices, Standard Methods, AOAC, ASTM, and others. Other concepts embodied in the methods at 40 CFR part 136 include, but are not limited to: 1) a method " range" that is usually not defined, but is often interpreted as either lower end of the range in which pollutants can be identified or quantified, 2) an " instrument detection limit" that has been defined by a variety of procedures, but is intended to capture instrument sensitivity only, 3) an " estimated detection limit" that may be based on best professional judgement, single laboratory data, or some other source of information, 4) a " practical quantitation limit," that has typically been determined according to one of the scenarios described in Section 2.3.1, and 5) " sensitivity" that is an undefined concept similar in result to the MDL.

The most obvious solution to this problem would be for the Office of Water to force all methods promulgated at 40 CFR 136 to contain single concepts for detection and quantitation. Unfortunately, taking such action would confound methods promulgation. Problems with this solution are that:

° To date, no single detection and quantitation limit concept has emerged a meeting the needs of all organizations for all applications ° If the Office of Water were to select a concept that differs from those of other organizations, those organizations would be required to conform their method to accommodate OW's concept. Doing so would mean that these organizations would have to invest additional laboratory resources to develop detection and quantitation limits that conformed to OW definitions. ° If outside organizations decided against conforming their concepts to that of OW, fewer methods would be promulgated at 40 CFR part 136. This would result in fewer options for the regulatory, permittee, and laboratory community. ° If EPA selected a concept that has burdensome procedures for develop detection and quantitation limits, it would have the effect of discouraging development of innovative technology or method modifications.

Given these issues, and EPA's desire to 1) encourage the development of improved measurement techniques, and 2) provide the stakeholder community with a variety of measurement options whenever Chapter 3 Draft Document for Peer Review  Do Not Circulate  August 2002 3 7 possible, EPA believes it would be impractical to force standardization on a single detection or quantitation limit concept on method developers and promulgate only those methods that contain this concept. We also believe, however, that there are real benefits to standardization, and that 1) all new methods developed by EPA for promulgation at 40 CFR part 136 should reflect such standardization, and 2) EPA should strongly encourage outside organizations to include these standardized concepts in their methods.

<IP_ADDRESS> Method Performance Verification at a Laboratory Just as sensitivity is important for evaluating measurement method performance, it is important to verify that a laboratory using a method can achieve acceptable levels of sensitivity for making measurements. Such demonstrations can take many forms and should be viewed in the context of the decision to be made. The analytical methods published at 40 CFR 136 are designed for monitoring compliance with Clean Water Act permits. Most pollutants in permits have a numeric limit, and compliance with these limits is determined by laboratory analysis of samples from the waste stream or water body regulated by these limits. The laboratory that conducts such analyses must be able to demonstrate that its detection or quantitation levels are low enough to assure reliable measurements.

Thus, even where a method describes the sensitivity measured or estimated by the developer or the organization that published the method, some means is needed to demonstrate that a give laboratory can achieve sufficient sensitivity to satisfy the regulatory decision ( e. g., monitoring compliance).

The EPA MDL procedure provides a means of verifying laboratory performance and has long been used in this fashion by EPA and various other Federal and State agencies. Other procedures may be employed, including analysis of reference materials containing the analytes of interest at concentrations that are at or below the regulatory limits of interest, spiked samples that are similarly prepared ( e. g., matrix spikes), or performance evaluation ( PE) samples such as those used in laboratory accreditation studies.

The IDE and IQE were advanced by the regulated industry and subsequently approved by ASTMInternational as means of characterizing the performance of a method in laboratories that participate in an interlaboratory study. The idea in developing these concepts was to establish detection and quantitation limits that could be met by any laboratory that participated in the study. An advantage of this approach is that individual laboratories do not have to demonstrate sensitivity. However, potential disadvantages also exist. For example, it may not be possible to develop a realistic IDE or IQE for a new methods involving a highly innovative technique because there may not be a sufficient number of laboratories practicing the technique to allow development of an IDE/ IQE. Also, establishing detection and quantitation limits that can be met by all laboratories that practice methods that are in widespread use can potentially lead to worst case limits that are significantly higher than limits that can be achieved by many affordable laboratories. This is somewhat analogous to buying a car at list price. This may be acceptable for those who choose to avoid the negotiation hassle, but will likely be unacceptable for others who prefer to get the best deal possible. Developers of the IDE/ IQE have recognized that an analogous concept is desirable for single laboratory application and have begun work on a within laboratory detection estimate ( WDE), to be followed by a within laboratory quantitation estimate ( WDE). As with the IDE/ IQE, these concepts will capture all sources of variability such as temporal variability, and will include a prediction or tolerance interval ( or both), but will not include interlaboratory variability. EPA's MDL remains appropriate for demonstrating method performance at an individual laboratory.

<IP_ADDRESS> National Pollutant Discharge Elimination System The National Pollutant Discharge Elimination System ( NPDES) serves as the primary means by which EPA, States, and Tribes control point source releases into the nation's waters. Under this system, Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 3 8 individual facilities are issued NPDES permits that provide limitations on the type, concentration, and volume of pollutants that may be legally discharged. Typically, these pollutant controls are based on technologybased standards. If, however, these technology based controls are not adequate to protect the water quality standard designated for the facility's receiving water, stricter controls are warranted. In such cases, NPDES permits contain water quality based controls.

Development and Implementation of Technology based Controls ( Effluent Guidelines)

EPA promulgates national effluent limitations guidelines and standards under the authority of Clean Water Act Sections 301, 304, 306, 307, 308, and 501. The regulations allow the discharge of pollutants from normal industrial processes when the discharges have been treated using various levels of available treatment technologies that are affordable. Functionally, these industry specific guidelines establish standards for the quality of wastewater discharges to waters of the United States. They are generally stated in the form of concentration based limit for selected substances that are is to be exceeded. For example, the maximum oil concentration in wastewater separated from oil pumped out of an offshore well and discharged on any single day shall not exceed 42 milligrams per liter ( mg/ L). This form is called a numeric effluent guideline limit or numeric limit.

Development and Implementation of Water Quality based Controls States designate water quality standards for various bodies of water within their boundaries. Each standard consists of a designated use, criteria to determine if the water quality supports that designated use, and an anti degradation polity. Possible designated uses include public water supply, recreation, and propagation of fish and wildlife. When the water quality standard is not met, waste load allocations are developed to indicate the maximum amount of a substance that can be discharged to a particular water body without impairing the designated use. EPA and delegated States calculate water quality based effluent limits based on the waste load allocation and the variability of the substance in the wastewater discharge. The concept is to prohibit discharge of a substance beyond the level at which a designated use would be impaired. Similar to effluent limitations guidelines and standards, permits generally specify the use of measurement methods promulgated at 40 CFR part 136 under the Clean Water Act Section 304( h).

The variability issues considered in the development of water quality based limits are similar to those considered in the development of numerical effluent guidelines limits and detection levels. All of these concepts use mathematical statistics to determine levels unlikely to be exceeded under specified conditions. Calculated water quality based limits consider all of the components of variation considered in the development of detection and quantitation limits, plus the additional sources of variability considered in the setting of effluent guidelines numeric limits. Those additional sources of variability include the variability of the substance in the industrial process producing the wastewater, variability of the wastewater treatment process, variability of the process for collecting samples, and a host of other issues.

A special case occurs when the water quality based effluent limit is less than the detection limit of the most sensitive analytical method. This case is addressed in Section 3.2.3 below on Compliance Evaluation Thresholds.

Permit Compliance Monitoring Under Clean Water Act Sections 318, 402, and 405, NPDES permits are issued to owners of facilities that discharge wastewater to waters of the United States ( coastal areas, lakes, rivers, streams, certain wetlands, etc.). Specific discharge limits are established either for individual facilities or for classes of facilities. Individual permits are established in industries with a lot of site specific issues that determine the substances Chapter 3 Draft Document for Peer Review  Do Not Circulate  August 2002 3 9 discharged, such as the pharmaceutical industry in which the specific drugs produced could influence the water quality. General permits are issued when the substances discharged are not so greatly related to the site, such as the coastal oil and gas extraction industry. The permit limits are typically established using technology based effluent guidelines, unless the facility is discharging into a water body that does not meet its designated use or that will not meet the designated use if a technology based limit permitted. In these situations, water quality based limits are used in the permit.

Detection plays a role because of concerns with measurement results at the low end of any measurement method. As discussed in the introduction, all measurement results are variable. At the low end of most measurement methods, there comes a point at which a particular measurement result is unacceptably likely ( a policy decision) to have come from a sample in which the substance of interest is absent ( zero concentration). Such a measurement result would be below the critical value defined by Currie ( 1995) and in common usage it would be called below detection. In practice, the reporting limit may be set equal to a critical value, detection limit, or quantitation limit. Assuming that the reporting limit is a detection limit of 1 mg/ L oil and grease, the measurement result would be reported as " less than 1 mg/ L of oil and grease."

<IP_ADDRESS> Non Regulatory Studies and Monitoring EPA conducts a variety of non regulatory studies and monitoring activities to support the These activities range from long term surveys, such as the Great Lakes Water Quality Surveys that are conducted each spring and summer to monitor trends in water quality against established baselines to short term studies that are used to establish baselines, model pollutant cycles, and guide direction for future study and policy. Examples of such studies include the National Study of Chemical Residues in Fish that was conducted in the late 1980s ( a follow up to that study is currently underway), and the Lake Michigan Mass Balance Study conducted in the early 1990s.

Assuming that EPA is actively designing a study or regimen, detection and quantitation concepts can be used along with information on the risks associated with substances ( pollutants) and the cost of measurement to select an appropriate measurement method before sample measurement begins. Accepting all positively valued measurement results and selecting a measurement method with a detection limit lower than the level of concern for a substance being measured would provide some assurance that measurement results associated with that concentration would be positively valued. Selecting a measurement method with a quantitation limit lower than the level of concern for a substance being measured would generate measurement results that are easier to explain.

3.2.2 Descriptive versus Prescriptive Uses of Lower Limits to Measurement The literature on detection and quantitation generally assumes that these procedures are descriptive, as opposed to prescriptive. In other words, detection and quantitation studies are described as characterizing the current performance of a laboratory or laboratories using a method to measure a substance. Two possible reasons for this treatment are: ( 1) the intended audience for the articles are laboratory staff and measurement methods developers who wish to make new methods useable to as many laboratories as possible and ( 2) the author may have an institutional reason for not attempting to control variability and thus lower detection and quantitation limits. On the other hand, the technology based and water quality based effluent limitations programs administered by EPA's Office of Water have an institutional requirement to protect human health and the environment. In order to provide this protection, the Agency must measure pollutants at ever lower concentrations. Establishing stringent standards and a compliance scheme for laboratories is one way to more rapidly develop the ability to measure at these concentrations. A prescriptive strategy concerning detection and quantitation limits would be to: Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 3 10 ° Determine the detection and quantitation limits at multiple laboratories. ° Establish detection limit and a quantitation limit for the method that is based on some performance of these laboratories. The limit could be established as the limits reported by the mean or median laboratory, or by some other criterion, such as the pooled value of the limits achieved by all laboratories, or the limit that is met by a certain percentage of the laboratories. ° Use the established detection and quantitation limit as a performance standard that must be demonstrated by laboratories that practice the method.

Such an approach is consistent with other performance standards included EPA methods, such as standards for instrument calibration, recovery of spiked reference and matrix samples, etc.

The use of such an approach would ensure that prescriptive detection and quantitation limits ( i. e., performance standards) reflect the capabilities of multiple laboratories, rather than a single state of the art research laboratory. Of course, it is possible that even when multiple laboratories are used to establish performance standards for detection and quantitation, some laboratories may not be able to achieve these standards using their current operations. However, most laboratories facing this problem will be able to achieve these standards by investing in staff training, improved equipment, a stronger quality assurance program, or higher quality maintenance and operations.

There is of course, a risk that some members of the laboratory community will not be able to meet the standard, either because they are not willing to invest the resources necessary to do so, or for other reasons. That risk needs to be considered when using a prescriptive approach to detection and quantitation ( i. e., establishing limits that act as performance standards.) Conversely, the risk of using a descriptive approach is that it can result in detection and quantitation limits that reflect a broad community of laboratories, including those that have made little if any effort to control variability at these levels. That can have the effect of raising detection and quantitation limits to a level that is higher than desired.

3.2.3 Compliance Evaluation Thresholds A situation that arises frequently in addressing water quality based limits is the setting of the permit limit below the detection/ quantitation limit of the most sensitive, approved analytical method. This subject was addressed in EPA's draft National Guidance for the Permitting, Monitoring, and Enforcement of Water Quality based Effluent Limitations Set Below Analytical Detection/ Quantitation Levels ( WQBEL guidance). The WQBEL guidance suggested use of the minimum level of quantitation ( ML) as the compliance evaluation threshold ( CET) when the water quality based effluent limit ( WQBEL) is below the detection/ quantitation limit of the most sensitive, approved analytical method. In comments on the WQBEL guidance, the regulated industry objected to the CET, claiming that it did not include interlaboratory variability and other sources of variability; the States objected to the CET, claiming that it would not allow them to be as protective as if the detection limit were used.

From a technical standpoint, a one sided limit that reduces false positives only, such as the Currie critical level or EPA's MDL, is the most appropriate concept for producing a CET for the situation in which the WQBEL is less than detection limit in the most sensitive analytical method because the one sided limit allows measurement to the lowest possible level while protecting a discharger from the risk of a false violation. For example, consider the situation in which 2,3,7,8 tetrachlorodibenzo p dioxin ( dioxin) is to be evaluated against the ambient water quality criterion of 13 parts per quintillion ( ppqt). The most sensitive analytical method is EPA Method 1613 with an MDL of 4 parts per quadrillion ( ppq) and an ML of 10 ppq. The MDL is more than 300 times greater than the ambient criterion. Therefore, if dioxin is detected in the receiving water as a result of a discharge ( at greater than an MDL of 4 ppq), there must have been an Chapter 3 Draft Document for Peer Review  Do Not Circulate  August 2002 3 11 exceedance of the ambient criterion. In the WQBEL guidance, EPA suggested use of the ML because it was the point at which the measurement could be considered reliable. However, from a purely technical standpoint, the MDL is the most appropriate.

3.2.4 Accepting the Procedures of Voluntary Consensus Standards Bodies In February 1996, Congress enacted Public Law 104 113, the National Technology Transfer and Advancement Act ( NTTAA). This act directs " federal agencies to focus upon increasing their use of ( voluntary consensus) standards whenever possible, thus reducing federal procurement and operating costs." The Act gives Federal agencies discretion to use other standards where the use of voluntary consensus standards would be " inconsistent with applicable law or otherwise impractical."

NTTAA is implemented by Federal agencies based on the policies described in Circular A 119 from the Office of Management and Budget ( OMB). The current version of this OMB circular was published in the Federal Register on February 19, 1998 ( 63 FR 8546).

Neither the NTTAA nor Circular A 119 require that agencies replace existing government standards with standards from a voluntary consensus standard body ( VCSB). In other words, if EPA already has standards in place for detection and quantitation concepts, EPA is not obligated by NTTAA to replace these with VCSB standards.

Circular A 119 also discusses the effect of the policy on the regulatory authorities and responsibilities of Federal agencies. The circular states that:

" This policy does not preempt or restrict agencies' authorities and responsibilities to make regulatory decisions authorized by statute. Such regulatory authorities and responsibilities include determining the level of acceptable risk; setting the level of protection; and balancing risk, cost, and availability of technology in establishing regulatory standards. However, to determine whether established regulatory limits or targets have been met, agencies should use voluntary consensus standards for test methods, sampling procedures, or protocols."

Thus, EPA is responsible for establishing the levels of risk and protection, not only for the regulatory limits applied to discharges, but also to the risks of decision errors ( e. g., false negatives or false positives) in the detection and quantitation concepts applicable under the Clean Water Act.

NTTAA recognizes that there are instances in which use of VCSB standards may not be appropriate. The Act includes the provision for agencies to not use consensus standards where they are " inconsistent with law or otherwise impractical." The stated objective of the Clean Water Act is " to restore and maintain the chemical, physical, and biological integrity of the Nation's waters." As a result, if adopting a VCSB standard for a detection or quantitation concept either would lead to raising the NPDES permit limits for dischargers, or prohibiting EPA or authorized States from enforcing those limits until new detection and/ or quantitation limits were developed for every method at 40 CFR 136, then EPA believes that such an approach would be inconsistent with the stated objectives of the Clean Water Act.

Finally, Circular A 119 describes two types of technical standards: a performance standard and a prescriptive standard. A performance standard is defined as " a standard ... that states requirements in terms of required results with criteria for verifying compliance but without stating the methods for achieving required results." In contrast, a Assessment of Detection and Quantitation Concepts Draft Document for Peer Review  Do Not Circulate  August 2002 3 12 prescriptive standard is one " which may specify design requirements, such as materials to be used, how a requirement is to be achieved, or how an item is to be fabricated or constructed."

Neither NTTAA nor Circular A 119 direct agencies to favor performance standards over prescriptive standards, or vice versa. EPA believes that the current MDL standard is a prescriptive one, in that it specifies both the design of the MDL study and how the requirement to establish method sensitivity be achieved. There is some obvious flexibility or opportunity for judgement in employing the MDL procedure, and much of the historical debate over the utility of the MDL procedure would suggest that it may not be prescriptive enough. All of the alternative concepts for establishing detection and quantitation are also prescriptive standards rather than performance standards.

One option that EPA may consider is to employ a performance based approach to establishing detection and quantitation limits in which method developers, laboratories, and others would be free to use any one of a variety of approaches to establishing these limits, including the existing MDL procedure or a VCSB standard. Thus, establishing method sensitivity could be considered a performance standard under NTTAA and Circular A 119, rather than a prescriptive standard. The fact that different approaches ( prescriptive standards) yield different answers would be immaterial if EPA evaluates the answers relative to a specific decision. That evaluation must not be divorced from knowledge of the decision to be made ( e. g., the regulatory limit for a given pollutant).

3.2.5 National versus Local Standards for Measurement In accordance with the Settlement Agreement, EPA is re examining the concepts of detection and quantitation used with methods approved for use at 40 CFR part 136. The Clean Water Act authorizes States and local governments to implement permits, with the requirement that they be at least as protective ( stringent) as the national standards established by EPA. Thus, EPA must take into account the impact of any revised or new detection/ quantitation limit concepts and procedures on State and local governments, as well as on those affected by State and local requirements. EPA also is aware that some States have implemented approaches to detection and quantitation that are either specific to that State, result in lower numerical limits in discharge permits, or both. Given the ability of State and local governments to use more stringent approaches, any decision by EPA with regard to this re evaluation of detection and quantitation concepts may not have an effect on those States and local governments.

3.2.6 Cost and Implementation Issues When discussing the implementation of a detection or quantitation procedure, we will make distinctions between the method developer and method user. Usually, method developers are governmental organizations such as EPA, NOAA, USGS, and DOE, or voluntary consensus standards bodies ( VCSBs) such as the American Public Health Association ( APHA), ASTM International, AOAC International, and ISO/ IUPAC. Method developers also may include manufacturers of instruments or supplies used in testing. Methods users are the laboratories performing tests to assess and assure product quality, to support regulatory compliance monitoring, or to support scientific studies.

Method development requires a more diverse set of skills than method use, in that a broad understanding of quality systems, statistics, and analytical technologies is required. Staff working for the method developer will usually include the project manager, measurement analysts, and statisticians. Method use requires a focus on obtaining reliable results in the analysis of a given sample. Staff working for the laboratory include the manager and measurement analysts. Chapter 3 Draft Document for Peer Review  Do Not Circulate  August 2002 3 13 <IP_ADDRESS> Implementation of a Detection/ Quantitation Limit Procedure by a Method Developer The basic resources available to the method developer are time, money, and the technical skills of its staff. The fundamental decision for implementing a detection or quantitation procedure is whether that procedure is intended to characterize the performance of the method at a well performing laboratory or if it is intended to characterize the performance of the method across a group of laboratories. If the procedure is intended to characterize the performance of the method across a group of laboratories it is further necessary to decide if there will be some way to compare the performance of individual laboratories to the group performance standard. There are serious time, cost, and skill issues with each of these decisions. Ordering these decisions from the least resource intensive to the most, they are characterizing the performance of the method: ( 1) at a well performing laboratory, ( 2) at a group of laboratories, or ( 3) at a group of laboratories with comparisons of individual laboratories. Other costs for the method developer could include planning, data management, reference laboratory services, and whether laboratories are willing to volunteer for the study or if their services must be purchased.

An independent decision is whether to assume a simple model for measurement variability and limit the number of test concentrations, iterate assuming a simple model, or to design a study of the relationship between measurement variation and the concentrations of the substances measured by the method. This decision will greatly influence the number of samples measured in the study. If the laboratories do not volunteer for the study, then the direct cost for measuring these samples or blanks ranges from a few dollars per sample to more than $ 1,000 per sample for measuring dioxins. Until such time as the relationship between measurement results and standardized concentrations becomes well known, such studies will require the active participation of professional statisticians in design, implementation, and analysis.