Technical Support Document for the Assessment of Detection and Quantitation Approaches February 2003 U. S. Environmental Protection Agency Office of Water ( 4303T) 1200 Pennsylvania Avenue, NW Washington, DC 20460 EPA 821 R 03 005 Technical Support Document for the Assessment of Detection and Quantitation Approaches Engineering and Analysis Division Office of Science and Technology U. S. Environmental Protection Agency Washington, DC 20460 February 2003 Acknowledgments and Disclaimer This document was prepared by Maria Gomez Taylor, Henry D. Kahn, William A. Telliard, Khouane Ditthavong, and Leonid Kopylev, of the Engineering and Analysis Division in EPA s Office of Water. Harry McCarty, Lynn Riddick, Ken Miller, and Joan Cuddeback, with DynCorp I& ET, and Dale Rushneck with Interface, Inc. provided assistance under EPA Contract No. 68 C 01 091. Sidina Dedah and Kathleen Stralka with Science Applications International Corporation provided assistance under EPA Contract No. 68 C 99 233.

Questions or comments about general aspects of this assessment should be addressed to:

William A. Telliard USEPA ( 4303T) 1200 Pennsylvania Avenue, NW Washington, DC 20460 telliard. william@ epa. gov Questions or comments about statistical issues related to this assessment should be addressed to:

Henry Kahn USEPA ( 4303T) 1200 Pennsylvania Avenue, NW Washington, DC 20460 kahn. henry@ epa. gov This document has been reviewed and approved for publication by the Engineering and Analysis Division, Office of Science and Technology. Neither the United States Government nor any of its employees, contractors, or their employees make any warranty, expressed or implied, or assumes any legal liability or responsibility for any third party s use of or the results of such use of any information, apparatus, product, or process discussed in this report, or represents that its use by such party would not infringe on privately owned rights.

February 2003 i ii February 2003 Table of Contents Acknowledgments and Disclaimer ....................................................... i Table of Contents .................................................................... iii Chapter 1 Introduction .......................................................... 1 1 1.1 Background .......................................................... 1 1 1.2 Clause 6 Settlement Agreement Requirements ............................... 1 1 1.2.1 Clause 6a...................................................... 1 1 1.2.2 Clause 6b...................................................... 1 2 1.2.3 Clause 6d...................................................... 1 2 1.2.4 Clause 6e...................................................... 1 2 1.2.5 Clause 6f ...................................................... 1 2 1.3 EPA s Approach to Conducting this Assessment ............................. 1 3 1.3.1 Study Plan..................................................... 1 4 1.3.2 Material and Data used in the Assessment ............................ 1 4 1.4 Peer Review of the Agency s Assessment................................... 1 9 1.5 Terminology used in this Document ...................................... 1 10 Chapter 2 Overview and History of Detection and Quantitation Limit Approaches ........... 2 1 2.1 Currie s Call for Standardization.......................................... 2 1 2.2 Development of the MDL and ML as Practical Embodiments of Currie s Proposal . . . 2 3 2.2.1 Method Detection Limit .......................................... 2 3 2.2.2 Minimum Level of Quantitation .................................... 2 4 2.3 Approaches Advanced by Other Organizations ............................... 2 5 2.3.1 EPA Approaches................................................ 2 5 2.3.2 Industry supported Approaches .................................... 2 7 2.3.3 Approaches Advocated by the Laboratory Community and Voluntary Consensus Standards Bodies ................................................ 2 8 2.3.4 Approaches Advocated by Other U. S. Government Agencies and Other Governments ................................................... 2 9 Chapter 3 Issues Pertaining to Detection and Quantitation .............................. 3 1 3.1 Analytical Chemistry Approaches to Detection and Quantitation ................. 3 2 3.1.1 Blank versus Zero Concentration ................................... 3 2 3.1.2 Lack of Instrument Response ...................................... 3 2 3.1.3 Matrix Effects .................................................. 3 4 3.1.4 Recovery Correction ............................................. 3 6 3.1.5 Measurement Quality over the Life of a Method....................... 3 8 3.2 CWA Regulatory Issues Affecting Detection and Quantitation .................. 3 9 3.2.1 Detection and Quantitation Limit Applications Under CWA .............. 3 9 3.2.2 Descriptive versus Prescriptive Uses of Lower Limits to Measurement .... 3 13 3.2.3 Compliance Evaluation Thresholds ................................ 3 14 3.2.4 Accepting the Procedures of Voluntary Consensus Standards Bodies ...... 3 14 February 2003 iii Assessment of Detection and Quantitation Approaches 3.2.5 National versus Local Standards for Measurement .................... 3 16 3.2.6 Cost and Implementation Issues ................................... 3 16 3.2.7 Use of a pair of related detection and quantitation procedures in all Clean Water Act applications. ............................................... 3 17 3.2.8 Alternative Procedures .......................................... 3 18 3.3 Statistical Issues ...................................................... 3 18 3.3.1 Sources of Variability ........................................... 3 19 3.3.2 Censoring Measurement Results................................... 3 20 3.3.3 Outliers ...................................................... 3 22 3.3.4 Criteria for the Selection and Appropriate Use of Statistical Models ....... 3 24 3.3.5 Methodology for Parameter Estimation ............................. 3 28 3.3.6 False Positives and False Negatives ................................ 3 28 3.3.7 Statistical Prediction and Tolerance ................................ 3 30 3.3.8 Design of Detection and Quantitation Studies ........................ 3 33 Chapter 4 Evaluation Criteria..................................................... 4 1 4.1 Criterion 1 ........................................................... 4 1 4.2 Criterion 2 ........................................................... 4 1 4.3 Criterion 3 ........................................................... 4 2 4.4 Criterion 4 ........................................................... 4 4 4.5 Criterion 5 ........................................................... 4 4 4.6 Criterion 6 ........................................................... 4 6 Chapter 5 Assessment........................................................... 5 1 5.1 Detection Limit Approaches ............................................. 5 1 5.1.1 Evaluation of the MDL ........................................... 5 1 5.1.2 Evaluation of the ASTM International Interlaboratory Detection Estimate ( IDE) ............................................................. 5 7 5.1.3 Evaluation of the ACS Limit of Detection ........................... 5 11 5.1.4 Evaluation of the IUPAC/ ISO Critical Value ( CRV) ................... 5 13 5.1.5 Evaluation of the IUPAC/ ISO Detection Limit ....................... 5 15 5.2 Quantitation Limit Approaches .......................................... 5 17 5.2.1 Assessment of the EPA Minimum level of Quantitation ( ML) ........... 5 17 5.2.2 Assessment of the IQE .......................................... 5 20 5.2.3 Assessment of the ACS Limit of Quantitation ........................ 5 24 5.2.4 Assessment of the IUPAC/ ISO Limit of Quantitation .................. 5 26 Chapter 6 Conclusions .......................................................... 6 1 References ........................................................................ 6 1 Appendix A Literature Search Regarding Detection and Quantitation Limit Approaches Appendix B Characterizing Measurement Variability as a Function of Analyte Concentration for a Variety of Analytical Techniques Appendix C Computation of Detection and Quantitation Limits iv February 2003 Chapter 1 Introduction 1.1 Background On June 8, 1999 ( 64 FR 30417), EPA promulgated ( i. e., published in a final rule) Method 1631B: Mercury in Water by Oxidation, Purge and Trap, and Cold Vapor Atomic Fluorescence Spectro metry ( the " method") for use in EPA's Clean Water Act programs. The method was developed specifically to measure mercury at ambient water quality criteria levels and includes a method detection limit ( MDL; see 40 CFR part 136, Appendix B) of 0.2 nanograms per liter ( ng/ L).

Following promulgation, a lawsuit was filed challenging EPA on the validity of the method. The basis of the challenge included several specific aspects of Method 1631 as well as the general procedures used to establish the MDL and minimum level of quantitation ( ML) published in the method. In order to settle the lawsuit, EPA entered into a settlement agreement ( the " Settlement Agreement") with the Alliance of Automobile Manufacturers, Inc., the Chemical Manufacturers Association, and the Utility Water Act Group ( collectively the   Petitioners ) and the American Forest and Paper Association (  Intervenor ) on October 19, 2000. Under Clause 6 of the Settlement Agreement, EPA agreed to perform an assessment of detection and quantitation limit concepts. The complete text of Clause 6 is provided in Exhibit 1 1 of this chapter. A summary of Clause 6 is provided in Section 1.2. The summary is followed by a description of EPA s approach to the assessment, including the material and data evaluated ( Section 1.3), the use of an independent peer review to evaluate the Agency's assessment ( Section 1.4), and a brief discussion of the terminology used in this document.

1.2 Clause 6 Settlement Agreement Requirements Clause 6 of the Settlement Agreement is titled Reassessment of Method Detection Limit and Minimum Level Procedures. Clause 6 consists of five subclauses, a  b and d  f. ( There is no subclause c).

1.2.1 Clause 6a Clause 6a broadly defines the scope of the assessment and provides a schedule for completing the initial phase. Specifically, Clause 6a requires EPA to:

  Sign and forward to the Office of Federal Register ( OFR) a notice inviting public comment on a reassessment of existing EPA procedures for determining the detection and quantitation limits of contaminants in aqueous samples.   Forward the notice to the OFR on or before February 28, 2003.   Provide a period of at least 120 days for public comment on the notice.   At a minimum, include the MDL procedure published at 40 CFR part 136, Appendix B, and the ML procedure described in Section 17.8 of Method 1631B, in the reassessment of detection and quantitation limits.   Invite comment on one or more alternative procedures for determining and describing test sensitivity.

Clause 6a also provides EPA with the option of proposing modifications to the existing procedures.

February 2003 1 1 Assessment of Detection and Quantitation Approaches 1.2.2 Clause 6b Clause 6b requires that EPA obtain a peer review of its reassessment, and describes six specific topics that must be included in the charge to the peer reviewers. Specifically, Clause 6b requires EPA to:

  Submit the reassessment of existing procedures ( including any proposed modifications thereof) and any evaluation of alternatives for peer review by experts in the field of analytical chemistry and the statistical aspects of analytical data interpretation.   Conduct the peer review in accordance with EPA s peer review policies.   Prepare a charge to the peer review panel that requests the peer reviewers to consider: < Criteria for selection and appropriate use of statistical models < Methodology for parameter estimation < Statistical tolerance and prediction < Criteria for design of detection and quantitation studies, including selection of concentration levels (  spiking levels ) < Interlaboratory variability, and < Incorporation of elements of probability design.

1.2.3 Clause 6d Clause 6d requires EPA to provide the Petitioners and Intervenor ( the   litigants ) with an opportunity for review of the Agency s assessment concurrent with the Clause 6b peer review.

1.2.4 Clause 6e Clause 6e requires EPA to provide the litigants with:

  An opportunity to meet periodically ( i. e., every six months) to discuss the Agency s progress during development of the assessment,   A plan for performing the assessment on or before the second of these meetings, and   Copies of relevant documents, where appropriate, in advance of these meetings.

1.2.5 Clause 6f Clause 6f establishes a schedule and requirements concerning final action on the notice described in Clause 6a. Specifically:

  On or before September 30, 2004, EPA is to sign and forward to the OFR a notice taking final action on the notice described in Clause 6a, and   Coincident with publication of this notice of final action, EPA is to provide the litigants with an opportunity to meet and discuss the implications of the final notice and/ or the need for any subsequent EPA action in light of the final notice.

1 2 February 2003 Chapter 1 Exhibit 1 1. Full Text of Clause 6 of the Settlement Agreement 6. Reassessment of Method Detection Limit and Minimum Level Procedures a. On or before February 28, 2003, EPA shall sign and forward to the Office of the Federal Register for prompt publication a notice inviting public comment on a reassessment of the existing Agency procedures for determination of sensitivity of analytic test methods for aqueous samples, specifically, EPA procedures for determining the detection limits and levels of quantitation of contaminants in aqueous samples, including, at a minimum, the   Definition and Procedure for Determination of the Method Detection Limit  published at 40 C. F. R. Part 136, Appendix B, as well as the   minimum level  procedures, which is described in section 17.8 of Method 1631B. The notice shall invite comment on EPA s evaluation of one or more alternative procedures for determining and describing test sensitivity. The notice also may propose modifications to the existing procedures. The notice shall invite public comment for a period of no less than one hundred twenty ( 120) days.

b. Prior to publishing the notice inviting public comment on EPA procedures for determining test sensitivity, EPA shall submit its reassessment of existing procedures ( including any proposed modifications thereof) and its evaluation of alternatives for peer review by experts in the field of analytical chemistry and the statistical aspects of analytical data interpretation. In its charge to the peer review panel, EPA shall request that the peer review consider: criteria for selection and appropriate use of statistical models; methodology for parameter estimation; statistical tolerance and prediction; criteria for design of detection and quantitation studies, including selection of concentration levels (  spiking levels ); interlaboratory variability; and incorporation of elements of probability design. EPA ( or its authorized representative) shall conduct the peer review in accordance with EPA s current peer review policies in the January 1998 Science Policy Council Handbook ( EPA 100 B 98 00) [ sic], including any subsequently developed EPA peer review documents that may revise or amend that Handbook.

[ Note  the correct document number for the Science Policy Council Handbook is EPA 100 B 98 001[

[ c. Note  there is no clause " 6. c" in the Settlement Agreement]

d. During the peer review period, EPA shall also provide an opportunity for concurrent review and comment by the Petitioners and Intervenor.

e. In the development of the reassessment/ assessment of alternatives, EPA shall provide the Petitioners and Intervenor with a periodic opportunity to meet ( i. e., every six ( 6) months) on the Agency s progress. EPA shall prepare and present the Petitioners and Intervenor with the Agency s   plan  for conducting the reassessment/ assessment of alternatives on or before the second such periodic meeting. Where appropriate, EPA shall provide the Petitioners and Intervenor with copies of relevant documents in advance of such meetings.

f. On or before September 30, 2004, EPA shall sign and forward to the Office of the Federal Register for prompt publication a notice taking final action on the notice described in subparagraph 6. a. Coincident with publication of the notice of final action, EPA shall provide Petitioners and Intervenor an opportunity to meet to discuss the implications of the final notice and/ or the need for any subsequent EPA action in light of the final notice.

1.3 EPA s Approach to Conducting this Assessment This document details the Agency s assessment of methodology for the determination of method sensitivity, specifically: detection and quantitation limits. This assessment is being conducted in accordance with a plan summarized in Section 1.3.1 and is based, in part, on an assessment of the data described in Section 1.3.2.

February 2003 1 3 Assessment of Detection and Quantitation Approaches 1.3.1 Study Plan EPA developed a technical approach for 1) conducting the assessment, and 2) complying with all applicable requirements of the Settlement Agreement. The approach was documented in a draft study plan that has since formed the general framework for the assessment described in this Assessment Document. EPA also conducted a literature search to identify and review issues and concepts that should be considered when developing the plan. A summary of this literature review is provided in Appendix A to this Assessment Document.

The study plan described roles and responsibilities for implementing the plan, provided a background discussion of detection and quantitation limit concepts, including the MDL and ML, and outlined a series of 11 events associated with the Agency s assessment of detection and quantitation limit approaches. The relationship between those planned events and this Assessment Document is summarized in Exhibit 1 2 at the end of this chapter.

Although the Settlement Agreement did not require that EPA seek formal peer review on its draft plan, the Agency chose to conduct a peer review of the draft plan. The peer review was initiated in December 2001, conducted in accordance with EPA s peer review policies, and performed by two statisticians and two chemists. EPA reviewed the comments and recommendations offered by these reviewers, and where appropriate, revised the plan to reflect the peer review comments. EPA also reviewed, and where appropriate, revised the plan to reflect comments provided by the litigants following their concurrent review.

1.3.2 Material and Data used in the Assessment In order to perform the assessment described in this document, EPA sought to collect documentation describing existing detection and quantitation limit concepts and procedures and data that could be used to evaluate these concepts and procedures.

Documentation concerning the existing concepts and procedures was obtained by performing a literature search as described in Appendix A to this Assessment Document, and where appropriate, by purchasing copies of documents describing concepts or procedures from the organizations that published them.

In performing this assessment, EPA hoped to identify a substantial amount of data containing results of direct relevance to the determination of detection and low level measurement capability. That is, measurement results in the low concentration region. To date, EPA has been able to identify only six data sets that were of use in fully evaluating variability in the range of analytical detection and quantitation. Three of the six were developed by EPA for the express purpose of studying the relationship between measurement variation and concentration across a wide variety of measurement techniques and analytes. EPA refers to these data sets as   EPA s ICP/ MS Study of Variability as a Function of Concentration,    EPA s Multi technique Variability Study  ( also referred to as the   Episode 6000 study ), and   EPA s GC/ MS Threshold Study  ( also referred to as   the Episode 6184 study ). In all three cases, replicate measurement results from each combination of analyte and measurement technique were produced by a single laboratory over a wide range and large number of concentrations. The fourth data set was developed by the American Automobile Manufacturer s Association ( AAMA) for the purpose of estimating one particular kind of quantitation value. That quantitation value is called an alternative minimum level ( AML; see Gibbons et al., 1997). In the AAMA study, replicate results were measured at a limited number of concentrations by multiple laboratories using EPA Method 245.2 ( cold vapor atomic absorption; CVAA) for mercury and EPA Method 200.7 ( inductively coupled plasma/ atomic emission spectroscopy; ICP/ AES) for twelve other metals. The final two data sets were 1 4 February 2003 Chapter 1 jointly gathered by EPA and the Electric Power Research Institute ( EPRI) to support interlaboratory validation of EPA Methods 1631 and 1638.

The studies from which these six data sets were obtained are summarized in sections <IP_ADDRESS>  <IP_ADDRESS> below. Additional information about these studies can be found in Appendices B and C to this Assessment Document.

Although the litigants offered specific suggestions for other data sets that they believed should be considered in this assessment, EPA found that these data sets did not include a sufficient number of results in the region of detection and quantitation to yield information for the assessment, overlapped with data already used in the assessment, or exhibited signs of significant contamination that made the data inappropriate for inclusion in the assessment. These data, and EPA s decisions regarding the data, are discussed in Section <IP_ADDRESS> below.

<IP_ADDRESS> EPA s ICP/ MS Study of Variability as a Function of Concentration The objective of the ICP/ MS study was to characterize variability as a function of concentration using EPA's draft Method 1638 for determination of nine metals by inductively coupled plasma with mass spectroscopy ( ICP/ MS). The nine metals were silver, cadmium, copper, nickel, lead, antimony, selenium, thallium, and zinc. The ICP/ MS instrument used in this study averages triplicate scans to produce a single measurement of each element at each concentration. Such averaging is typical of ICP/ MS design and use.

In preparation for the study, the ICP/ MS was calibrated using triplicate scans averaged to produce a single measurement of 100, 1,000, 5,000, 10,000, and 25,000 nanograms per liter ( ng/ L) for each element. Originally, the instrument was calibrated using unweighted least squares estimates under the assumption of linearity. Subsequently, the analytical results were adjusted with weighted least squares estimates. Weighted least squares estimates are based on the knowledge that variability ( expressed as the standard deviation) increases with increasing analyte concentration.

Although the instrumentation has the capability to provide intensity results for each of the three scans at each concentration, averaging the three scans to produce a single measurement is the normal operating mode, and the average was used to produce the measurements in this study. Draft Method 1638 specifies the use of average response factors rather than least squares estimation of a linear calibration, although it does allow for the use of such procedures.

All nine metals were spiked into reagent water to produce solutions at concentrations of: 0, 10, 20, 50, 100, 200, 500, 1,000, 2,000, 5,000, 10,000, and 25,000 ng/ L. Each solution was divided into seven replicate aliquots for subsequent analysis. The aliquots were analyzed beginning with the blank ( zero concentration) followed by analyses from the highest to the lowest concentration. This sequence was chosen to minimize carry over effects and to allow the analyst to stop at the concentration that returned zero results. Carry over is caused by residual sample remaining in the inlet system of the instrument, in this case, the ICP/ MS. Carry over can occur when analysis of a high concentration sample is followed by analysis of a relatively low concentration sample, as could occur if the replicates were analyzed in random order. Use of the highest to lowest analytical sequence ensured that each successive concentration analyzed was close enough to the previous concentration that any effects of carryover would be negligible and, therefore, would not compromise study results. ( A more in depth discussion of the randomized design and the effects of carry over issues is provided in Chapter 3, Section <IP_ADDRESS>).

Results at multiple mass to charge ratios, or m/ z's, were reported for each metal, although draft Method 1638 specifies only one m/ z for eight of the nine metals. For lead, m/ z's 206, 207, and 208 are February 2003 1 5 Assessment of Detection and Quantitation Approaches specified. Only data associated with m/ z's specified in draft Method 1638 were used in the ICP/ MS study.

<IP_ADDRESS> EPA s Multi technique Variability Study ( the   Episode 6000 Study )

In 1997 and 1998, EPA conducted a study of variability vs. concentration for a number of analytical methods. Five laboratories were employed for the analyses; each analyte and method combination was tested by one of these laboratories. Details of the study design are described in EPA s Study Plan for Characterizing Variability as a Function of Concentration for a Variety of Analytical Techniques ( July 1998). Based on the sampling episode number assigned to the study by the EPA Sample Control Center, the study and results have become known as the Episode 6000 study and data. The analytes and analytical techniques studied were:

  Total suspended solids ( TSS) by gravimetry   Metals by graphite furnace atomic absorption spectroscopy ( GFAA)   Metals by inductively coupled plasma atomic emission spectrometry ( ICP/ AES)   Hardness by ethylene diamine tetraacetic acid ( EDTA) titration   Phosphorus by colorimetry   Ammonia by ion selective electrode   Volatile organic compounds by purge and trap capillary column gas chromatography with a photoionization detector ( GC/ PID) and electrolytic conductivity detector ( GC/ ELCD) in series   Volatile organic compounds by gas chromatography with a mass spectrometer ( GC/ MS)   Available cyanide by flow injection/ ligand exchange/ amperometric detection   Metals by inductively coupled plasma spectrometry with a mass spectrometer ( ICP/ MS)

In this study, an initial ( range finding) MDL was determined for each combination of analyte and analytical technique using minor modifications to the MDL procedure at 40 CFR part 136. Specifically. the modifications made the optional iterative step 7 of the MDL procedure mandatory and required the spike concentration to be no more than a factor of three times the determined MDL ( instead of a factor of five times). During the study, however, two of the laboratories found that the reduction in the allowable spike range necessitated an unreasonably large number of iterations. In continuing the study, EPA returned to the spike to MDL ratio of five published in the 40 CFR part 136, Appendix B procedure.

After determining the initial MDL, each laboratory analyzed 7 replicate samples spiked at concentrations that were 100, 50, 20, 10, 7.5, 5.0, 3.5, 2.0, 1.5, 1.0, 0.75, 0.50, 0.35, 0.20, 0.15, and 0.10 times the initial MDL. In a few instances, laboratories analyzed more than 7 replicates. As often as possible, the replicate analyses at each concentration level were produced using the same calibration that was used in determining the initial MDL. Where laboratory reports indicated that multiple calibrations were conducted, each result was associated with its calibration in the data analysis.

Spiked aqueous solutions were analyzed in order from the highest concentration ( 100 times the MDL) to the concentration at which 3 or more non detects ( zeros) were encountered among the 7 replicates, or the lowest concentration specified ( 0.1 times the MDL), whichever occurred first. This analysis order ( 1) minimized carryover that could occur in some methods if a low concentration sample had followed a high concentration sample ( as may happen when samples are analyzed in random order), and ( 2) prevented collection of a large number of zeros if the signal disappeared.

For methods that do not produce a signal for a blank, the signal will disappear somewhere below the MDL, i. e., a zero will be reported. Laboratories were instructed that when three nondetects ( out of seven measurements) were reported, it was not necessary to move to the next lower concentration,

1 6 February 2003 Chapter 1 because it would be of no practical value to have laboratories measure seven zeros, move to a lower level, measure seven zeros, etc.

A variant of the iterative procedure for determining the MDL was used for organic compounds determined by chromatographic methods. Methods for organics normally list many ( 15 to 100) analytes, and the response for each analyte is different. Therefore, to determine an MDL for each analyte, the concentration of the spike would need to be inversely proportional to the response. Making a spiking solution with 15 to 100 different concentrations is cumbersome and error prone. The approach used in the study was to run seven replicates at decreasing concentrations until signal extinction, then select the concentration( s) appropriate for the determining the MDL for each analyte according to the MDL procedure. In some cases, the laboratories selected the concentrations, in others cases, EPA did. This approach was generally applied for organics analysis. However, laboratories also had the option of using some combination of the monotonically decreasing concentrations described above and a few selected concentrations to achieve the desired spiking levels.

<IP_ADDRESS> EPA s GC/ MS Threshold Study ( the   Episode 6184 Study )

Data from the Episode 6184 study of variability vs. concentration were used to evaluate the effect of GC/ MS thresholds on the ability to identify semivolatile organic compounds at low concentrations. Details of the design of this study are described in EPA s Study Plan for Characterizing Error as a Function of Concentration for Determination of Semivolatiles by Gas Chromatography/ Mass Spectrometry ( December 1998). Data were generated for 82 semivolatile organic compounds using EPA Method 1625C ( semivolatile organic compounds by GC/ MS). MDLs were not determined for these compounds. Instead, solutions of the analytes were prepared and analyzed at concentrations of 50.0, 20.0, 10.0, 7.50, 5.00, 3.50, 2.00, 1.50, 1.00, 0.75, 0.50, 0.35, 0.20, 0.15, 0.10, 0.075 and 0.050 ng/: L ( or : g/ mL). Each solution was injected into the GC/ MS in triplicate with the mass spectrometer threshold set to 0, and again in triplicate with the mass spectrometer threshold set to a level typical of that used in routine environmental analyses. As with the ICP/ MS study and the Episode 6000 study, and for the same reasons described in Section <IP_ADDRESS>, samples were analyzed in order from the highest to the lowest concentration.

<IP_ADDRESS> AAMA Metals Study of Methods 200.7 and 245.2 The American Automobile Manufacturer s Association conducted an interlaboratory study of EPA Method 200.7 ( metals by ICP/ AES) and Method 245.2 ( mercury by CVAA). The study was designed to estimate a quantitation value based on a concept termed the alternative minimum level ( AML) that had been described in the literature ( Gibbons et al., 1997). Nine laboratories participated in the study, and each reported data for the following 13 metals: aluminum, arsenic, cadmium, chromium, copper, lead, manganese, mercury, molybdenum, nickel, selenium, silver and zinc. Study samples were analyzed by EPA Method 200.7 for 12 of the metals. Mercury was determined by EPA Method 245.2.

As part of the study design, the nine laboratories were randomized prior to the start of the study. Five sample matrices ( including reagent water) were studied, including four wastewater matrices that are representative of the automotive industry. Starting from a blank, or unspiked sample, all target analytes were spiked at four concentrations to yield a total of five concentrations per matrix. Concentrations ranged from 0.01 to 10 : g/ L for mercury and selenium on the low end, and from 2.0 and 1000 : g/ L for mercury and selenium on the high end. In addition, the concentrations were matrix dependent. The same concentration ranges for each metal by matrix combination were used for all five weeks of the study.

Matrix A ( reagent water) was analyzed in all nine laboratories, and three laboratories analyzed each of the other four matrices. All analyses were repeated weekly over a five week period. As a result,

February 2003 1 7 Assessment of Detection and Quantitation Approaches a total of 6,825 observations were obtained, which includes 2,925 observations for matrix A ( 9 labs × 13 metals × 5 spike concentrations × 5 weeks), and 975 observations ( 3 labs × 13 metals × 5 spike concentrations × 5 weeks) for each of the other four matrices ( 6,825 = 2,925 + ( 975 × 4)). There were two missing values for chromium in matrix A from laboratories 1 and 9.

<IP_ADDRESS> Method 1631 Interlaboratory Validation Study The Method 1631 interlaboratory validation study was conducted by EPA to evaluate performance of the method and to gather data to evaluate existing performance specifications, including detection and quantitation limits. To accommodate stakeholder interests and expand the scope of the study, the Electric Power Research Institute ( EPRI) funded the distribution of additional samples to study participants.

This jointly funded study involved an international community of twelve participating laboratories and one referee laboratory. Each participating laboratory analyzed four different matrices, each containing mercury at a concentration selected to allow for characterization of method performance across the measurement range of the method. Each of the 12 participating laboratories was provided with 13 sample pairs ( a total of 26 blind samples). These included 1 filtered effluent pair, 1 unfiltered effluent pair, 4 filtered freshwater pairs, 1 filtered marine water pair, 1 unfiltered marine water pair, and 5 spiked reagent water pairs. All 12 laboratories received and analyzed the same sample pairs ( a total of 312 analyses). To measure the recovery and precision of the analytical system, and to monitor matrix interferences, the laboratories were instructed to analyze matrix spike and matrix spike duplicate samples on specified field samples for each filtered and unfiltered matrix, spiked at 1 5 times the background concentration of mercury determined by analysis of an unspiked aliquot of the sample. The laboratories were instructed to perform all other QC tests described in Method 1631, including the analysis of blanks, and to conduct MDL studies in reagent water following the procedure at 40 CFR part 136.

<IP_ADDRESS> Method 1638 Interlaboratory Validation Study The Method 1638 interlaboratory validation study was conducted by EPA to evaluate performance of the method and to gather data that would allow revision of existing performance specifications, including detection and quantitation limits. To accommodate stakeholder interests and expand the scope of the study, the Electric Power Research Institute funded the distribution of additional samples to study participants.

A total of eight laboratories ( and a referee laboratory) participated in the study. The study was designed so that each participating laboratory would analyze sample pairs of each matrix of interest at concentrations that would span the analytical range of the method. Each laboratory was provided with 11 sample pairs ( a total of 22 blind samples). These included 1 filtered effluent pair, 1 unfiltered effluent pair, 4 filtered freshwater pairs, and 5 spiked reagent water pairs. All eight laboratories received and analyzed the same sample pairs ( a total of 176 analyses). To measure the recovery and precision of the analysis, and to monitor matrix interferences, the laboratories were instructed to analyze a matrix spike and matrix spike duplicate of specified field samples in each filtered and unfiltered matrix, spiked at 1 5 times the background concentration of the analytes determined by analysis of an unspiked aliquot of the sample. The laboratories were instructed to perform all other QC tests described in Method 1638, including the analysis of blanks, and to conduct MDL studies in reagent water following the procedure at 40 CFR part 136.

1 8 February 2003 Chapter 1 <IP_ADDRESS> Data Considered but not Used in this Assessment The Petitioners and Intervenor to the Settlement Agreement suggested ten specific data sets that EPA should consider in its assessment of detection and quantitation limits. EPA evaluated each of these data sets to determine if the design of the study, including the concentrations targeted in the study, would provide sufficient data for evaluating measurement variability in the region of interest ( i. e., at concentrations below, at, and above the region of detection and quantitation). If such data were determined to be present, EPA further evaluated the data set to ensure that it was of sufficient quality to support the Agency s assessment. Four of the ten data sets met these requirements and were used in EPA s assessment. Table 1 identifies each of the data sets suggested by the petitioners along with EPA s rationale for using or excluding the data from this assessment.

Table 1. Dataset Source and Year Analytes and technology EPA Decision Regarding Use AAMA 1996 1997 Metals by ICP/ AES ( 200.7) Used in this assessment and described in Section <IP_ADDRESS> AAMA 1996 1997 Mercury by CVAA ( 245.2) Used in this assessment and described in Section <IP_ADDRESS> EPA/ EPRI 1997 1998 Mercury by CVAF ( 1631) Used in this assessment and described in Section <IP_ADDRESS> EPA/ EPRI 1997 1998 Metals by ICP/ MS ( 1638) Used in this assessment and described in Section <IP_ADDRESS> EPRI 1987 Metals by GFAA ( EPA 200) Not used in this assessment because of insufficient low level data EPRI 1990 Metals by ICP/ AES ( EPA 200.7) Not used in this assessment because of insufficient low level data EPRI 1994 Al, Be, Tl by GFAA ( EPA 200) Not used in this assessment because of overlap with EPA's Episode 6000 Study, which provides data on the same analytes but covers a larger number of concentrations in the region of interest AAMA 1996 1997 PCBs by GC/ ECD ( 608.2) Not used in this assessment because of overlap with EPA's Episode 6000 Study, which provides data on the same analytes but covers a larger number of concentrations in the region of interest EPRI 1996 Cd, As, Cr by GFAA ( EPA 200) Not used in this assessment because of overlap with EPA's Episode 6000 Study, which provides data on the same analytes but covers a larger number of concentrations in the region of interest MMA 2000 2001 Aroclors 1016 and 1260 by GC/ ECD Not used in this assessment. hough the study examined the region of detection and quantitation, samples spiked with low levels of Aroclors exhibited average recoveries > 500%, with RSDs > 200% across 10 laboratories, indicating contamination of the samples from an unknown source. Data Sets Suggested by Petitioners Alt 1.4 Peer Review of the Agency s Assessment In August 2002, EPA conducted a formal peer review of the Agency s assessment. This peer review, which satisfied requirements in Clause 6b of the Settlement Agreement, was conducted in accordance with EPA s peer review policies described in the Science Policy Council Handbook ( EPA 100 B 00 001). The review was performed by two experts in the field of analytical chemistry and two experts in the statistical aspects of analytical data interpretation. Each reviewer was provided with a draft February 2003 1 9 Assessment of Detection and Quantitation Approaches version of this Assessment Document, which documented the Agency s approach to the assessment and the Agency s preliminary findings and conclusions. Reviewers also were provided with copies of all data evaluated in the assessment, statistical programs used to analyze the data, and copies of the detection and quantitation concepts and procedures evaluated by EPA. In accordance with the Agency s peer review policies, the reviewers also were provided with a written   charge  intended to ensure the evaluation would meet EPA needs.

In its charge to the peer reviewers, EPA requested a written evaluation of whether the assessment approach described by EPA is valid and conceptually sound. Reviewers also were asked to consider and address eight specific questions pertaining to the adequacy of the concepts and issues considered, the evaluation criteria developed by EPA, EPA s assessment and conclusions, the data used to perform the assessment, suggested improvements to the procedures discussed, and EPA s consideration of interlaboratory vs. intralaboratory issues. Comments from peer reviewers were generally supportive of EPA s assessment and its presentation of the assessment in the draft Assessment Document. Where appropriate, EPA revised the Assessment Document to reflect specific suggestions and comments offered by the peer reviewers. This version of the Assessment Document reflects those revisions. Copies of all materials associated with the peer review, including the peer review charge, the materials provided to the peer reviewers for review, complete copies of the peer reviewers  comments, and detailed EPA responses to each of the comments are provided in the public docket supporting the Agency s assessment.

1.5 Terminology used in this Document We use the term " quantitation" in this document because of its common usage among analytical chemists, even though we recognize that the term " quantification" ( i. e., the act of quantifying) is the term listed in most dictionaries. Also, when referring to detection and quantitation, we use the words " approach" or " concept" to refer, generically, to the procedures used to establish detection and quantitation limits or the theories on which those procedures are based. We use the word " limit" rather than " level" to indicate that the detection and quantitation concepts are directed at the lowest concentration or amount at which an analyte is determined to be present ( detection) or may be measured ( quantitation). In choosing the word   limit  we do not mean to imply any sense of permanence. We recognize that measurement capabilities generally improve over time, and that detection or quantitation   limits  established today may be superseded by future developments in analytical chemistry.

1 10 February 2003 Chapter 1 Exhibit 1 2. Relationship of Assessment Document to Assessment of Detection and Quantitation Limit Approaches Event 1, Develop a detailed plan for responding to Clause 6 the Settlement Agreement: This event was completed in April 2002 when the draft plan was revised to reflect peer review and Litigant comments.

Event 2, Identify and explore issues to be considered: The Settlement Agreement identified six specific issues that should be considered during the assessment of detection and quantitation limit concepts, and subjected to formal peer review. During development of the technical approach, EPA identified a number of other issues that should be considered during the assessment. EPA listed and described each of these issues in the study plan and noted that identification of issues is likely to be a dynamic process, in that as a suite of issues is identified and discussed, other issues may surface. Finally, EPA stated its intent to prepare an   issue paper  that fully explained and discussed each of the identified issues. Chapter 3 of this Assessment Document serves the function of the issue paper described in the plan.

Event 3, Develop criteria against which concepts can be evaluated: After fully considering all relevant issues, EPA developed a suite of criteria that could be used to evaluate the suitability of various detection and quantitation procedures for use in CWA programs. Chapter 4 of this Assessment Document provides and describes the criteria selected by EPA after its consideration of all pertinent issues.

Event 4, Evaluate existing procedures for establishing detection and quantitation levels: EPA evaluated existing detection and quantitation limit concepts used or advanced 1) by voluntary consensus standards bodies ( VCSBs), 2) in the published literature, 3) by EPA. As per the terms of the Settlement Agreement, the MDL and ML were explicitly targeted for inclusion. EPA committed to evaluating concepts published by ASTM International and ISO and to consider approaches and procedures offered by other organizations such as the American Chemical Society ( ACS) and the International Union of Pure and Applied Chemistry ( IUPAC), as well as other approaches that have been adopted by EPA for use in other programs or that were identified during EPA s review of the published literature. Chapter 2 describes the concepts that EPA evaluated in the assessment. Where appropriate, these approaches also are discussed in context to the issues that are identified and discussed in Chapter 3. Chapter 5 presents the results of EPA s assessment of each approach against the evaluation criteria established in Chapter 4. Appendices B and C of this document present additional details of EPA's assessment of each approach, using the data described in Chapter 1, Section 1.3.

Event 5, Develop and evaluate alternative procedures: EPA planned to develop and evaluate alternative procedures and modifications to existing procedures only if the Agency s assessment of existing procedures suggested that modifications or alternatives to the existing procedures were needed. EPA noted that its primary objective in developing such alternatives ( or modifications) would be to address deficiencies noted in Event 4 and improve the performance of the procedures that best meet the criteria established in Event 3. In accordance with the plan and with EPA s findings during the assessment, this Assessment Document includes suggested modifications to the existing MDL and ML procedures.

Event 6, Conduct peer review of the Agency s assessment: EPA documented results of the Agency s assessment in a draft Assessment Document that was completed in August, 2002. EPA conducted a formal peer review of the assessment in accordance with the Agency s peer review policies and guidance. The peer review was performed by two experts in the field of analytical chemistry and two experts in the statistical aspects of analytical data interpretation.

Events 7  11, Actions taken following peer review. After considering peer review comments, EPA revised its assessment and the draft Assessment Document to reflect peer review comments. EPA also finalized its strategy regarding the FR notices to be published per the terms of Settlement Agreement Clause 6a and took the actions necessary to ensure publication of those notices.

February 2003 1 11 Chapter 2 Overview and History of Detection and Quantitation Limit Approaches It is not possible to measure the concentration of a substance in water all the way down to zero. As an analogy, consider the following example: imagine measuring an object less than 16th of an inch in length with a ruler marked in 1/ 16th inch increments. How well can the length of the object be measured using only the ruler? Similar issues arise as chemists try to measure ever smaller concentrations of substances in water. In response to the challenges associated with measuring low concentrations, chemists have defined numerical values that provide points of reference for reporting and using measurement results. These values are usually referred to as detection and quantitation limits. This chapter provides an overview of detection and quantitation approaches and procedures in analytical chemistry and their use in Clean Water Act applications.

2.1 Currie s Call for Standardization Since 1968, most of the literature regarding detection and quantitation has referenced the work of Dr. Lloyd Currie, recently retired from the National Institutes of Science and Technology ( NIST, formerly the National Bureau of Standards). In 1968, Currie published a paper in which he reviewed the then current state of the art regarding detection and quantitation, presented a three tiered concept, and demonstrated his concept with operational equations for a single laboratory. In his paper, Currie reviewed eight existing definitions for the concept of detection, and reported that when these eight operational definitions were applied to the same data, they resulted in numerical values that differed by nearly three orders of magnitude. These results made it impossible to compare the detection capabilities of measurement methods using available publications. Currie proposed standardizing the terminology using theoretical definitions that he called the critical value, the detection limit, and the determination limit. ( In 1995, writing on behalf of International Union of Pure and Applied Chemistry ( IUPAC), Currie used the term   quantification limit  instead of his original term   determination limit.  Substantial agreement with the International Organization for Standardization ( also known as " ISO") on the meaning and language of detection and quantitation was achieved later, although some   subtle differences in perspective  remain [ Currie, 2000]). His purpose for these definitions was to create a system in which the standard documentation of any measurement method would include a statement of capabilities that were directly comparable to any other method for measuring the same substance.

Currie used terms from statistical decision theory as the basis for his three tiered system. In 1968 and 1995, Currie defined the critical value as the measured value at which there is a small chance that the concentration in the sample is zero. Consequently, any measured result greater than or equal to the critical value is considered evidence that the sample contains the substance of interest. Currie was careful to emphasize that the decision as to whether the substance has been detected is made bycomparing the measurement result to the critical value. Figure 2 1 shows a critical value selected such that measurements greater than the critical value have less than a 1% chance of being associated with a sample that does not contain the substance of interest. The area under the curve to the right of the critical value represents the probability that a Figure 2 1 measured value will exceed the critical value. The area February 2003 2 1 Assessment of Detection and Quantitation Approaches under the curve to the left of the critical value represents the ( much greater) probability of observing a value that is less than the critical value when the true concentration is zero.

Currie ( 1968 and 1995) used the term detection limit to refer to a true concentration that has a high probability of generating measured values greater than the critical value. That is, measurements on samples that contain concentrations equal to the detection limit have a high probability of exceeding the critical value and are, therefore, unlikely to result in a decision that the substance is not detected in the sample. In Currie s concept, the critical value and the detection limit are related and functionally dependent, but it is clear that the detection decision is made on the basis of comparing sample by sample measurements to the critical value. While Currie s terminology is consistent with standard statistical decision theory, it is in all likelihood responsible for a great deal of confusion among chemists and others who may associate the term   limit  with some sort of decision point. Currie ( 1995) states:   The single, most important application of the detection limit is for planning. It allows one to judge whether the CMP Figure 2 2 ( Chemical Measurement Process) under consideration is adequate for the detection requirements.  Figure 2 2 shows a detection limit selected such that 99% of the measurements on a sample containing this concentration are expected to be above the critical value. The bell shaped curve centered at the detection limit illustrates how likely various measurement responses are when the concentration of the substance in a sample is equal to the detection limit. That is, the figure shows the probability density of values measured in a sample with a true concentration equal to the detection limit. The area under the curve to the left of the critical value is equal to 1% of the total area, while the area to the right is equal to 99%.

Currie ( 1968, 1995) defined the determination limit, later renamed the quantification limit, as ( quoting Currie, 1995)   performance characteristics that mark the ability of a CMP to adequately   quantify  an analyte.  Quantification limits   serve as benchmarks that indicate whether the CMP can adequately meet the measurement needs. The ability to quantify is generally expressed in terms of the signal or analyte ( true) value that will produce estimates having a specified relative standard deviation ( RSD) commonly 10%.  This translates into a quantification limit equal to a multiplier of 10 times the standard deviation ( a measure of measurement variability) at the limit. The multiplier of 10 ( equal to the inverse of the 10% RSD) is arbitrary, but has been used widely. IUPAC selected 10 as a   default value  ( Currie, 1995), implying other values are possible. In papers published in 1980 and 1983, the American Chemical Society s Committee on Environmental Improvement also recommended the use of a multiplier of 10 for determining quantitation limits ( see MacDougall, et al., 1980 and Keith, et al., 1983). Measured concentrations greater than the quantitation limit are considered to be reliable by chemists, although from a statistical perspective, any measured value, along with knowledge of the precision of the measurement, is useful.

Currie s goal of having method developers publish directly comparable descriptions of detection and quantitation capability remains elusive more than thirty years after publication of his first paper on this topic. Even if Currie s three tiered concept were used, the treatment of related issues causes difficulty in comparing methods. Some of these issues include interlaboratory variability, selection of appropriate statistical models, design of detection and quantitation capability studies, and statistical prediction and tolerance. These and other issues are discussed in Chapter 3 of this document.

2 2 February 2003 Chapter 2 2.2 Development of the MDL and ML as Practical Embodiments of Currie s Proposal In 1981, staff at EPA s Environmental Monitoring and Support Laboratory in Cincinnati, Ohio, published a procedure for determining what they referred to as a method detection limit ( MDL) ( Glaser et al., 1981). The MDL functions as a practical, general purpose version of Currie s critical value. The MDL was subsequently promulgated for use in CWA programs on October 26, 1984 ( 49 FR 43234) at 40 CFR part 136, Appendix B. Prior to formal development of the MDL in 1981, the EPA Office of Water had included the term   minimum level  ( ML) or   minimum level of quantitation  in some methods for analysis of organic pollutants. These methods were proposed on December 3, 1979 and subsequently promulgated on October 26, 1984, along with the MDL. Additional information about the MDL and ML is provided below in Sections 2.2.1 and 2.2.2.

2.2.1 Method Detection Limit Conscious of the definitions provided by Currie and others, Glaser et al. ( 1981) stated  [ t] he fundamental difference between our approach to detection limit and former efforts is the emphasis on the operational characteristics of the definition. [ The] MDL is considered operationally meaningful only when the method is truly in the detection mode, i. e., [ the] analyte ( the substance of interest) must be present.  Expanding on this reasoning, Glaser et al. ( 1981) developed MDL estimates for methods that produce a result of zero for blanks, such as EPA Methods 624 and 625 for determination of organic pollutants by gas chromatography/ mass spectrometry ( GC/ MS). Blank variability exists, whether or not it can be detected by measurement processes. Failure to detect this variability may be attributed to insufficient sensitivity of the measurement process or, as is the case with some measurement processes, thresholds that are built into equipment which censor measurements below certain levels. Currie s critical value is dependent on the ability to estimate measurement variability of blank samples. In cases where the substance is not detected in direct measurements on blanks, an alternative approach to estimating blank variability must be used. One option is to estimate measurement variability at concentrations that represent the lowest possible levels where a signal can be detected. This is the basic approach of the MDL, which provides a general purpose, straightforward, operational procedure for estimating a quantity analogous to the Currie critical value when measurement processes applied to blank samples do not produce detectable signals. More complex statistical procedures for estimating blank variability are possible and may be preferable from a rigorous statistical perspective, but the MDL has been found to be satisfactory by chemists in a wide range of applications.

In 1984, the MDL became a regulatory option for wastewater discharge permits authorized under the Clean Water Act. To determine the MDL, at least seven replicate samples with a concentration of the pollutant of interest near the estimated detection capabilities of the method are analyzed. The standard deviation among the replicate measurements is determined and multiplied by the t distribution for n 1 degrees of freedom ( in the case of 7 replicates, the multiplier is 3.143, which is the value for 6 degrees of freedom). The decision to base the MDL on a minimum of seven replicates reflected a consensus among EPA chemists and statisticians that a requirement of seven replicates is not overly burdensome for laboratories and that laboratories could reasonably be expected to perform the analyses in a single batch.

Both the MDL concept and the specific definition at part 136 have been used within EPA by the Office of Ground Water and Drinking Water ( OGWDW), the Office of Solid Waste ( OSW), the Office of Emergency and Remedial Response ( OERR), and others. The MDL also has been used outside of EPA in Standard Methods for the Examination of Water and Wastewater, published jointly by the American Public Health Association ( APHA), the American Water Works Association ( AWWA), and the Water Environment Federation ( WEF), and in methods published by the ASTM International, and elsewhere.

February 2003 2 3 Assessment of Detection and Quantitation Approaches Despite such widespread use, some members of regulated industry and others have claimed that the MDL is a less than ideal concept for detection. Specifically, critics have faulted the MDL because:

  There are some inconsistencies between the definition and the procedure   It does not account explicitly for false negatives   It does not account for bias   A prediction or tolerance limit adjustment is not provided, and   It does not account for interlaboratory variability These issues are discussed later in this document.

2.2.2 Minimum Level of Quantitation The minimum level of quantitation ( ML) was originally proposed on December 5, 1979 ( 44 FR 69463) in footnotes to Table 2 of EPA Method 624 and to Tables 4 and 5 of EPA Method 625. The ML was defined as the " level at which the entire analytical system must give recognizable mass spectra and acceptable calibration points" ( in the footnote to Table 2 in Method 624) and as the " level at which the entire analytical system must give mass spectral confirmation" ( in the footnotes to Tables 4 and 5 in EPA Method 625).

Between 1980 and 1984, EPA also developed Methods 1624 and 1625 and promulgated these methods along with the final versions of EPA Methods 624 and 625 on October 26, 1984 ( 49 FR 43234). The definitions of the ML in the promulgated versions of EPA Methods 1624 and 1625 were the " level at which the analytical system shall give recognizable mass spectra ( background corrected) and acceptable calibration points" ( in footnote 2 to Table 2 in Method 1624) and as the " level at which the entire GC/ MS system must give recognizable mass spectra ( background corrected) and acceptable calibration points" ( in footnotes 2 to Tables 3 and 4 in Method 1625).

As EPA developed additional methods over the next decade, the definition of the ML was generalized to " the lowest level at which the entire analytical system must give a recognizable signal and acceptable calibration point for the analyte" ( see, e. g., Section 24.2 of EPA Method 1613 at 40 CFR part 136, Appendix A). In generating actual numerical values for MLs, the lowest calibration point was estimated from method development studies and included in the methods, although a specific calculation algorithm was not used. EPA methods that include the ML generally specify the number of calibration standards to be used and the concentrations of those standards. As a result, laboratories using those methods calibrate their analytical systems with a multi point calibration ( i. e., calibrate using a series of standards at different concentrations over the range of the instrument) that includes a standard at the lowest calibration point listed in the method ( i. e., the ML).

In response to a need top establish a compliance evaluation threshold when the water quality based permit limit is below the detection limit of the most sensitive analytical method published at 40 CFR part 136, EPA refined the definition of the ML in 1994 as 10 times the same standard deviation used to calculate the MDL1 . Because the MDL is commonly determined as 3.14 times the standard deviation of seven replicate measurements, the ML was commonly calculated as 3.18 times the MDL. ( The figure of 3.18 was derived by dividing 10 by 3.14; if more than 7 replicates were used to determine the MDL,

1The refined definition of the ML first appeared in EPA's 1994 draft National Guidance for the Permitting, Monitoring, and Enforcement of Water Quality based Effluent Limitations Set Below Analytical Detection/ Quantitation Levels The draft guidance was very controversial and never finalized. However, the refined definition of the ML has remained in use or newer analytical methods.

2 4 February 2003 Chapter 2 both the MDL and the ML multipliers are adjusted accordingly, based on values from the t distribution.) This calculation makes the ML analogous to Currie s quantification limit and the American Chemical Society s limit of quantitation ( LOQ), which is defined as ten times the standard deviation of replicate or low concentration measurements ( MacDougall, et al., 1980 and Keith, et al., 1983).

To simplify implementation of the ML, the definition also was expanded to state that the calculated ML is rounded to the whole number nearest to ( 1, 2, or 5), times 10n , where n is an integer. The reason for this simplification is that calibration of an analytical system at some exact number ( e. g., 6.27) is difficult and prone to error, whereas rounding to the whole number nearest to ( 1, 2, or 5) x 10n provides a practicable value. The most recent definition of the ML is " the lowest level at which the entire analytical system must give a recognizable signal and acceptable calibration point for the analyte. It is equivalent to the concentration of the lowest calibration standard, assuming that all method specified sample weights, volumes, and cleanup procedures have been employed. The ML is calculated by multiplying the MDL by 3.18 and rounding the result to the number nearest to ( 1, 2, or 5) x 10n , where n is an integer," and this definition was contained in the version of EPA Method 1631 that was promulgated on June 8, 1999 ( 64 FR 30417) ( see Section 17.8 of EPA Method 1631 Revision B).

The ML will generally be somewhat lower than Currie s quantitation limit, even when similar sample sizes and estimation procedures are used. This is because the standard deviation used to calculate the ML will generally be smaller than the standard deviation at the lowest concentration at which the relative standard deviation is 10%. This is due to the fact that, in almost all cases, standard deviation is non decreasing with increasing concentration, e. g., it generally tends to increase as concentration increases.

Although the ML has been used successfully in EPA methods for more than 20 years, some members of the regulated industry and others have claimed that the ML is less than an ideal concept for quantitation because it:

C Does not account for interlaboratory variability, and C Is based on a multiple of the standard deviation rather than a fitted model These concerns are discussed later in this document.

2.3 Approaches Advanced by Other Organizations To expand somewhat on Currie ( 1968), standardizing the operational definitions of detection and quantitation would benefit society by making it easier to compare and select measurement methods based on low level measurement capability and requirements in particular applications. Unfortunately, in spite of agreement on general principles and definitions advanced by Currie and his supporters, consensus on procedures that would result in comparable detection and quantitation estimates has been elusive. Sections 2.3.1  2.3.3, which are by no means an exhaustive list of the various approaches advanced to date, highlight approaches that have been most widely advanced for environmental applications.

2.3.1 EPA Approaches Over the years, a number of detection and quantitation limit approaches have been developed, suggested, or used by EPA among the various organizations charged with responding to differing program mandates. In part, this situation reflects actual differences in the mandates, and in part, it reflects the fact that no concept advanced to date has emerged as a clear   winner  that meets all needs for all people. Approaches that have been used or suggested by EPA include the:

February 2003 2 5 Assessment of Detection and Quantitation Approaches   MDL and ML ( described in Sections 2.2.1 and 2.2.2)   Instrument detection limit ( IDL)   Practical quantitation limit ( PQL)   Estimate quantitation limit ( EQL)   Contract required detection limit ( CRDL) and contract required quantitation limit ( CRQL)

Instrument Detection Limit: EPA methods for analysis of metals have historically included an instrument detection limit, or IDL. Functionally, the IDL is similar to the MDL except that the IDL includes temporal variability ( it is determined on 3 non consecutive days) and does not include all sample processing steps ( the IDL characterizes the detection capabilities of the instrument as opposed to the method). Because IDLs do not reflect the entire measurement process and, for the most part, have been used only for measurement of metals, EPA did not consider the IDL as a potential alternate to the MDL when conducting the assessment described in this Assessment Document.

Practical Quantitation Limit: The practical quantitation limit, or PQL, was established in the 1980s by EPA s drinking water program as the lowest concentration at which reliable measurements can be made. The PQL is defined as " the lowest concentration of an analyte that can be reliably measured within specified limits of precision and accuracy during routine laboratory operation conditions" ( 52 FR 25690, July 8, 1987). The PQL is a means of integrating information on the performance of approved analytical methods into the development of a drinking water regulation. The PQL incorporates the following:

  Quantitation,   Precision and bias,   Normal operations of a laboratory, and   The programmatic need to have a sufficient number of laboratories available to conduct compliance monitoring analyses of drinking water samples.

EPA uses two main approaches to determine a PQL for an analyte under the Safe Drinking Water Act ( SDWA). One approach is to use the data from Water Supply ( WS) studies ( e. g., laboratory performance evaluation studies conducted by the Agency as part of the certification process for drinking water laboratories). The PQL is established at the concentration at which at least 75% of the laboratories in the study, or the subset representing EPA Regional laboratories and state laboratories, obtain results within some predetermined percentage of the true value of the test samples ( e. g., ± 30%). This approach is used in most cases when WS data are available to calculate a PQL. The WS data approach was used to determine the PQLs for Phase V inorganic chemicals such as antimony, beryllium, cyanide, nickel and thallium ( July 17, 1992; 57 FR 31776), as well as many other contaminants regulated under the SDWA.

In the absence of WS data, the second approach that EPA uses is the multiplier method. In this approach, the PQL is calculated by multiplying the EPA derived MDL by a factor between 5 and 10. The exact multiplier varies and sometimes depends on the degree of concern about the specific contaminant ( i. e., based on a human health risk assessment for consumption of drinking water).

Application of the PQL has been traditionally limited to drinking water. Furthermore, the PQL may not be related to the lowest quantitation limit because 1) the PQL is associated with the analyte and may have been determined irrespective of a specific analytical method ( e. g., using data from a variety of methods approved for that analyte at 40 CFR part 141), 2) the performance evaluation ( PE) samples from which it is derived contain pollutant concentrations that may be well above the true limit of quantitation, 3) the multiplier used to calculate a PQL when PE data are not available is somewhat dependent on concerns about risks from human exposure to contaminants in drinking water, and 4) the resulting PQLs may be too high for purposes other than the Safe Drinking Water Act ( e. g., other EPA programs). In addition, because EPA has privatized the performance evaluation program for drinking water laboratory 2 6 February 2003 Chapter 2 certification, it is not yet clear that appropriate data will be available in the future. Based on these facts, EPA did not conduct an assessment of the PQL for CWA applications.

In the late 1980s, EPA s Office of Solid Waste ( OSW) adopted a different version of the PQL as a quantitation limit. No procedure for establishing the limits was given; instead values were extrapolated from the Contract Laboratory Program CRQLs ( see below). Since 1994, OSW has actively removed the term " PQL" from its revised methods, replacing it with the term " estimated quantitation limit" ( EQL). The term PQL and the original numerical values remain in a few older OSW guidance documents.

Estimated Quantitation Limit: EPA's Office of Solid Waste has defined the EQL as:

" The lowest concentration that can be reliably achieved within specified limits of precision and accuracy during routine laboratory operating conditions. The EQL is generally 5 to 10 times the MDL. However, it may be nominally chosen within these guidelines to simplify data reporting. For many analytes the EQL analyte concentration is selected as the lowest non zero standard in the calibration curve. Sample EQLs are highly matrix dependent. The EQLs in SW 846 are provided for guidance and may not always be achievable." ( see SW 846, Chapter 1).

As noted in most newer SW 846 methods, the EQLs are provided for guidance and may not always be achievable. Because the EQL is not rigorously defined and is guidance, because the EQL may be based on the MDL, and because the EQL can be the lowest calibration point and would, therefore, overlap the ML, EPA did not consider the EQL further in its assessment of detection and quantitation approaches.

Contract Required Detection and Quantitation Limits: EPA s Superfund program has adopted the use of contractually required limits that are based on consensus among analytical chemists about levels that can realistically be achieved in commercial laboratories using a contractually specified method. Laboratories that participate in the Superfund Contract Laboratory Program ( CLP) are required to demonstrate that they can achieve the specified CRDLs and CRQLs. The CRDLs are consensus values that apply to the analyses of metals using CLP methods. The CRQLs apply to organic analytes and are based on the concentration of the lowest non zero calibration standard specified in the CLP methods, in a fashion analogous to the original derivation of the ML. Because few CWA applications involve the use of the CLP methods, EPA did not consider the CRDL or the CRQL as viable alternatives to the MDL and ML when conducting the assessment described in this document.

2.3.2 Industry supported Approaches The regulated community has demonstrated an interest in detection limit approaches since EPA first promulgated the MDL and ML for use in CWA programs in 1984 ( 49 FR 43234). As part of that rule, EPA promulgated Methods 601 through 613, 624, 625, 1624, and 1625 for organic compounds at 40 CFR part 136, Appendix A and EPA Method 200.7 for metals by inductively coupled plasma spectrometry ( ICP) at 40 CFR part 136, Appendix C. EPA also promulgated the MDL procedure at 40 CFR part 136, Appendix B. The Virginia Electric Power Company ( VEPCO) brought suit against EPA, challenging the Agency's use of the MDL in the promulgated methods. In a settlement, EPA agreed that the MDL would be applicable only to the 600 series organic methods, as these methods already contained MDL values; i. e., it would not be applicable to EPA Method 200.7. The settlement agreement did not preclude future use of the MDL by EPA or the right of VEPCO to bring suit in such future use.

After the VEPCO settlement, the regulated community, mainly through efforts of the Electric Power Research Institute ( EPRI), remained involved in detection and quantitation approaches to be used under EPA's CWA programs. The first approaches that industry advanced were the compliance February 2003 2 7 Assessment of Detection and Quantitation Approaches monitoring detection level ( CMDL) and compliance monitoring quantitation level ( CMQL) ( Maddalone, et al., 1993). The CMDL/ CMQL were variants of EPA's MDL/ ML that attempted to adjust for interlaboratory variability.

The regulated community continued its efforts to develop alternative detection and quantitation approaches with development of the " alternate minimum level" ( AML) in the mid 1990s ( Gibbons et al., 1997). The AML is based on statistical modeling of standard deviation versus concentration, which requires large amounts of data.

Most recently, the regulated community has funded development of the interlaboratory detection estimate ( IDE) and interlaboratory quantitation estimate ( IQE). The IDE and IQE have been balloted and approved by ASTM's Committee D 19 for water as Standard Practices D 6091 and D 6512, respectively. These approaches take into account nearly all sources of variability to arrive at detection and quantitation limits that are higher, on average, than the limits produced by other approaches ( see Appendix C of this Assessment Document). Because the regulated community has shifted support from the CMDL/ CMQL and the AML to the IDE and IQE, and because EPA is not aware of other organizations that currently advocate the earlier approaches, EPA did not consider industry approaches other than the IDE/ IQE in its assessment of possible alternatives to the MDL and ML.

As with all other approaches advocated to date, the IDE and IQE have fallen short of being ideal approaches for detection and quantitation for all organizations and applications. To date, EPA is not aware of a demonstrated implementation of the IDE or IQE in the development of an analytical method. Specific concerns that have been raised about the IDE and IQE are that:

  They contain an allowance for false negatives that may be inappropriate,   The IDE and IQE are based on the use of prediction and/ or tolerance intervals, which may be inappropriate,   The IDE and IQE require a large amount of data in order to be able to model variability versus concentration, including data generated in multiple laboratories, and   The complex statistical procedures involved in calculating an IDE and IQE would place a heavy burden on the analytical chemists that typically develop, modify, and use methods.

These concerns are discussed in detail later in this document.

2.3.3 Approaches Advocated by the Laboratory Community and Voluntary Consensus Standards Bodies In 1980 ( MacDougall et al., 1980) and 1983 ( Keith et al., 1983), the American Chemical Society's Committee on Environmental Improvement ( CEI) advanced approaches for the Limit of Detection ( LOD) and Limit of Quantitation ( LOQ). The ACS LOD is defined as the lowest concentration level that can be determined to be statistically different from a blank. The recommended value for the LOD is three times the standard deviation of replicate measurements of a blank or low level sample. The LOD is roughly equivalent to the MDL in numerical terms and conceptually equivalent to Currie s critical value.

The ACS LOQ is defined as the level above which quantitative results may be obtained with a specified degree of confidence. The recommended value for the LOQ is 10 times the standard deviation of replicate measurements of blanks or low level samples. Because the LOD and LOQ are still used by the analytical community, they have been included in EPA's reassessment of detection and quantitation approaches.

2 8 February 2003 Chapter 2 In the mid 1980s, the ACS CEI introduced the concept of the Reliable Detection Limit ( RDL) and the Reliable Quantitation Limit ( RQL). The RDL and RQL were attempts at simplification of the LOD and LOQ. Both the RDL and the RQL involved applying a multiplier to the standard deviation derived from replicate measurements of a low level sample. Neither concept received acceptance by the analytical community. Because the RDL and RQL are no longer being advanced by ACS, they were not considered for evaluation in EPA's assessment of detection and quantitation approaches.

In 1999 ( Currie, 1999a and 1999b), IUPAC and ISO reached substantial agreement on the terminology and approaches documented by Currie ( 1995), although   subtle differences in perspective  of the organizations remain ( Currie, 2000). IUPAC and ISO have not, to date, published methods that include limits reflecting these standards. Similarly, although ASTM International adopted the IDE in 1997 and the IQE in 2000, ASTM International has not included any IDE or IQE values in methods approved through the ASTM ballot process. On the other hand, ISO and ASTM International have published methods that employ the MDL. Because IUPAC and ISO have approved the critical value, detection limit, and quantification limit, and because ASTM International has approved through ballot the IDE and IQE, EPA has included these approaches in its assessment of detection and quantitation approaches.

At the ACS Annual Meeting held in August, 2002, CEI members discussed the issue of detection and quantitation, with the objective of determining if the LOD and LOQ approaches should be re visited. At that meeting, several members suggested that the committee consider adopting a sample specific detection limit approach in which the ratio of instrument signal to background noise is used to estimate a detection limit for each analyte in each sample analyzed. EPA did not include the signal to noise ratio concept in this assessment because its application is limited to specific types of measurement techniques, such as gas chromatography/ mass spectrometry. Limitations of this concept for use in general environmental chemistry are best illustrated by the fact that it would not apply to any of the techniques traditionally used to determine the " conventional pollutants" cited in the Clean Water Act ( the only pollutants cited by name in the Act), i. e., biochemical oxygen demand ( BOD), total suspended solids ( TSS), fecal coliforms, and pH.

2.3.4 Approaches Advocated by Other U. S. Government Agencies and Other Governments Within the U. S., EPA found that other Federal agencies tend to rely on the detection and quantitation limit approaches described above or on variants of those procedures. For example, the USGS National Water Quality Laboratory ( NWQL) began using the EPA MDL procedure in 1992. USGS has since developed a variant of the MDL called the long term MDL ( LT MDL) that employs at least 24 spiked samples prepared and analyzed by multiple analysts on multiple instruments over a 6 to 12 month period. ( Unlike EPA programs that rely on hundreds of commercial, Federal, state, and local laboratories for sample analysis, nearly all samples analyzed for USGS programs are analyzed by the USGS National Water Quality Laboratory.) As described by USGS, the long term MDL is based on many of the same fundamental assumptions as the MDL, namely:

1. Normal data distribution, 2. Constant standard deviation, and 3. Best case detection condition ( because LT MDLs typically are determined by spiking the analyte in a clean matrix, e. g., reagent water).

The primary differences between the EPA MDL and the USGS LT MDL are the longer time period and mixing of instruments and analysts ( Oblinger Childress, et al., 1999). Because the MDL and LT MDL approaches otherwise are so similar, EPA did not evaluate the long term MDL approach in this February 2003 2 9 Assessment of Detection and Quantitation Approaches assessment. Instead, EPA considered the underlying differences between the two approaches ( namely the effects of temporal, instrument, and analyst variability) in its assessment of issues ( see Chapter 3).

Outside the U. S., EPA found that the European Union ( EU) relies on the terminology and conventions developed by Currie, IUPAC, and others ( Eurachem, 2000). The EU advocates reporting all results along with an estimate of the uncertainty associated with each value. In its discussion of the issue, the EU indicates that use of the term   limit of detection  only implies a level at which detection becomes problematic and is not associated with any specific definition. Instead, the EU focuses its attention on ways to estimate uncertainty, basing its approach on the ISO Guide to the Expression of Uncertainty in Measurement ( 1993). However, the EU also notes that the use of uncertainty estimates in compliance statements and the expression and use of uncertainty at low levels may require additional guidance. The United Kingdom s Valid Analytical Measurement Programme ( VAM) has adopted a similar approach that is based on both the ISO and the Eurachem guidance ( Barwick and Ellison, 2000). Because these approaches are focused on estimating uncertainty rather than at establishing or defining limits for detection and quantitation, EPA did not consider the European approaches in this assessment.

2 10 February 2003 Chapter 3 Issues Pertaining to Detection and Quantitation As part of the Settlement Agreement concerning EPA's reassessment of detection and quantitation limit approaches, the Agency agreed to consider several specific issues pertaining to these approaches. These issues included:

  Criteria for selection and appropriate use of statistical models,   Methodology for parameter estimation,   Statistical tolerance and prediction,   Criteria for design of detection and quantitation studies, including selection of concentration levels (  spiking levels ),   Interlaboratory variability, and   Incorporation of elements of probability design.

In developing its plan for conducting this assessment, EPA identified a number of other issues that should be considered. These issues include:

  Concepts of the lower limit of measurement,   The need for approaches that can support CWA programs, including:  method performance verification at a laboratory,  method development and promulgation,  National Pollutant Discharge Elimination System ( NPDES) applications,  non regulatory studies and monitoring,  descriptive versus prescriptive uses of lower limits to measurement, and  use of a pair of related detection and quantitation procedures in all OW applications   Censoring of measurement results,   Sources of variability ( including, but not limited to interlaboratory variability),   False positives and false negatives,   Measurement quality over the life of a method,   Matrix effects,   Background contamination,   Outliers,   Instrument non response,   Accepting the procedures of voluntary consensus standards bodies ( VCSBs),   National versus local standards for measurement,   Ease of use ( i. e., ability of study managers, bench chemists, and statisticians to do what is required by a detection or quantitation limit procedure),   Cost to implement the procedures, and   Laboratory specific applications.

Approaches to establishing the lower limits of measurement were discussed in Chapter 2. For clarity and brevity, EPA has organized the remaining issues into three subsections that follow. Section 3.1 discusses the issues that are primarily driven by analytical chemistry concerns, Section 3.2 discusses the issues that are primarily driven by CWA regulatory considerations, and Section 3.3 discusses issues that are primarily driven by statistical concerns. Table 3 1, at the end of this chapter, provides a summary of each issue discussed in Sections 3.1  3.3.

February 2003 3 1 Assessment of Detection and Quantitation Approaches 3.1 Analytical Chemistry Approaches to Detection and Quantitation 3.1.1 Blank versus Zero Concentration Analytical chemists rarely, if ever, say that a sample contains zero concentration of a substance of interest. Even when the sample is created in a laboratory for the purpose of containing as little substance of interest as possible ( a blank), analytical chemists recognize the possible contribution of the blank to the final measurement result. The ability of a laboratory to reduce the concentration of a substance in the blank is often the limiting factor in attempts to make measurements at ever lower levels.

A classic example of the potential problem is illustrated by the seminal works of Patterson in the late 1960s and 1970s ( e. g., Patterson and Settle, 1976). Patterson demonstrated that the majority of concentrations of lead reported in the literature for such diverse matrices as urban dust, open ocean waters, and biological tissues were in error by several orders of magnitude. The source of the " gross positive errors" was contamination introduced during sample collection, handling, and analysis. Interlaboratory studies of the day designed to determine consensus values for reference materials were, in fact, determining the consensus values for background contamination across laboratories. Patterson recognized the value in running blank samples ( samples thought not to contain the substance of interest) to demonstrate that the sample collection, handling, and analysis processes were not introducing contamination. Patterson subsequently developed the techniques for " evaluating and controlling the extent and sources of industrial lead contamination introduced during sample collecting, handling, and analysis" that form the basis of the " clean techniques" used for metals analysis today, and that are incorporated in EPA Method 1631 among others.

The most common analytes for which contamination problems are encountered in environmental measurements are metals, primarily zinc because of its ubiquity in the environment. On the other hand, it is rare to find contamination in the measurement of organic compounds, except for methylene chloride, acetone, and a few other volatile organic compounds used as solvents in analytical laboratories. Therefore, for determination of metals, a blank is usually included or compensated in the calibration whereas, for organics, except for the solvents, the concentration in the blank is generally assumed to be zero and there is no compensation of the calibration.

Measurement methods designed to determine substances at very low concentrations may include requirements for the preparation and analysis of a variety of blanks that are designed to identify the extent and the sources of contamination. Analysts understand that " blank" does not mean zero concentration, but that through careful control and evaluation, it is possible make measurements for which the blank contribution is sufficiently small to be considered negligible.

Useful detection and quantitation limit approaches should address the potential contribution of the blank through both the design of the study that generates the detection and quantitation limit estimates and the evaluation of the study results.

3.1.2 Lack of Instrument Response Instruments do not always produce a result from an appropriately prepared sample. Sometimes this is attributable to uncontrollable instrument limitations, sometimes it is attributable to controllable instrument settings ( thresholds) established by the manufacturer or the laboratory, and sometimes it occurs randomly. As an example, gas chromatograph/ mass spectrometer ( GC/ MS) instruments often contain thresholds below which no instrument signal is reported. With no instrument signal reported, no measurement result can be reported, and the instrument will report zero to indicate the lack of a signal.

3 2 February 2003 Chapter 3 To understand how instrument thresholds are used, it may be helpful to think of background static heard on a citizen band ( CB) radio or a walkie talkie. The static is present, but it has no meaning. Turning the " squelch" knob to the point at which the static is filtered out also may make it impossible to hear the caller. In the context of detection, increasing the instrument threshold may cause the instrument to miss a substance of interest at a low level.

In 1997, EPA conducted study of 82 semivolatile ( acid and base/ neutral) organic compounds measured by EPA Method 1625 in order to observe the performance of a GC/ MS instrument both with and without an instrument threshold ( see Chapter 1, Section <IP_ADDRESS>). In the study, solutions at up to 17 concentration levels were analyzed with the threshold on ( i. e., low level signals are automatically suppressed ) and with the threshold off ( i. e., there is no suppression of signals). Samples were analyzed at decreasing concentrations, including a blank, with triplicate determinations at each concentration. For measurement results obtained with the threshold turned on, all of the measurements made on the blank were reported as zero. This was not a surprising result, given the purpose of the instrument threshold. For measurements obtained without the threshold, 27 of 230 measurements on the blank ( 11%) were reported as 0.000 ng/ mL and no measurement results were reported lower. Instrument non response at a low concentration has both direct and indirect impacts on estimating detection and quantitation limits.

The main direct impact of non response at low concentrations is that it is not possible to estimate the standard deviation of measurements at zero concentration. By definition, however, this standard deviation is required to calculate the Currie critical value. The EPA MDL procedure was constructed to deal with this problem by providing for a means estimation of a standard deviation at a low concentration. The MDL procedure includes step by step instructions for determination of a concentration as close to zero as is possible that will generate a measurement.

In order to meet the requirements of the MDL definition, it is necessary to find the concentration at which the measurement method ceases to generate measurement results, and many laboratories have run repeat measurements in order to find this concentration. This problem manifested itself in EPA's variability versus concentration ( Episode 6000) studies. The 40 CFR part 136, Appendix B procedure suggests iteration until the calculated MDL is within a factor of 5 of the spike level. For the Episode 6000 studies, EPA instructed laboratories to use a factor of 3 instead of 5 in an attempt to more narrowly define the lowest spike level at which measurements could be made.

This change to a factor of 3 also was suggested by one of the peer reviewers charged with evaluating EPA's assessment of detection and quantitation limits, who noted:

" However, the use of as much as five times the critical level for the spike concentrations could be problematic. The inflation of the MDL by using a spike at the critical level is only 25% for a method with a high level CV of 20% ( this and other calculations here are done with the Rocke and Lorenzato 1995 variance function assuming a sample size of 7). A spike concentration of 3 times the critical level inflates the MDL to a value 140% higher, which even there may be tolerable. Use of a value 5 times the critical level gives an inflation of over 280%. ..."

Following some theoretical example calculations that are not reproduced here, the reviewer's comment continues with:

" Thus, I would recommend that the procedure be altered to use concentrations that are no more than 3 times the detection limit, and perhaps to permit concentrations lower then the critical level, including possibly blanks" ( Rocke, 2002).

February 2003 3 3 Assessment of Detection and Quantitation Approaches The reviewer's calculations suggest that MDL may be strongly inflated for a spike level of 5 times the MDL, but only moderately inflated at a spike level of 3 times the MDL. However, during the Episode 6000 studies, several laboratories asked for relief from the requirement, and EPA relented after learning of the difficulties in attempting to achieve the factor of 3. If the reviewer's example calculations are correct and a practical procedure for determining the MDL using the factor of 3 could be implemented, it could exacerbate the concern from the regulated community that MDL values are too low.

Given the competing theoretical and practical considerations, one conclusion that can be drawn is that detection limits are somewhat variable and not easy to define. Further details are in the results of the studies given in Appendices B and C to this Assessment document.

In summary, both Currie's approach and EPA's approach have theoretical problems with addressing instrument non response. Any operational approach to detection or quantitation should take this issue into account.

3.1.3 Matrix Effects " Sample matrix" is a term used to describe all of the substances, other than the substance( s) of interest, present in an environmental sample. In the case of a wastewater sample, this would include the water itself, as well as any other dissolved or suspended materials. For any given measurement, some of the substances may interfere with the measurement, while others may be substances that have no effect on the measurement. Interferences in the sample may act either positively ( i. e., increasing the measured result), negatively ( i. e., decreasing the measured result), or even preventing the measurement from being made.

" Matrix effect" is a term used to describe a situation in which a substance or combination of substances in the sample ( other than the substance[ s] of interest) influence the results of the measurement. Positive interferences may inflate the results for the substance or make it difficult to distinguish one substance from another. However, unless the positive bias is consistent and predictable, the measurement result may be unreliable. Negative interferences may suppress the results for the substance to the point that the results cannot be distinguished from background instrument noise.

In some cases, finding a matrix effect indicates that the analyst should select a more appropriate method. For example, a colorimetric method for the measurement of sulfide may be a poor choice for the analysis of a sample that is very cloudy or darkly colored. In other cases, characteristics of the sample such as its pH may destroy the substance of interest, effectively preventing analysis for that substance.

Nearly all of the newer analytical methods approved at 40 CFR part 136 describe the preparation and analysis of quality control samples that are designed to indicate the presence of matrix effects ( e. g., matrix spike and/ or matrix spike duplicate samples). Many of these methods also contain techniques for addressing matrix effects. Further, EPA has developed guidance documents that amplify the discussions in those methods ( e. g., Guidance on Evaluation, Resolution, and Documentation of Analytical Problems Associated with Compliance Monitoring, June 1993, EPA 821 B 93 001). For determination of mercury by EPA Method 1631 that is the subject of the Settlement Agreement, additional guidance on resolving matrix interferences to achieve specified detection and quantitation limits is provided in EPA s Guidance for Implementation and Use of EPA Method 1631 for the Determination of Low Level Mercury ( March 2001, EPA 821 R 01 023). Following the techniques in the methods and guidance will usually reduce adverse effects of the sample matrix on detection/ quantitation limits and measurement results.

3 4 February 2003 Chapter 3 <IP_ADDRESS> Allowance for Matrix Effects in Detection and Quantitation Limits There are those who believe that detection and quantitation limits should be determined in   real world  matrices, rather than in reference matrices intended to simulate method performance in a particular matrix type. Problems with such an approach, however, are that:

  Many   real world  matrices contain the target pollutant at levels well above the detection or quantitation limit, making it impossible to characterize what can and cannot be detected at low levels. Diluting the sample to dilute the target pollutant concentration is an option. However, this also has the potential to dilute any interferences that might be present, thereby defeating the purpose of using the real world matrix.   It is not possible to anticipate and obtain samples of every possible matrix on which a method might be used when the method is being developed and detection/ quantitation limits are being established.   Although use of a reference matrix to establish detection and quantitation limits allows the results to be reproduced ( i. e., confirmed) by an independent party, such a confirmation may not be possible with many real world matrices that may be subject to seasonal, diurnal, or other types of variability.   The cost of determining detection and quantitation limits in every possible matrix would be prohibitive.

Given these difficulties, EPA believes that a reference matrix or reference matrices should be used to establish method detection and quantitation limits, but that the procedures for defining these limits should allow for evaluation of data collected in particular matrices of concern. EPA also believes that such matrix specific determinations should only be used when all efforts to resolve matrix interferences have been exhausted.

<IP_ADDRESS> Repository of Reference Matrices Two of the four peer reviewers charged with evaluating EPA s assessment of detection and quantitation limit approaches suggested that EPA create a repository of reference matrices, similar to those developed by NIST, and that these reference matrices be used to challenge a test method and to establish detection and quantitation limits ( Cooke, 2002 and Wait, 2002). EPA has considered such a repository from time to time and again in response to this suggestion, but has been unable to resolve all of the issues surrounding such a repository. Some of these issues are:

C The stability of aqueous samples, C The holding times necessary to assure stability, C The argument that no matrix from a given industrial discharge in industrial category or subcategory reflects the characteristics of another discharge in that or other industrial categories or subcategories, C The cost of maintaining such a repository, and C The potential conflict with NIST and with non governmental organizations that provide reference matrices.

Given these issues, EPA believes that the development and maintenance of standard reference materials ( SRMs) and certified reference materials ( CRMs) are best left to NIST and the commercial marketplace. EPA agrees that such reference materials are a useful means of challenging a test method and has suggested in recent methods that reference matrices be analyzed, when available, as an additional QC measure. For example, when EPA developed an appendix to Method 1631 for application to matrices other than water, EPA specified use of a quality control sample ( QCS) with the statement that " many certified reference materials ( CRMs) are available for total mercury in plants, animals, fish, sediments, soils, and sludge" and the requirement that " recovery and precision for at least one QCS per batch of samples must meet the performance specifications provided by the supplier."

February 2003 3 5 Assessment of Detection and Quantitation Approaches Although EPA agrees that SRMs and CRMs could be useful in establishing detection and quantitation limits, EPA believes that practical considerations are likely to preclude their use for this purpose in most situations. This is because the materials would need to contain the analytes of interest at levels that are near the detection limit ( e. g., within 1 to 5 times the concentration of a determined MDL). Such concentrations are unlikely to occur in an SRM produced by NIST or a CRM produced by a vendor, and diluting the CRM/ SRM would diminish matrix effects, as indicated in Section <IP_ADDRESS>.

As an alternative to using standard reference materials, EPA commonly tests its analytical methods on a variety of real world matrices, and allows for this variability in the QC acceptance criteria for the matrix spike ( MS) and matrix spike duplicate ( MSD) samples. For example, EPA published performance data in Table 3 of EPA Method 1631B for reagent water, fresh water, unfiltered and filtered marine water, and unfiltered and filtered secondary effluent, and allowed for the variability among these matrices in the QC acceptance criteria for the MS/ MSD in the method. ASTM Committee D 19 allows this approach in development of QC acceptance criteria for methods ( see Section <IP_ADDRESS> of ASTM D 5847: Standard Practice for Writing Quality Control Specifications for Standard Test Methods for Water Analysis.)

3.1.4 Recovery Correction This section addresses correction for recovery in detection and quantitation limits. The purpose of a recovery correction is to adjust a measured concentration in a sample for the amount by which the measured concentration differs from the true concentration ( if known).

To illustrate the potential need for recovery correction, consider the case of certain compounds such as organic bases ( e. g., benzidine) and acids ( e. g., phenols) that are either not totally ( 100%) recovered in the extraction process, or are adsorbed on the surface of a GC column at low ( nanogram) levels. As a result, the measured concentration of such compounds is always less than the true concentration in the water sample. These incomplete recoveries have led some developers of detection and quantitation limit approaches to believe that these limits should be recovery corrected ( i. e., that the detection or quantitation limit should be adjusted inversely proportional to the recovery). For example, if an analyte is recovered at 50%, the detection and/ or quantitation limit should be doubled. EPA believes that recovery correction may be appropriate if ( 1) the recovery is consistent across laboratories, matrices, and conditions, and ( 2) the relative variability ( as relative standard deviation) remains constant as the recovery decreases. These two requirements are rarely met; therefore, recovery correction would be appropriate only in rare circumstances.

The first requirement ( consistent recovery) would need to be tested under a variety of conditions because, if the recovery varies among laboratories, matrices, and analytical conditions, then a detection and/ or quantitation limit would need to be developed for each of these conditions. EPA's experience is that poor recovery is rarely consistent; i. e., if one laboratory measures a recovery of 40%, another laboratory may measure 20%, or 60%, but not exactly 40%.

The normal condition in environmental analytical measurements is that the variability ( as standard deviation) remains approximately constant as the recovery decreases ( i. e., the relative precision [ as RSD] is poorer at low recovery). For example, if the RSD is 10% at 100% recovery, the RSD may be 50% at 50% recovery, and may be 100% at 10% recovery. This increase in relative variability is not the result of measurements being made at lower levels, as is the normal case, but as a result of variability in the extraction ( partitioning) process. For examples of the effect of poor recovery on precision, see the quality control ( QC) acceptance criteria for the semivolatile organic compounds in Table 8 of EPA Method 1625 ( see 40 CFR part 136, Appendix A). Because nearly all detection and quantitation limits 3 6 February 2003 Chapter 3 are based on precision ( as standard deviation), including a recovery correction for an analyte with poor precision at low recovery is, in effect, a double counting for poor precision.

A third concern with the issue of recovery correction is   where does it stop?  It makes little sense to recovery correct those measurements made in the region of detection and quantitation if similar recovery correction steps are not employed for measurements at higher concentrations. EPA has traditionally viewed recovery correction with great caution, and has preferred to require that laboratories analyze quality control samples to demonstrate that analytes are recovered within an acceptable level. For example, EPA's Office of Water methods require that laboratories prepare and analyze both a reference matrix and a sample matrix that have been spiked with the analytes of interest, and that these analytes be recovered within method specified acceptance criteria. If the recovery criteria are met, then samples analyzed in the batch are considered to be reliable within the overall level of error associated with the method, and results are reported without correcting for the recovery. EPA believes that it would be inconsistent to correct for recovery in measurements made at the detection or quantitation limit, if such corrections are not made to results obtained at higher concentrations ( e. g., during the routine analysis of samples).

EPA acknowledges that recovery correction techniques are employed in some Agency methods. Most notably are those methods that employ isotope dilution techniques, in which a stable, isotopically labeled analog of each target analyte is spiked into each sample. Because of their structural similarity to the analytes of interest, the labeled analogs are assumed to behave exactly like their unlabeled analogs ( the target analytes). Because the recovery of the labeled analog will be similar to that of the target analyte, the technique allows for recovery correction of each target analyte and is particularly useful in highly complex matrices. In these methods, recovery correction techniques are specified as part of the procedures for calculating and reporting results and are dependent on the one to one relationship of the target analyte and the labeled analog. Inclusion of a further procedure for recovery correction in a detection and quantitation limit approach could result in double counting of bias.

Few of the   traditional  approaches to establishing detection and quantitation limits include procedures for recovery correction. For example, the issue was not addressed by Currie in his original proposal of a critical value or quantitation limit. Similarly, neither EPA's MDL and ML nor the American Chemical Society s LOD and LOQ, all of which are based on the approaches advanced by Currie, include a mechanism for recovery correction. When Currie introduced his critical value, he defined it as " the minimum significant value of an estimated net signal or concentration, applied as a discriminator against background noise" ( Currie, 1995). Because the critical value is defined as a measured concentration rather than a true concentration, a recovery correction is not included.

The use of recovery correction has been included in several of the most recently developed approaches for detection and quantitation. For example, the minimum detectable value ( MDV) recently adopted by ISO and IUPAC, and the interlaboratory detection estimate ( IDE) and interlaboratory quantitation estimate ( IQE) adopted by ASTM include procedures for recovery correction. The IQE also contains a further correction that we have termed a " bias" correction.

In the MDV approach, recovery is treated as a linear function versus concentration, and an extrapolation is used to estimate the recovery at zero concentration. EPA has found that this projection of the regression line to zero concentration can lead to errors because, depending on the intercept ( in concentration units), the recovery at zero concentration can be positive, zero, or negative, resulting in an inflated MDV, an MDV very close to zero, or a negative MDV. For further details, see the section titled " Negative detection limits for the ISO/ IUPAC MDV" in Appendix C to this Assessment Document, and the data in Table 2 of that appendix.

February 2003 3 7 Assessment of Detection and Quantitation Approaches The IDE and IQE fit recovery versus concentration in a way analogous to the fitting in the MDV. The difference between the treatment of recovery in the MDV and the IDE/ IQE is that an unweighted model is used in the MDV, whereas the linear model in the IDE and IQE is weighted as determined by the model of standard deviation versus concentration that is used in calculating the IDE and IQE. ( If this model is the constant model, the weighting is the same as for the MDV.)

The IQE, but not the IDE, includes an additional correction for the bias associated with an estimate of the true standard deviation at each concentration as compared to the measured standard deviation at each concentration. In this context ( a " bias" correction to the IQE), the word " bias" means the amount by which the estimated sample standard deviation differs from the true population standard deviation, and should not be confused with common use of the word " bias" in analytical chemistry measurements ( the deviation of a result from the true value, usually expressed as percent).

The effect of these corrections on detection and quantitation limits was calculated using data generated in EPA s Multi technique Variability Study ( the   Episode 6000 Study ). Details of these effects are discussed in Appendix C.

3.1.5 Measurement Quality over the Life of a Method We have all heard the expression " Practice makes perfect." Although there is no such thing as a   perfect  measurement, the idea that results get better with practice applies to the quality of measurements made with a given method over time. We can demonstrate it using simple techniques like laboratory control charts. The improvements are a result of experience, as well as improvements in equipment over time. EPA expects changes in performance when new staff are trained. For this reason, many EPA methods specify that " start up tests" be repeated each time new staff arrive. It is not unusual to see slight increases in measurement variability as new staff are trained. However, when new staff become as good as the existing staff, control charts should show it.

The use of quality control ( QC) charts as a means of tracking method and laboratory improvement as a function of time is described in EPA's Handbook for Analytical Quality Control in Water and Wastewater Laboratories ( referenced in the 40 CFR part 136, Appendix A methods). Although these charts are instructive in tracking improvement, they have two significant drawbacks: ( 1) they do not establish an absolute limit within which an analysis must be operated and ( 2) continued improvement can lead to unusually stringent limits that, eventually, will not be met. As long as absolute QC acceptance criteria ( limits), such as those found in EPA methods, are established for the determination, and as long as there is a recognition that stringent limits may be an artifact of improvement beyond what is routinely achievable, QC charts can be instructive in identifying statistically significant losses of, or improvements in, analyte responses in the region of interest. ASTM Committee D 19 adopted the philosophy of establishing absolute limits for analytical methods in approving Standard Practice D 5847.

As with most other areas of technology, measurement instruments continue to improve. Instrument manufacturers and laboratories are increasing data processing power, speed of analysis, and the reduction of chemical or electronic " noise." Any of these instrument improvements can be expected to improve the measurement method in determining the concentrations of environmental pollutants. This process can be illustrated for a variety of EPA methods. A case in point is EPA Method 1613 for determination of polychlorinated dibenzo p dioxins and polychlorinated dibenzofurans. Development of this method began in 1988. At the time, high resolution mass spectrometer systems that were commercially available were able to achieve a detection limit of approximately 4 pg/ L and an ML of 10 pg/ L. By the time that EPA proposed the method in 1991, the Canadian government published its own version that included a quantitation limit 5 pg/ L. By the time EPA officially promulgated Method 1613 3 8 February 2003 Chapter 3 in 1997, many laboratories performing the analysis had replaced or supplemented their old instruments with newer models. As a result, many laboratories performing analyses using Method 1613 routinely measure sample results at levels 10 times lower than those analyzed routinely only 10 years earlier.

Given that measurement capabilities tend to improve over time, EPA believes that a detection and quantitation limit approach should be supported by procedures that will allow individual laboratories and other organizations to affordably characterize such improvements.

3.2 CWA Regulatory Issues Affecting Detection and Quantitation Section 3.2.1 below provides a brief overview and a discussion of Clean Water Act activities that involve chemical measurements and are, therefore, directly impacted by detection and quantitation limit approaches. Specific issues that must be considered in the context of these CWA applications and EPA s regulatory obligations are discussed in Sections 3.2.2  3.2.6.

3.2.1 Detection and Quantitation Limit Applications Under CWA The Clean Water Act directs EPA, States, and local governments to conduct a variety of data gathering, permitting, and compliance monitoring, and enforcement activities. Many of these activities depend directly on environmental measurements and, therefore, are affected by detection and quantitation limit approaches as discussed in the subsections that follow.

<IP_ADDRESS> Method Development and Promulgation Section 304( h) of the Clean Water Act ( CWA; the " Act") requires EPA to promulgate test procedures ( analytical methods) to be used for data gathering to support certification, permitting, and monitoring under the Act. These methods are promulgated at 40 CFR part 136, and include methods developed by EPA as well as those developed by other organizations, such as the publishers of Standard Methods for the Examination of Water and Wastewater, as well as AOAC International, ASTM International, the U. S. Geological Survey, instrument manufacturers, and others. Upon request by a laboratory, permittee, instrument manufacturer, or other interested party, EPA considers alternate testing procedures ( ATPs). If EPA deems these ATPs to be acceptable for nationwide use, they too, may be published at 40 CFR part 136. A primary objective in promulgating methods developed by EPA and by other organizations is to provide the regulatory community, permittees, and laboratories with multiple options so that they may choose the method that yields the best performance at the lowest cost for the application.

In recent years, EPA has focused on developing methods for promulgation at 40 CFR part 136 where no other methods are available that meet an immediate or anticipated regulatory need. The National Technology Transfer and Advancement Act of 1995 ( NTTAA) urges government agencies to consider methods published by voluntary consensus standards bodies ( VCSBs), such as Standard Methods and ASTM International, when VCSB methods are available. EPA accepts that many of these methods have been through a sufficient level of testing, peer review, and scientific acceptance to warrant proposal if they meet EPA's regulatory needs. When an individual laboratory, permittee, or other organization submits a request for approval of an alternate test procedure, however, EPA generally requires that the procedure be subjected to a level of testing that demonstrates that the method provides sensitivity, accuracy, and other measures of performance comparable to an approved method.

The lack of widespread consensus on detection limits has obvious impacts on EPA s responsibility to promulgate methods under CWA. Most organizations that develop methods use February 2003 3 9 Assessment of Detection and Quantitation Approaches different approaches, and many organizations have changed approaches over the years. The result is that a number of different approaches for detection and quantitation are embodied in the methods approved at 40 CFR part 136. The vast majority of the approved methods include the MDL which, as noted in Section 2.2.1, has been used by several EPA Offices, Standard Methods, AOAC, ASTM, and others. Other approaches embodied in the methods at 40 CFR part 136 include, but are not limited to: 1) a method   range  that is usually not defined, but is often interpreted as the lower end of the range in which pollutants either can be identified or quantified, 2) an   instrument detection limit  that has been defined by a variety of procedures, but is intended to capture instrument sensitivity only, 3) an " estimated detection limit" that may be based on best professional judgement, single laboratory data, or some other source of information, 4) a " practical quantitation limit," that has typically been determined according to one of the scenarios described in Section 2.3.1, and 5) " sensitivity" that is an undefined concept similar in result to the MDL.

The most obvious solution to this problem would be for the Office of Water to force all methods promulgated at 40 CFR part 136 to contain uniform approaches for detection and quantitation. Unfortunately, taking such action would confound methods promulgation. Problems with this solution are that:

  To date, no single detection and quantitation limit approach has emerged to meet the needs of all organizations for all applications.   If the Office of Water were to select an approach that differs from those of other organizations, those organizations would be required to conform their method to accommodate OW s approach. Doing so would mean that these organizations would have to invest additional laboratory resources to develop detection and quantitation limits that conformed to OW definitions.   If outside organizations decided against conforming their approaches to that of OW, fewer methods would be promulgated at 40 CFR part 136. This would result in fewer options for the regulatory, permittee, and laboratory communities.   If EPA selected an approach that has burdensome procedures for developing detection and quantitation limits, it could discourage development of innovative technology or method modifications.

Given these issues, and EPA s desire to 1) encourage the development of improved measurement techniques, and 2) provide the stakeholder community with a variety of measurement options whenever possible, EPA believes it would be impractical to force standardization on a single detection or quantitation limit approach on method developers and promulgate only those methods that contain this approach. The Agency also believes, however, that there are real benefits to standardization, and that 1) all new methods developed by EPA for promulgation at 40 CFR part 136 should reflect such standardization, and 2) EPA should strongly encourage outside organizations to include these standardized approaches in their methods.

<IP_ADDRESS> Method Performance Verification at a Laboratory Just as sensitivity is important for evaluating measurement method performance, it is important to verify that a laboratory using a method can achieve acceptable levels of sensitivity for making measurements. Such demonstrations can take many forms and should be viewed in the context of the decision to be made. The analytical methods published at 40 CFR part 136 are designed for monitoring compliance with CWA permits. Most pollutants in permits have a numeric limit, and compliance with this limit is determined by laboratory analysis of samples from the waste stream or water body regulated by the limit. The laboratory that conducts such analyses must be able to demonstrate that its detection or quantitation limits are low enough to assure reliable measurements.

3 10 February 2003 Chapter 3 Thus, even where a method describes the sensitivity measured or estimated by the developer or the organization that published the method, some means are needed to demonstrate that a given laboratory can achieve sufficient sensitivity to satisfy the regulatory decision ( e. g., monitoring compliance).

The EPA MDL procedure provides a means for verifying laboratory performance and has long been used in this fashion by EPA and various other Federal and state agencies. Other procedures may be employed, including analysis of reference materials containing the analytes of interest at concentrations that are at or below the regulatory limits of interest, spiked samples that are similarly prepared ( e. g., matrix spikes), or performance evaluation ( PE) samples such as those used in laboratory accreditation studies.

The IDE and IQE were advanced by the regulated industry and subsequently approved by ASTM International as a means of characterizing the performance of a method in laboratories that participate in an interlaboratory study. The idea in developing these approaches was to establish detection and quantitation limits that could be met by any laboratory that participated in the study. An advantage of this approach is that individual laboratories do not have to demonstrate sensitivity. However, potential disadvantages also exist. For example, it may not be possible to develop a realistic IDE or IQE for a new method involving a highly innovative technique because there may not be a sufficient number of laboratories practicing the technique to allow development of an IDE/ IQE. Also, establishing detection and quantitation limits that can be met by all laboratories that practice methods that are in widespread use can potentially lead to worst case limits that are significantly higher than limits that can be achieved by many commercial laboratories.

Developers of the IDE/ IQE have recognized that an analogous approach is desirable for single laboratory application and have begun work on a within laboratory detection estimate ( WDE), to be followed by a within laboratory quantitation estimate ( WQE). As with the IDE/ IQE, these approaches will capture a wide range of sources of variability such as temporal variability, and will include a prediction or tolerance limit ( or both), but will not include interlaboratory variability. EPA would consider such single laboratory approaches if and when they are adopted by ASTM International.

<IP_ADDRESS> National Pollutant Discharge Elimination System The National Pollutant Discharge Elimination System ( NPDES) serves as the primary means by which EPA, States, and Tribes control point source releases into the nation s waters. Under this system, individual facilities are issued NPDES permits that provide limitations on the type, concentration, and volume of pollutants that may be legally discharged. Typically, these pollutant controls are based on technology based standards. If, however, these technology based controls are not adequate to protect the water quality standard designated for the facility's receiving water, stricter controls are warranted. In such cases, NPDES permits generally contain water quality based controls.

Development and Implementation of Technology based Controls ( Effluent Guidelines)

EPA promulgates national effluent limitations guidelines and standards under the authority of Clean Water Act Sections 301, 304, 306, 307, 308, and 501. The regulations allow the discharge of pollutants from normal industrial processes when the discharges have been treated using various levels of available treatment technologies that are affordable. Functionally, these industry specific guidelines establish standards for the quality of wastewater discharges to waters of the United States. They are generally stated in the form of concentration based limits for selected substances that are not to be exceeded. For example, the maximum oil concentration in wastewater separated from oil pumped out of an offshore well and discharged on any single day shall not exceed 42 milligrams per liter ( mg/ L). This form is called a numeric effluent guideline limit or numeric limit.