United States Office of Water EPA 821 B 03 004 Environmental Protection 4303T Agency EPA Microbiological Alternate Test Procedure ( ATP) Protocol for Drinking Water, Ambient Water, and Wastewater Monitoring Methods Guidance July 2003 ACKNOWLEDGMENTS This protocol was prepared under the direction of the U. S. Environmental Protection Agency's ( EPA's) Office of Water. This document was prepared by the DynCorp Biology Studies Group.

DISCLAIMER The Engineering and Analysis Division, of the Office of Science and Technology, has reviewed and approved this report for publication. The Office of Science and Technology directed, managed, and reviewed the work of DynCorp in preparing this report. Neither the United States Government nor any of its employees, contractors, or their employees make any warranty, expressed or implied, or assumes any legal liability or responsibility for any third party's use of or the results of such use of any information, apparatus, product, or process discussed in this report, or represents that its use by such party would not infringe on privately owned rights. Mention of company names, trade names, or commercial products in this protocol does not constitute endorsement or recommendation for use.

Questions concerning this report should be addressed to:

Robin K. Oshiro Engineering and Analysis Division ( 4303T) U. S. EPA Office of Water, Office of Science and Technology 1200 Pennsylvania Avenue, NW Washington, DC 20460 oshiro. robin@ epa. gov 202 566 1075 202 566 1053 ( facsimile) FOREWORD This document describes a process for seeking EPA approval of microbiological alternate test procedures ( ATPs) or new methods for use in monitoring drinking water, ambient water, and wastewater. This document serves as a supplement to the ATP guidelines at 40 CFR 136.4, 136.5, and 141.27.

With this guidance document, EPA has revised and combined the three existing microbiological ATP protocols to make the ATP process more transparent to applicants while maintaining the same level of data quality. The guidelines in Quantitative Membrane Filter Methods ( Reference 10.16), Presence/ Absence Liquid Culture Methods for Finished Water ( Reference 10.17), and Presence/ Absence Membrane Filter Methods for Finished Waters ( Reference 10.15) have been incorporated into this single protocol, which is applicable to a wider range of analytes and techniques.

This revised ATP protocol describes a process for conducting side by side method comparisons and for conducting quality control ( QC) acceptance criteria based method studies for EPA approved reference methods with QC acceptance criteria. Additionally, in some cases the revised protocol provides applicants an opportunity to demonstrate comparability by meeting QC acceptance criteria associated with the EPA approved reference methods for different combinations of analyte and determinative technique.

Under EPA's ATP program, any person may apply for approval of the use of an ATP or new method to test for a regulated analyte. EPA anticipates that the standardized procedures described herein should generally expedite the approval of ATPs, encourage the development of innovative technologies, and enhance the overall utility of the EPA approved methods for compliance monitoring under the National Pollution Discharge Elimination System ( NPDES) permit program and national primary drinking water regulations ( NPDWRs).

This document is not a legal instrument and does not establish or affect legal obligations under Federal regulations. EPA reserves the right to change this protocol without prior notice.

Questions or comments regarding this document should be directed to:

Robin K. Oshiro Engineering and Analysis Division ( 4303T) U. S. EPA Office of Water, Office of Science and Technology 1200 Pennsylvania Avenue, NW Washington, DC 20460 oshiro. robin@ epa. gov 202 566 1075 202 566 1053 ( facsimile) EPA Microbiological Alternate Test Procedure ( ATP) Protocol i TABLE OF CONTENTS Section 1.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1.1 Background and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1.2 Types of Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3 1.2.1 Limited Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3 1.2.2 Nationwide Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3 1.3 Types of Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 1.4 Scope of Alternate Test Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 1.4.1 EPA Approved Reference Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 1.4.2 Modifications to Sample Preparation Techniques . . . . . . . . . . . . . . . . . . . . . . 1 5 Section 2.0 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 2.1 Submission Addresses and Approval Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 2.2 Application Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 2.3 Reason for ATP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 2.4 Standard EPA Method Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 2.5 Method Comparison Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 2.6 Method Development Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 2.7 Study Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 2.8 Study Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4 2.9 Proprietary Information in Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4 Section 3.0 Method Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 3.1 Scope and Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 3.2 Summary of Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 3.3 Method Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 3.4 Interferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 3.5 Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 3.6 Equipment and Supplies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 3.7 Reagents and Standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 3.8 Sample Collection, Preservation, and Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 3.9 Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 3.10 Calibration and Standardization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 3.11 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 3.12 Data Analysis and Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 3.13 Method Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3.14 Pollution Prevention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3.15 Waste Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3.16 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3.17 Tables, Diagrams, Flowcharts, and Validation Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 Section 4.0 Study Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1 4.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1 4.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1 4.3 Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1 4.4 Coordination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 4.4.1 Study Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 4.4.2 Technical Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 4.5 Data Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 EPA Microbiological Alternate Test Procedure ( ATP) Protocol ii Section 5.0 Quality Assurance/ Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1 5.1 Quality Assurance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1 5.2 Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1 5.2.1 Analyst Counting Variability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 5.2.2 Autoclave Sterilization Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 5.2.3 Dilution/ Rinse Water Blanks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 5.2.4 Incubator/ Waterbath Temperatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 5.2.5 Initial Demonstration of Capability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 5.2.6 Initial Precision and Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4 5.2.7 Matrix Spike and Matrix Spike Duplicate Samples . . . . . . . . . . . . . . . . . . . . . 5 4 5.2.8 Media Sterility Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4 5.2.9 Method Blank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4 5.2.10 Ongoing Demonstration of Capability ( ODC) Samples . . . . . . . . . . . . . . . . . . 5 5 5.2.11 Ongoing Precision and Recovery ( OPR) Samples . . . . . . . . . . . . . . . . . . . . . . 5 5 5.2.12 Positive/ Negative Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 5 5.2.13 Preparation Blanks ( PB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 6 5.2.14 Refrigerator/ Freezer Temperatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 6 5.2.15 Sample Processing Equipment Sterility Checks . . . . . . . . . . . . . . . . . . . . . . . . 5 6 Section 6.0 Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1 6.1 Side by Side Comparison Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1 6.1.1 Number of Laboratories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1 6.1.2 Number of Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1 6.1.3 Verification of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 6.2 QC Acceptance Criteria Based Comparison Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4 6.2.1 Number of laboratories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4 6.2.2 Number of Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4 6.2.3 Number of Replicates per Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4 Section 7.0 Sample Preparation and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1 7.1 Collection of Samples for Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1 7.1.1 Source Water Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1 7.2 Sample Spiking and " Stressing" Procedures for Bacteriological Methods . . . . . . . . . . 7 1 7.2.1 Drinking Water: Spiking and Chlorine Stressing . . . . . . . . . . . . . . . . . . . . . . . 7 2 7.2.2 Preparation of Enumerated Spiking Suspension . . . . . . . . . . . . . . . . . . . . . . . . 7 4 7.2.3 Log Phase Growth Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 5 7.2.4 Commercially Available Enumerated Spikes . . . . . . . . . . . . . . . . . . . . . . . . . . 7 5 7.3 Spiking Procedures for Virus Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 6 7.3.1 Cell Monolayer Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 6 7.3.2 Propagation of Virus Stock Suspension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 6 7.3.3 Titering of the Virus Stock Suspension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 6 7.4 Spiking Procedures for Cryptosporidium and Giardia . . . . . . . . . . . . . . . . . . . . . . . . . 7 6 7.5 Analysis of Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 7 7.5.1 Side by Side Comparison Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 7 7.5.2 QC Acceptance Criteria Based Comparison Studies . . . . . . . . . . . . . . . . . . . . 7 7 7.6 Verification of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 7 7.6.1 Verification of Results from Bacteriological Methods . . . . . . . . . . . . . . . . . . . 7 8 7.6.2 Verification of Results from Virus Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 8 7.6.3 Verification of Results from Cryptosporidium and Giardia Methods . . . . . . . 7 8 EPA Microbiological Alternate Test Procedure ( ATP) Protocol iii Section 8.0 Review of Study Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1 8.1 Assessment of Compliance with Approved Study Plan . . . . . . . . . . . . . . . . . . . . . . . . . 8 1 8.2 Data Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1 8.3 Data Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1 8.4 Development of Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1 8.4.1 Mean Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1 8.4.2 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2 8.4.3 False Positive Rates, False Negative Rates, Sensitivity, and Specificity . . . . . 8 2 8.5 Statistical Assessment of Method Comparability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3 8.5.1 Presence / Absence Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3 8.5.2 Quantitative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5 8.5.3 QC Acceptance Criteria Based Comparison Studies . . . . . . . . . . . . . . . . . . . 8 10 8.6 Method Recommendation and Approval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 12 Section 9.0 Study Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1 9.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1 9.2 Study Objectives and Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 9.3 Study Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 9.4 Data Reporting and Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 9.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 9.6 Data Analysis and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 9.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 9.8 Appendix A  Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 9.9 Appendix B  Study Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 9.10 Appendix C  Supporting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 9.10.1 Raw Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 9.10.2 Electronic Data Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4 9.10.3 Example Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4 9.11 Appendix D  Supporting References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4 Section 10.0 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1 LIST OF APPENDICES Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Glossary Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ATP Application Form Appendix C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application Inventory Form Appendix D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Elements and Example Bench Sheets Appendix E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . EPA Headquarters and Regional ATP Contacts EPA Microbiological Alternate Test Procedure ( ATP) Protocol 1 1 LU WW & AW ATP application Apply through State Director RA provides letter of approval or disapproval NW DW, WW, & AW Regional review Forward Forward to OW for review OW review OW recommends approval or disapproval Acronyms: AW = Ambient Water DW = Drinking Water LU = Limited use NW = Nationwide use OW = Office of W ater RA = Regional Administrator WW = Wastewater OW recommendation Optional EPA Administrator approves through rulemaking SECTION 1.0 INTRODUCTION 1.1 Background and Objectives In accordance with the Clean Water Act ( CWA) and Safe Drinking Water Act ( SDWA), the U. S. Environmental Protection Agency ( EPA) promulgates guidelines establishing test procedures ( analytical methods) for data gathering and compliance monitoring under National Pollution Discharge Elimination System ( NPDES) permits and national primary drinking water regulations ( NPDWRs). The approved test procedures can be found in the Code of Federal Regulations ( CFR) at 40 CFR Part 136 for wastewater and ambient water and 40 CFR Part 141 for drinking water. In addition, EPA's regulations at 40 CFR 136.4, 136.5, and 40 CFR 141.27, allow entities to apply for Agency permission to use an alternate test procedure ( ATP) in place of an EPA approved reference method. Figure 1.1 below summarizes the ATP or new method review process within EPA. These regulations are the basis for the Agency's alternate test procedure ( ATP) program for water methods that is administered by the Office of Water, Office of Science and Technology, Director of Analytical Methods.

An ATP is a modification of an EPA approved reference method or a procedure that uses the same determinative technique ( i. e., the physical and/ or chemical process used to determine the identity and concentration of an analyte) and measures the same analyte( s) of interest as the EPA approved reference method. The use of a different determinative technique to measure the same analyte( s) of interest as an EPA approved reference method is considered a new method.

Under the ATP program, an organization or individual may apply for approval of an ATP or new method to be used as an alternate to an EPA approved reference method. The applicant is generally responsible for characterizing method performance of its proposed alternate test procedure prior to submission to the ATP program. EPA can provide assistance to applicants in the development of a study plan to demonstrate comparability with the EPA approved reference method. Figure 1.1 summarizes the ATP or new method review process within EPA. The Agency reviews the ATP package, approves or disapproves the application, and, for nationwide applications, will generally propose to include successful ATPs in the CFR ( unless the ATP is for limited use or constitutes a minor modification See Appendix A).

Figure 1 1. Summary of the ATP or New Method Review Process EPA Microbiological Alternate Test Procedure ( ATP) Protocol 1 2 To make the ATP program more accessible to applicants while maintaining data quality, EPA has revised and combined the three microbiological ATP protocols into this comprehensive protocol. The guidelines in Quantitative Membrane Filter Methods ( Reference 10.16), Presence/ Absence Liquid Culture Methods for Finished Water ( Reference 10.17), and Presence/ Absence Membrane Filter Methods for Finished Waters ( Reference 10.15) have been incorporated into a single protocol applicable to a wider range of analytes and techniques. Additionally, the revised protocol illustrates how applicants can demonstrate comparability by meeting quality control ( QC) acceptance criteria associated with EPA approved reference methods for which those criteria have been developed.

The ATP program provides laboratories and regulated facilities with an opportunity to enhance compliance monitoring and encourages the use of innovative technologies. Approval for an ATP or new method may be sought when, for example, the alternate procedure reduces analytical costs, overcomes matrix interferences problems, improves laboratory productivity, or reduces the amount of hazardous materials used and/ or produced in the laboratory.

Any person or organization may apply to gain approval for the use of an ATP or new method for determination of a specific constituent that is regulated under the NPDES permit program or the NPDWRs. The ATP applicants generally may demonstrate comparability of its proposed ATP or new method with the EPA approved reference method using the procedures described in this protocol. Other possible method comparison procedures include those provided by organizations such as ASTM ( Reference 10.5), AOAC International ( Reference 10.1), and ISO ( Reference 10.8).

1.2 Types of Applications The types of applications submitted may depend on the intended use of the ATP. Methods intended for use in demonstrating compliance with the NPDES permit program ( wastewater or ambient water ATPs) may be submitted for approval for limited use ( single laboratory) or for nationwide use ( all laboratories). Because only the Administrator has the authority to approve an alternate analytical technique for SWDA purposes, EPA will generally consider proposed methods intended for use in demonstrating compliance with NPDWRs ( drinking water ATPs) that are submitted for approval for nationwide use only.

1.2.1 Limited Use The primary intent of the limited use ATP is to allow use of an ATP or new method by a single laboratory. Limited use ATPs can be applied to one or more matrix types, excluding drinking water matrices; limited use applications generally will not apply to Office of Ground Water and Drinking Water ( OGWDW) ATP applications ( Reference 10.18). If a method developer intends to apply the method to more than one matrix type, method studies should be conducted on each matrix type. Generally, nine different wastewater types should be analyzed to demonstrate the ATP or new method will be applicable to most other matrix types. If method modifications are within the specified flexibility of the EPAapproved reference method and all QC acceptance criteria are met, it generally will not be necessary to submit the modification to the ATP program.

1.2.2 Nationwide Use The primary intent of a nationwide use ATP is to allow use of an ATP or new method by all regulated entities and laboratories for one or more matrix types including drinking water. Nationwide use approval allows vendors to establish that new devices and reagents produce results that are acceptable for compliance monitoring purposes, and allows environmental laboratories across the United States to apply new technologies or modified techniques throughout their chain of laboratories to one or more matrix types. If a method developer intends to apply the method to more than one matrix type, method studies should be conducted on each matrix type. Generally, nine different wastewater types should be analyzed EPA Microbiological Alternate Test Procedure ( ATP) Protocol 1 3 to demonstrate the ATP or new method will be applicable to most other wastewaters. If method modifications are within the specified flexibility of the EPA approved reference method and all QC acceptance criteria are met, it generally will not be necessary to submit the modification to the ATP program.

1.3 Types of Studies The type of study most useful in seeking approval of an ATP or new method generally depends on whether or not the EPA approved reference methods contain QC acceptance criteria. There are two basic types of studies described in this protocol: ° Side by Side Method Comparison Study. A side by side method comparison study generally consists of parallel testing of an ATP or new method along side an EPA approved reference method to determine whether the performance of the new or modified method is acceptable compared to the reference method. ° QC Acceptance Criteria Based Method Comparison Study. For EPA approved reference methods that contain ( or are supplemented with) QC acceptance criteria for most combinations of analyte( s) and determinative technique( s), the goal of the study is for the applicant to demonstrate that its ATP or new method is able to meet the QC acceptance criteria of the EPA approved reference method ( or other EPA specified document) for the applicable combination of analyte and determinative technique through a QC acceptance criteria based comparison study.

Specific guidelines for the studies can be found in Section 5.0: Quality Control, Section 6.0: Study Design, Section 7.0: Sample Preparation and Analysis, and Section 9.0: Review of Study Results.

1.4 Scope of Alternate Test Procedures This protocol for demonstration of comparability, submission, and approval of an ATP or new method offers flexibility to modify EPA approved reference methods. Generally, an applicant should demonstrate and document that the modified method produces results better than or equal to those produced by an appropriate EPA approved reference method for the applicable combination of analyte and determinative technique.

1.4.1 EPA Approved Reference Methods The ATP process is based on comparing the performance of an ATP or new method to an EPA approved reference method through a side by side comparison study or a QC acceptance criteria based comparison study. Method comparability is demonstrated when results produced by an ATP meet or exceed the performance criteria associated with the EPA approved reference method. Table 1 1 below lists the EPAapproved reference methods for the analytes covered by this protocol. This table will be updated as necessary as additional pathogens are added to the list or advances in technology merit a change in the EPA approved reference method. When performing a study, the applicant should use the reference method that uses the same determinative technique ( e. g., MF, MPN) as the ATP for the analyte( s) of interest. If the applicant is validating a new method, which generally will use a determinative technique that is not currently approved for use with the analyte( s) of interest, then the applicant should consult EPA prior to commencing the study to determine which is the most appropriate reference method. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 1 4 Table 1 1. EPA Approved Reference Methods Analyte Method Format1 EPA Approved Reference Method2 40 CFR Citation Total coliforms MPN SM 9221B 136.3, 141.21, 141.74 MF SM 9222B Presence/ Absence SM 9221D 141.21 Fecal coliforms MPN SM 9221E 136.3, 141.21, 141.74, 503.8( b) MF SM 9222D E. coli MPN SM 9221F 136.3, 141.21, 141.74 MF SM 9222G, SM 9213D HPC Pour Plate SM 9215B 141.74 Fecal streptococcus MPN SM 9230B 136.3 MF SM 9230C Enterococcus MPN SM 9230B 136.3 MF SM 9230C Salmonella MPN Kenner and Clark 136.3, 503.8( b)

Enteric virus Plaque Assay EPA Document 3 503.8( b)

Helminth ova Microscopy EPA Document 4 503.8( b)

Aeromonas MF USEPA Method 1605 141.40 Coliphage Plaque Assay USEPA Method 1601 141.403 Two Step Enrichment USEPA Method 1602 Cryptosporidium Filtration/ IMS/ FA USEPA Method 1622/ 1623 136.3, 141.74 Giardia Filtration/ IMS/ FA USEPA Method 1623 1MPN = most probable number, MF = membrane filtration, IMS/ FA = immunomagnetic separation/ fluorescent antibody 2" SM" refers to Standard Methods for the Examination of Water and Wastewater. For the edition( s) approved for a specific method, consult the CFR sections referenced in the column headed " 40 CFR Citation." ( References 10.2, 10.3, and 10.4) 3 See References 10.12 and 10.13 4 See References 10.12 and 10.19 1.4.2 Modifications to Sample Preparation Techniques A sample preparation technique is any technique in the analytical process conducted at the laboratory that precedes the determinative technique ( i. e., the physical and/ or chemical process by which measurement of the identity and concentration of an analyte is made). Sample preparation techniques include the procedures, equipment, reagents, etc., that are used in the preparation and cleanup of a sample for analysis. Laboratories generally may modify sample preparation techniques, provided the modification is not explicitly prohibited in the EPA approved reference method that is being modified and provided the modification can be demonstrated to produce results equal or superior to results produced by an EPAapproved reference method for each combination of analyte and determinative technique. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 2 1 SECTION 2.0 APPLICATION ATP applications should be submitted in triplicate to EPA to facilitate the review process. The application consists of a completed ATP application form ( a sample application form is provided in Appendix D) with any attachments. Electronic submissions are also generally acceptable and often may accelerate the review process.

2.1 Submission Addresses and Approval Authority A summary of ATP submission information and approval authorities is provided in Table 2 1.

Table 2 1. Submission of Alternate Test Procedure Applications Level of Use Applicant Submit Application To1 Approval Authority Limited Use for Wastewater or Ambient Water EPA Regional laboratories EPA Regional Administrator ( Regional ATP Coordinator) 2 EPA Regional Administrator States, commercial laboratories, individual dischargers, or permittees in States that do not have the authority to administer Clean Water Act and Safe Drinking Water Act monitoring programs EPA Regional Administrator ( Regional ATP Coordinator) 2 States, commercial laboratories, individual dischargers, or permittees in States that have the authority to administer Clean Water Act and Safe Drinking Water Act monitoring programs Director of State Agency issuing the NPDES permit2 Nationwide Use for Drinking Water, Wastewater, Ambient Water All applicants Director, Analytical Methods, Attn: ATP Program Coordinator, EPA Headquarters EPA Administrator 1 See Appendix E for EPA addresses. 2 The Regional Administrator or the Director of State Agency issuing the NPDES permit may choose to forward limited use applications to the Director of Analytical Methods, Attn: ATP Program Coordinator for an approval recommendation. Generally, the Regional Administrator or the Director of State Agency issuing the NPDES permit will forward a copy of the approval to the Director of Analytical Methods, Attn: ATP Program Coordinator.

Generally, upon receipt, the application will be assigned an identification number, and a confirmation letter referencing this identification number will be sent to the applicant. The applicant should use the identification number in all future communications concerning the application.

2.2 Application Information The following information should be provided on the ATP application form ( Appendix B): ° Name, mailing address, phone number, and email address of the applicant ° Date of submission of the application ° Method number, title, and revision date of the ATP or new method submitted for review ° The analyte( s) included in the ATP or new method submitted for review EPA Microbiological Alternate Test Procedure ( ATP) Protocol 2 2 ° The matrix or matrices to which the ATP applies ° EPA approved reference method used for demonstration of comparability ° Type of application ( i. e., wastewater, drinking water, ambient water, point source categories regulated at 40 CFR Parts 400 499) ° The level of use desired ( i. e., limited use or nationwide use) ° Type of study ( side by side comparison or QC acceptance criteria based comparison study) ° Applicant's NPDES permit number, the issuing agency, the type of permit and the discharge serial number ( if applicable)

The following items should be submitted as attachments to the initial application: ° Reason for proposing the ATP or new method ° The proposed ATP or new method prepared in standardized format ( Section 3.0) ° A method comparison table that gives a side by side comparison of the steps of the proposed ATP or new method and the EPA approved reference method ( Section 2.5) ° Method development information ° Study plan for EPA review and comment ( Section 4.0)

A study plan is generally not needed with the application if an applicant is unsure whether or not a modification is allowed within the method specified flexibility. In such cases, the applicant may request that EPA determine the usefulness of a full ATP comparability assessment based on the other information submitted with the application. From this information, EPA can determine whether a full ATP assessment will be helpful, whether the proposed modification is considered to be a minor modification ( i. e., employs the same chemistry and/ or biological principles as the EPA approved reference method to determine the presence/ absence or to quantify the amount of the target organism in a sample), or whether the proposed modification is considered to be within the specified flexibility of the EPA approved reference method.

The elements of a complete application are presented in Table 2 2. A list of the information discussed in detail in Sections 2.3 to 2.8 is provided in Appendix C. EPA will generally seek all application information and attachments before the application is considered complete.

Table 2 2. Application Information Application Information ° Completed application form ° Reason for ATP ° Method in EPA format ° Method comparison table ° Method development information ° Study plan ( to be approved by EPA before proceeding with study) ° Study report ( final report generally considered part of a complete application)

Note: Although the application process generally begins with the initial submission of ATP materials, the application is not usually considered to be complete until the final study report has been submitted, and all EPA questions on the report have been resolved.

2.3 Reason for ATP The entity that proposes an ATP should indicate why the ATP is being proposed. Examples include, but are not limited to, the following: ° The ATP improves method performance ° The ATP provides equivalent method components for a lower cost ° The ATP enables laboratories to perform analyses more efficiently EPA Microbiological Alternate Test Procedure ( ATP) Protocol 2 3 ° The ATP successfully overcomes some or all of the interferences associated with the EPA approved reference method ° The ATP significantly reduces the amount of hazardous wastes generated by the laboratory ° The ATP provides another means for measuring a contaminant ( i. e., provides more choices)

2.4 Standard EPA Method Format In accordance with the standard EPA format advocated by EPA's Environmental Monitoring Management Council ( EMMC), methods should contain 17 specific topical sections in a designated order. The 17 sections are listed in Section 3.0 of this document. Additional numbered sections may be inserted starting with Section 11.0, Procedure, as appropriate for a particular method. For detailed information on the EPA format for proposed methods, see the Guidelines and Format document ( Reference 10.14).

2.5 Method Comparison Table As part of the application, the applicant should provide a two column table comparing the proposed ATP or new method to the EPA approved reference method. The two column method comparison table should include the number and title of each method, the latest revision date of the proposed ATP, and a detailed discussion of each of the 17 topics specified by the standard EPA method format ( as applicable). Each topic should be discussed on a separate row in the method comparison table. The applicant should highlight any differences between the proposed ATP and the EPA approved reference method.

2.6 Method Development Information Before EPA reviews the study plan and works with applicants on ATP studies, the applicant should provide data on performance of the modified or new method in the water matrix for which the ATP or new method is being applied. These data may have been generated during method development by the vendor, or through independent tests by third party laboratories. Examples include, but are not limited to, replicate spiked reagent water or replicate spiked matrix water tests. It is the responsibility of the applicant to provide sufficient data to demonstrate that the ATP or new method performs sufficiently at a preliminary level in the matrix of interest to merit evaluation of an ATP. If sufficient data is not available, EPA may request additional studies be conducted prior to the review of the ATP study plan.

In addition to data, the following descriptive method information will facilitate EPA's evaluation of the ATP application: ° The purpose and intended use of the method ° The analytical basis for the method, noting any relationship of the method to other existing analytical methods and indicating whether the method is associated with a sampling method ° Method limitations and an indication of any means of recognizing cases where the method may not be applicable to specific matrix types ( e. g., turbidity greater than 50 NTU) ° The basic steps involved in performing the test and data analysis ° Options to the method, if applicable This information also will aid EPA in preparing the docket and the preamble for the proposed rule that will be published in the Federal Register if EPA proposes to approve the ATP for nationwide use.

2.7 Study Plan Prior to conducting all studies, the applicant should submit a study design for EPA review and comment. A detailed procedure ( Section 3.0) for the new method or the modification should be included as an attachment to the study plan. Generally, EPA will evaluate the study plan to ensure that the appropriate EPA Microbiological Alternate Test Procedure ( ATP) Protocol 2 4 data quality objectives identified in this protocol are defined and addressed. EPA comments will be incorporated into the study design and this process will be repeated until EPA has approved the study design. Generally, the study plan should contain the elements listed below: ° Background ° Objectives ° Study Design ° Coordination ° Data Reporting These elements are further described in Section 4.0.

2.8 Study Report The applicant should conduct a study and provide a comprehensive study report with the ATP or new method application. The study report should include the following elements: ° Background ° Study Objectives and Design ° Study Implementation ° Data Reporting and Validation ° Results ° Data Analysis and Discussion ° Conclusions ° Appendix A  Method ° Appendix B  Study Plan ° Appendix C  Supporting Data ° Appendix D  Supporting References These elements are further described in Section 9.0.

2.9 Proprietary Information in Applications All information provided to the Federal government is subject to the requirements of the Freedom of Information Act. Therefore, any proprietary information submitted with the proposed ATP application should be marked as confidential. However, EPA prefers that supporting documentation labeled as confidential business information not be submitted as part of the ATP application. If proprietary information is determined to be essential to the application, EPA staff will request the information and will handle such information according to the regulations in subparts A and B of 40 CFR Part 2.

Specifically, in accordance with 40 CFR § 2.203, a business that submits information to EPA may assert a business confidentiality claim covering the information by placing on ( or attaching to) the information at the time it is submitted to EPA, a cover sheet, stamped or typed legend, or other suitable form of notice employing language such as trade secret, proprietary, or company confidential. Allegedly confidential portions of otherwise non confidential documents should be clearly identified by the business, and may be submitted separately to facilitate identification and handling by EPA. If the business desires confidential treatment only until a certain date or until the occurrence of a certain event, the notice should so state. Please be advised, however, that any methods proposed in the Federal Register cannot be claimed as confidential business information.

If a claim of business confidentiality is not made at the time of submission, EPA will make such efforts as are administratively practicable to associate a late claim with copies of previously submitted information in EPA files. However, EPA cannot ensure that such efforts will be effective due to the nature of application review that may already be in progress. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 3 1 SECTION 3.0 METHOD FORMAT Because alternate test procedures may be approved by EPA as comparable to the reference methods, and may be implemented by multiple laboratories, it is important that the written procedures include all of the information necessary to use the technique in the laboratory, including but not limited to: reagents and equipment, sample collection and preservation procedures, quality control, and a detailed description of the procedure. The information described below should be provided in the method. In addition, EPA recommends the Environmental Monitoring Methods Council ( EMMC) format described below be used.

Sections 3.1 through 3.17 provide a list of EMMC method sections and a general description of the type of information that should be included in each section. The date and revision number of the method should be included on the cover page. In addition, the date should be included as a footer on each page of the method. A detailed description of method format guidelines, as well as an example of a formatted method, is provided in Reference 10.14. The detailed information in Reference 10.14 is provided as guidance for the method write up and as such, specific suggestions for font size, margins, etc. are optional.

3.1 Scope and Application Include a list of target organisms ( by common name), taxonomic group and their CAS registry numbers or other accepted numbering systems ( if available), the matrices to which the method applies, a generic description of method sensitivity ( the minimum number of organisms the method can detect per unit volume or mass, if known), and the data quality objectives that the method is designed to meet or monitoring programs for which the method was designed to support.

3.2 Summary of Method Summarize the method in a few paragraphs. The purpose of the summary is to provide a succinct overview of the method procedure to aid the reviewer or data user in understanding the method and how the results are generated. Include a general description of the method procedure, sample volume, type of media used, preparation steps, incubation time and temperatures, and the techniques used for qualitative or quantitative determinations.

3.3 Method Definitions Provide definitions of terms that are necessary to understand how the method is used or what the results represent. This should include a definition of the target organism or group of organisms, relative to the determinative step of the method. For extensive lists of definitions, this section may simply refer to a glossary attached at the end of the method document.

3.4 Interferences This section should discuss any known method interferences such as toxic materials, particulates, nontarget organisms, etc. If known interferences in the reference method are not interferences in the alternate method, this also should be clearly stated.

3.5 Safety This section should discuss only those safety issues specific to the method and beyond the scope of routine laboratory practices. Target analytes or reagents that pose specific health, toxicity, or safety issues should be addressed in this section. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 3 2 3.6 Equipment and Supplies For critical equipment that may affect the performance of the method, cite the manufacturer, model name, and catalog or product number of the equipment that was used to develop or validate the method; note that equivalent equipment can be used, if applicable. Use generic language for standard laboratory glassware and disposables.

3.7 Reagents and Standards Provide sufficient details on the concentration and preparation of reagents and standards to allow the work to be duplicated, but avoid lengthy discussions of common procedures. If only pre prepared proprietary reagents can be used, specify this. Include catalog and/ or product numbers where appropriate. Indicate shelf life of packaged materials and special storage specifications.

3.8 Sample Collection, Preservation, and Storage Provide information on sample collection, preservation, shipment, storage conditions, and holding times. If effects of holding time were specifically evaluated, provide reference to relevant data.

3.9 Quality Control Describe specific quality control ( QC) measures that enable one to establish the sensitivity, specificity, false positive rates, false negative rates, bias, and precision of measurements using the method, and that the measurements are free from contamination. Specific QC measures may include positive and negative controls, duplicate samples, method blanks, and media sterility checks. Indicate which QC measures are appropriate initially, before a laboratory uses the method, and which are appropriate on an ongoing basis. Indicate frequencies for each QC measure and list minimum specifications or acceptance ranges ( see Section 5.0). Indicate corrective actions that should be taken when QC measures are not met. Define all terms in method definitions section.

3.10 Calibration and Standardization Discuss initial calibration specifications for instruments used in the method ( e. g., water baths, refrigerators, thermometers, balances, pH meters, microscopes, etc.). Indicate frequency of such calibrations; refer to performance specifications; and indicate corrective actions that should be taken when performance specifications are not met. This section may also include procedures for calibration, verification, or continuing calibration, or these steps may be included in the procedure section.

3.11 Procedure Provide a detailed description of the sample processing and analysis steps. Avoid unnecessarily restrictive instructions, but provide sufficient detail for manual procedures so that analysts in other laboratories perform the method consistently. Ranges should be provided for temperature requirements, time requirements, etc.

3.12 Data Analysis and Calculations Identify qualitative and quantitative aspects of the method. List criteria for the identification of target organism( s) and interpretation of results for all steps of the method, including criteria for presumptive and confirmed results. Provide equations used to derive final sample results. Provide discussion of estimating detection limits, recoveries, specificity, false positive/ false negative rates, etc., if appropriate. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 3 3 3.13 Method Performance Provide detailed information on method performance, including data on precision, bias ( for quantitative methods), specificity, detection limits ( including the method by which they were determined and matrices to which they apply), statistical procedures used to develop performance specifications ( i. e., recovery, precision, specificity, false positive/ false negative rates, etc.). Where performance is tested relative to the reference method, provide a summary of the side by side comparison of performance versus reference method specifications.

3.14 Pollution Prevention Describe aspects of this method that minimize or prevent pollution that may be attributable to the reference method.

3.15 Waste Management Cite how waste is minimized and the proper disposal of samples and waste.

3.16 References Include source documents, publications, etc.

3.17 Tables, Diagrams, Flowcharts, and Validation Data Additional information may be presented at the end of the method. Lengthy tables may be included here and referred to elsewhere in the text by number. Diagrams should only include new or unusual equipment or aspects of the method. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 4 1 SECTION 4.0 STUDY PLAN Applicants should submit a study design for EPA review, comment, and approval prior to conducting the side by side comparison study or the QC acceptance criteria based comparison study. This process protects the applicant by providing written approval of the study design before resources are spent to conduct the study. The ATP program is intended to be flexible, and thus EPA may modify the study design for a particular proposed method. Data from studies conducted without EPA review and approval may not meet EPA's criteria, and may not adequately address the applicant's study objectives. A detailed procedure ( Section 3.0) for the ATP or new method should be included as an attachment to the study plan. EPA will generally evaluate the study plan to verify that the appropriate data quality objectives identified in this protocol are defined and addressed. EPA comments are incorporated into the study design. This review/ revision process is repeated until EPA has approved the study design.

Generally, the study design should include the information described in Sections 4.1 through 4.5.

4.1 Background This section of the study plan should include the following information: ° A statement identifying the ATP as a new method or a modification of a reference method ° The EPA program( s) to which the ATP or new method applies ( e. g., drinking water, wastewater, ambient water, point source categories regulated at 40 CFR Parts 400 499, etc.) ° A short ( one paragraph) summary of the ATP or new method ° The organization and method number of the reference method if applicable ° A description of the reasons for the extent of the modification, the logic behind the technical approach to the modification, and the result of the modification ° The matrices ( e. g., finished water, wastewater, ambient water, etc.), matrix types ( e. g., turbidity greater than 10 NTU, etc.), and/ or media to which the ATP or new method is believed to be applicable ° A list of the analytes measured by the ATP or new method, including the corresponding CAS registry number ( if available) or other identification number 4.2 Objectives Include a description of the new method or modification, describe the goals of the study, and define data quality objectives.

4.3 Study Design The following information should be included in the study design: ° Laboratories that will participate in the study ( Sections 5.0 and 6.0) ° Number and type of samples to be analyzed ( Section 6.0) ° Description of the matrices that will be used ( Sections 6.0 and 7.0) ° Description of the spikes that will be used ( Section 7.0) ° Description of the spiking procedure ( Section 7.0) ° Positive and negative control organisms ( Section 5.0) ° Quality control procedures that will be followed ( Section 5.0) EPA Microbiological Alternate Test Procedure ( ATP) Protocol 4 2 4.4 Coordination Describe how the study will be coordinated, how the spikes will be shipped to the laboratories, and who will compile the data for submission. Data compilation should not be performed by any of the analysts conducting the sample analyses.

4.4.1 Study Management This section of the study plan should include the following information: ° The organization responsible for managing the study ° The laboratories, facilities, and other organizations that will participate in the study ° A delineated study schedule including, but not limited to, sample collection, start of sample analysis, interpretation of sample results, completion of study, etc.

4.4.2 Technical Approach This section of the study plan should include the following: ° A description of how sample matrices and participating laboratories will be selected ° A description of how samples will be collected and distributed ° The numbers and types of analyses to be performed by the participating laboratories ° A description of sample spiking procedures ° A description of how analyses are to be performed 4.5 Data Reporting List the data elements that will be collected and provide sample bench sheets ( see Appendix D) that will be used to record raw data during the study. Raw data should be submitted as an attachment to the Study Report ( Section 9.0). Address the statistical analysis of the study results that will be performed, if the statistical analyses will differ from those described in Section 8.0. Please note, however, that EPA's evaluation of method performance will generally be based on the statistical analyses described in Section 8.0. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 5 1 SECTION 5.0 QUALITY ASSURANCE/ QUALITY CONTROL For side by side comparison studies in which only one laboratory is performing analyses, the comparability study should be conducted at an independent laboratory and the laboratory should be certified to perform microbiological analyses under EPA's drinking water laboratory certification program. A laboratory with a vested interest in the method, instrumentation, apparatus, reagents, media, or associated kits may not participate in the side by side comparison study.

For QC acceptance criteria based comparison studies, at least three independent laboratories should participate. A laboratory with a vested interest in the ATP also may participate in the study, but in such instances there should be at least three independent laboratories participate and the majority of the laboratories participating are independent laboratories. All laboratories should be certified to perform microbiological analyses under EPA's drinking water certification program, or a comparable certification program, if a laboratory located outside of the U. S. is included. At least three independent laboratories participating in the study should to be certified for microbiological analyses under EPA's drinking water certification program. If more than three laboratories participate, the majority of the laboratories should be certified for microbiological analyses under EPA's drinking water certification program.

5.1 Quality Assurance The laboratory should have a comprehensive quality assurance ( QA) program in place and operating at all times during the performance of the comparability study. General QA program guidance is provided at http:// www. epa. gov/ quality/ qs docs/ r2 final. pdf ( Reference 10.11). The laboratory should adhere closely to all QA and quality control ( QC) measures in this protocol as well as the QC measures in the method( s). Laboratory QA/ QC criteria for facilities, personnel, and laboratory equipment are included in Standard Methods 9020 Quality Assurance ( Reference 10.4) and the U. S. EPA Manual for the Certification of Laboratories Analyzing Drinking Water, Fourth Edition ( March 1997) ( Reference 10.13).

The laboratory should adhere to standard laboratory practices for cleanliness and environment, and to the methods for glassware and apparatus, reagents, solvents, and safety. Additional guidelines regarding general laboratory procedures should generally be followed, as specified in Sections 4 and 5 of the Handbook for Analytical Quality Control in Water and Wastewater Laboratories, EPA 600/ 4 79 019 ( Reference 10.19).

5.2 Quality Control Laboratories participating in a comparability study should perform all QC procedures specified in the methods, except where explicitly stated in the approved study plan. The QC procedures listed in Table 5 1 and described below should be performed, as appropriate, based on the technique used in the method. Other QC procedures may be necessary, based on the approved study plan. Laboratories participating in side by side comparison studies or QC acceptance criteria based comparison studies should perform all QC procedures specified in the methods, except where explicitly stated in the approved study plan. The laboratory should maintain records to define the quality of data that are generated. The laboratory should maintain a record of the date and results of all QC sample analyses. Laboratories should maintain reagent and material lot numbers along with samples analyzed using each of the lots. Laboratories should also maintain media preparation records.

Table 5 1 lists quality control measures for each laboratory participating in side by side comparison studies and QC acceptance criteria based comparison studies. Detailed descriptions of the QC measures are provided in Sections 5.2.1 to 5.2.15. If contamination is detected in any of the blanks or sterility checks described below, the source of contamination should be identified and corrected. The EPA Microbiological Alternate Test Procedure ( ATP) Protocol 5 2 blank/ sterility check and all samples associated with that contaminated blank/ sterility check should be reprepared and reanalyzed. Measures taken to eliminate contamination should be reported.

Table 5 1. Quality control measures for each laboratory involved in the study Quality Control Measure Frequency Suggested for Study Side by Side Comparison Study QC acceptance criteria based Comparison Study Analyst counting variability ( Section 5.2.1) 2 samples per study T T Autoclave sterilization verification ( Section 5.2.2) Within one week prior to the start of the study T T Dilution/ rinse water blanks ( Section 5.2.3) 1 per every 20 samples or 1 per day of study, whichever is greater T T Incubator/ waterbath temperatures ( Section 5.2.4) 2 times per day when used in study T T Initial demonstration of capability ( IDC) ( Section 5.2.5) or Initial precision and recovery ( IPR) ( Section 5.2.6) Each laboratory participating in the study should generate acceptable IDC/ IPR data. T Matrix spike/ matrix spike duplicate ( Section 5.2.7) Per approved study plan T Media sterility checks ( Section 5.2.8) 1 per each batch of media used in the study or per each test run, whichever is greater T T Method blank ( Section 5.2.9) Per approved study plan T T Ongoing demonstration of capability ( ODC) ( Section 5.2.10) or Ongoing precision and recovery ( OPR) ( Section 5.2.11) Included as part of the study; not needed as separate QC. Optional Positive and negative controls ( Section 5.2.12) 1 positive control and 1 negative control for each media/ stain used in the study. See 5.2.12 below for more details. T T Preparation blanks ( Section 5.2.13) Frequency depends on the type of method. See 5.2.13 below for more details. T T Refrigerator/ freezer temperatures ( Section 5.2.14) Once per day when used in study T T Sample processing equipment sterility checks ( Section 5.2.15) Prior to the analysis of samples. See 5.2.15 below for more details. T T EPA Microbiological Alternate Test Procedure ( ATP) Protocol 5 3 5.2.1 Analyst Counting Variability If the laboratory has two or more analysts, each should count colonies, plaques, or positive wells on the same plate/ tray from one positive field sample per month. Compare each analyst's count of the colonies, plaques, or wells. Counts should fall within 10% between analysts. If counts fail to fall within 10% of each other, analysts should perform additional sets of counts, until the number of target colonies, plaques, or positive wells counted fall within 10% between analysts for at least three consecutive samples. If there is only one analyst replicate counts should be done and be within 5% of original counts.

5.2.2 Autoclave Sterilization Verification Autoclave sterilization verification should be performed within one week of the start of the study by placing Bacillus stearothermophilus spore suspensions or strips inside glassware. Autoclave at 121 ° C for 15 minutes. Place Bacillus stearothermophilus spore suspensions in trypticase soy broth tubes and incubate at 55 ° C for 48 hours. Check for growth to verify that sterilization was adequate. If sterilization was inadequate, determine appropriate time for autoclave sterilization. Repeat spore test. The laboratory should have historical data verifying that at a minimum, autoclave sterilization is performed on a monthly basis.

5.2.3 Dilution/ Rinse Water Blanks The laboratory should analyze dilution/ rinse water blanks to demonstrate freedom from contamination. An aliquot of dilution/ rinse water which is analyzed exactly like a field sample should be analyzed for every day of the study or for every 20 samples, whichever is more frequent, and observed for contamination with agent of interest.

5.2.4 Incubator/ Waterbath Temperatures Incubator or waterbath temperatures should be measured and recorded two times per day when in use. Temperatures should be taken at least 4 hours apart and should be within the range of the desired temperature as specified in each method. Thermometers used to measure " in use" temperatures should be calibrated yearly against an NIST traceable thermometer.

5.2.5 Initial Demonstration of Capability Laboratories participating in a QC acceptance criteria based comparison study should have successfully performed an initial demonstration of capability ( IDC) test for the new or modified method under evaluation in the study. An IDC test is performed when QC acceptance criteria are available for the evaluation of precision or recovery, but not both. The laboratory should perform an IDC as specified in the method to demonstrate acceptable performance. The laboratory should complete any additional analyses as specified in the study plan.

For the IDC test, the laboratory spikes and analyzes reference matrix ( e. g., reagent water, buffered water, etc.) samples to demonstrate acceptable performance with the method prior to the analysis of field samples. The number of samples involved in the IDC varies by method. If the results of the IDC test meets all IDC acceptance criteria ( e. g., RSD, minimum number of samples positive, etc.), system performance will generally be acceptable. If any of the IDC test results fail to meet the acceptance criteria, system performance will generally be unacceptable. In this event, the laboratory should identify and correct the problem and repeat the test. IDC tests should be accompanied by a method blank ( Section 5.2.9). EPA Microbiological Alternate Test Procedure ( ATP) Protocol 5 4 5.2.6 Initial Precision and Recovery Laboratories participating in a QC acceptance criteria based comparison study should have successfully performed an initial precision and recovery ( IPR) test for the method using the modified version of the method under evaluation in the study. An IPR test is performed when QC acceptance criteria are available for the evaluation of both precision and recovery. The laboratory should perform an IPR as specified in the method to demonstrate acceptable performance. The laboratory should complete any additional analyses as specified in the study plan.

For the IPR test, the laboratory spikes and analyzes reference matrix ( e. g., reagent water, buffered water, etc.) samples to establish the laboratory's ability to generate acceptable precision and recovery prior to the analysis of field samples. Using results of the analyses, the laboratory calculates mean percent recovery and relative standard deviation ( RSD) of the recoveries for the analyte( s) and compares them with the corresponding limits for the IPR test criteria in the method. If the RSD and the mean percent recovery meet the acceptance criteria, system performance is acceptable and analysis of samples may begin. If the RSD or the mean percent recovery are unacceptable, system performance will generally be unacceptable. In this event, the laboratory should identify and correct the problem and repeat the test. IPR tests should be accompanied by method blank tests ( Section 5.2.9).

5.2.7 Matrix Spike and Matrix Spike Duplicate Samples Matrix spike ( MS) and matrix spike duplicate ( MSD) samples are spiked matrix water samples analyzed by the laboratory to verify acceptable method performance in the matrix being monitored. During routine performance of the method, MS/ MSD samples are analyzed by the laboratory for the first sample of any new matrix that will be monitored, and on the 21st sample thereafter. During a QC acceptance criteriabased comparison study for some methods ( e. g., Cryptosporidium and Giardia methods), the laboratory should analyze MS samples as part of routine laboratory QC, however these analyses may not be necessary during this study because MS/ MSD samples using blinded spiking suspensions distributed by the study coordinator will be used for the study.

5.2.8 Media Sterility Checks Before using newly prepared media, a representative portion of each media batch needs to be checked for sterility. The laboratory should test media sterility by incubating one unit ( tube or plate) from each batch of medium specified in the method or per each test run, whichever is more frequent, at the appropriate temperature for the length of the method specified incubation time and observing for growth.

5.2.9 Method Blank Method blanks are reagent water blanks or other blanks including but not limited to buffered water, tap water, etc., depending on the method analyzed to demonstrate freedom from contamination. Method blanks should be analyzed at the frequency specified in the approved study plan. For QC acceptance criteria based comparison studies, the laboratory needs to analyze a method blank with the IPR and IDC tests. During routine performance of the method, the laboratory should analyze at least one method blank per every 20 test samples or per every week. During a QC acceptance criteria based comparison study for some methods ( e. g., Cryptosporidium and Giardia methods), the laboratory should analyze method blanks as part of routine laboratory QC, but method blanks also should be shipped to the laboratory as double blind samples by the study coordinator. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 5 5 5.2.10 Ongoing Demonstration of Capability ( ODC) Samples Ongoing demonstration of capability samples are spiked reference matrix ( e. g., reagent water, buffered water, etc.) samples that are analyzed to demonstrate that the analytical system is in control on an ongoing basis.

5.2.11 Ongoing Precision and Recovery ( OPR) Samples Ongoing precision and recovery samples are spiked reference matrix ( e. g., reagent water, buffered water, etc.) samples that are analyzed by the laboratory to verify that method performance criteria are being met. During a performance based comparison study for some methods ( e. g., Cryptosporidium and Giardia methods), the laboratory should analyze OPR samples as part of routine laboratory QC because generally OPR samples using blinded spiking suspensions distributed by the spiking coordinator will be used for the study.

5.2.12 Positive/ Negative Controls Positive and negative controls are target and non target organisms processed to ensure the laboratories are familiar with the identification of the target organism and to ensure that confirmation test results are appropriate.

<IP_ADDRESS> Positive/ Negative Culture Controls ( Culture Based Methods)

Positive and negative culture controls refer to cultures that, when analyzed exactly like field samples, produce a known positive or a known negative result, respectively, for a given type of media. One positive culture control and one negative culture control should be prepared and analyzed for every media ( including confirmation media) used in the method whenever a new batch of medium or reagents is used, every day of the study, or every 20 samples, or as specified in the method, whichever is more frequent. Each control should be carried through the entire procedure and should exhibit the expected positive or negative result.

<IP_ADDRESS> Positive/ Negative Staining Controls ( Cryptosporidium and Giardia Methods)

A positive staining control for Cryptosporidium and Giardia methods is a slide containing positive antigen or intact Cryptosporidium oocysts and Giardia cysts, and that is stained using the same procedure as used for field samples or test samples. A negative staining control is a slide containing only phosphate buffered saline ( PBS) that is stained using the same procedure as used for field samples or test samples. The laboratory should prepare and examine positive and negative staining controls with each batch of slides the laboratory prepares during the study. Positive staining controls should exhibit acceptable fluorescence and negative staining controls should not exhibit fluorescence.

<IP_ADDRESS> Positive/ Negative Staining Controls ( Other)

At a minimum, the laboratory should prepare and examine a positive and negative control using the same procedure as used for field or test samples whenever a new batch of media or reagents is used every day of the study or every 20 samples, whichever is more frequent. Each control should be carried through the entire procedure and should exhibit the expected positive or negative result. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 5 6 5.2.13 Preparation Blanks ( PB)

<IP_ADDRESS> Membrane Filter Preparation Blank ( PB MF)

If membrane filtration is used, at the beginning and the end of each filtration series, a PB MF is performed by filtering 20 30 mL of dilution water through the membrane filter and testing for growth. If the control indicates contamination with the target organism, all data from affected samples should be rejected. A filtration series ends when 30 minutes or more elapse between sample filtrations.

<IP_ADDRESS> Multiple Tube Fermentation Test Preparation Blank ( PB MTF)

If a multiple tube fermentation test is used, a volume of sterile buffered water that is analyzed exactly like a field sample should be analyzed for every day of the study or every 20 samples, whichever is more frequent. The preparation blank should be incubated with the sample batch and observed for growth of the target organism. If the control indicates contamination with the target organism, all data from affected samples should be rejected. If buffered water is not used for dilutions, only the multiple tube fermentation media should be included.

<IP_ADDRESS> Other Preparation Blank ( PB Other)

A volume of sterilized water ( e. g., reagent grade as defined in Specification D 1193, Annual Book of ASTM Standards) that is analyzed exactly like a field sample should be analyzed for every day of the study or every 20 samples, whichever is more frequent. The preparation blank should be incubated with the sample batch and observed for growth of the target organism. If the control indicates contamination with the target organism, all data from affected samples should be rejected.

5.2.14 Refrigerator/ Freezer Temperatures Refrigerator and freezer temperatures should be measured and recorded once per day when in use. Refrigerator temperature should be maintained at 1 ° C to 4 ° C. Freezer temperatures should be maintained at  15 ° C to  20 ° C. Special freezers capable of long term storage of cultures or virus should be maintained at  70 ° C to  80 ° C. Thermometers used to measure " in use" temperatures should be calibrated yearly against an NIST traceable thermometer.

5.2.15 Sample Processing Equipment Sterility Checks A representative portion of non disposable items such as sample containers, blender jars, etc., used to collect or process samples should be checked for sterility prior to use in analyses. To test for sterility add approximately 500 mL ( or appropriate volume based on the size of the equipment being used) of a sterile non selective broth ( e. g., tryptic soy, trypticase soy, or tryptone broth) to the non disposable item and incubate at 35 ° C ± 0.5 ° C for 24 hours and check for growth. Depending on the incubation times specified in the method, the length of the incubation time for the sample processing equipment sterility check may be increased. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 6 1 SECTION 6.0 STUDY DESIGN This section provides a description of the study design that is important for assessing comparability using either side by side method comparison studies ( Section 6.1) or QC acceptance criteria based studies ( Section 6.2). The sections below address the number of laboratories, number of matrices, and the number of samples that should be analyzed in the studies.

6.1 Side by Side Comparison Studies Methods are compared using the following parameters: ° Recovery. Does the new method have similar, better or worse recoveries of the target organism as the reference method? ° Precision. Are the recoveries by the new method significantly less or more variable than the reference method? ° False positive rate/ specificity. Is the new method significantly more likely or less likely to detect non target organisms or other sample constituents that would be reported as the target organism by the analyst when compared to the reference method? ° False negative rate/ sensitivity. Is the new method significantly more likely or less likely to exhibit non detects for samples with the target organism or to exhibit results that are biased low when compared to the reference method?

To generate these parameters, samples are analyzed by a single laboratory ( 6.1.1). The number of samples ( and matrix types) used in the study are determined using a historical EPA standard ( <IP_ADDRESS>)

6.1.1 Number of Laboratories A single laboratory should be used for a side by side comparability study. Since the study should be conducted in a single independent laboratory with no conflict of interest; the laboratory selected cannot be the method developer's laboratory and cannot be affiliated with the method developer.

6.1.2 Number of Samples The following standards generally provide the minimum number of samples that should be analyzed. Additional data are generally acceptable and may be very helpful when reviewing an ATP or new method.

<IP_ADDRESS> Method Comparison Study Design Summary Table 6.1 provides a summary of the method comparability requirements for nationwide microbiological ATPs and new methods. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 6 2 EPA Microbiological Alternate Test Procedure ( ATP) Protocol 6 3 Number of Matrices Ten different water/ wastewater matrices from geographically diverse locations should be included to obtain, as much as is practical, a good presentation of the wide range of water types with an even wider range of target organisms to which the method should appropriately respond. Each water or wastewater sample should be collected in sufficient volume to complete all replicate analyses of sample or dilution volumes by both the ATP or new method and the EPA approved reference method. For ambient water studies, the turbidity of at least one matrix should be greater than 10 NTU. Generally, for matrices other than finished drinking water or ambient water, matrix composition will be addressed on a study specific basis.

Number of Samples and Replicates Twenty replicate analyses should be performed by each method for each of the 10 matrices for a total of 200 replicate analyses per method. The replicate analyses should be performed on the same day for both the proposed and reference methods.

Verification of Results For quantitative reference methods, 10 typical colonies must be verified from 4 randomly chosen replicates ( of the 20 replicates) of each of the 10 samples for a total of 400 colony verifications. For presence/ absence tests, all positive samples should be verified for up to 200 samples. See Section 6.1.3 for additional information pertaining to the verification of results. Calculations for false positive rates and false negative rates are described in Section 8.4.3.

6.1.3 Verification of Results <IP_ADDRESS> False Positives To assess whether the false positive rates are significantly different between methods, replicates known to contain non target organisms that could be falsely identified as the target organism should be analyzed by both the ATP or new method. The determination that the samples do not contain the target organism should be based on a third independent standard method ( see Section 7.6) rather than by the EPAapproved reference method being used in the comparison. This is because it should not be assumed that the accepted method has a false positive or negative rate of zero. In side by side comparison studies using the EPA standard for determining the number of matrices and the number of samples, at least 200 positive results should be verified for the ATP or new method in order to adequately compare false positive rates.

<IP_ADDRESS> False Negatives To assess whether the false negative rates are significantly different between methods, replicates known to contain target organisms should be analyzed by both the ATP or new method. The determination that the samples do not contain the target organism should be based on a third independent standard method ( see Section 7.6) rather than by the EPA approved reference method being used in the comparison. This is because it should not be assumed that the accepted method has a false positive or negative rate of zero. In side by side comparison studies using the EPA standard for determining the number of matrices and the number of samples, at least 200 negative results should be verified for the ATP or new method in order to adequately compare false negative rates. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 6 4 6.2 QC Acceptance Criteria Based Comparison Studies QC acceptance criteria based comparison studies are generally conducted to demonstrate that the ATP or new method is able to meet the QC acceptance criteria of the EPA approved reference method. In some instances, the quality control ( QC) acceptance criteria specified in a method may not be sufficient to demonstrate comparability between the ATP and the EPA approved reference method and a side by side comparison study should be conducted. Generally, EPA will make this decision based on review of the application materials.

6.2.1 Number of laboratories A minimum of three laboratories should be used for a QC acceptance criteria based comparison study. All three laboratories should meet all QC acceptance criteria in the EPA approved reference method. If more than three laboratories participate, at least 75% of the participating laboratories should meet all QC acceptance criteria in the EPA approved reference method.

6.2.2 Number of Matrices For all QC acceptance criteria based comparison studies, there should be one matrix per laboratory. For ambient water studies, the turbidity of at least one matrix should be greater than 10 NTU. Generally, for matrices other than finished drinking water or ambient water, matrix composition should be addressed on a study specific basis.

6.2.3 Number of Replicates per Matrix The number of replicates per matrix and laboratory should be specific to the QC criteria to which the results will be compared. Generally, the number of reagent results should be 4 and the number of source water results per matrix should be 2. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 7 1 SECTION 7.0 SAMPLE PREPARATION AND ANALYSIS 7.1 Collection of Samples for Analysis Each sample should be collected in sufficient volume to complete all replicate analyses by both the ATP or new method and the EPA approved reference method. Samples should be spiked ( if necessary) and analyzed as soon as possible after collection.

7.1.1 Source Water Characterization Source water characterization information should be collected at the time of sample collection. This information will be useful in characterizing the matrix to identify potential interferences and generally includes, but is not limited to, the following: ° Sample collection location ° Source of water ( e. g., ground water, stream, river, lake, etc.) ° Plant treatment processes ( if sample is collected from a water treatment facility) ° Temperature ° pH ° Turbidity ° Total organic carbon ( TOC) ° Free and total disinfectant residual at time of sample collection ° Heterotrophic plate count ( HPC) unless heterotrophic bacteria are the target analytes, in which case total coliforms should be measured In addition to this information, data on the concentration of potential interferences ( e. g., competing bacteria, interfering chemicals, etc.) in the water collected for analysis may be necessary. Known interferences should be discussed in the method and addressed in the study plan. The final results, as well as bench sheets, log sheets, instrument printouts, and any associated quality control analyses should be submitted as part of Appendix C of the study report ( Section 9.10).

7.2 Sample Spiking and " Stressing" Procedures for Bacteriological Methods Depending on the matrix and the analyte of interest, it may not always be necessary to spike samples prior to analysis. Rangefinding analyses should be performed to assess the ambient concentration of target organism( s) in the matrix of interest to determine whether sample spiking will be useful. Samples are collected and dilutions are prepared in order to enumerate the number of organisms in the sample without qualifiers ( i. e. less than, greater than, or too numerous to count). In addition, rangefinding can also be used to obtain environmental isolates for use in sample spiking or to determine the concentration of target organism( s) in a spiking suspension or spiked sample.

If samples are spiked, environmental isolates should be used, as pure strains may exhibit different recovery and precision characteristics than natural flora. NELAC ( http:// www. epa. gov/ ttn/ nelac/ standard/ 5qs 15 0 jhrev corr. pdf, Reference 10.9) and ATCC ( http:// www. atcc. org/ SearchCatalogs/ faqBacteriology. cfm# Q15, Reference 10.6) recommend that bacterial cultures be transferred monthly and passed no more than five times before returning to the original culture.

Sections 7.2.1 through 7.2.4 below, detail several different procedures for sample spiking and stressing. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 7 2 7.2.1 Drinking Water: Spiking and Chlorine Stressing Drinking water samples should be spiked and stressed with chlorine as described in Sections <IP_ADDRESS> and <IP_ADDRESS> below.

<IP_ADDRESS> Drinking Water Spiking Because finished drinking water does not typically contain the target analyte( s) of interest, finished drinking water should be spiked. For the evaluation of drinking water samples, a single finished drinking water matrix is spiked with target organism( s) from other matrices, such as non chlorinated secondary sewage effluents or polluted surface waters. Sewage effluent generally has the advantage of providing a wide range of strains, whereas surface waters typically have the advantage of providing organisms more variable in quality. Depending on the study design ( Section 6.0), the source of spiking suspensions ( i. e., wastewaters) should come from geographically dispersed sites.

To prepare spiked drinking water samples:

( 1) Collect at least five liters of each non chlorinated secondary sewage effluents or polluted surface water to be used as spiking suspensions.

( 2) Perform rangefinding analyses as described above on each spiking suspension to determine target organism density as soon as possible after receipt of the sample to ensure that target organism density and diversity are not reduced.

( 3) For each spiking suspension, spike a sufficient volume of drinking water with a sufficient volume of spiking suspension ( based on rangefinding) to obtain 103 105 target organisms/ 100 mL. Please note: A single drinking water, with negligible concentrations of oxidants and reductants ( e. g., chlorine and sodium thiosulfate, respectively) should be used for the entire study. Refrigerate all spiked drinking waters at 1 ° C  4 ° C for use in preliminary chlorination study and comparability study. High oxidant or reductant levels in the drinking water could interfere with organism stressing.

<IP_ADDRESS> Preliminary Chlorination Study to Determine Appropriate Exposure Time Microorganisms in the spiked drinking water samples ( from Section <IP_ADDRESS>) should be stressed by chlorination at ambient temperatures under conditions similar to those in drinking water treatment facilities. The goal of chlorinating the spiked samples is to simulate drinking water treatment by reducing the number of organisms in the spiked drinking water samples from 103 105 target organisms/ 100 mL to 1 10 chlorine stressed target organisms/ 100 mL for most probable number methods or to 20  100 chlorine stressed target organisms/ 100 mL for membrane filtration methods ( i. e., a 2 4 log removal).

After chlorination, no two samples are expected to produce the same levels of injured ( stressed) target organisms because the disinfection process is impacted by physical and biological factors. These include: the type of spiked drinking water sample to be disinfected ( e. g., spiked with sewage effluent or polluted source water), the initial concentration of the target organism( s), the chlorine demand of the spiked sample, the type and concentration of chlorinating agent, the exposure time, the sample mixing , pH, and temperature. As a result, a preliminary chlorination study should be performed to establish the exposure time necessary to reduce the number organisms in the spiked drinking water samples from 103 105 target organisms/ 100 mL to 1 10 chlorine stressed target organisms/ 100mL ( or a 2 4 log removal). During this preliminary chlorination exposure time study, the physical and biological parameters in Section 7.1.1 should be carefully monitored and recorded for each sample. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 7 3 The exposure time is directly dependent upon the initial concentration of the target organism present, the matrices' chlorine demand, and the form of chlorine present. For testing, the spiked sample is generally exposed to 2.0 2.5 mg total chlorine/ L for over a range of times such as 10, 20, and 30 minutes to reduce the density of target organisms from 103 105 CFU/ 100mL to 1 10 CFU/ 100 mL sample. However, the period of exposure of a sample with a low chlorine demand may be significantly shorter than 20 30 minutes.

Suggested Preliminary Chlorination Study ( 1) Determine and record the total residual and free residual chlorine concentrations using an EPAapproved N, N diethyl p phenylenediamine ( DPD) colorimetric method ( e. g., Standard Method 4500 Cl G) initially, at midpoint, and at the end of the exposure time just prior to dechlorination.

( 2) For each exposure time, place 2 L of each spiked drinking water sample ( from Section <IP_ADDRESS>) in a glass container.

( 3) Add an appropriate volume of a diluted solution of reagent grade sodium hypochlorite ( e. g., a 1: 20 dilution of 5% ( w/ v) stock solution), to achieve the desired level of chlorinating agent and stir the sample continuously during exposure to chlorination. If the spiked sample has an appreciable chlorine demand ( e. g., spiked with a primary effluent or a sewage sample), add dilute sodium hypochlorite solution until a total residual chlorine level between 2.0 and 2.5 mg/ L is maintained in the absence of free chlorine. If a sample has a low chlorine demand, avoid overstressing or killing the organisms by prolonged exposure to free residual chlorine. The free residual chlorine concentration should not exceed 0.5 1.0 mg/ L.

( 4) Stop the chlorine oxidation ( dechlorinate) at the end of the exposure period by adding 0.8 mL of a 10% ( w/ v) sodium thiosulfate solution/ L sample.

( 5) Enumerate the target organism density in an aliquot of the spiked, chlorine stressed, dechlorinated drinking water sample using the appropriate EPA approved reference method from Table 1 1.

( 6) For each exposure time, repeat Steps 3  5, above.

<IP_ADDRESS> Suggested Procedure for Chlorination and Dilution of Samples for Comparability Study ( 1) Determine and record the total residual and free residual chlorine concentrations initially, at midpoint, and at the end of the exposure time just prior to dechlorination using an EPAapproved N, N diethyl p phenylenediamine ( DPD) colorimetric method ( e. g., Standard Method 4500 Cl G).

( 2) Place a sufficient volume of each spiked drinking water in a glass container to perform sufficient repeat analyses at multiple dilutions.

( 3) Immediately prior to chlorination, perform enumeration to determine target organism density as described in <IP_ADDRESS>( 2). This value should be used to determine the log reduction due to chlorination. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 7 4 ( 4) Add reagent grade sodium hypochlorite to achieve the same concentration, as in the preliminary chlorination exposure study.

( 5) To reduce the density of target organisms from 103 105 CFU/ 100mL to 1 10 CFU/ 100 mL, chlorinate each spiked drinking water sample for the appropriate time, based upon the preliminary chlorination exposure study. Stir the sample continuously during the chlorination.

( 6) Stop the chlorine oxidation ( dechlorinate) at the end of the exposure period by adding 0.8 mL of a 10% ( w/ v) sodium thiosulfate solution/ L sample.

( 7) Enumerate target organism density in an aliquot of the spiked, chlorine stressed, dechlorinated drinking water sample using the appropriate EPA approved reference method from Table 1 1.

( 8) Refrigerate the spiked, chlorine stressed, dechlorinated drinking water samples at 1 ° C  4 ° C for use in comparability testing.

( 9) Read the plates to determine the approximate density of the target organisms. Use these results to estimate the appropriate dilution necessary to reach the target organism density of 1 10 CFU/ 100 mL.

( 10) Evaluate three dilutions of each spiked drinking water sample. Samples should be diluted with the same, original, oxidant free and reductant free drinking water, as necessary to reach a target organism density of 1 10 target organisms/ 100 mL for most probable number methods or 20 100 target organisms/ 100 mL for membrane filtration methods. Make the dilution and at least two others that bracket the target density, for example, half and double that dilution. One of these dilutions should contain the desired 1 10 target organisms per 100 mL. Immediately conduct the comparability analyses with each sample using these three dilutions.

Please note: For the evaluation of presence/ absence methods, the data used for comparison should be from the dilution( s) which produces results closest to an equal number of positive and negative results for the reference method. A 25% to 75% split in responses ( in either direction) should be sought. For comparability, the evaluated results for the ATP or new method should be from the same dilution as the reference method. If one of the dilutions does not produce an acceptable split in positive and negative results for the reference method, the applicant should return to the original, spiked sample.

7.2.2 Preparation of Enumerated Spiking Suspension This dilution scheme is adapted from Standard Methods for the Examination of Water and Wastewater, 19th Edition, Section 9020 B ( Reference 10.3). This entire process should be performed quickly to avoid loss of viable organisms. There should be approximately 1010 organisms per slant. Therefore, dilution bottles " A" through " E," below contain approximately 1010, 108, 106, 104, and 103 organisms per dilution bottle, respectively. Depending on the growing conditions, these numbers may vary. As a result, until experience has been gained, more dilutions should be filtered to determine the appropriate dilution.

Inoculate bacterial culture onto the entire surface of several nutrient agar slants with a slope approximately 6.3 cm long in a 125 × 16 mm screw cap tube. Incubate for 24 ± 2 hours at 35 ° C ± 0.5 ° C. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 7 5 From the slant that has the best growth, prepare serial dilutions using four dilution bottles with 99 mL of sterile buffered dilution water ( bottles A, B, C, and D) and one dilution bottle containing 90 mL of sterile buffered dilution water ( bottle E).

Pipette 1 mL of buffered dilution water from bottle " A" to one of the slants. Emulsify the growth on the slant by gently rubbing the bacterial film with the pipette, being careful not to tear the agar. Pipette the suspension back into dilution bottle " A." Repeat this procedure a second time to remove any remaining growth on the agar slant, without disturbing the agar.

Make serial dilutions as follows: ( 1) Shake bottle " A" vigorously and pipette 1 mL to bottle " B" containing 99 mL buffer ( 2) Shake bottle " B" vigorously and pipette 1 mL to bottle " C" containing 99 mL buffer ( 3) Shake bottle " C" vigorously and pipette 1 mL to bottle " D" containing 99 mL buffer ( 4) Shake bottle " D" vigorously and pipette 10 mL to bottle " E" containing 90 mL buffer; this should result in a final dilution of approximately 10 organisms / mL. If it is more convenient for your laboratory, an acceptable alternative to the dilution scheme presented for this step, is to pipette 11 mL of dilution D into dilution bottle E, which contains 99 mL of dilution water.

Filter 1 to 5 mL portions in triplicate from bottles " D" and " E" according to standard membrane filtration methods to determine the number of CFU in the dilutions. The recommended target dilution and spike volume depends on the method and the target analyte( s). Typically, dilutions should be stored at 1 ° C to 5 ° C and may be used throughout the day they are prepared, however, storage conditions should be adjusted as necessary since both storage conditions and viability may vary from organism to organism.

7.2.3 Log Phase Growth Curve Inoculate 100 mL of broth media with a single isolated colony and incubate organisms at optimum temperature. If possible, shake at 200 RPM during incubation.

Take optical density ( OD) measurements at 550 600 nm at 30 minute intervals for the first 8 hours and record readings. In addition to OD readings at 30 minute intervals, using aseptic technique, remove a 0.1 mL portion of the culture and make a series of 1: 10 dilutions in sterile buffer. Initially plate 0.1 mL of the 1.0, 10 1, and 10 2 dilutions in duplicate and incubate overnight at optimum temperature. Count colonies and record data. As the optical density increases, evaluate serial dilutions to accommodate the increased numbers of bacteria ( i. e., when the optical density exceeds 1.0 then plate 0.1 mL from dilutions 10 1, 10 2, and 10 3).

Based on OD and the CFU/ mL results, the OD of fresh cultures ( in log phase growth) can be used to determine the concentration of bacteria in the tubes by a simple graphic representation of the combined OD and CFU/ mL results.

Commercially available McFarland standards may be used to determine the bacterial density instead of actually doing a growth curve within the lab. In order to determine bacterial densities using McFarland standards, the OD of the standards are compared to the OD of the log phase culture.

7.2.4 Commercially Available Enumerated Spikes One time use, commercially available spiking suspensions may be obtained from a variety of vendors. Prepare spiking suspensions according to manufacturer's instructions. It should be noted that a different spiking volume than that recommended by the manufacturer may be necessary to achieve the target density. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 7 6 7.3 Spiking Procedures for Virus Methods 7.3.1 Cell Monolayer Propagation In order to propagate virus stock suspensions cell monolayers should be propagated. The type of assay cytopathic effect ( CPE) or plaque assay ( PA), being performed determines the size of the flask and the number of days the monolayer incubates prior to use for the assay. Inoculate T75 or T25 flasks with 1 x105 cells containing cell specific growth medium. Flasks for CPE should incubate for 5 7 days at 37 ° C ± 0.5 ° C with 95% relative humidity ( and 5% CO2 concentration, if necessary). One CPE flask should be stained with crystal violet and microscopically checked to ensure a 95% confluent cell monolayer prior to use in the assay. Flasks used for PA incubate for 7 10 days under the same conditions as the monolayers used for CPE. PA flasks should also be checked to ensure 95% confluency before use for assay.

7.3.2 Propagation of Virus Stock Suspension For ( CPE) analysis, utilizing the appropriate cell line ( e. g., Buffalo green monkey kidney cells; BGMK), inoculate the cell monolayer with virus and incubate culture flasks at 37 ° C ± 0.5 ° C until the entire monolayer has been destroyed by virus replication ( approximately 72  96 hours). Freeze ( to approximately  80 ° C) and thaw the flasks three times, then pool contents of the flasks and spin at 10,000 x g for 30 minutes. Filter supernatant using a 0.2 µ m pore nylon filter to remove any remaining cell debris. The filtrate is the virus stock suspension.

7.3.3 Titering of the Virus Stock Suspension Titer the virus stock suspension ( from Section 7.3.2) by performing a plaque assay. Inoculate monolayers with 0.2 mL of serial diluted viral stock suspension. After rocking and rinsing the monolayers, add the agar overlay to the monolayer. Incubate the culture flasks for seven days while reading the number of plaques each day for the entire seven days. After the viral stock suspension has been titered, appropriate volumes of the suspension can be used to spike test matrices to obtain plaque forming units ( PFU)/ mL.

7.4 Spiking Procedures for Cryptosporidium and Giardia Enumerated spiking suspensions are needed for initial and ongoing precision and recovery ( IPR and OPR) samples ( often referred to as positive controls) and matrix spike ( MS) samples. Flow cytometer   sorted organisms are necessary for these spiking suspensions, rather than manual techniques. Flow cytometer   sorted spikes generally are characterized by lower variability than manually enumerated spikes.

Spiking suspensions should be prepared using unstained organisms that have not been irradiated, heatfixed or formalin fixed. Immediately before sorting spiking suspensions, initial calibration of the flow cytometer should be performed by conducting a series of sequential sorts directly onto membranes or well slides. These initial sorts should be stained and counted microscopically to verify the accuracy of the system. When sorting the spiking suspensions, ongoing calibration samples should also be prepared and counted at regular intervals. The mean of the ongoing calibration counts should be used as the estimated spike dose. Flow cytometer sorted spiking suspensions should be used by the expiration date noted on the suspension. Flow cytometer sorted spiking suspensions containing live organisms should be used within two weeks of the preparation date. General procedures for preparing flow cytometer counted spikes for Cryptosporidium and Giardia can be found in EPA Method 1622 and 1623 ( April 2001). EPA Microbiological Alternate Test Procedure ( ATP) Protocol 7 7 A potential commercial source of flow sorted Cryptosporidium and Giardia spiking suspensions is:

Wisconsin State Laboratory of Hygiene Flow Cytometry Unit 2601 Agriculture Drive Madison, WI 53718 Phone: ( 608) 224 6260 Fax: ( 608) 224 6213 The Wisconsin State Laboratory of Hygiene prepares and distributes live Cryptosporidium parvum oocysts and Giardia intestinalis cysts that have not been treated to reduce viability.

7.5 Analysis of Samples Samples should be analyzed in accordance with the EPA approved study plan. Any deviations from the study plan may suggest the need for additional analyses and potentially, rejection of data generated from the study. If any deviations from the approved study plan are necessary prior to or during the study, the applicant should consult EPA and receive approval for the modification to the study plan. Deviations from the approved study plan should be documented in the study report ( Section 9.0).

Example analysis schemes are provided in Sections 7.5.1 and 7.5.2 below. Generally, the analysis schemes will differ depending on the type of organism and the type of method. The study plan ( Section 4.0) should detail the order in which samples will be analyzed.

7.5.1 Side by Side Comparison Studies Method Blank 1 Sample 1 up to Sample 20 Method Blank 2 Positive Control Negative Control Method Blank 3 Media Sterility Checks 7.5.2 QC Acceptance Criteria Based Comparison Studies Replicate 1 for IPR Replicate 2 for IPR Replicate 3 for IPR Replicate 4 for IPR Method Blank Matrix Spike Matrix Spike Duplicate Unspiked Matrix Sample 7.6 Verification of Results The number of positive and negative results to be verified in a side by side comparison study is discussed in Section 6.1.3. Sections 7.6.1 to 7.6.3 below discuss the types of independent standards that may be used for the verification of results from bacteriological methods, virus methods, and Cryptosporidium and Giardia methods, respectively. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 7 8 7.6.1 Verification of Results from Bacteriological Methods <IP_ADDRESS> Biochemical Tests that May be Used for Verifications Oxidase Citrate Catalase Lysine decarboxylase ONPG Methyl Red and Voges Proskauer ( MRVP) Indole Triple sugar iron ( TSI) Coagulase Lysine iron agar ( LIA) Esculin Urease Sugars ( e. g., Trehalose, lactose, mannitol, and sorbitol)

It is recommended that multiple biochemical tests be utilized to verify colony identification. The above list of biochemical tests is not exhaustive. The choice of which biochemical tests to use is based on type of organism ( i. e., gram stain results) being identified/ verified. Biochemical tests used for verification should be discussed in the study plan ( Section 4.0). It may be appropriate to perform a gram stain prior to biochemical identification. A description of the biochemical tests listed above, as well as additional tests is provided in Standard Methods for the Examination of Water and Wastewater, Method 9225.

<IP_ADDRESS> Commercially Available Biochemical Testing Products Commercial biochemical test systems incorporate multiple biochemical tests to allow for identification to the genus and/ or species level, which may be difficult when using individual biochemical tests prepared in house.

Commercial biochemical test systems are available in two formats: systems that depend on the analyst to manually interpret the results ( e. g., API strips, BBL crystal, and enterotube) and systems that automate the interpretation of results ( e. g., Vitek and Biolog).

7.6.2 Verification of Results from Virus Methods Viral protocol testing confirmations tend to be method specific, therefore a detailed description of the confirmation/ verification procedure should be included in the study plan and reviewed on a study specific basis.

7.6.3 Verification of Results from Cryptosporidium and Giardia Methods Verification of results may not be necessary for Cryptosporidium and Giardia IFA methods beyond the analyst's microscopic examination of the organism. If verification is necessary, a detailed description of the confirmation/ verification procedure should be included in the study plan and reviewed on a studyspecific basis. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 1 SECTION 8.0 REVIEW OF STUDY RESULTS Generally, upon receipt of the applicant's data, EPA will perform the following reviews discussed in Sections 8.1 to 8.5. 1. Assessment of compliance with the approved study plan 2. Data review 3. Data validation 4. Development of descriptive statistics 5. Statistical assessment of method comparability Methods that are deemed acceptable will generally be recommended for approval ( Section 8.6).

8.1 Assessment of Compliance with Approved Study Plan Generally, EPA will review the study report and associated data to ensure the study was conducted according to the approved study plan. The applicant should explain and justify ( possibly with additional studies) any deviations from the study plan. Deviations from the approved study plan that occur without prior approval from EPA may result in the rejection of some or all study data.

8.2 Data Review Upon receipt of the applicant's data, EPA will generally verify that all raw data described in Section 9.10 are present and complete. Generally, all calculations used in the method will be verified. This may include calculations used for spiking enumeration, preliminary or presumptive stages of the method, and the determination of the final result.

8.3 Data Validation After verifying data completeness and reviewing of all calculations, EPA will generally verify that all measurements were performed in accordance with the method. This may include, but is not limited to, the following: ° Temperature logs for incubator/ waterbath/ refrigerator ° Media preparation records ° Sample incubation times ° Associated QC samples ( as described in Section 5.0) ° Method blanks ° Preparation blanks ° Sterility checks ° Positive and negative controls 8.4 Development of Descriptive Statistics 8.4.1 Mean Recovery To determine if matrix characteristics effect method performance, mean recoveries should be calculated separately for each matrix and method. Mean recoveries should also be calculated for each method over all matrices. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 2 8.4.2 Precision Precision can be expressed both on an absolute scale ( i. e., standard deviation) and on a relative scale ( i. e., relative standard deviation). The RSD ( sometimes referred to as coefficient of variation) is calculated as the standard deviation divided by the mean, expressed as a percent. For the purpose of summarizing the data, both standard deviations and RSDs should be calculated, and the one which is most appropriate for assessing comparability will be used in analyses. Generally, RSDs are most appropriate for summarizing precision when variability increases as concentration increases.

To give an indication of the effect of multiple matrices on precision, standard deviations should be calculated separately for each matrix and method. Standard deviations also should be calculated for each method over all matrices.

8.4.3 False Positive Rates, False Negative Rates, Sensitivity, and Specificity False positive ( FP) and false negative ( FN) rates of approved and reference methods should be evaluated when assessing comparability. An independent standard ( described in Section 7.6) may sometimes be necessary to confirm positives and negatives of both the ATP or new method and the EPA reference method. The confirmation methods used should be discussed in the validation study plan.

Generally, performance of the ATP or new method and EPA approved reference methods will be defined in terms of false positive rates and false negative rates. For the purposes of the ATP protocol, false positive rates and false negative rates are equivalent to ( 1 Specificity) and ( 1 Sensitivity), respectively. Specificity is defined as the percent of negative samples correctly identified as negative, and sensitivity is defined as the percent of positive samples correctly identified as positive ( see equations on the next page). In order to calculate estimates of false positives, false negatives, sensitivity, and specificity for each method, 2 by 2 tables for each matrix, and over all matrices, should be set up as follows in Table 8 1.

Table 8 1. Standard Format for 2 by 2 Tables Independent Standard +  Total ATP or New Method + TP1 FP1 TP1 + FP1  FN1 TN1 FN1 + TN1 Total TP1 + FN1 FP1 + TN1 TP1 + FP1 + TN1 + FN1 Independent Standard +  Total EPAApproved Reference Method + TP2 FP2 TP2 + FP2  FN2 TN2 FN2 + TN2 Total TP2 + FN2 FP2 + TN2 TP2 + FP2 + TN2 + FN2 EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 3 Estimates of sensitivity, specificity, false positive rates, and false negative rates as percentages for the two methods should be calculated as follows:

Sensitivityi = TPi * 100 % TPi + FNi Specificityi = TNi * 100 % TNi + FPi False positive ratei = FPi * 100 % = ( 1  TNi ) * 100 % = 1  Specificityi TNi + FPi TNi + FPi False negative ratei = FNi * 100 % = ( 1  TPi ) * 100 % = 1  Sensitivityi TPi + FNi TPi + FNi Where F = False, N = Negative, T = True and " i" refers to the specified method ( i= 1 for new method, i= 2 for reference method)

8.5 Statistical Assessment of Method Comparability 8.5.1 Presence / Absence Methods For presence/ absence methods, the chi square test and Breslow Day test will generally be used to compare the percent of false positives and false negatives in the proposed method analyses with the percent of false positives and false negatives in the reference method analyses.

Generally, tests for significant differences in false positive and false negative rates between methods will be run using the results summarized in Table 8 1 in Section 8.4. To perform the chi square and Breslow Day tests, it may help to first rearrange the data as follows in Tables 8 2 and 8 3: EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 4 Table 8 2. False Negative Rate Comparison Method New Reference Total Result True + TP1 TP2 TP1 + TP2 False  FN1 FN2 FN1 + FN2 Total TP1+ FN1 TP2 + FN2 TP1 + TP2 + FN1 + FN2 Table 8 3. False Positive Rate Comparison Method New Reference Total Result False + FP1 FP2 FP1 + FP2 True  TN1 TN2 TN1 + TN2 Total FP1 + TN1 FP2 + TN2 FP1 + FP2 + TN1 + TN2 Before running the chi square tests, positives and negatives should be confirmed using the independent standard method. The chi square test will be used to determine whether the false positive rate and false negative rate have a statistically significant difference between the ATP or new method and the EPAapproved reference method.

<IP_ADDRESS> Assessing Method Differences ( Chi Square Test)

In order to assess whether the false positive and false negative rates differ between methods, chi square tests should be run over all matrices; additional tests in each matrix also may be necessary depending on the presence of matrix interactions ( see Section <IP_ADDRESS>). For false negative rates, the chi square test( s) indicate whether the proportions of negative samples correctly identified as negative by the two methods are significantly different, and for false positive rates, the chi square test( s) indicate whether the proportions of true positive results correctly identified as positive for the two methods are significantly different.

<IP_ADDRESS> Assessing Method/ Matrix Interactions ( Breslow Day Test)

In order to assess whether there are false positive and false negative rate differences between methods, it is also necessary to establish if there is a matrix effect on these parameters. The effect of matrices on false positive and false negative rate differences between methods is assessed using the Breslow Day test. ( Reference 10.7). The Breslow Day test is used to test whether there is an interaction between matrix and method in terms of the likelihood of a false positive result ( for specificity) and in terms of the likelihood of a false negative result ( for sensitivity). If a significant interaction is found between method and matrix, then the chi square tests for a difference between methods for that attribute should be done separately for each matrix. Otherwise, separate chi square tests for each matrix are not necessary, and a single chisquare test can be run using the data from all matrices.

<IP_ADDRESS> Method Comparability Conclusions A decision on whether the ATP or new method is comparable to the EPA approved reference method will generally be made based on the results of the false positive and false negative rate comparisons. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 5 Generally, the decision on acceptability of the new method should be made based on the chi square test using data from all matrices. However, if the results of the Breslow Day test indicate that there is a significant interaction between method and matrix for false positive and/ or false negative rates, then further review should be made for those matrices which yielded a higher false positive and/ or false negative rate for the proposed method.

If the results of the chi square tests indicate that the false positive and false negative rates of the ATP or new method are not significantly different from the EPA reference method, this generally will be interpreted as not having enough evidence to conclude that the performance of the ATP or new method is worse than the EPA approved reference method. However, if the results of the chi square test indicate that the false positive and/ or false negative rates of the new method are significantly less than that of the EPA approved reference method, this will generally be interpreted as worse performance and would lead to rejection of the ATP or new method.

8.5.2 Quantitative Methods <IP_ADDRESS> Testing for Normality Many of the statistical analyses used to assess differences in recovery and precision between methods require certain assumptions about the data be met. Because the validity of these assumptions affect which statistical tests will be used to assess recovery and precision differences, testing these assumptions is a prerequisite as a first step. There are two assumptions that should be evaluated prior to comparing the two methods statistically. One assumption is that the variability of the replicates is constant for each matrix and method. This assumption is discussed in Section <IP_ADDRESS>. The other main assumption that should be met is that the data follow a normal distribution.

An assessment of normality can be done either graphically or with statistical tests, or both. A normal distribution looks like a bell curve ( i. e., symmetric around the mean). A graphical assessment can be done using a histogram, stem and leaf or Normal probability plot. Appropriate statistical tests include the Shapiro Wilk test, D'Agostino test, and Kolmogorov Smirnov test. These tests are computation intensive, and it may be necessary to use software ( e. g., SAS) to run them. The Shapiro Wilk test is generally inappropriate for testing the normality assumption of a data set with greater than 50 results.

For environmental data ( especially field data, though sometimes spiked data as well), the distribution of results will often be positively skewed ( i. e., with a few unusually high results). A common corrective action of this that can be utilized is the logarithmic transformation, base e ( natural log). If the data are positively skewed, this transformation should be attempted, assuming that none of the results are negative or equal to zero. When zero valued results are present in a skewed data set, a common approach is to add a small constant to every result prior to transformation. However, the arbitrary choice of this constant will have a large effect on the transformed data ( the log of 1 and the log of 0.1 are very different, for example), and this approach is not recommended. The use of non parametric tests, discussed later in this guidance, are considered to be the appropriate alternatives.

If no zero valued results are present in a positively skewed data set, then the log transformed data should be tested for normality. If the log transformed data do follow a Normal distribution, then all analyses should be done using the log transformed results.

<IP_ADDRESS> Evaluating Precision The decision of what statistical test to use to evaluate precision will generally be made on the results of the normality test described in Section <IP_ADDRESS>. If the assumption of normality is met, then the F test for differences in variance should be used. If the assumption of normality is not met, a non parametric test EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 6 such as the Conover Squared Rank test should be used. The F test is based on comparisons of variances ( i. e., the squared standard deviations). The Conover Squared Rank test is based on ranking absolute deviations from the mean. Figure 8 1 provides a summary of the procedure for evaluating precision.

Prior to testing whether there is a precision difference between methods, the precision of the different matrices should be compared for each method.

If the results of the F test indicate that variances differ significantly by matrix, or if the results of the Conover Squared Rank test indicate that absolute deviations from the mean differ significantly by matrix, then comparisons of precision between methods should be done separately for each matrix. Otherwise, a single comparison of method precision can be done using the data from all matrices. If the results of the F test indicate that there is not enough evidence to conclude that the variances differ significantly by matrix, then the pooled within matrix variance ( i. e., the average of the matrix variances for the given method) should be calculated for each method. The F test should then be used to compare the pooled within matrix variance for the two methods. If the Conover Squared Rank test was used and there was not enough evidence to conclude that the mean absolute deviation differed significantly between matrices, then the Conover Squared Rank test should be used to compare the mean absolute deviations from the overall method mean for the two methods. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 7 Compare absolute deviations from method mean for two methods using Conover squared rank test. Run test for normality Data follow Normal distribution Data do not follow Normal distribution Compare variances between matrices for each method using F test.

Are all matrices equal? Yes No Pool variances over matrices for each method. Compare pooled variances for two methods using the F test. Compare variances between methods separately for each matrix using the F test.

Is the new method at least as precise for at least 80% of matrices?

Evaluate Recovery Yes Failure No Is the new method significantly less precise?

No Yes Evaluate Recovery Failure Is the new method significantly less precise?

No Failure Yes Evaluate Recovery Figure 8 1. Evaluating Precision EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 8 <IP_ADDRESS> Evaluating Recovery Comparisons of mean recovery of the different methods should be done using either an analysis of variance ( ANOVA) model or non parametric test such as the Wilcoxon Mann Whitney ( WMW) test. If the assumption of normality is not met, or if the precision differs between methods or matrices based on the precision evaluation described in the previous section, the WMW test should be used. If all assumptions are met, the ANOVA model should be used. Figure 8 2 provides a summary of the procedure for evaluating recovery.

If an ANOVA model is used, method by matrix interactions should be tested for significance first using the corresponding ANOVA F test. If the method by matrix interaction is significant, it would mean that the difference between methods is not the same for each matrix. In this case, no conclusion regarding an overall difference between the recovery of methods can be made. If the interaction is not significant, then the test for a significant difference between methods can be run in the ANOVA model, using the F test for a significant method main effect.

If the WMW test is used, between matrix and between laboratory variability cannot be separated from replicate variability. Therefore, significant interactions between method and laboratory or matrix cannot be tested when the WMW test is used. Instead the WMW model only tests for an overall difference between methods. The model can be run separately for each matrix, but the diminished power will limit the value of this approach. It is therefore recommended that a single WMW test be run comparing method recoveries, without stratifying by or controlling for matrix type. EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 9 Evaluate false positive and false negative rates. Is recovery significantly lower for the new method? Run test for normality Data do not follow Normal distribution Failure Run ANOVA model testing for method difference, controlling for matrix. No interaction Interaction No Yes Run single WMW test to compare method recovery, disregarding matrices.

Is recovery significantly lower for the new method?

Failure No Yes Evaluate false positive and false negative rates. Evaluate false positive and false negative rates.

Failure No Yes Run ANOVA model testing for method difference separately for each matrix. Count number of matrices where the new method is equal to or significantly better.

Is recovery significantly lower for the new method for at least 20% of all matricies? Data follow Normal distribution Using analysis of variance ( ANOVA), test for method/ matrix interaction. Figure 8.2 Evaluating Recovery EPA Microbiological Alternate Test Procedure ( ATP) Protocol 8 10 <IP_ADDRESS> False Positive Rates, False Negative Rates, Sensitivity, and Specificity Comparisons of false positive rates, false negative rates, sensitivity, and specificity should be conducted according to the methods described in Sections <IP_ADDRESS> and <IP_ADDRESS>.

<IP_ADDRESS> Method Comparability Conclusions If there is not enough evidence to conclude that there is a significant interaction between method and matrix for recovery, or if there is not enough evidence to conclude that there is a statistically significant difference in precision between matrices for either method, then it can be concluded that any such differences of the method will be consistent. Therefore, lack of sufficient evidence to conclude a significant difference between methods will generally be interpreted as equal or better performance, and might lead to acceptance of the ATP or new method, pending review of false positive and false negative rates. However, a statistically significant test ( worse) statistic will be interpreted as worse performance of the ATP or new method, and would generally lead to rejection of the new method.

In cases where significant interactions between matrix and method are found when assessing recovery, or in cases where differences in precision between matrices for at least one method ( and therefore a comparison of method precision could not be done over all matrices), some judgment will be necessary in deciding whether the proposed method should be deemed acceptable. The decision should be based on the attribute ( i. e., recovery or precision) for each matrix. As a general rule, if there was not enough evidence to conclude that the new method was similar or better than the EPA approved reference method for that attribute for at least 80% of the matrices used in the study, then the new method can generally be recommended for approval, pending review of false positive and false negative rates.