Page 1 of 20 Response to Peer Review Comments on the Draft Plan for the Assessment of Detection and Quantitation Limits under Section 304( h) of the Clean Water Act Reviewer A: The assessment approach is conceptually sound. The list of issues in Item [ Event] 3 is sufficient. The plan itself seems comprehensive and well thought out. All important issues will be considered in the process. Of course, the most important proof of the soundness of the plan for assessment will be the product of the assessment to follow. At this stage, I believe that EPA has provided a procedure that can address all of the facets necessary to produce a scientifically and statistically valid re examination of the issue of lower limits for analytical measurements.

Reviewer C: Events 3 through 8 give chief steps in the assessment approach which is proposed. These seem to thoroughly cover many relevant issues. Key to the assessment plan is Event 3, since so many important issues are identified. Giving adequate attention to these issues up front should help to insure that the key issues are understood so that careful choices can be made before the evaluation and selection begins. I can't think of anything to add to the list for Event 3. At the same time, I'd be hesitant to delete anything at this point, although I think that it will be the case that some issues will be found to be not nearly as complex as other issues, and that these simpler issues can be addressed somewhat quickly.

Reviewer E: I think the Plan is sufficient [ and . . . see 1b]

Response: Thank you. EPA appreciates the positive feedback that all three of these reviewers provided on the study plan Reviewer F: Conceptual soundness cannot be evaluated due to insufficient detail. No description is provided on what parameters and/ or criteria will be used to assess methods for identifying detection limit or quantitation limit. If one technique/ method is to be better than another, what makes it better?

Response: EPA sought input on the soundness of the Agency's plan for identifying criteria that will be used to assess detection and quantitation limit concepts, not on the criteria themselves. EPA intends to develop such criteria after identifying and fully considering relevant issues. EPA believes that establishing criteria before such an evaluation would be premature, and in the later words of this reviewer ( see Question 5) would create " the specter that EPA has simply selected criteria that support the conclusion( s) they wish to make." EPA would like to add that the Agency's entire assessment process, including the plan, the identification of issues and appropriate criteria, and the evaluation of various concepts against these criteria will be subjected to formal peer review prior to proposal. ( See Events 9 and 10 of the plan.) 1a. Evaluate the conceptual soundness of the assessment approach presented in the study plan. Is the list of issues included in Item [ Event] 3 of the study plan sufficient? Are there other significant issues that EPA should add or delete? Page 2 of 20 Action/ Revision Implemented based on Peer Reviewer Response to Question 1a: Revise plan to include a statement clarifying that it would be premature to identify evaluation criteria in the plan itself because issues that might affect these criteria have not yet been fully identified.

Reviewer A: I have some comments below on what I think to be the most significant, highpriority issues.

The entire issue of lower limits of analytical methods is one that has been fraught with intellectual confusion, even when it has not been politically and economically controversial. The term " detection limit" has been used for a number of dissimilar concepts, and this has led to further confusion. For the purpose of this review, I will use a version of Currie's terminology.

The Critical Level is an observed value of the response ( such as peak area) or equivalently, of the estimated concentration which would occur by chance in a sample in which the analyte was not present only a specified fraction of the time, say 1%. As with any such limit or level, it can depend on the entire analytical method, but is a well defined construct. This is the limit that tells the analyst when detection has occurred. Such detection is almost always quantitative, so long as there is a measured value and a standard error.

The Minimum Detectable Quantity is the smallest true concentration of the analyte such that with specified probability, such as 99%, the estimated concentration after measurement will exceed the critical level. This level has no interpretation once measurement has occurred. It is useful for planning, but not for interpretation of existing measurements.

For example, if the standard deviation at zero or a low level of an analyte is 2 ppt exactly, then the critical level at 99% confidence is, using the normal percentage point, ( 2.326)( 2) = 4.652. If the standard deviation is estimated, say by a sample of 7 replicates, then one would use the t distribution with 6 degrees of freedom so that the critical level would be ( 3.143)( 2) = 6.286. Using the latter value for illustration, the practical use of the critical level of about 6 ppt is that any measured concentration exceeding that demonstrates that the analyte is present in non zero quantities. If we have a measured value of 8 ppt, for example, then the standard deviation ( assuming that it is constant over the range) is 2 ppt, so a 95% confidence interval for the true concentration is ( using the t distribution again) is 8 ppt ± ( 2.447)( 2) ppt or approximately ( 3ppt, 13ppt). This is a quantitative assessment. We know the true concentration to within 5 ppt, which is pretty accurate. The coefficient of variation ( 2/ 8 = 25%) is not a relevant measure as to whether we have a quantitative measurement of the concentration.

If we use 99%/ 1% also for the minimum detectable quantity, and if we assume that within the range up to this level the standard deviation of measurement is approximately constant, then the minimum detectable quantity is twice the critical level. In the case illustrated above, this would be twice 6.286, or about 11ppt. This value of 11ppt might be a good spike concentration for quality control, because it should almost always generate a detection signal. However, it provides no information useful in interpreting a measured value of, say, 8 ppt. 1b. How should EPA prioritize the issues [ included in Event 3 of the study plan and suggested by commenters]? Page 3 of 20 Glaser et al. ( 1981), along with many others, have seriously confused the issue. Consider the oft quoted definition of the MDL: "  the minimum concentration of a substance that can be identified, measured, and reported with 99% confidence that the analyte concentration is greater than zero " The first half of this definition refers to an actual concentration, like the minimum detectable quantity, but the second half treats it as a measured value like the critical level. Clearly, we understand this better than we did in 1981. What the MDL estimates is the critical level, which is the threshold of detection, and it therefore should not be described as an actual concentration.

EPA should use the review that they are engaged in to clarify these issues, and to provide lower limits that are scientifically and statistically valid. In the first place, this means distinguishing carefully between the critical level and the minimum detectable quantity, and specifying when each one is to be used.

Another issue that deserves a hard look and a scientific analysis is that of the so called limit of quantitation. For an analytical measurement to be quantitative, there are two requirements. First, there needs to be an estimated quantity produced by a known analytical procedure. Second, there needs to be an estimate of the precision of this measurement. Modern metrological techniques ( Rocke and Lorenzato 1995; Zorn, Gibbons, and Sonzogni 1997) can provide the precision estimates at all measured values of an analyte. Thus, any measured value by a calibrated system is quantitative, and there is no such thing as a limit of quantitation if it is defined by standard deviation, coefficient of variation and the like. The only scientifically valid definition is akin to the original ML of 1984. If the instrument cannot produce a number, then we are below the limit of quantitation. This will usually be because the peak cannot be identified that should be integrated to produce the estimated concentration.

The procedures provided by EPA and the questions to be examined will be sufficient to deal with these and other issues, so long as the attention is focused on scientifically and statistically valid methods. This review provides an opportunity for EPA to clarify these vexed issues, and provide unambiguous, practical, interpretable, and valid methods for lower limits of analytical methods.

Response: The reviewer has suggested the Agency's prioritization of issues include ( 1) a clarification of confusion regarding existing detection limit concepts and terminology, including the critical level and the minimum detectable quantity, ( 2) identification of " lower limits that are scientifically and statistically valid", and ( 3) examination and scientific analysis of quantitation limits, with the intent of providing " unambiguous, practical, interpretable, and valid methods for [ characterizing] lower limits of analytical methods." The reviewer has further commented on the plan that the procedures provided by EPA and the questions to be examined will be sufficient to deal with these and other issues, provided that attention is focused on scientifically and statistically valid methods.

EPA appreciates these comments. EPA agrees that the numerous concepts and definitions proposed by a number of different individuals and organizations have resulted in widespread confusion, misinterpretation, and misapplication.

EPA's primary goal in performing this review is to identify a scientifically valid concept that best meets the Agency's needs under the Clean Water Act. In doing so, EPA will attempt to Page 4 of 20 address and clarify the confusion that exists among historical concepts and definitions. EPA also will prioritize this reviewer's recommendations concerning the need for statistically and scientifically valid lower limits and unambiguous, practical, interpretable, and valid methods for characterizing these limits.

Reviewer C: With regard to prioritizing the issues listed for Event 3, items a [ concepts of the lower limit to measurement], b [ concepts in relation to Office of Water applications], f [ criteria for design of detection and quantification studies, including selection of concentration levels], and o [ accepting the procedures of consensus organizations] seem to be important and may also require more time than some of the other items. However I suspect that at least some parts of the literature review are well underway ( or complete). Still, it will be very important to take the results of the literature review and carefully organize and summarize the material ( as opposed to just providing a lengthy listing of issues and proposed methods), and this may prove to be a rather difficult job. Item f deals with the experimental design of the studies that will be performed to test various methods. This seems to be a very important part of the overall plan, since the design should take into account many important issues that are identified in completing the many other items of Event 3. Overall, it seems that a large scale and carefully performed experimental comparison of a good number of well selected methods will be how one finds out which methods are the most accurate. Since time and resource constraints always tend to limit the size of studies that are to be performed, it is important to use a good design in order to produce the most effective results. Items b [ concepts in relation to Office of Water applications] and o [ accepting the procedures of consensus organizations] touch upon legal and regulatory concerns, and thus may be somewhat complex.

Important items which might not be so time consuming include items e and i. Item e deals with the development of detection/ quantitation procedures based on statistical tolerance and prediction. Item i, which calls for consideration of false positives and false negatives, shouldn't take much time, but it is of the upmost importance since the definition of MDL utilizes the probability of a false positive. False negatives are also important, since they are cases where contaminants are present, but a nondetect is recorded. Since false results are such a key issue, it will be important that they are carefully considered when the experimental design is developed. For example, if the false positive rate is to be controlled to be no greater than 0.01, then a large number of trials under the null hypothesis condition of no analyte present must be performed to accurately assess whether or not the false positive rate is adequately controlled. Also, one needs to carefully consider the various ways in which hypothesis testing can be used in the assessment. If the null hypothesis is taken to be p <= 0.01, where p is the probability of a false positive resulting from a particular MDL method, then one has to keep in mind that a failure to reject does not provide statistically significant evidence that p <= 0.01, since a failure to reject could be due to low power, resulting from too few experimental trials. To truly protect against a method having a false positive rate exceeding 0.01, one may wish to make p > 0.01 the null hypothesis and p <= 0.01 the alternate hypothesis. It can get to be a bit confusing, since there are two different null hypothesis situations ( at two different levels): the null situation of no analyte present, for which we are concerned about the false positive rate of an MDL method, and the null hypothesis that when no analyte is present, the false positive rate is greater than 0.01, in which case a type I error would be a claim that the rate is less than or equal to 0.01 when in fact it is not. Page 5 of 20 While I do not think that items c [ criteria for the selection and appropriate use of statistical models], d [ methodology for parameter estimation], and m [ outliers] are by themselves unimportant, I do think that it's important to view them as being linked, and to keep in mind that the main concern is to determine which methods for determining the MDL have the greatest accuracy, and not which methods and models are best for a particular stage in the overall determination of the MDL. This is particularly true if the model upon which a method is based is only a loose approximation of reality, in which case the interpretation of parameters may be viewed as being rather fuzzy. In the end, it's whether or not a particular method is reasonably accurate  viewing the performance of the overall method, and not examining too closely each of it's components. Rather than spend so much time on model selection and assessment, and parameter estimation, it may be better to put more resources into the testing stage, and thoroughly test a large number of methods which have been proposed by others. However, I do think that some thought can go into parameter estimation. For example, if one is uncertain about how to identify and handle outliers, one could replace the commonly used sample standard deviation ( it's used in Method 1631B) with a more robust measure of scale. But again, it's important to keep in mind the overall goal of finding an accurate way to determine the MDL. It's not so important to know whether or not a robust measure of scale is a better estimator in the traditional sense of estimating a true distribution parameter accurately. Rather, what is important is finding out whether or not replacing the sample standard deviation with a robust measure improves the performance of the overall determination of the MDL. Fortunately, in some cases, using alternative estimators, in addition to the one originally proposed for use with a particular method, won't require that additional observations be made, but will only require that alternate determinations of the MDL be made using the same set of data.

Response: EPA appreciates the reviewer's comments and recommendations concerning the prioritization of issues and the suggestion that statistical issues be viewed as being linked with the other criteria. EPA will integrate the suggestions into the issue paper and evaluation criteria.

Reviewer E: [ I] would prioritize the need for MDLs that are performance driven by the labs. EPA only provides some guidance with ranges of MDLs in a method.

Response: EPA agrees that detection limit capabilities may vary among laboratories. EPA believes, however, that many factors, including laboratory capability, may drive MDL performance. These factors may include, but are not limited to, variations in instrumentation, technological advances, analyst and/ or laboratory experience, and even regulatory needs for lower levels of detection. EPA will address this issue when developing its issue paper.

Action/ Revision Implemented based on Comment 1b: Include a discussion of existing concepts and definitions in the Agency's issue paper. Prioritize the need for:

° Statistically and scientifically valid lower limits ( Reviewer A) ° Unambiguous, practical, interpretable, and valid methods for characterizing these limits ( Reviewer A) ° Concepts of the lower limit of measurement ( Reviewer C) ° Criteria for the design of detection and quantification studies, including selection of concentration levels ( Reviewer C) Page 6 of 20 ° Accepting procedures of consensus standards organizations, which may be complex because of legal and regulatory concerns ( Reviewer C) ° Concepts in relation to Office of Water applications, which may be complex because of legal and regulatory concern ( Reviewer C) ° Statistical and tolerance levels, which are important issues, but probably not very timeconsuming ( Reviewer C) ° Consideration of false positives and false negatives, which the reviewer notes will likely require little time but is of the upmost importance and offers specific issues to consider with respect to this item ( Reviewer C) ° Performance driven detection limit concepts ( Reviewer E)

EPA also will incorporate Reviewer C's comment concerning the linked nature of statistical issues into its assessment process.

Reviewer A: Here are some of what I think to be the most important criteria for lower limits of analytical procedures:

a) Definitions should be consistent, scientifically meaningful, and statistically valid. For some specific comments on this issue, see below. b) Procedures that implement the definitions should be clear and well defined. They should also implement the definition. c) The data used to establish specific limits should be realistic. For example, if matrix effects are presumed to be important, then matrix spikes rather than reagent water spikes should be used. d) There should be no artificial limits. Each defined and implemented limit should be based on a scientific and statistical analysis of the true limits of knowledge, not on preferences of involved parties for higher or lower levels. If we can measure something and provide an estimated standard deviation, then we should be able to use the data in science and in regulation.

Response: Reviewer A has provided criteria that largely reflect the reviewer's prioritization of issues in response to question 1B. EPA agrees these are reasonable criteria and will include them in the list of criteria against which to evaluate detection and quantitation limit concepts.

Reviewer C: I think that the main focus should be the probability of a false positive result in the null case of no analyte present. This probability should be as close as possible to the allowable limit ( say 0.01) without going over the limit. While exceeding the limit is particularly bad, since false detections could result in lawsuits and also contribute to a lack of confidence in water quality studies, being too far under the limit, which results from MDL determinations that are larger than they need to be, hurts the power of the method to detect the analyte when it is 2. Based on your understanding of the issues, are there any types of criteria that you suggest EPA include to evaluate detection and quantitation procedures [ Event 4, Criteria Development]? If yes, please identify the criterion and provide a rationale for its inclusion. Page 7 of 20 present. Alternatively, and perhaps nearly equivalently, one can view the situation in terms of both false positives and false negatives, with a goal being to find the method with the smallest false negative rate that also satisfies the false positive criterion. The trouble with this approach is that the false negative rate will almost surely depend on how much analyte is present, and one method may be better than another at low levels while the opposite is true at higher levels. ( Putting it in statistical terms, there may not be a uniformly most powerful test.)

It should be kept in mind that the methods will be used in a variety of settings, and so the null situation of no analyte present should be tested with a variety of water sources. If one observes differing false positive rates for the variety of water sources ( all having no analyte present), then rather than average all of the observed false positive rates, I think it may be prudent to focus on the highest value and require that it satisfy the false positive criterion, since one will want to believe that the false positive rate limit is adhered to no matter what the circumstances ( within reason) may be.

Response: EPA agrees that the criteria must reflect concerns about false positives and false negatives. However, the Settlement Agreement schedule for this evaluation does not allow EPA to implement the reviewer's suggestion to test the null hypothesis in a variety of water sources ( all having no analyte present). Indeed, EPA is not sure how such tests could reasonably be conducted unless the variety of water sources tested were first manipulated to remove of all possible traces of the pollutant of interest from the matrices in order to ensure that any positive results obtained are, indeed, false positives. The problem with such an approach, however, is that removing the target pollutant from the matrices could also remove any matrix interferences.

Reviewer C: In addition to focusing on the false positive rate, which is tied to the definition of the MDL, I also think that the MDL values from the various methods investigated should be compared to an estimate of the 99th percentile of the null distribution, by which I mean the distribution of determinations obtained in the case of no analyte present. It seems to me that the 99th percentile of this distribution could be viewed as a value for the MDL. Using any value less than this 99th percentile as the MDL would mean that if no analyte were present, the probability of obtaining a measured concentration greater than the MDL would exceed 0.01, and thus violate Glaser's definition ( which is also the definition given in Appendix B to Part 136). To estimate the 99th percentile of the null distribution, a simple nonparametric quantile estimator could be used ( with the choice of estimator being dependent on the shape of the distribution and the number of observations ( where here each observation is a determination of the concentration of analyte from a sample in which no analyte is present)). It should be kept in mind that in order to get a decent estimate of the 99th percentile of a distribution, it may take several hundred observations. So this method of determining the MDL may be impractical for ordinary use. But perhaps it can be used as a spot check in at least a small number of cases.

Response: EPA agrees with the reviewer that obtaining a decent estimate of the 99th percentile of a distribution may take several hundred observations and would be impractical for ordinary use. For that reason alone, EPA is not likely to establish such a test as a criterion for selecting an appropriate detection limit concept. If, however, EPA is able to obtain an existing data set that would allow for such an investigation during the course of its detection/ quantitation limit assessment, EPA would attempt to perform the analyses recommended by this reviewer. At this time, EPA is not aware of any detection limit studies containing hundreds of analyses conducted Page 8 of 20 on samples that were known to be free of the target pollutant of interest yet retained matrix effects.

Reviewer C: I think claims made in the literature about various methods [ for establishing detection/ quantitation limit concepts] should not be used to judge them, since previous investigations of accuracy were not done the same way for all of the methods. Instead, the determination of the most reliable methods should be based on the results of a large welldesigned and properly executed experiment ( or series of experiments).

Response: Again, to the extent that EPA can use existing data to evaluate the various concepts, EPA will do so. Unfortunately, EPA has found that very few data sets exist that would allow for such assessments. In part, this is because historical studies typically were performed with very few concentrations, very few replicates at each concentration, or concentrations that were not in the region of interest ( i. e., the region of detection). Compounding the problem ( and adding to the confusion), is the fact that many of the claims in the published literature are based on information extrapolated from data sets that were not appropriately suited to support these claims. We interpret the reviewer's comment to mean that s/ he shares this concern. At this time, EPA is aware of only one data set that includes multiple replicates of varying known concentrations above, at, and below the estimated detection limit. Even this data set, which was collected by EPA in 1998 1999 as part of its detection/ quantitation limit assessments, is limited by the fact that it involves only single laboratory determinations. If data, time, and resources become available for such an assessment prior to proposal, EPA will be happy to implement this reviewer's suggestion.

Reviewer C: Finally, with regard to criteria, some weight has to be put on simplicity and ease of use. The method( s) selected will be used by a large number of people, and some will be more skilled than others.

Response: EPA agrees with the reviewer's comment that the criteria should give weight to simplicity and ease of use by a large number of people with varying skills.

Reviewer E: I do not have any criteria in mind.

Response: No response required.

Reviewer F: EPA has not provided a single criterion for performing this evaluation. Without some idea of what they are planning to do, it is not possible to suggest alternatives.

Response: As noted previously, EPA does not believe it is appropriate to specify criteria before fully assessing related issues. EPA does, however, intend to seek formal peer review on its selected criteria at a later date. The purpose of this peer review was to seek comment on EPA's plan for identifying issues, developing criteria that reflect these issues, and evaluating concepts against those criteria. EPA also sought to use this peer review to seek preliminary recommendations on issues, criteria, and concepts that EPA should not overlook when implementing the plan. Page 9 of 20 Action/ Revision Implemented based on Peer Reviewer Comments on Item 2: EPA adopted each of the criteria recommended by the peer reviewers into its final selection of criteria as follows:

Reviewer Reviewer's Recommended Criteria Maps to EPA Criterion Number( s) A A  Consistent, scientifically meaningful and statistically valid 1 B Clear and well defined procedures that allow for implementation of the definition 1 and 3 C  Rely on realistic data, such as matrix samples when matrix issues are a concern 3 D  Establish limits on scientific and statistical analysis, not on personal or organizational preferences for higher or lower limits 1, 2 and 4 C Criteria should address appropriate risk of false positives and false negatives 4 Criteria should recognize the need for simplicity and ease of use by a large number of people of varying skills 3 Reviewer A: I believe that the EPA should take a fresh look at all of the suggested definitions and procedures It is correct to evaluate suggestions from the scientific literature, from standards organizations, and from other organizations. However, it may be that none of these procedures meets the criteria of 2) [ i. e., criteria suggested in commenter's response to charge question 2], and meeting those criteria is far more important than precedent.

Response: EPA appreciates this feedback. EPA will attempt to evaluate each unique concept or definition available in the literature. ( EPA believes that there is significant overlap in concept among articles describing different detection limit terms.) EPA agrees with the reviewer's comment about the possibility that none of the existing definitions and procedures will meet the selected criteria and that meeting those criteria is the primary objective of the study.

Reviewer C: Of course, methods currently used by the EPA, such as Method 1631B, should be included. In addition, I'm in favor of including several promising methods that are to be identified from the open scientific literature, as well as perhaps a small number of alternate procedures that are used by other organizations, provided that some justification can be found for them other than that they are sometimes used. ( I fear that sometimes a method can become popular for no real good reason. One organization may select a method for use somewhat haphazardly, and then another organization can pick up the method just because others have used it. Since methods are sometimes selected based in part on political considerations, or concerns about practicality that may not be pertinent in all cases, I tend not to give a lot of weight to the 3. What existing detection and quantitation procedures should EPA evaluate or focus on? Procedures from the open scientific literature? Voluntary consensus standards procedures? Procedures from other organizations? Page 10 of 20 popularity of methods, and I'm much more impressed by studies, performed by people other than the originators of the method, that demonstrate that a method is reasonably accurate.)

Methods from the literature are nice to consider, because as time passes, refinements seem to be made upon previously proposed methods, and one can be hopeful that the class of methods that get the most attention of this sort are the ones that perform best, and thus hope that recent papers giving the latest adjustments to an established general scheme will include some that are ( near) state of the art. Of course, one has to be wary of authors who are too eager to publish and develop a small adjustment in hope of getting a quick and easy publication. Also, since many published methods seem to be highly derivative of previously published ones, there might not be enough variety found in the literature and so one may want to consider other sources.

Response: EPA believes that this reviewer may have misunderstood the Agency's question. EPA intended to seek input on procedures for determining detection and quantitation limits, not on measurement methods for specific analytes.

Reviewer C: With regard to identifying alternate methods from the literature review, I think that the 1998 memorandum from Raphael Kuznetsovski of SAIC should be of great help. There are 27 papers and books described in the method detection limits and quantification section. The summaries provided for each of the 27 works can be used to narrow the focus to perhaps a dozen or fewer key entries worthy of careful consideration. In general, I recommend examining recent articles, slightly older articles from the chemistry literature by authors who are well established in the field, and also articles from the statistical literature in order to perhaps gain a slightly different perspective.

Response: EPA agrees that the 1998 literature review assembled by SAIC under contract to EPA will be of help in identifying pre 1998 methods from the published literature. EPA has supplemented these articles with a literature search to identify post 1998 articles that may be relevant to this evaluation. As noted above, EPA agrees that many of the articles can be consolidated into a smaller subset of unique concepts or procedures.

Reviewer C: I'll now make some brief comments about some of the works described in the SAIC memo. ( Since complete citations are provided in that memo, I won't duplicate such information here.) Several of the papers pertain to the Hubaux and Vos approach: the original 1970 paper, the 1978 paper by Bayley, Cox, and Springer, and the 1997 paper by Coleman, Auses, and Grams, with this last paper being by far the most recent of the three, and perhaps worthy of a close look. Gibbons has written a lot of pertinent articles, and I think several of his works should be studied, particularly those which compare various methods, like his 1994 book. Similarly, Currie has written a lot, but some of his work is rather dated, and some of it deals with related issues and not particular methods. But his 1997 paper may be of interest, since it touches on assumptions and approximations. Davis' 1994 article, the 1991 paper of Lambert, Peterson, and Terpenning, and the 1997 paper of Spiegelman and Tarlow are examples of works from the statistical literature that could be of interest, and the 1987 paper of Clayton, Hines, and Elkins and the 1988 paper of Garner and Robertson are papers from the chemical literature that could be considered, with the later paper of particular interest since it touches on the issue of assumptions.

Response: EPA plans to consider the specific papers cited by this reviewer in its evaluation. Page 11 of 20 Reviewer C: Finally, I think that it will be important to consider variety with regard to the complexity of the methods to be chosen. It might be good to include some simple methods in addition to those with are rather involved. Not only are the more complex methods more difficult for some to accurately use, but it may be the case that some of the simpler methods could outperform some of the complex ones, due to poor assumptions underlying some of the methods.

Response: EPA agrees that concepts relying on simple procedures may outperform those involving complex ones, and that the Agency's assessment should include an evaluation of both simple and complex methodologies.

Reviewer E: I think EPA should focus on the MDL approach and the detection limit should be performance driven. Each lab should develop their own MDLs and the EPA method should provide some examples and some expectations for each analytes and matrix but let the labs demonstrate their own MDLs. The " market" will then drive the MDLs. If there is a need for more sensitive MDLs then the successful labs will improve their MDLs.

The " environmental chemistry" community has been using the Fed. Reg. MDL method for about 10 years to demonstrate detection limits. I feel this is an acceptable approach.

Response: While EPA agrees with the reviewer's suggestion that allowing the market to drive detection limits would be beneficial, the Agency does not believe that it is appropriate to limit its assessment to the MDL procedure. Doing so would violate the spirit of the Settlement Agreement.

Reviewer E: Another decision EPA needed to make is to define the factor that is used to convert the MDL to a Reporting Limit, or what every it will be called. For example if the RL or ML is defined as 3.18 times the MDL then everyone could understand and use it even though they may not agree.

Response: While EPA does not necessarily agree ( at least without further evaluation) that reporting limits should be based on a simple multiplier of the MDL, EPA agrees that the issue of reporting limits differs from detection limit concepts and will consider the use of multipliers and other suggested approaches. Given the limited schedule and resources available, however, EPA will necessarily limit any discussion of reporting limits to their context within Clean Water Act programs.

Reviewer F: I believe the charge to EPA is that they look at all existing procedures, see if they can come up with any new ones and then, with some predefined rationale, pick the best one. This sounds like a justifiable charge. If the intent of this question is to determine if I have a favorite that I want to be sure is on the list, I do not.

Response: EPA believes that significant overlap among the existing procedures allows for classification into broad groups, much as described by Reviewer C above. Given the limited schedule, EPA will attempt to review as many concepts as possible, including each of those recommended by the Peer Reviewers and any other concepts that the Agency identifies as being likely to meet the selected evaluation criteria. Page 12 of 20 Action/ Revision Implemented based on Peer Review Comments to Question 3: Evaluate concepts from the published literature against the selected criteria that represent a range of complexity, from simple to complex. At a minimum, include the following papers:

° Hubaux and Vos approach ( the original 1970 paper, the 1978 paper by Bayley, Cox, and Springer, and the 1997 paper by Coleman, Auses, and Grams ° The Gibbons papers, including his 1994 book that compares various methods ° Currie's work, particularly his 1997 paper ° Davis' 1994 article ( statistics) ° Lambert, Peterson, and Terpenning's 1991 article ( statistics) ° Spiegelman and Tarlow's 1997 paper ( statistics) ° Clayton, Hines, and Elkins 1987 paper ( chemistry) ° Garner and Robertson 1988 paper ( chemistry) ° EPA's method detection limit and similar concepts ° Literature concerning detection limit multipliers or other approaches to derive acceptable reporting limits Reviewer A: I am familiar with some of the data sets proposed, but not all of them. Data sets for a procedure like this one should have several to many replicates at a number of concentrations from zero or near zero, to near the critical level to high concentrations.

Response: EPA agrees.

Reviewer C: Based on my rather limited knowledge about the data sets referred to, they do seem appropriate for use. At this time, I don't have any other data sets to recommend. However, it may be good to produce some new data for this important study, since existing data may not be ideal.

Response: Unfortunately, EPA does not have the time or resources needed to produce new data to support this evaluation. EPA will continue to consider any new data made available to the Agency for this use, however.

Reviewer E: I am aware of MDL data sets for the MDLs in method 1638 that could be evaluated.

Response: MDL data sets will generally not meet the needs of this project. As pointed out by Reviewer A, data sets for a procedure like this one should have several to many replicates at a number of concentrations from zero or near zero, to near the critical level to high concentrations.

Reviewer F: The Plan indicates that Episode 6000 data will be reviewed. Are these the only data that will be reviewed? Are these data associated with Method 1631B? Are there other data that could be used in addition? If so, will they be used or excluded? 4. Are the data sets listed in the plan appropriate for use in the detection and quantitation reassessment? Are there other data sets that you recommend EPA evaluate? Page 13 of 20 There is only one data set identified in the Plan, Episode 6000 data. If these are mercury data derived from Method 1631B, they are appropriate for that method. However, because I haven't seen them, I don't know. However, I would be surprised to hear that Episode 6000 data are applicable to all EPA methods. If they are, how was this determined? I know EPA has other data associated detection studies because I've seen EPA statisticians present them at various technical meetings. Also, based on the references identified in the memo to Chuck White from Raphael Kuznetsovski, it would appear that there are a number of other data sets outside EPA. Perhaps the authors would be willing to share them. What I'm not sure of is whether or not the Plan is intended to address other EPA methods.

Response: EPA agrees that the Episode 6000 data set was not adequately described in the draft plan. The Episode 6000 data set consists of a series of iterative MDL studies performed on 11 measurement techniques, including ICP/ AES, ICP/ MS, GC/ MS, and classical wet chemistry. EPA also agrees that additional data should be reviewed as part of this assessment, provided that those data are from studies directed at characterizing variability in the region of detection. EPA will revise the plan to include an improved description of the Episode 6000 data set and of the Agency's strategies for identifying and selecting other data to be included in the study.

Action/ Revision Implemented based on Peer Review Comments to Question 4: Revise the draft plan to clarify the nature of the Episode 6000 data set and the Agency's strategy for identifying and selecting additional data sets. Also, we should solicit additional data sets in the proposal.

Reviewer A: It is better to develop selection criteria during the development of the project. However, possible criteria should be included in the plan.

Response: EPA is pleased that the reviewer agrees it is better to develop selection criteria during project development. With respect to documenting possible criteria in the plan, EPA believes that modifying the plan to include criteria that are selected at a much later date ( i. e., after fully considering the issues) will cause confusion. Therefore, EPA will document the selected criteria separately.

Reviewer C: I believe that it is better to firm up the selection criteria as the study progresses, thinking that people will develop a better understanding of the issues as they put more thought into them. Plus, instead of specifying the criteria up front, it may be better to wait and see what happens and then examine the pros and cons of the various methods when making specific comparisons.

Response: EPA is pleased that this reviewer also agrees it is better to develop selection criteria during project development. 5. Is it appropriate to include specific selection criteria in the plan or is it more appropriate to develop selection criteria based on the analysis of issues that will be conducted as part of the plan? Page 14 of 20 Reviewer E: I think the selection criteria should be included so each lab demonstrates their MDLs by analysis of replicate samples of actual field samples , either spiked or unspiked depending on the concentration in the matrix. EPA would provide the guidance in terms of number of replicates Response: In developing it's selection criteria, EPA will consider this reviewer's preference for criteria that include performance in real world matrices. The existing MDL procedure provides guidance on application to matrices other than reagent water; e. g., to field sample matrices. EPA may produce a separate guidance document to clarify the detection/ quantitation procedures ultimately selected.

Reviewer F: It is not just " appropriate" to include specific selection criteria, it is essential if an objective assessment is to be conducted. Otherwise there is the specter that EPA has simply selected criteria that support the conclusion( s) that they wish to make. This is likely to be obvious to the plaintiffs in the suit against EPA.

Response: EPA disagrees. EPA believes that it is not appropriate to select criteria without a rationale, and the rationales for various criteria cannot be fully developed without a full evaluation of the related issues. Other arguments against defining the criteria in the planning stage were provided by Reviewer C ( see above).

Action/ Revision Implemented based on Peer Review Comments to Question 5: None.

V. OTHER COMMENTS A. Summary Level Comments Reviewer C: I think the plan is very thorough, with the various events called for touching on many important issues, and I can't find anything that seems to have been overlooked. The issues considered deal with the legal aspects of the situation, and also with doing good science and searching for a reliable MDL determination procedure.

Response: EPA appreciates the positive feedback on the plan.

Reviewer F: The Plan is too vague and lacking in technical detail to generate an assessment of its scientific merit. The approach described in the Plan does not contain sufficient detail to determine whether or not it is valid. Neither does it contain enough information with respect to how the assessment will be done to determine the likelihood that it will be objective and/ or useful.

Response: EPA is disappointed that Reviewer F finds the plan too vague, yet Reviewer F offers no specific details to enhance the plan.

Action/ Revision Implemented based on Summary Level Peer Review Comments: None. Page 15 of 20 C. Concern that Plan Does Not Meet Requirements of Clause 6 of the Settlement Agreement Reviewer F: This reviewer has read and reread these documents. The second paragraph of the Charge to Peer Review Panel requests an opinion regarding the validity of the Plan and the likelihood that it will result in an " objective and useful assessment of detection ( limit) and quantitation ( limit) procedures". None of the five Questions to Peer Reviewers ( included in the Charge to Peer Reviewers) address the purpose indicated in the Plan's introduction as the reason the plan was developed, namely, responding to Clause 6 of the settlement agreement. In addition, only the Plan document identifies Method 1631B; the other documents seem to address EPA methods in general. Consequently, I will try to address all the review considerations starting with the Plan.

The Plan identifies 4 key items associated with satisfying the requirements of Clause 6 of the settlement agreement. These are: Reassessment of EPA's existing MDL and ML procedures; External review of the reassessment; External review of any alternate procedures EPA has under consideration; and Proposal of revised/ alternate procedures, if warranted. This is to be accomplished by the end of February 2003. To this end, EPA has established a series of tasks and milestones leading up to the due date. Although not specifically stated, it is presumed that a document will be generated which addresses the four items in Clause 6.

Response: The study plan contains a section near the end titled, " Relationship of the Plan to the Settlement Agreement," that specifies which plan events will meet the requirements of each subpart of Clause 6 of the Settlement Agreement. EPA believes this section is sufficiently clear, but in response to Reviewer F's concerns, will also integrate these linkages directly into the plan where possible. With respect to the reviewer's question concerning documentation, EPA intends to develop one or more documents to detail the Agency's implementation of this plan. This documentation will include the Agency's identification and assessment of issues, the criteria selected for evaluating various concepts, the evaluation of various concepts, and the rationale for the final selection. EPA expects to incorporate each of these documents into a Technical Development Document that will support the FR notice.

Reviewer F: At the end of the Introduction to the Plan, there is a statement that the " document contains the detailed plan for the reassessment called for in Clause 6  ". In fact the Plan contains very little detail. The only thing that comes close to detail is the listing of " issues" that are part of Task 3. There is no description of the processes and/ or criteria that will be used in the reassessment. EPA should identify the specific criteria that will be used to select the most appropriate mechanism( s) for determining detection limit( s) and/ or quantitation limit( s). If the criteria are not identified up front, it may appear that EPA first selected mechanisms, then created justification to support them. This is what Clause 6 seems to be trying to avoid.

Response: EPA disagrees for reasons explained in responses to Issues 2 and 5 in this document.

Reviewer F: Presumably the people involved have the skills and knowledge, and EPA will allow them the time and provide the resources, needed to do the job. Unfortunately there is no way to assess this based on the Plan. EPA identifies a " Core Workgroup consisting of measurement analysts and statisticians within the Office of Science and Technology" but does Page 16 of 20 not provide information on their background or experience. The success of this project will depend on the people involved, so this is critical information. In summary, it is very difficult to see how this Plan will result in EPA being where they want to be when they want to be there. They may succeed, but working to this Plan will not be the reason for their success.

Response: The purpose of this peer review was to seek input on EPA's plan for evaluating concepts and methodologies supporting detection and quantitation limit concepts. The plan outlines a series of tasks and a schedule for accomplishing those tasks. EPA will provide the resources and staff necessary to implement this plan. These resources include staff who are experienced and trained in the areas of analytical chemistry, environmental statistics, and implementation of Clean Water Act programs, and are familiar with many of the detection and quantitation limit issues identified in the plan. In addition, materials developed by this team will be reviewed by other individuals throughout the Agency, including those that represent Regional perspective and legal perspectives, as well as those that offer insights from other environmental programs ( such as the Safe Drinking Water Act).

Action/ Revision Implemented based on Peer Review Comments concerning adherence to the Settlement Agreement: Clarify the linkages between the Settlement Agreement and the planned activities.

D. Unclear whether Plan Applies to All Methods or Only Method 1631B Reviewer F: It is also not clear whether the Plan is intended to address only Method 1631B or EPA methods in general. If it is EPA methods in general, the magnitude of the job is much, much greater and it seems unlikely that a valid technical effort could be conducted in the time frame identified.

Also, because Method 1631B is cited in the Introduction along with Clause 6, it is assumed that the Plan document is intended to address only this method.

Response: Although the Settlement Agreement was established in response to a legal challenge to Method 1631B, the terms of the Agreement dictate that the Agency evaluate detection and quantitation limit concepts as they apply to all CWA programs. EPA will modify the plan to indicate more clearly that the scope of this effort is to address methods used in CWA programs.

Action/ Revision Implemented based on Peer Review Comments concerning Scope: Revise the plan to clarify its scope.

E. Confusion over Milestones/ Event Sequence Reviewer C: In the schedule given on page 12 some events seem out of sequence. From pages 8 and 9 it seems as though Events 5 and 6, which deal with evaluation and reassessment, should follow Event 4, which is criteria development, and this seems sensible to me. Yet on page 12, it appears as though Event 4 lags behind Events 5 and 6.

Response: EPA agrees. EPA has revised the schedule to address these and other concerns. Page 17 of 20 Reviewer F: EPA has organized this effort into 16 tasks/ milestones that are identified in a chart on page 12. The first two tasks have been completed, the resulting output being the documents under review. The organization of the remaining tasks, however, is somewhat confusing. First, there seem to be six first draft reports, excluding the Federal Register Notice, plus two seconddraft reports. I suspect these are sections in the final report document but this is not clear. It appears that the drafts are assembled into a single report as the interim, and only milestone, of Task 9. External Peer Review is then preformed over a three month period shown as Task 10, starting in May of ` 02. Final selection of the " Recommend Option" takes place as a milestone ( task 9) after External Peer Review is complete.

However, even though the report has been sent out for external peer review, development tasks ( Tasks 3, 4 and 7), evaluation tasks ( Tasks 5 and 8), and the reassessment task ( Task 6), continue to run on through Feb ` 03. Consequently, it appears that the material developed in these tasks after April ' 02 will not be subject to external peer review. This appears to be inconsistent with the requirements of Clause 6. Further, why should these tasks continue past the time the Recommended Option has been selected? If the tasks continue does that mean some other Option could replace the one identified in task 11?

In addition, some of the tasks identified in the Plan do not appear to be clearly related to dealing with the four points of Clause 6. The first point of Clause 6 is addressed by Task 6, the second point by Tasks 9 and 10, the third point by Tasks 8, 9 and 10, and the fourth point by Task 7. Tasks 1 and 2 were used to create the documents reviewed here. That leaves Tasks 12 through 16, which appear to be EPA administrative issues that must fit into the time line prior to the completion date. If this is the case, it is difficult to understand how the development, evaluation and reassessment tasks identified above could continue while EPA is writing a Federal Register Notice ( Task 12), the Agency is reviewing ( Task 13) and OMB is reviewing ( Task 14).

Response: EPA agrees and has revised the schedule to address these and other concerns.

Action/ Revision Implemented based on Peer Review Comments concerning Schedules: Include revised schedule in the plan.

F. Comments Supporting Ability to Change MDL Over Time Reviewer E: [ Study Plan, Page 4, Section ii. Method Development and Promulgation] I strongly believe that detection limits should not be cast in stone when the method is promulgated. The DL's often improve radically as labs become more proficient and improvements are made to techniques. I have experienced clients and regulators who flag data as below the DL because it is below the Method specified DL but well above our achieved DL!

Response: EPA agrees that detection limits are subject to change and will bear this in mind when defining criteria for evaluating concepts. Again, this is an implementation issue that will affect any concepts chosen by EPA. How we ultimately address this issue may not be part of this effort but, as noted earlier, may be addressed as part of a guidance document. Alternatively, we can provide improved guidance for application of whatever procedure goes into 40 CFR 136 in place of the current Appendix B. We will point out that states and others can still do differently. Page 18 of 20 Reviewer E: [ Study Plan, Page 5, Section vii. Descriptive versus Prescriptive Uses of Lower Limits to Measurement] I am very much in agreement with this statement which I assume means that the detection limit can change depending on the measurement method and the proficiency of the lab.

Response: EPA appreciates the positive feedback and will bear this in mind when defining criteria for evaluating concepts.

Reviewer E: [ Study Plan, Page 6, Section j. Measurement Quality Over the Life of a Method] I agree with this statement that the measurement quality will usually improve over the life of the method. This is a very important issue and therefore the DL for a method should be allowed to change as the equipment and analyst change.

Response: Thank you for the positive feedback. Again, EPA will bear this in mind when defining criteria for evaluating concepts.

Reviewer E: [ General comments] Each EPA method should include a statement about the range of MDLs that have been demonstrated during the validation of the method. Each EPA method should state that any lab using this method must demonstrate their own MDL, which may be higher or lower than the range mentioned in the method.

Response: EPA will consider this recommendation as part of its overall evaluation.

Reviewer E: The labs can inform their " clients" about their lab MDLs and the " environmental chemistry market" will drive these MDLs. If a regulation requires low detection limit then labs will be motivation to demonstrate the needed MDL.

Response: EPA will consider this recommendation when defining criteria for evaluating concepts.

Action/ Revision Implemented based on Peer Review Comments concerning the ability of the MDL to Change over Time: When identifying and addressing issues, include Peer Reviewer E's concerns about the need for concepts that allow detection limits to be adjusted according to changes in technology and lab proficiency.

G. Clarification Needed Regarding the Term " Outliers"

Reviewer E: [ Study Plan, Page 7, Section m. Outliers] The term " outliers" is confusing as used in the statement. My understanding of outliers are data points that can be separated from the other data points with a statistical test called an " outlier test." There should be an acceptable procedure to determine when data is not appropriate to be included in an MDL.

Response: For the purposes of EPA's evaluation of detection/ quantitation limit concepts, EPA will investigate the various definitions of " outlier" and choose the definition most consistent with application to these concepts. Page 19 of 20 H. Recommend Exploration of Procedure to Develop Theoretical MDLs for Solids Based on Liquid MDLs Reviewer E: [ Study Plan, Page 8, Section t. Cost to Implement Limit Procedures] A procedure for developing " theoretical MDL's" for solids, such as sediment and tissue, based on liquid MDL's should be explored. For example a true MDL for mercury in fish tissue cannot be experimentally determined because all fish tissue has relatively high levels of mercury compare to the sensitivity of the method. However, using water derived MDL it is possible to calculate the MDL for fish. This situation is true for many metals that are naturally abundant in sediment, soil and tissue.

Response: EPA will address the issue of " theoretical MDL's" in its evaluation. EPA would like to note, however, that the Agency has successfully used matrices other than water to simulate performance in tissue. For example, EPA has found that egg and chicken breast can be used to establish ` tissue' detection limit studies for mercury using Method 1631.

I. Typos Reviewer F: [ Study Plan, Page 10] There are a few typos, judgment is misspelled, and Task 16 on page 10 should identify December 2002, not 2001. It's not the nits that need to be addressed, it's organization and the lack of detail.

Response: EPA has corrected the typos on Page 10 of the plan and has revised the milestone/ event schedule to correct dates as appropriate.

J. Concern Regarding Errors in Current Appendix B to Part 136 Reviewer C: This doesn't pertain explicitly to the 12 page plan. But in reviewing supplementary material that was sent to me along with the plan, I noticed a few typos in Appendix B to Part 136. The most serious mistake is in part ( b) of Step 7. If the ratio of variances is less than 3.05, then one would compute the pooled standard deviation. So the first > 3.05 should be < 3.05. ( One can check this in the original description of the method given in the 1981 paper by Glaser, Foest, McKee, Quane, and Budde.) Less serious are two minor typos: ( i) below the sum at the bottom of page 2 it should just be i = 1, not i = 1S; ( ii) on the 3rd line of page 3, S should be replaced by a sigma.

Response:. The procedure as published in the code at 40 CFR 136, Appendix B appears to be correct.

K. Concern Regarding Method 136B Reviewer C: I'll also comment that I'm a bit suspicious of Method 136B. The justification given for the approach in the paper by Glaser et al makes use of assumptions and approximations. Since the final formula given for the MDL doesn't seem quite right, I wonder if perhaps too many assumptions were made along the way. Also, distribution skewness could harm the accuracy of the method. Page 20 of 20 Response: EPA will consider these concerns when evaluating concepts.


[Federal Register Volume 77, Number 96 (Thursday, May 17, 2012)]
[Notices]
[Pages 29339-29340]
[FR Doc No: 2012-11951]

-----------------------------------------------------------------------

ENVIRONMENTAL PROTECTION AGENCY

[EPA-HQ-OPP-2011-0766; FRL-9517-3]

Agency Information Collection Activities; Submission to OMB for
Review and Approval; Comment Request; Foreign Purchaser Acknowledgement
Statement of Unregistered Pesticides

AGENCY: Environmental Protection Agency (EPA).

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: In compliance with the Paperwork Reduction Act (44 U.S.C. 3501
et seq.), this document announces that the following Information
Collection Request (ICR) has been forwarded to the Office of Management
and Budget (OMB) for review and approval: Foreign Purchaser
Acknowledgement Statement of Unregistered Pesticides; EPA ICR No.
0161.12, OMB Control No. 2070-0027. The ICR, which is abstracted below,
describes the nature of the information collection activity and its
expected burden and costs.

DATES: Additional comments may be submitted on or before June 18, 2012.

ADDRESSES: Submit your comments, referencing Docket ID No. EPA-HQ-OPP-
2011-0766, to (1) EPA online using www.regulations.gov (our preferred
method), or by mail to: EPA Docket Center, Environmental Protection
Agency, Office of Pesticide Programs Regulatory Public Docket (28221T),
1200 Pennsylvania Ave. NW., Washington, DC 20460, and (2) OMB by mail
to: Office of Information and Regulatory Affairs, Office of Management
and Budget (OMB), Attention: Desk Officer for EPA, 725 17th Street NW.,
Washington, DC 20503.

FOR FURTHER INFORMATION CONTACT: Scott Drewes, Field and External
Affairs Division, (7506P), Environmental Protection Agency, 1200
Pennsylvania Ave. NW., Washington, DC 20460; telephone number: (703)
347-0107; fax number:<PHONE_NUMBER>; email address:
<EMAIL_ADDRESS>
SUPPLEMENTARY INFORMATION: EPA has submitted the following ICR to OMB
for review and approval according to the procedures prescribed in 5 CFR
1320.12. On November 30, 2011 (76 FR 74051), EPA sought comments on
this ICR pursuant to 5 CFR 1320.8(d). EPA received no comments during
the comment period. Any additional comments on this ICR should be
submitted to EPA and OMB within 30 days of this notice.
    EPA has established a public docket for this ICR under Docket ID
No. EPA-HQ-OPP-2011-0766, which is available for online viewing at
www.regulations.gov, or in person viewing at the OPP Regulatory Public
Docket in the EPA Docket Center (EPA/DC), EPA West, Room 3334, 1301
Constitution Ave. NW., Washington, DC. The EPA/DC Public Reading Room
is open from 8 a.m. to 4:30 p.m., Monday through Friday, excluding
legal holidays. The telephone number for the Reading Room is 202-566-
1744.
    Use EPA's electronic docket and comment system at
www.regulations.gov, to submit or view public comments, access the
index listing of the contents of the docket, and to access those
documents in the docket that are available electronically. Once in the
system, select ``docket search,'' then key in the docket ID number
identified above. Please note that EPA's policy is that public
comments, whether submitted electronically or in paper, will be made
available for public viewing at www.regulations.gov as EPA receives
them and without change, unless the comment contains copyrighted
material, confidential business information (CBI), or other information
whose public disclosure is restricted by statute. For further
information about the electronic docket, go to www.regulations.gov.
    Title: Foreign Purchaser Acknowledgement Statement of Unregistered
Pesticides.
    ICR numbers: EPA ICR No. 0161.12, OMB Control No. 2070-0027.
    ICR Status: This is a request to renew an existing approved
collection. This ICR is scheduled to expire on July 31, 2012. Under OMB
regulations, the Agency may continue to conduct or sponsor the
collection of information while this submission is pending at OMB.
    Abstract: This information collection program is designed to enable
the Environmental Protection Agency (EPA) to provide notice to foreign
purchasers of unregistered pesticides exported from the United States
that the pesticide product cannot be sold in the United States. Section
17(a)(2) of the Federal Insecticide, Fungicide, and Rodenticide Act
(FIFRA) requires an exporter of any pesticide not registered under
FIFRA section 3 or sold under FIFRA section 6(a)(1) to obtain a signed
statement from the foreign purchaser acknowledging that the purchaser
is aware that the pesticide is not registered for use in, and cannot be
sold in, the United States. A copy of this statement, which is known as
the Foreign Purchaser Acknowledgement Statement, or FPAS, must be
transmitted to an appropriate official of the government in the
importing country. This information is submitted in the form of annual
or per-shipment statements to the EPA, which maintains original records
and transmits copies, along with an explanatory letter, to appropriate
government officials of the countries which are importing the
pesticide.
    In addition to the export notification for unregistered pesticides,
FIFRA requires that all exported pesticides include appropriate
labeling. Export labeling requirements meet the definition of third-
party notification. In the interests of consolidating various related
information collection requests, this ICR includes burden estimates for
the FPAS requirement for unregistered pesticides, as well as the
labeling requirement for all exported pesticides, both registered and
unregistered. These burdens have been consolidated in this information
collection since the implementation of the 1993 pesticide export policy
governing the export of pesticides, devices, and active ingredients
used in producing pesticides.
    Responses to the collection of information are mandatory (see 40
CFR 168, subpart D). An Agency may not conduct or sponsor, and a person
is not required to respond to, a collection of information, unless it
displays a currently valid OMB control number. The OMB control numbers
for EPA's regulations in title 40 of the CFR, after appearing in the
Federal Register when approved, are listed in 40 CFR part 9, are
displayed either by publication in the Federal Register or by other
appropriate means, such as on the related collection instrument or
form, if applicable. The display of OMB control numbers in certain EPA
regulations is consolidated in 40 CFR part 9.
    Burden Statement: The annual public reporting and recordkeeping
burden for this collection of information is estimated to average from
one to eight hours per response. Burden means the total time, effort,
or financial resources expended by persons to generate, maintain,
retain, or disclose or provide information to or for a Federal agency.
This includes the time needed to review instructions; develop, acquire,
install, and utilize technology and systems for the purposes of
collecting, validating, and verifying information, processing and
maintaining information, and disclosing and providing information;
adjust the existing ways to comply with any previously applicable
instructions and requirements which have subsequently changed; train
personnel to be able to respond to a collection of

information; search data sources; complete and review the collection of
information; and transmit or otherwise disclose the information.
    Respondents/Affected Entities: Entities potentially affected by
this ICR are individuals or entities that either manufacture and export
or that reformulate or repackage and export unregistered pesticides.
The North American Industrial Classification System (NAICS) code
assigned to the parties responding to this information is 325300.
    Estimated Number of Respondents: 50.
    Frequency of Response: On occasion.
    Estimated Total Annual Hour Burden: 24,470.
    Estimated Total Annual Cost: $1,461,658, includes $0 annualized
capital or O&M costs.
    Changes in the Estimates: There is a decrease of 22 hours in the
total estimated burden currently identified in the OMB Inventory of
Approved ICR Burdens. This decrease reflects EPA's updating of burden
estimates for this collection based upon historical information on the
number of foreign purchaser acknowledgement statements submitted
annually. Based upon revised estimates, the average number of foreign
purchaser acknowledgement statements submitted annually has decreased
from 2,304 to 2,283, with a corresponding decrease in the associated
burden from 2,442 hours in the previous renewal to 2,420 hours in the
current renewal. This change is an adjustment.

John Moses,
Director, Collection Strategies Division.
[FR Doc. 2012-11951 Filed 5-16-12; 8:45 am]
BILLING CODE 6560-50-P


[Federal Register: January 11, 2011 (Volume 76, Number 7)]
[Notices]
[Page 1610]
[DOCID:fr11ja11-51]

-----------------------------------------------------------------------

DEPARTMENT OF ENERGY

Federal Energy Regulatory Commission

[Docket No. ER11-2589-000]

Evraz Claymont Steel, Inc.; Supplemental Notice That Initial
Market-Based Rate Filing Includes Request for Blanket Section 204
Authorization

January 4, 2011.
    This is a supplemental notice in the above-referenced proceeding
Evraz Claymont Steel, Inc.'s application for market-based rate
authority, with an accompanying rate tariff, noting that such
application includes a request for blanket authorization, under 18 CFR
part 34, of future issuances of securities and assumptions of
liability.
    Any person desiring to intervene or to protest should file with the
Federal Energy Regulatory Commission, 888 First Street, NE.,
Washington, DC 20426, in accordance with Rules 211 and 214 of the
Commission's Rules of Practice and Procedure (18 CFR 385.211 and
385.214). Anyone filing a motion to intervene or protest must serve a
copy of that document on the Applicant.
    Notice is hereby given that the deadline for filing protests with
regard to the applicant's request for blanket authorization, under 18
CFR part 34, of future issuances of securities and assumptions of
liability, is January 24, 2011.
    The Commission encourages electronic submission of protests and
interventions in lieu of paper, using the FERC Online links at http://
www.ferc.gov. To facilitate electronic service, persons with Internet
access who will eFile a document and/or be listed as a contact for an
intervenor must create and validate an eRegistration account using the
eRegistration link. Select the eFiling link to log on and submit the
intervention or protests.
    Persons unable to file electronically should submit an original and
14 copies of the intervention or protest to the Federal Energy
Regulatory Commission, 888 First Street, NE., Washington, DC 20426.
    The filings in the above-referenced proceeding are accessible in
the Commission's eLibrary system by clicking on the appropriate link in
the above list. They are also available for review in the Commission's
Public Reference Room in Washington, DC. There is an eSubscription link
on the Web site that enables subscribers to receive e-mail notification
when a document is added to a subscribed dockets(s). For assistance
with any FERC Online service, please e-mail<EMAIL_ADDRESS>
or call<PHONE_NUMBER> (toll free). For TTY, call<PHONE_NUMBER>.

Kimberly D. Bose,
Secretary.
[FR Doc. 2011-298 Filed 1-10-11; 8:45 am]
BILLING CODE 6717-01-P


Bill Wehrum/ DC/ USEPA/ US 05/ 25/ 2005 03: 11 PM To afraas@ omb. eop. gov, aflynn@ omb. eop. gov cc Lydia Wegman/ RTP/ USEPA/ US@ EPA, Kathy Kaufman/ RTP/ USEPA/ US@ EPA Subject Fw: CAIR Better than BART Write Up Art and Amy  Here's our explanation of why we think the new IPM runs support the conclusion that CAIR is better than BART. Note that this write up focuses only on the technical question of whether we can rely on the existing air quality runs in light of the new IPM runs that more accurately reflect the final CAIR rule and what we expect from the final BART rule. We think the answer is yes, the new IPM runs in conjunction with the existing air quality runs provide a solid basis for concluding that CAIR is better than BART.

 Forwarded by Bill Wehrum/ DC/ USEPA/ US on 05/ 25/ 2005 03: 04 PM

Bill Grantham/ RTP/ USEPA/ US 05/ 24/ 2005 05: 38 PM To Jeff Holmstead/ DC/ USEPA/ US@ EPA cc Bill Wehrum/ DC/ USEPA/ US@ EPA, Lydia Wegman/ RTP/ USEPA/ US@ EPA, Peter Tsirigotis/ RTP/ USEPA/ US@ EPA, Brian Timin/ RTP/ USEPA/ US@ EPA, Joe Paisie/ RTP/ USEPA/ US@ EPA, Lea Anderson/ DC/ USEPA/ US@ EPA, Chet Thompson/ DC/ USEPA/ US@ EPA, Kathy Kaufman/ RTP/ USEPA/ US@ EPA, Steve Page/ RTP/ USEPA/ US@ EPA Subject CAIR Better than BART Write Up Jeff,

As requested on yesterday's call, here is a write up summarizing the results of the new CAIR better than BART analysis, which Brian Timin and I put together, for purposes of informing the interagency review. The spreadsheet referred to within will be forwarded tomorrow ( it is done but needs some formatting).


[Federal Register Volume 80, Number 56 (Tuesday, March 24, 2015)]
[Proposed Rules]
[Pages 15521-15522]
[FR Doc No: 2015-06579]

========================================================================
Proposed Rules
                                                Federal Register
________________________________________________________________________

This section of the FEDERAL REGISTER contains notices to the public of
the proposed issuance of rules and regulations. The purpose of these
notices is to give interested persons an opportunity to participate in
the rule making prior to the adoption of the final rules.

========================================================================

Federal Register / Vol. 80, No. 56 / Tuesday, March 24, 2015 /
Proposed Rules

DEPARTMENT OF TRANSPORTATION

Federal Aviation Administration

14 CFR Part 39

[Docket No. FAA-2015-0494; Directorate Identifier 2014-NM-160-AD]
RIN 2120-AA64

Airworthiness Directives; Bombardier, Inc. Airplanes

AGENCY: Federal Aviation Administration (FAA), DOT.

ACTION: Notice of proposed rulemaking (NPRM).

-----------------------------------------------------------------------

SUMMARY: We propose to adopt a new airworthiness directive (AD) for
certain Bombardier, Inc. Model DHC-8-400 series airplanes. This
proposed AD was prompted by reports of inadvertent deployment of a
single outboard spoiler during flight. This proposed AD would require
replacement of the power control units (PCUs) for the outboard spoilers
with upgraded PCUs. We are proposing this AD to prevent leakage of the
piston head seal and piston rod seals of the outboard spoiler PCUs,
which could result in inadvertent spoiler deployment and reduced
controllability of the airplane.

DATES: We must receive comments on this proposed AD by May 8, 2015.

ADDRESSES: You may send comments, using the procedures found in 14 CFR
11.43 and 11.45, by any of the following methods:
     Federal eRulemaking Portal: Go to http://www.regulations.gov. Follow the instructions for submitting comments.
     Fax:<PHONE_NUMBER>.
     Mail: U.S. Department of Transportation, Docket
Operations, M-30, West Building Ground Floor, Room W12-140, 1200 New
Jersey Avenue SE., Washington, DC 20590.
     Hand Delivery: U.S. Department of Transportation, Docket
Operations, M-30, West Building Ground Floor, Room W12-140, 1200 New
Jersey Avenue SE., Washington, DC, between 9 a.m. and 5 p.m., Monday
through Friday, except Federal holidays.
    For service information identified in this proposed AD, contact
Bombardier, Inc. Q-Series Technical Help Desk, 123 Garratt Boulevard,
Toronto, Ontario M3K 1Y5, Canada; telephone<PHONE_NUMBER>; fax 416-375-
4539; email<EMAIL_ADDRESS>Internet http://www.bombardier.com. You may view this referenced service information at
the FAA, Transport Airplane Directorate, 1601 Lind Avenue SW., Renton,
WA. For information on the availability of this material at the FAA,
call<PHONE_NUMBER>.

Examining the AD Docket

    You may examine the AD docket on the Internet at http://www.regulations.gov by searching for and locating Docket No. FAA-2015-
0494; or in person at the Docket Management Facility between 9 a.m. and
5 p.m., Monday through Friday, except Federal holidays. The AD docket
contains this proposed AD, the regulatory evaluation, any comments
received, and other information. The street address for the Docket
Operations office (telephone<PHONE_NUMBER>) is in the ADDRESSES section.
Comments will be available in the AD docket shortly after receipt.

FOR FURTHER INFORMATION CONTACT: Cesar Gomez, Aerospace Engineer,
Airframe and Mechanical Systems Branch, ANE-171, FAA, New York Aircraft
Certification Office, 1600 Stewart Avenue, Suite 410, Westbury, NY
11590; telephone<PHONE_NUMBER>; fax<PHONE_NUMBER>.

SUPPLEMENTARY INFORMATION:

Comments Invited

    We invite you to send any written relevant data, views, or
arguments about this proposed AD. Send your comments to an address
listed under the ADDRESSES section. Include ``Docket No. FAA-2015-0494;
Directorate Identifier 2014-NM-160-AD'' at the beginning of your
comments. We specifically invite comments on the overall regulatory,
economic, environmental, and energy aspects of this proposed AD. We
will consider all comments received by the closing date and may amend
this proposed AD based on those comments.
    We will post all comments we receive, without change, to http://www.regulations.gov, including any personal information you provide. We
will also post a report summarizing each substantive verbal contact we
receive about this proposed AD.

Discussion

    Transport Canada Civil Aviation (TCCA), which is the aviation
authority for Canada, has issued Canadian Airworthiness Directive CF-
2014-22, dated July 16, 2014 (referred to after this as the Mandatory
Continuing Airworthiness Information, or ``the MCAI''), to correct an
unsafe condition for certain Bombardier, Inc. Model DHC-8-400 series
airplanes. The MCAI states:

    Although [Canadian] AD CF-2009-26 [dated May 21, 2009 (http://wwwapps3.tc.gc.ca/Saf-Sec-Sur/2/CAWIS-SWIMN/attachment.asp?aiid=CF-2009-26&revid=0&cntr=CF&file=CFCF-2009-26.pdf&type=PDE), which
corresponds to FAA AD 2009-25-05, Amendment 39-16124 (74 FR 63574,
December 4, 2009)] was issued to mandate the upgrade of the spoiler
lift/dump valve, it did not reduce the rate of inadvertent single
spoiler deployment occurrences. Further investigation revealed that
the outboard spoiler PCUs may also be subject to pressure reversals
at the PCU main control valve seal, resulting in leakage at the
piston head seal and piston rod seals. If not corrected, this
condition may result in [inadvertent spoiler deployment and] reduced
controllability of the aeroplane.
    This [Canadian] AD mandates the replacement of the existing
outboard spoiler PCUs with the upgraded PCUs with re-designed seals
for better leakage protection.

    You may examine the MCAI in the AD docket on the Internet at http://www.regulations.gov by searching for and locating Docket No. FAA-2015-
0494.

Related Service Information Under 1 CFR Part 51

    Bombardier, Inc. has issued Service Bulletin 84-27-63, dated
October 17, 2013. The service information describes procedures for
replacement of the existing outboard spoiler PCUs with upgraded PCUs.
The actions described in this service information are intended to
correct the unsafe condition identified in the MCAI. This service
information is reasonably available; see ADDRESSES for ways to access
this service information.

FAA's Determination and Requirements of This Proposed AD

    This product has been approved by the aviation authority of another
country, and is approved for operation in the United States. Pursuant
to our bilateral agreement with the State of Design Authority, we have
been notified of the unsafe condition described in the MCAI and service
information referenced above. We are proposing this AD because we
evaluated all pertinent information and determined an unsafe condition
exists and is likely to exist or develop on other products of the same
type design.

Costs of Compliance

    We estimate that this proposed AD affects 82 airplanes of U.S.
registry.
    We also estimate that it would take about 4 work-hours per product
to comply with the basic requirements of this proposed AD. The average
labor rate is $85 per work-hour. Required parts would cost about $0 per
product. Based on these figures, we estimate the cost of this proposed
AD on U.S. operators to be $27,880, or $340 per product.

Authority for This Rulemaking

    Title 49 of the United States Code specifies the FAA's authority to
issue rules on aviation safety. Subtitle I, section 106, describes the
authority of the FAA Administrator. ``Subtitle VII: Aviation
Programs,'' describes in more detail the scope of the Agency's
authority.
    We are issuing this rulemaking under the authority described in
``Subtitle VII, Part A, Subpart III, Section 44701: General
requirements.'' Under that section, Congress charges the FAA with
promoting safe flight of civil aircraft in air commerce by prescribing
regulations for practices, methods, and procedures the Administrator
finds necessary for safety in air commerce. This regulation is within
the scope of that authority because it addresses an unsafe condition
that is likely to exist or develop on products identified in this
rulemaking action.

Regulatory Findings

    We determined that this proposed AD would not have federalism
implications under Executive Order 13132. This proposed AD would not
have a substantial direct effect on the States, on the relationship
between the national Government and the States, or on the distribution
of power and responsibilities among the various levels of government.
    For the reasons discussed above, I certify this proposed
regulation:
    1. Is not a ``significant regulatory action'' under Executive Order
12866;
    2. Is not a ``significant rule'' under the DOT Regulatory Policies
and Procedures (44 FR 11034, February 26, 1979);
    3. Will not affect intrastate aviation in Alaska; and
    4. Will not have a significant economic impact, positive or
negative, on a substantial number of small entities under the criteria
of the Regulatory Flexibility Act.

List of Subjects in 14 CFR Part 39

    Air transportation, Aircraft, Aviation safety, Incorporation by
reference, Safety.

The Proposed Amendment

    Accordingly, under the authority delegated to me by the
Administrator, the FAA proposes to amend 14 CFR part 39 as follows:

PART 39--AIRWORTHINESS DIRECTIVES

0
1. The authority citation for part 39 continues to read as follows:

    Authority: 49 U.S.C. 106(g), 40113, 44701.

Sec.  39.13  [Amended]

0
2. The FAA amends Sec.  39.13 by adding the following new airworthiness
directive (AD):

Bombardier, Inc.: Docket No. FAA-2015-0494; Directorate Identifier
2014-NM-160-AD.

(a) Comments Due Date

    We must receive comments by May 8, 2015.

(b) Affected ADs

    None.

(c) Applicability

    This AD applies to Bombardier, Inc. Model DHC-8-400, -401, and -
402 airplanes, certificated in any category, serial numbers 4001 and
4003 through 4453 inclusive.

(d) Subject

    Air Transport Association (ATA) of America Code 27, Flight
Controls.

(e) Reason

    This AD was prompted by reports of inadvertent deployment of a
single outboard spoiler during flight. We are issuing this AD to
prevent leakage of the piston head seal and piston rod seals of the
outboard spoiler power control units (PCUs), which could result in
inadvertent spoiler deployment and reduced controllability of the
airplane.

(f) Compliance

    Comply with this AD within the compliance times specified,
unless already done.

 (g) Replacement of PCUs for the Outboard Spoilers

    Within 6,000 flight hours or 36 months after the effective date
of this AD, whichever occurs first: Replace the outboard spoiler
PCUs with upgraded PCUs having re-designed seals, in accordance with
the Accomplishment Instructions of Bombardier Service Bulletin 84-
27-63, dated October 17, 2013.

(h) Other FAA AD Provisions

    The following provisions also apply to this AD:
    (1) Alternative Methods of Compliance (AMOCs): The Manager, New
York Aircraft Certification Office (ACO), ANE-170, FAA, has the
authority to approve AMOCs for this AD, if requested using the
procedures found in 14 CFR 39.19. In accordance with 14 CFR 39.19,
send your request to your principal inspector or local Flight
Standards District Office, as appropriate. If sending information
directly to the ACO, send it to ATTN: Program Manager, Continuing
Operational Safety, FAA, New York ACO, 1600 Stewart Avenue, Suite
410, Westbury, NY 11590; telephone<PHONE_NUMBER>; fax<PHONE_NUMBER>.
Before using any approved AMOC, notify your appropriate principal
inspector, or lacking a principal inspector, the manager of the
local flight standards district office/certificate holding district
office. The AMOC approval letter must specifically reference this
AD.
    (2) Contacting the Manufacturer: For any requirement in this AD
to obtain corrective actions from a manufacturer, the action must be
accomplished using a method approved by the Manager, New York ACO,
ANE-170, FAA; or Transport Canada Civil Aviation (TCCA); or
Bombardier, Inc.'s TCCA Design Approval Organization (DAO). If
approved by the DAO, the approval must include the DAO-authorized
signature.

(i) Related Information

    (1) Refer to Mandatory Continuing Airworthiness Information
(MCAI) Canadian Airworthiness Directive CF-2014-22, dated July 16,
2014, for related information. This MCAI may be found in the AD
docket on the Internet at http://www.regulations.gov by searching
for and locating Docket No. FAA-2015-0494.
    (2) For service information identified in this AD, contact
Bombardier, Inc., Q-Series Technical Help Desk, 123 Garratt
Boulevard, Toronto, Ontario M3K 1Y5, Canada; telephone<PHONE_NUMBER>;
fax<PHONE_NUMBER>; email<EMAIL_ADDRESS>Internet
http://www.bombardier.com. You may view this service information at
the FAA, Transport Airplane Directorate, 1601 Lind Avenue SW.,
Renton, WA. For information on the availability of this material at
the FAA, call<PHONE_NUMBER>.

    Issued in Renton, Washington, on March 12, 2015.
Jeffrey E. Duven,
Manager, Transport Airplane Directorate, Aircraft Certification
Service.
[FR Doc. 2015-06579 Filed 3-23-15; 8:45 am]
 BILLING CODE 4910-13-P