Peer reviewed work products: An example of a major work product undergoing peer review is the IRIS Documentation: Reference Dose for Methylmercury. Methylmercury contamination is the basis for fish advisories. It is necessary to determine an intake to humans that is without appreciable risk in order to devise strategies for decreasing mercury emissions into the environment. After EPA derived a reference dose ( RfD) of 0.0001 mg/ kg day in 1995, industry argued that it was not based on sound science. Congress ordered EPA to fund an National Research Council/ National Academy of the Sciences panel to determine whether our RfD was scientifically justifiable. The panel concluded that the 0.0001 mg/ kg day was an appropriate RfD, based on newer studies than the 1995 RfD. The information in this document was evaluated, incorporated, and subjected to comment by the Office of Water, where it contributed in large part to Chapter 4 of Drinking Water Criteria for the Protection of Human Health: Methylmercury ( EPA/ 823/ R 01/ 001) January 2001. The peer review mechanism was an external peer review workshop and public comment session held on November 15, 2000, accompanied by a public comment period from October 30 to November 29, 2000.

Case by base determination   PBT Chemicals Rule: An example of a case by case determination is the Guidance Document for Reporting Releases and Other Waste Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 45 Management Activities of Toxic Chemicals: Dioxin and Dioxin like Compounds ( December, 2000). In a final rule published October 29, 1999, EPA lowered the reporting thresholds for certain persistent bioaccumulative toxic ( PBT) chemicals that are subject to reporting under Section 313 of the Emergency Planning and Community Right to Know Act of 1986 ( EPCRA) and Section 6607 of the Pollution Prevention Act of 1990 ( PPA). We also added a category of dioxin and dioxin like compounds to the EPCRA Section 313 list of toxic chemicals and established a 0.1 gram reporting threshold for the category. In addition, EPA added certain other PBT chemicals to the EPCRA Section 313 list of toxic chemicals and established lower reporting thresholds for these chemicals. As a result of this rulemaking, we developed a guidance document on the reporting requirements for the dioxin and dioxin like compounds category, as well as a number of other guidance documents. The dioxin guidance document provides guidance on how to estimate annual releases and other waste management quantities of dioxin and dioxin like compounds to the environment from certain industries and industrial activities. Due to the high interest level of stakeholders, we solicited public comments on the draft guidance document and formed a workgroup of interested stakeholders. The workgroup reviewed all public comments, provided their own comments, and then reviewed and commented on the final draft.

Case by case determination   National Water Quality Inventory Report: A second example of a case by case determination is the National Water Quality Inventory Report to Congress. The National Water Quality Inventory Report to Congress is a biennial report to Congress and the public about the quality of our nation's waters. It is prepared under Section 305 ( b) of the Clean Water Act ( CWA), which requires States and other jurisdictions to assess the health of their waters and the extent to which water quality supports State water quality standards and the basic goals of the CWA. States' Section 305 ( b) assessments are an important component of their water resource management programs. These assessments help States: implement their water quality standards by identifying healthy waters that need to be maintained and impaired waters that need to be restored, prepare their Section 303 ( d) lists of impaired waters, develop restoration strategies such as total maximum daily loads and source controls, and evaluate the effectiveness of activities undertaken to restore impaired waters and protect healthy waters.

A number of commenters said that EPA created a limited definition of what types of information are to be considered " influential," and that we have no rational basis to do so. A number of commenters also stated that " all Agency information should be considered influential"; that " all data relied upon by the Agency should meet a high standard of quality regardless of the type"; or that "` influential' information includes information used to support any EPA action, not just ` top' Agency actions." EPA followed OMB's guidelines in establishing a definition for " influential" information that was not all encompassing. OMB stated " the more important the information, the higher the quality standards to which it should be held, for example, in those situations involving " influential scientific, financial or statistical information...". OMB narrowed the definition of " influential" in their final guidance as follows: Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 46 In this narrower definition, " influential", when used in the phrase " influential scientific, financial, or statistical information", is amended to mean that " the agency can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions" ( 67 FR 8455).

OMB also amended their definition to say that " each agency is authorized to define " influential" in ways appropriate for it given the nature and multiplicity of issues for which the agency is responsible" ( 67 FR 8455). We adopted OMB's " influential" definition. Once the Agency reviewed the wide range of information disseminated to the public, such as major rulemakings, risk assessments, rule related guidance, health advisories, annual reports, fact sheets, and coloring books, it became apparent that there were reasons to distinguish between " influential" information and other information. EPA adopted OMB's definition for " influential" and used types of information the Agency disseminates to further explain what information is included.

Another commenter suggested that EPA should not indicate whether disseminated information is " influential" when it is first disseminated but should wait to designate information as " influential" until either an information correction request is made or a final agency action is taken. We intend to consider this point, as well as other comments made about when disseminated information becomes influential, as the Agency implements the Guidelines.

One commenter suggests that the definition of the term " influential" should be more narrow. Specifically, the commenter states the following:

Within the relatively narrow sphere of " disseminated" information, an agency should reserve the designation of " influential" for information disseminated in support of agency actions that are " major" regulations under Executive Order 12866, provide a " significant" opportunity to advance the agency's mandate by other means, or involve precedent setting or reasonably controverted issues. This designation recognizes that procedures to promote the quality of information have significant costs, and that the most significant ( and therefore the most costly) of such procedures should be reserved for information that is the most important in terms of the agency's mission.

EPA agrees with the commenter that there are significant costs associated with ensuring that information disseminated by the Agency is of high quality. Consequently, EPA chose a definition of the term " influential" to cover information that, when disseminated, will result in a clear and substantial impact on important public policies and private sector decisions. We believe that this definition balances the costs associated with implementing the Guidelines, the need to ensure high quality information, and the Agency's mission to protect human health and safeguard the natural environment.

Several commenters indicated that it is inappropriate for EPA to base its definition of " influential" on categories of actions. They suggest that the definition be based instead on the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 47 content of the information. We consider our definition to be based on information content, given that those categories of disseminated information we defined as influential are those that EPA can reasonably determine will or do have a clear and substantial impact on important public policies or private sector decisions. We note here that, in addition to the specific classes of disseminated information we have defined as " influential," EPA has reiterated the " case bycase portion of the OMB " influential" definition. This general provision is intended to capture disseminated information, based on its content, that would not otherwise rise to the level of " influential" under the other parts of our definition ( i. e., top Agency actions, Economically Significant actions, major peer reviewed products).

Several commenters assert that EPA should categorically state that certain specific types of disseminated information products are influential, and that we should categorically state that certain specific types of disseminated information products are not influential. Given the vast array of information disseminated by the Agency, and given the fact that certain information may have a clear and substantial impact on important public policies or private sector decisions at one time, but not have such an impact later on ( and vice versa), classifying types of information as " influential" or otherwise upfront is difficult and could be misleading. We intend to rely on our definition in determining whether specific types of disseminated information products are to be considered " influential" for purposes of the Guidelines.

A. 3.5 Reproducibility Some commenters stated that there needs to be more clarity in the definition of " reproducibility" and related concepts. We have tried to provide definitions that are consistent with OMB guidelines. Also, our Guidelines now include that EPA intends to ensure reproducibility for disseminated original and supporting data according to commonly accepted scientific, financial, or statistical standards. Many commenters thought there should be some kind of method to consider reproducibility when proprietary models, methods, designs, and data are used in a dissemination. Some commenters discourage all use of proprietary models; others suggest proprietary model use be minimized with application limited to situations in which it is absolutely necessary. We understand this concern, but note that there are other factors that are appropriately considered when deciding whether to use proprietary models, including feasibility and cost considerations ( e. g., it may be more cost effective for the Agency to use a proprietary model in some situations than to develop its own model). In cases where the Agency relies on proprietary models, these model applications are still subject to our Peer Review Policy. Further, as recently directed by the Administrator, the Agency's Council on Regulatory Environmental Modeling is now revitalizing its development of principles for evaluating the use of environmental models with regard to model validation and certification issues, building on current good modeling practices. In addition, these Guidelines provide for the use of especially rigorous " robustness checks" and documentation of what checks were undertaken. These steps, along with transparency about the sources of data used, various assumptions employed, analytic methods applied, and statistical procedures employed should assure that analytic results are " capable of being substantially reproduced." Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 48 Regarding robustness checks, commenters were concerned that the EPA did not use the term " especially rigorous robustness checks." We have modified our Guidelines to include this term. Some commenters speculated on the ability of the Agency's Peer Review program to meet the intent of the Guidelines and were concerned about the process to rebut a peer review used to support the objectivity demonstration for disseminated information. Our Peer Review program has been subject to external review and we routinely verify implementation of the program. Affected persons wishing to rebut a formal peer review may do so using the complaint resolution process in these Guidelines, provided that the information being questioned is considered to be " disseminated" according to the Guidelines.

Regarding analytic results, some commenters indicated that the transparency factors identified by EPA ( section 6.3 of the Guidelines) are not a complete list of the items that would be needed to demonstrate a higher degree of quality for influential information. EPA agreed with the list of four items that was initially provided by the OMB and recognizes that, in some cases, additional information regarding disseminated information would facilitate increased quality. However, given the variety of information disseminated by the Agency, we cannot reasonably provide additional details for such a demonstration at this time. Also, in regards to laboratory results, which were mentioned by several commenters, these Guidelines are not the appropriate place to set out for the science community EPA's view of what constitutes adequate demonstration of test method validation or minimum quality assurance and quality control. Those technical considerations should be addressed in the appropriate quality planning documentation or in regulatory requirements.

EPA has developed general language addressing the concept of reproducibility and may provide more detail after appropriate consultation with scientific and technical communities, as called for by OMB in its guidelines. We have already begun to consult relevant scientific and technical experts within the Agency, and also have planned an expedited consultation with EPA's Science Advisory Board ( SAB) on October 1, 2002. Based on these initial consultations, EPA may seek additional input from the SAB in 2003. These consultations will allow EPA to constructively and appropriately refine the application of existing policies and procedures, to further improve reproducibility. In the interim, EPA intends to base the reproducibility of disseminated original and supporting data on commonly accepted scientific, financial, or statistical standards.

A. 3.6 Influential Risk Assessment General Risk Assessment Risk assessment is a process where information is analyzed to determine if an environmental hazard might cause harm to exposed persons and ecosystems ( paraphrased from Risk Assessment in the Federal Government, National Research Council, 1983). That is:

Risk = hazard x exposure For a chemical or other stressor to be " risky," it must have both an inherent adverse effect on an Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 49 organism, population, or other endpoint and it must be present in the environment at concentrations and locations that an organism, population, or other endpoint is exposed to the stressor. Risk assessment is a tool to determine the likelihood of harm or loss of an organism, population, or other endpoint because of exposure to a chemical or other stressor. To assist those who must make risk management decisions, risk assessments include discussions on uncertainty, variability and the continuum between exposure and adverse effects.

Risk assessments may be performed iteratively, with the first iteration employing protective ( conservative) assumptions to identify possible risks. Only if potential risks are identified in a screening level assessment is it necessary to pursue a more refined, data intensive risk assessment. The screening level assessments may not result in " central estimates" of risk or upper and lower bounds of risks. Nevertheless, such assessments may be useful in making regulatory decisions, as when the absence of concern from a screening level assessment is used ( along with other information) to approve the new use of a pesticide or chemical or to decide whether to remediate very low levels of waste contamination. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA 34 Safe Drinking Water Act Amendments of 1996, 42 U. S. C. 300g 1( b)( 3)( A) & ( B).

35 In section III. 3. ii. C. of its guidelines, OMB states that: " With regard to analysis of risks to human health, safety and the environment maintained or disseminated by the agencies, agencies shall either adopt or adapt the equality principles applied by Congress to risk information used and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996 ( 42 U. S. C. 300g 1( b)( 3)( A) & ( B)). Agencies responsible for dissemination of vital health and medical information shall interpret the reproducibility and peer review standards in a manner appropriate to assuring the timely flow of vital information from agencies to medical providers, patients, health agencies, and the public. Information quality standards may be waived temporarily by agencies under urgent situations ( e. g., imminent threats to public health or homeland security) in accordance with the latitude specified in agency specific guidelines".

36Because the assessment of " environmental risk" is being distinguished in OMB's adaptation of the SDWA quality principles from " human health risk", the term " environmental risk" as used in these Guidelines does not directly involve human health concerns. In other words, " environmental risk assessment" is, in this case, the equivalent to what EPA commonly refers to as " ecological risk assessment".

3740 CFR part 160 for FIFRA and 40 CFR part 792 for TSCA.

Appendix 50 OMB Guidelines In its guidelines OMB stated that, with respect to influential information regarding health, safety or environmental risk assessments, agencies should either adopt or adapt the quality principles in the Safe Drinking Water Act ( SDWA) Amendments of 1996.34,35 In the background section of the OMB guidelines, OMB explains that " the word ` adapt' is intended to provide agencies flexibility in applying these principles to various types of risk assessment."

Guidelines Development Consideration EPA carefully and practically developed the adaptation of the SDWA quality principles using our considerable experience conducting human health and ecological36 risk assessments as well as using our existing policies and guidance.

EPA conducts many risk assessments every year. Some of these are screening level assessments based on scientific experts' judgments using conservative assumptions and available data and can involve human health, safety, or environmental risk assessments. Such screening assessments provide useful information that are sufficient for regulatory purposes in instances where more elaborate, quantitative assessments are unnecessary. For example, such assessments could indicate, even with conservative assumption, the level of risk does not warrant further investigation. Other risk assessments are more detailed and quantitative and are based on research and supporting data that are generated outside EPA. For example, pesticide reviews are based on scientific studies conducted by registrants in accordance with our regulations and guidance documents. Our test guidelines and Good Laboratory Practices ( GLPs) 37 describe sound scientific practices for conducting studies needed to assess human and environmental hazards and exposures. Such studies are not required to be peer reviewed. Risk assessments based on these studies can include occupational, dietary, and environmental exposures. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA 38http:// www. epa. gov/ OSP/ spc/ rcpolicy. htm 39 Ibid.

40US EPA( 1998). Guidelines for ecological risk assessment ( Federal Register 63( 93): 26846 26924). http:// www. epa. gov/ ncea/ raf.

41US EPA ( 1992). Guidelines For Exposure Assessment. Federal Register 57( 104): 22888 22938. http:// www. epa. gov/ ncea/ raf/.

42US EPA ( 1998). Guidelines For Neurotoxicity Risk Assessment. Federal Register 63( 93): 26926 26954. http:// www. epa. gov/ ncea/ raf/.

43US EPA ( 1996). Guidelines For Reproductive Toxicity Risk Assessment. Federal Register 61( 212): 56274 56322. http:// www. epa. gov/ ncea/ raf .

44 The Office of Solid Waste and Emergency Response has developed Tools for Ecological Risk Assessment for Superfund Risk Assessment. One example is the Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting Ecological Risk Assessments  Interim Final. http:// www. epa. gov/ oerrpage/ superfund/ programs/ risk/ ecorisk/ ecorisk. htm http:// www. epa. gov/ oerrpage/ superfund/ programs/ risk/ tooleco. htm Appendix 51 The results of these risk assessments are conducted and presented to policy makers to inform their risk management decisions. EPA currently has numerous policies that provide guidance to internal risk assessors on how to conduct a risk assessment and characterize risk. The EPA Risk Characterization Policy38 and associated guidelines are designed to ensure that critical information from each stage of a risk assessment is used in forming conclusions about risk and that this information is communicated from risk assessors to policy makers.

EPA Existing Policies and Guidance Current EPA guidance and policies incorporate quality principles. These are designed to ensure that critical information from each stage of a risk assessment is used in forming conclusions about risk and that this information is communicated from risk assessors to policy makers. One example is the EPA Risk Characterization Policy39 which provides a single, centralized body of risk characterization implementation guidance to help EPA risk assessors and risk managers make the risk characterization process transparent and risk characterization products clear, consistent and reasonable ( TCCR). These principles have been included in other Agency risk assessment guidance, such as the Guidelines for Ecological Risk Assessment. 40 Other examples of major, overarching guidelines for risk assessments include: Guidelines For Exposure Assessment 41, Guidelines For Neurotoxicity Risk Assessment, 42 and Guidelines For Reproductive Toxicity Risk Assessment. 43 Each of these documents has undergone external scientific peer review as well as public comment prior to publication. Additionally, individual EPA offices have developed more specific risk assessment policies to meet the particular needs of the programs and statutes under which they operate. 44 EPA's commitment to sound science is evidenced by our ongoing efforts to develop and continually improve Agency guidance for risk assessment. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA 4551 FR 33992 34054, 24 September 1986.

46Framework For Ecological Risk Assessment, U. S. EPA, Risk Assessment Forum, 1992, EPA/ 630/ R 92/ 001.

47Guidelines For Ecological Risk Assessment, U. S. EPA, Risk Assessment Forum, 1998, EPA/ 630/ R 95/ 002F. http:// cfpub. epa. gov/ ncea/ cfm/ ecorsk. cfm Appendix 52 EPA's Experience Conducting Risk Assessments The first EPA human health risk assessment guidelines45 were issued in 1986. In 1992, the Agency produced a Framework for Ecological Risk Assessment46 which was replaced by the 1998 Ecological Risk Assessment Guidelines. 47 As emphasized elsewhere in this document, the statutes administered by EPA are diverse. Although the majority of risk assessments conducted within the Agency are for chemical stressors, we also assess risks to biological and physical stressors. In addition to risk assessment guidelines, both the EPA Science Policy Council and the EPA Risk Assessment Forum have coordinated efforts to address the complex issues related to data collection and analysis for hazard and exposure assessments. Thus, the Agency has considerable experience in conducting both screening level and in depth assessments for a wide array of stressors.

Most environmental statutes obligate EPA to act to prevent adverse environmental and human health impacts. For many of the risks that we must address, data are sparse and consensus about assumptions is rare. In the context of data quality, we seek to strike a balance among fairness, accuracy, and efficient implementation. Refusing to act until data quality improves can result in substantial harm to human health, safety, and the environment.

Public Comments We received a range of public and stakeholder comments on the adaptation of the SDWA principles for " influential" human health, safety, and environmental risk assessments that are disseminated by EPA. Some commenters stated that we should adopt the SDWA quality principles for human health risk, safety and environmental risk assessments. Many commenters sought clarification on reasons for EPA's adaptation of the SDWA quality principles for human health risk assessments and additional information on how we plan to address this process. Others urged us to adapt the SDWA principles rather than adopt, because of certain elements in the SDWA principles that may not be applicable to all risk assessments such as a " central estimate of human risk for the specific populations affected." Others stated that we should neither adapt nor adopt SDWA principles because the " Data Quality Act" does not authorize importing decisional criteria into statutory provisions where they do not apply. The decisional criteria set forth in SDWA are expressly limited to SDWA. We also received comments at a level of detail that are more appropriate for implementation of the Guidelines than for the formulation of the Guidelines. These include comments regarding the use of clinical human test data, and comments regarding the use of particular types of assumptions in risk assessments. To the extent that an affected person believes that our use of data or assumptions in a particular Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 53 dissemination of information is inconsistent with these Guidelines, the issue can be raised at that time.

A few commenters raised a question regarding a conflict between EPA's existing policies and the SDWA principles and asked us to identify the conflicting specific risk assessment standards and make every effort to reconcile the conflicting standards with the SDWA principles. A few commenters stated that EPA should not have two separate standards for risk assessments ( i. e., one for influential and one for non influential), but that all risk assessments should be considered influential. Another stated that if there is a conflict between existing policies and the SDWA principles, EPA should identify the conflicting specific risk assessment standards and make every effort to reconcile the conflicting standards with the SDWA principles. Some commenters have questioned why the " best available, peer reviewed science and supporting studies" language of SDWA was conditioned by terms such as " to the extent practicable" or " as appropriate."

Adaptation of SDWA Quality Principles Public comments received by the Agency on the draft Guidelines were widely divergent. As no obvious consensus could be drawn, we carefully considered comments and arguments on adoption and adaptation. We also reviewed our experience with the SDWA principles, existing policies, and the applicability and appropriateness of the SDWA language with regard to the variety of risk assessments that we conduct and have determined that, to best meet the statutory obligations of the many statutes EPA implements, it remains most appropriate to adapt the SDWA principles to human health, safety, and environmental risk assessments.

In response to public comments we have removed " as appropriate" from these Guidelines in our SDWA adaptation. EPA agrees that the phrase peer reviewed science " as appropriate" was unclear. We revised this statement in part ( A) to " including, when available, peer reviewed science and supporting studies." EPA introduced such adaptations in order to accommodate the range of real world situations we address in the implementation of our diverse programs.

Numerous commenters expressed that EPA did not provide adequate clarifications of how we adapted the principles and what our thinking was on each adaptation. In these Guidelines we have provided detailed clarifications regarding each adaptation made to the original SDWA language and other remarks regarding our intent during the implementation of the SDWA adaptation for influential disseminations by EPA. We direct reader to the Guidelines text for such clarifications.

A. 3.7 Complaint Resolution A few commenters noted that EPA should outline how an affected person would rebut the presumption of objectivity afforded by peer review. EPA believes this determination would be made on a case by case basis considering the circumstances of a particular peer review and has Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 54 decided not to provide specific suggestions for affected persons on how to rebut the presumption of objectivity afforded by a peer review.

OMB and other commenters noted that agencies' guidelines needed to make clear that a request for correction can be filed if an affected person believes that information does not comply with the EPA Guidelines and the OMB guidelines. EPA has added language in the EPA Guidelines to make sure this is more clear to readers.

EPA received numerous comments on the EPA definition of affected persons. In the draft Guidelines, EPA had adopted OMB's definition. EPA agrees with comments suggesting that, instead of elaborating on the definition of " affected person," a more open approach would be to ask complainants to describe how they are an affected person with respect to the information that is the subject of their complaint. EPA is asking that persons submitting requests for correction provide, among other things, such an explanation. EPA has revised the Guidelines accordingly, so that we may consider this information along with other information in the complaint in deciding on how to respond.

Some commenters noted that the EPA Guidelines do not state how the process will work, specifically, for States, municipalities, and EPA. They expressed concern of being " caught in the middle," so to speak, on trying to get their own information corrected. EPA does not believe that the Guidelines needed greater details on how States will work with EPA to address complaints, but intends to work closely with States to better ensure timely correction. EPA does appreciate the frustration of an information owner in seeing what they deem " incorrect" information in a disseminated document or web site. However, EPA notes that this is a very complex issue that cannot be addressed with general language in the Guidelines for all cases.

Several comments indicated that EPA appears to have given itself " carte blanche" authority to " elect not to correct" information. The commenters stated that there was no valid reason why EPA would opt out of correcting information and that all errors should be corrected. To the contrary, EPA like every Federal agency wants to correct wrong information. The issue is not as simple as the correction of an improper zip code or phone number on the EPA web site. Even these simple errors may be very complex if it would involve changing data in an EPA and/ or State database. Furthermore, EPA is not certain of the volume of complaints it will receive after October 1 and therefore needed to provide a general provision in the Guidelines to recognize that once EPA approves a request, the corrective action may vary depending on the circumstances. On a case by case basis, EPA will determine the appropriate corrective action for each complaint. EPA determined that this was the most reasonable approach. The revision also recognizes practical limitations on corrective action for information from outside sources.

Several commenters noted that EPA needs to establish time frames for the complaint process. Commenters stated that EPA should establish time frames for when affected persons can submit a complaint on an information product, when EPA needs to responds to affected persons with a decision on discrete, factual errors, when EPA would respond to affected persons with a decision on more complex or broader interpretive issues, and when an affected person should submit a Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 55 request for reconsideration. One commenter suggested that EPA solicit all complaints at one time during a 6 month window or another time frame. EPA notes that commenters provided helpful examples and well thought out proposals for such a suite of time frames and appreciates the public input.

EPA did not agree on the need to develop two separate time frames for complaints that are more factual in nature versus those that are more complex. One commenter suggests a 15 day time line for discrete factual errors and a 45 day time line for all other complaints. Another commenter recommended 30 days for factual errors and 60 days for all other complaints. Another commenter advised EPA to model this complaint process according to the FOIA process. This commenter also suggested a 3 week time line for more numeric corrections and 60 days for " broader interpretive issues or technical questions." While EPA appreciates the value of these approaches, they might be problematic to implement. However, as EPA learns more about the nature of this complaint process following some period of implementation, these suggested approaches could be revisited.

EPA also agreed with commenters that a window of opportunity for commenters to submit a request for reconsideration made sense. EPA has advised affected persons in these Guidelines to submit a request for reconsideration within 90 days of the initial complaint decision by EPA.

Some commenters asked that EPA establish time lines for when EPA would take corrective action. EPA does not anticipate that there would be any value in applying a specific time frame for this action and prefers to look at each complaint and appropriate corrective action on a case by case basis, as discussed above.

Commenters suggested that 45 days was a reasonable time frame for EPA to get back to the affected person with either a decision or a notice that EPA needs more time. One group noted that HHS, SSA, and NRC adopted the 45 day window. EPA disagreed with this approach and instead opted for a 90 day time frame similar to the DOT Guidelines.

EPA received many comments on how EPA should structure its internal processes for the complaint resolution process. Several comments specifically discussed the role that OEI should play in the initial complaint and the requests for reconsideration. EPA does not agree that OEI should be the arbiter on all requests for reconsideration, but does view the role of OEI in the process as an important one. Namely, OEI may work to help ensure consistent responses to complaints and requests for reconsideration. Other comments recommending specific internal implementation processes are being considered as EPA designs the correction and request for reconsideration administrative processes in greater detail.

Many commenters argued that Assistant Administrators and Regional Administrators should not decide requests for reconsideration because they would be biased or would have a conflict of interest when deciding complaints regarding information disseminated by their own Offices or programs, or if they had to reconsider decisions made by their own staffs. EPA does not agree. This type of decision making is within the delegated decision making authority of EPA's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 56 officials, and these decisions should be presumed to be unbiased absent a specific showing that a decision maker is not impartial in a particular case. EPA does agree with commenters who noted that it is important to make consistent decisions on cross cutting information quality issues. In order to achieve appropriate consistency of response to affected persons on requests for reconsideration and to ensure that cross cutting information quality issues are considered across the Agency at a senior level, EPA intends for an executive panel to make the final decisions on all requests for reconsideration. Furthermore, we felt it important to add greater detail on the time frame within which EPA would respond to a requestor on their request for reconsideration. We have added that it is EPA's goal to respond to requesters regarding requests for reconsideration within 90 days.

EPA received many recommendations in public comments to include the public in the EPA complaint process. Specifically, commenters requested that EPA notify the public about all pending requests to modify information and one commenter stated that EPA should allow the public to comment on information corrections requests for information that are considered " central to a rulemaking or other Final Agency Action" before EPA accepts or rejects the request. As a general matter, EPA does not intend to solicit public comment on how EPA should respond to requests for correction or reconsideration. EPA also does not intend to post requests for correction and requests for reconsideration on the EPA web site, but we plan to revisit this and many other aspects of the Guidelines within one year of implementation.

EPA also received many comments on how information that is currently being reviewed by EPA in response to a complaint appears to the public on the EPA web site or some other medium. Some commenters recommended the use of flags for all information that has a complaint pending with a note that where appropriate, challenged information will be pulled from dissemination and removed from EPA's web site. Other commenters stated that the information in question should be removed from public access until the resolution process has been completed. Still other commenters requested that EPA not embark on self censorship. As a general rule, EPA has decided not to flag information that has a complaint pending. EPA believes that information that is the subject of a pending complaint should not necessarily be removed from public access based solely on the receipt of a request for correction.

A. 4 Next Steps EPA is actively developing new policies and procedures, as appropriate, to improve the quality of information disseminated to the public. Some activities specifically support ensuring and maximizing the quality, objectivity, utility, and integrity of information. For instance, we are consulting with the scientific community on the subject of reproducibility. The EPA Science Advisory Board ( SAB) is performing an expedited consultation on the subject on October 1, 2002. Based on this initial consultation, EPA and the SAB may consider a full review of reproducibility and related information quality concepts in 2003. Furthermore, as noted earlier, the EPA Science Policy Council has commissioned a workgroup to develop assessment factors for consideration in assessing information that EPA collects or is voluntarily submitted in support of various Agency decisions. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA Appendix 57 As new processes, policies, and procedures are considered and adopted into Agency operations, we will consider their relationship to the Guidelines and determine the extent to which the Guidelines may need to change to accommodate new activity.