Response: The EPA clarified 40 CFR 30.5 to give equal consideration to dose-response data that are publicly available or available through restricted access. As discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The final rule is limited to dose-response data and does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate the dose-response data underlying pivotal science. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data available. Researchers may choose to make more data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate. Further, EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
Further, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's SAB, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes.
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results As discussed in the preamble to the final rule, the rule requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Radiation Protection Scientific Peer Review
Comment: Commenter (6149) states that optimization of technical resources is critical in evaluating radiation protection scientific bases due to the limited availability of experts in the fields of radiobiology, radiation epidemiology, and radiation health physics. The commenter contends that the current system of review utilizing the National Academies, National Research Council, and National Council on Radiation Protection (NCRP) to leverage radiation protection expertise provides sufficient independent scientific review for regulatory action. The commenter states that, while the field of study is extensive and the breadth of regulatory impact is large, the number of experts in the fields of radiobiology, radiation epidemiology, and related health physics areas is extremely limited. The commenter notes that the National Academies, National Research Council, and NCRP have proven the capability to establish committees and provide qualified independent reviews of the scientific basis for radiation protection within this limited population of technical experts. According to the commenter, the EPA, nor any single agency, lacks adequate expertise of its own necessary to provide a meaningful independent review without engaging the resources available to National Academies, National Research Council, and NCRP. According to the commenter, these organizations are already performing this function, mandating an additional independent review by the EPA represents an additional regulatory burden without a commensurate benefit.
Response: As described in 40 CFR 30.5 of the final rule, the EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies -- based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. As described in preamble discussion of costs and benefits, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small.
Further, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's Science Advisory Board, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes.
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
<IP_ADDRESS>.2 Studies Can Be Judged Without Data Being Made Public
Comment: Commenters (0040, 0569, 1433, 1879, 1943, 2477, 2906, 4845, 5173, 6098, 6111, 6133, 6152, 6188, 6194, 6446, 6909, 6922, 8281-PH26, 9227) do not agree with the proposal's premise that open access to data needs to be included in the Scientific Review process to validate the quality of a scientific study. Some of these commenters provide examples (e.g., American Cancer Society's (ACS) Cancer Prevention Study II) of reviewed and replicated study results that support their position. Commenters (1973, 2170, 5165, 6046, 6122, 6133) assert that key information about studies is already publicly available under existing peer review and that studies relying on data that cannot be made publicly available can still be judged with regards to the quality and transparency of the interpretation of findings with existing peer review protocols. Comments include:
 While the rule's premise is that availability of individual level data will improve the quality of the science, validation and analysis of data represent only one aspect of the overall criteria used to judge the credibility of science. (0569, 5173)
 Public access to the underlying data is not necessary in order to further EPA's proclaimed interests in ensuring the validity of the scientific information relied upon and "the public's ability to understand and meaningfully participate in the regulatory process." (6922)
 Transparency is not a key test of the quality and legitimacy of a scientific study. The commenter states that legitimacy and quality is determined through peer review, reliance on the modern scientific method, and robust collaboration and debate within the scientific community. The commenter contends that the fact that the underlying data cannot be made public does not impact the validity of the results of these peer-reviewed studies. (1433)
 While researchers rely heavily and critically on primary research papers, there is no issue whatsoever evaluating the science in those papers without having access to the primary data. (1879)
 The process of good science does not require all data to be available, just a good experimental design to test the hypotheses with a thorough review of the methods used to reach the conclusions. (2477)
 The scientific journals that have implemented data availability policies have found that mere availability of data, without documentation of the methods used to collect, process, and analyze such data, provides little to help the stated aims of promoting reproducibility or public involvement in policymaking. The commenter notes that interoperability standards for collection and reporting must first be agreed upon within the scientific community and the burden of making data available in such a way as to meaningfully advance the goals of the rule must be assessed. (2906)
 The unspoken premise of the proposal is that unless EPA can guarantee full public access to a study's underlying data, the study must be deemed unreliable and should play no role in assessing a pollutant or chemical's effects on human health. The commenter argues that this premise ignores the many ways in which the scientific community, regulators, and the public have traditionally determined the quality and relevance of study results. The commenter provides that study reports typically explain the protocols used to gather data, the methods used for data analysis, the doses or exposure concentrations at which effects were and were not observed, the nature, severity, and incidence of such effects, and any unusual occurrences that may affect interpretation of the results. According to the commenter, this information plays an important role in the peer review process, informing the judgment of independent reviewers as to whether a study is worthy of publication in the scientific literature. The commenter states that Agency reviewers likewise consider these indicators of reliability in deciding how much weight a study deserves in making judgments about hazard and risk. The commenter states that, in its narrow focus on a single criterion for study acceptability, the proposal departs from this comprehensive, multi-faceted approach for determining the "best available science" to inform decision-making. (4845)
 Verification of method requires collection of new data, not use of the raw data. The commenter states that independent studies test the same method across many data sets. The commenter asserts that when you validate a protocol you are not testing the raw data within the system you are testing the system for accuracy using new data obtained in the same manner. (6098)
 The professional public health, medical, and scientific research community have long recognized that whether or not the raw data underlying a study is released does not determine the quality of the study. The commenter states that, rather, it is the scientific method that is determinative. The commenter adds that the proposed rule fails to take into account the fact that studies are reliable and constitute the best available science when they comply with professionally-established best practices for describing the methodology, sampling size, sampling procedure and assumptions utilized and the results are consistent with those of other studies. (6111)
 The proposed rule would replace EPA's existing successful process, which is in step with established scientific procedures, with one driven by the public availability of data, and unconcerned with well-established hallmarks of reliable scientific research, like peer review requirements. The commenter argues that this emphasis is inappropriate, given the courts' agreement that publicizing the data underlying studies upon which EPA relies "would be impractical and unnecessary." The commenter contends that the proposed rule would reduce, not enhance, the quality of data available to EPA as it considers important regulatory actions. (6188)
 EPA proposed this rule under the false premise that only scientific studies for which all underlying data is made publicly available can be adequately evaluated. The commenter states that this is simply not the case, and any discussion about mandating transparency through a bright line rule such as the one currently being proposed should be grounded in this fact. The commenter adds that, in addition, where necessary and appropriate, reviewers (as well as other researchers seeking to reproduce or extend scientific analysis) can have confidential access to key data in conformity with privacy requirements. (6909)
 The Agency's arbitrary, single-minded focus on considering studies for which certain data and models are publicly available (but only the dose-response studies relevant to health protective regulation, not the ones supporting registration of chemicals) stands in stark contrast to the way the scientific community validates research findings. The commenter states that the scientific community, and scientific journals look to a range of attributes when assessing the quality of a scientific study, including whether the study has been peer reviewed, whether the scientists used rigorous scientific methods, and whether the study's results have been reproduced or replicated. The commenter states that the scientific community recognizes that the quality of a study is not determined by whether the underlying data is publicly available and has long utilized a variety of tools for ensuring the integrity and rigor of research findings. (9227)
 The proposal fails to recognize is that disclosure of data addresses only one method of validating scientific research -- and a relatively less important aspect at that. According to the commenter, disclosure of data for a given study -- the focus of the proposal -- permits independent researchers to determine whether the data and methodology used in that study can be applied to generate the same results. The commenter notes that this may help protect against sources of error or misrepresentation in a particular study, however, both EPA and independent researchers have recognized that such reanalysis does not by itself validate a particular study. The commenter provides that, rather, a study's evidentiary weight rests both on the strength of its methodology, as well as whether similar results can be obtained by applying the study's methodology to a relevant, but different dataset or population, or by using a distinct methodology to interrogate the same hypothesis. (9227)
 Commenters (1973, 6122) assert that scientists already include model selection, assumptions, and uncertainty in work that is peer reviewed. Commenter (1973) adds that studies that have undergone the peer review process required for journal publication have also had to address comments based on review and comment by a group of experts unrelated to the study. Commenter (6122) asserts that the main point of peer review that is addressed under current peer review mechanisms is to validate the methodology and ensure results were interpreted correctly under the given assumptions.
 Commenters (2170, 6133) cite the following from a Science magazine article in support of their position that the merits of studies relying on data that is not publicly available can still be judged:
 Importantly, the merits of studies relying on data that cannot be made publicly available can still be judged. Reviewers can have confidential access to key data and as a core skill, scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results.
 Peer-review can and does occur without the study data being made public, including confidential data whereby the researcher can work with the peer-reviewers to provide datasets with study participant confidentiality ensured. (6046)
 While many peer-reviewed scientific journals have recently adopted policies that support data sharing, these standards recognize the array of workflows across scientific fields and make the case for data sharing at different levels of stringency that consider exceptional circumstances, such as data sets featuring personal identifiers and other data that cannot be shared openly. (6133)
 The scientific community has already adopted several protocols and guidelines specifically designed to improve the reporting and evaluation of studies and to improve the quality and transparency of the interpretation of findings. According to the commenter, these protocols and guidelines, such as CONSORT, ARRIVE and STROBE, do not require public access to all study data but still improve the scientific basis of evaluating studies. (1973)
 The following ESCAPE study as an example where the peer review process is completely transparent, without the need for privacy violations. (1973)
 For example, the ESCAPE study consisted of 22 cohorts for which the methodology was well documented in papers describing how the recruitment was done, who was eligible to be recruited, who was not eligible to be recruited, how many people were recruited, where the recruitment was done, what kind of data were collected on the people, including medical data and data on risk factors for disease, and they have all produced detailed summary statistics of those data, and their correlations. Separate papers describe in detail how the land use regression models that produced the exposure were fit, what the protocol was for collecting monitoring data, locating the monitors, what land use variables were used and where they were obtained from, etc. The statistical methods for fitting the land use regressions, and what variables were used were similarly described in detail. The papers that matched these two sets of data to examine the association of air pollution with health also described which data were used, how they were used, and what statistical models were fit. This is a completely transparent process, without the need for privacy violations. The same is true for the other cohort studies that EPA has relied on. They are perfectly transparent as to the nature of the population studied, how the recruitment was done, what data was collected on the individuals, and how the analysis was done.
Response: EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
As discussed in the preamble to the final rule, the requirements of the final rule build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty.
This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
Comment: Commenter (6922) notes that, for decades, EPA has relied upon epidemiological studies to help determine what pollutants or chemicals need to be regulated, and how rigorously to do so. The commenter states that the EPA's reliance on those studies was reasonable  -  even though the underlying data was not public  -  because the studies underwent peer review to ensure these studies and their results were sound, replicable, and reliable. In other words, the commenter asserts that not having the raw data did not, and does not, make the science unsound. To the contrary, the commenter contends that the studies that include or rely on confidential health information often are among the best science that the Agency has to evaluate the effects of pollutant exposure on humans, and the costs imposed on individuals and society as a result of that exposure. Similarly, the commenter provides that the EPA routinely relies on data provided by regulated industries that is treated as confidential business information (CBI) and is therefore protected from public disclosure because it may include customer or other personally-identifiable information, including electric and gas consumption. According to the commenter, this is no different from relying on public health studies that rely on data that is protected from disclosure for various reasons.
Response: EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty. As discussed in the preamble to the final rule, the requirements of the final rule build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. In addition, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (6133) states that the proposed rule does not require that any information actually be independently validated before EPA may consider it or base regulatory decisions on such verification. Accordingly, there is an irrational disconnect between EPA's insistence that information be "publicly available for independent validation" and the proposal's claim that this ensures EPA will consider and use the "best available science." The commenter notes that the EPA itself has not outlined a process by which "dose response data and models" would be validated, and the proposal does not seriously consider the methodological complications of partial redaction of underlying study data.
Response: As discussed in the preamble of the final rule, where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (6194) states that, where EPA alleges that the proposal is consistent with the focus on transparency in these and other laws, regulations, and policies, the Agency is really conflating standards of quality with public dissemination of data. The commenter states that, rather than recommend foregoing the use of the information based on confidential information, OMB provides agencies mechanisms for validating information when data are not available. The commenter states that, when it comes to its own research, EPA "strives to increase access to its research results" but recognizes that, "[w]hether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications."
Response: EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
In addition, the EPA clarified 40 CFR 30.5 to give equal consideration to dose-response data that are publicly available or available through restricted access. As discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
This rule build[s] upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (9227) asserts that the absence of publicly available underlying data does not make the results of a study invalid or even suggest that the study is likely to be invalid, nor has EPA presented evidence to suggest that studies with publicly available underlying data are more likely to represent strong science than studies without such data availability. The commenter states that key reasons why researchers do not make data for some studies publicly available have nothing to do with scientific quality. The commenter states that, while reanalyzing study results using the same data is one way to help validate those results, it is neither the primary nor a sufficient way to do so. The commenter states that EPA's apparent conflation of data availability and best available science is not based on any evidence cited by EPA, is contrary to the evidence before EPA, and is simply arbitrary. The commenter provides the following additional information regarding data availability and study quality:
   EPA's Preamble to the Integrated Science Assessments provides another discussion of how EPA evaluates study quality, and similarly, does not call out publicly available data:
 [T]he individual study quality is evaluated by considering the design, methods, conduct, and documentation of each study, but not the study results. This uniform approach aims to consider the strengths, limitations, and possible roles of chance, confounding, and other biases that may affect the interpretation of individual studies and the strength of inference from the results of the study.
 A statement by the American Statistical Association on p-Values: Context, Process, and Purpose further emphasizes the multiple considerations related to quality, stating "Researchers should bring many contextual factors into play to derive scientific inferences, including the design of a study, the quality of the measurements, the external evidence for the phenomenon under study, and the validity of assumptions that underlie the data analysis." Similarly, the letter filed by the Presidents of the NASEM in this docket lists multiple reports conducted since 2007 that have examined EPA's scientific assessment processes and "that advise EPA on the scientific bases of regulatory decisions related to human health and the environment." According to the NASEM Presidents, These reports encourage EPA to consider all available science in the rule-making process and provide guidance about how the agency could be more transparent in describing how evidence is gathered and evaluated. . . . Individual study quality should be evaluated based on information that is available in standard journal articles, such as the study design elements, analytical techniques, and statistical methods. Researchers may be contacted to answer questions about the conduct of the study or be asked to provide additional data. If the study data are not available, their absence may affect how the study is rated and used in the analysis, but the study should not necessarily be eliminated from the assessment.
 The commenter asserts that the proposal conflates transparency with quality, which is inconsistent with OMB's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which does not consider transparency an absolute requirement, provides that the objectivity of information that cannot be made publicly available can be verified through other types of "robustness checks," and provides an example of the Harvard Six Cities Study where reanalysis was achieved even when underlying data could not be made publicly available.
Response: This rulemaking is designed to build upon open data initiatives in the federal government and scientific community and advances the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
<IP_ADDRESS> Consistency with Existing Peer Review Policies
 <IP_ADDRESS>.1 Proposed Rule is Not Aligned with Existing Review Policies
Comment: Commenters (0559, 6117, 6375, 6464, 9227) express concern that the proposal is not aligned with existing policies.
Commenter (9227) states that EPA's Peer Review Handbook acknowledges that "EPA strives to ensure that the scientific and technical bases of its decisions meet two important criteria: (1) they are based upon the best current knowledge from science, engineering, and other domains of technical expertise; and (2) they are credible." The commenter states that this policy clearly impacts EPA's regulatory actions, and thus will be impacted by the Proposal. According to the commenter, the EPA fails to analyze the impact the Proposal will have on its ability to comply with this policy and fails to explain why it is changing course or justify its decision to do so.
Commenter (6375) suggests that EPA consider updating and utilizing its own peer-review policy in addition to that provided in the peer-reviewed scientific literature. The commenter suggests that, for proprietary data/CBI that are submitted to EPA to support new chemical registrations and other regulatory actions, EPA has full data access from which to conduct its own peer reviews in accordance with its own policy.
Commenters (0559, 6464) express that policies should be consistent with standards for determining accepted science that is used by the scientific community and other research agencies. Commenter (6464) states that establishing a different and arbitrary set of standards, would be overstepping the federal government's role.
Response: As discussed in the preamble to the final rule,  the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's Science Advisory Board, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes.
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
<IP_ADDRESS>.2 Requirements of this Rule as Compared to Peer Review in Environmental Statutes
Comment: Commenter (6133) states that under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). According to the commenter, not only are there no implied grants of authority to an agency in the other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language.
Commenter (6133) asserts that the EPA approach proposed in §30.7 is even more unlawful than would be the case, independently, under this case law. The commenter provides that the proposed §30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." 83 Fed. Reg. at 18,774/2. The commenter adds that the EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB Final Information Quality Bulletin for Peer Review that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. To the contrary, the commenter contends that treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference.
Commenter (6133) asserts that the Proposal identifies no statutory authority for EPA to conduct independent peer review for any, much less all, "pivotal regulatory science" consistent with the dense content (and exceptions) of the OMB bulletin. The commenter provides that the Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The commenter further provides that the Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB bulletin, notwithstanding that Congress has known about this bulletin since 2005.
Commenter (6133) states that when Congress writes statutes it knows how to create the legal authority for peer review and identify who shall conduct that peer review; including what role, if any, that EPA or other parties will play, and how that peer review may be conducted. The commenter asserts that not only do no provisions in the law authorize EPA's Proposal in §30.7, but existing environmental statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in §30.7.
Commenter (6133) provides the following examples in support of their assertion:
 CAA.
      §7511b. Federal ozone measures.
      [...]
      (g) Ozone design value study. The Administrator shall conduct a study of whether the methodology in use by the Environmental Protection Agency as of November 15, 1990, for establishing a design value for ozone provides a reasonable indicator of the ozone air quality of ozone nonattainment areas. The Administrator shall obtain input from States, local subdivisions thereof, and others. The study shall be completed and a report submitted to Congress not later than 3 years after November 15, 1990. The results of the study shall be subject to peer and public review before submitting it to Congress.
      42 U.S.C. §7511b.
      §7412. Hazardous air pollutants.
      [...]
      (p) Mickey Leland National Urban Air Toxics Research Center.
      [...]
      (3) Scientific Advisory Panel. The Board of Directors shall be advised by a Scientific Advisory Panel, the 13 members of which shall be appointed by the Board, and to include eminent members of the scientific and medical communities. The Panel membership may include scientists with relevant experience from the National Institute of Environmental Health Sciences, the Center for Disease Control, the Environmental Protection Agency, the National Cancer Institute, and others, and the Panel shall conduct peer review and evaluate research results. The Panel shall assist the Board in developing the research agenda, reviewing proposals and applications, and advise on the awarding of research grants.
      42 U.S.C. §7412.
 CWA.
      §1321. Oil and hazardous substance liability.
      (a) Definitions.
      [...]
      (27) the term "best available science" means science that-- (A) maximizes the quality, objectivity, and integrity of information, including statistical information; (B) uses peer-reviewed and publicly available data; and (C) clearly documents and communicates risks and uncertainties in the scientific basis for such projects;
      33 U.S.C. §1321.
 SDWA.
      §300g-1. National drinking water regulations.
      [...]
      (b) Standards.
      [...]
      (3) Risk assessment, management, and communication.
      (A) Use of science in decision-making. In carrying out this section, and, to the degree that an Agency action is based on science, the Administrator shall use--
      (i) the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and(ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
      (B) Public information. In carrying out this section, the Administrator shall ensure that the presentation of information on public health effects is comprehensive, informative, and understandable. The Administrator shall, in a document made available to the public in support of a regulation promulgated under this section, specify, to the extent practicable--
      (i) each population addressed by any estimate of public health effects;
      (ii) the expected risk or central estimate of risk for the specific populations;
      (iii) each appropriate upper-bound or lower-bound estimate of risk;
      (iv) each significant uncertainty identified in the process of the assessment of public health effects and studies that would assist in resolving the uncertainty; and
      (v) peer-reviewed studies known to the Administrator that support, are directly relevant to, or fail to support any estimate of public health effects and the methodology used to reconcile inconsistencies in the scientific data.
      [...]
      (12) Certain contaminants.
      [...]
      (B) Sulfate.
      (i) Additional study. Prior to promulgating a national primary drinking water regulation for sulfate, the Administrator and the Director of the Centers for Disease Control and Prevention shall jointly conduct an additional study to establish a reliable dose-response relationship for the adverse human health effects that may result from exposure to sulfate in drinking water, including the health effects that may be experienced by groups within the general population (including infants and travelers) that are potentially at greater risk of adverse health effects as the result of such exposure. The study shall be conducted in consultation with interested States, shall be based on the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices, and shall be completed not later than 30 months after August 6, 1996.
      42 U.S.C. §300g-1.
      §300j-2. Grants for State programs.
      [...]
      (d) New York City watershed protection program.
      (1) In general The Administrator is authorized to provide financial assistance to the State of New York for demonstration projects implemented as part of the watershed program for the protection and enhancement of the quality of source waters of the New York City water supply system, including projects that demonstrate, assess, or provide for comprehensive monitoring and surveillance and projects necessary to comply with the criteria for avoiding filtration contained in 40 C.F.R. 141.71. Demonstration projects which shall be eligible for financial assistance shall be certified to the Administrator by the State of New York as satisfying the purposes of this subsection. In certifying projects to the Administrator, the State of New York shall give priority to monitoring projects that have undergone peer review.
      42 U.S.C. §300j-2.
 RCRA.
      §6939a. Exposure information and health assessments.
      [...]
      (b) Health assessments.
      (2) Whenever in the judgment of the Administrator, or the State (in the case of a State with an authorized program), a landfill or a surface impoundment poses a substantial potential risk to human health, due to the existence of releases of hazardous constituents, the magnitude of contamination with hazardous constituents which may be the result of a release, or the magnitude of the population exposed to such release or contamination, the Administrator or the State (with the concurrence of the Administrator) may request the Administrator of the Agency for Toxic Substances and Disease Registry to conduct a health assessment in connection with such facility and take other appropriate action with respect to such risks as authorized by section 9604(b) and (i) of this title. If funds are provided in connection with such request the Administrator of such Agency shall conduct such health assessment.
      [...]
      (e) Periodic reports. The Administrator of such Agency shall issue periodic reports which include the results of all the assessments carried out under this section. Such assessments or other activities shall be reported after appropriate peer review.
      42 U.S.C. §6939a.
 CERCLA.
      §9604. Response authorities.
      [...]
      (i) Agency for Toxic Substances and Disease Registry; establishment, functions, etc.
      [...]
      Any toxicological profile or revision thereof shall reflect the Administrator of ATSDR's assessment of all relevant toxicological testing which has been peer reviewed. The profiles required to be prepared under this paragraph for those hazardous substances listed under subparagraph (A) of paragraph (2) shall be completed, at a rate of no fewer than 25 per year, within 4 years after October 17, 1986. A profile required on a substance listed pursuant to subparagraph (B) of paragraph (2) shall be completed within 3 years after addition to the list. The profiles prepared under this paragraph shall be of those substances highest on the list of priorities under paragraph (2) for which profiles have not previously been prepared. Profiles required under this paragraph shall be revised and republished as necessary, but no less often than once every 3 years. Such profiles shall be provided to the States and made available to other interested parties.
      [...]
      (7)(A) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of a health assessment, the Administrator of ATSDR shall conduct a pilot study of health effects for selected groups of exposed individuals in order to determine the desirability of conducting full scale epidemiological or other health studies of the entire exposed population. (B) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of such pilot study or other study or health assessment, the Administrator of ATSDR shall conduct such full scale epidemiological or other health studies as may be necessary to determine the health effects on the population exposed to hazardous substances from a release or threatened release. If a significant excess of disease in a population is identified, the letter of transmittal of such study shall include an assessment of other risk factors, other than a release, that may, in the judgment of the peer review group, be associated with such disease, if such risk factors were not taken into account in the design or conduct of the study.
      [...]
      (13) All studies and results of research conducted under this subsection (other than health assessments) shall be reported or adopted only after appropriate peer review. Such peer review shall be completed, to the maximum extent practicable, within a period of 60 days. In the case of research conducted under the National Toxicology Program, such peer review may be conducted by the Board of Scientific Counselors. In the case of other research, such peer review shall be conducted by panels consisting of no less than three nor more than seven members, who shall be disinterested scientific experts selected for such purpose by the Administrator of ATSDR or the Administrator of EPA, as appropriate, on the basis of their reputation for scientific objectivity and the lack of institutional ties with any person involved in the conduct of the study or research under review. Support services for such panels shall be provided by the Agency for Toxic Substances and Disease Registry, or by the Environmental Protection Agency, as appropriate.
      42 U.S.C. §9604.
 EPCRA.
      No mentions of peer review.
 FIFRA.
      §136w. Authority of Administrator.
      [...]
      (e) Peer review. The Administrator shall, by written procedures, provide for peer review with respect to the design, protocols, and conduct of major scientific studies conducted under this subchapter by the Environmental Protection Agency or by any other Federal agency, any State or political subdivision thereof, or any institution or individual under grant, contract, or cooperative agreement from or with the Environmental Protection Agency. In such procedures, the Administrator shall also provide for peer review, using the advisory panel established under subsection (d) of this section or appropriate experts appointed by the Administrator from a current list of nominees maintained by such panel, with respect to the results of any such scientific studies relied upon by the Administrator with respect to actions the Administrator may take relating to the change in classification, suspension, or cancellation of a pesticide. Whenever the Administrator determines that circumstances do not permit the peer review of the results of any such scientific study prior to the Administrator's exercising authority under section 136d(c) of this title to immediately suspend the registration of any pesticide to prevent an imminent hazard, the Administrator shall promptly thereafter provide for the conduct of peer review as provided in this sentence. The evaluations and relevant documentation constituting the peer review that relate to the proposed scientific studies and the results of the completed scientific studies shall be included in the submission for comment forwarded by the Administrator to the advisory panel as provided in subsection (d). As used in this subsection, the term "peer review" shall mean an independent evaluation by scientific experts, either within or outside the Environmental Protection Agency, in the appropriate disciplines.
7 U.S.C. §136w.
      §136w-8. Pesticide registration service fees.
      (a) Definition of costs. In this section, the term "costs", when used with respect to review and decision-making pertaining to an application for which registration service fees are paid under this section, means --
      (1) costs to the extent that --
      (A) officers and employees provide direct support for the review and decision-making for covered pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses;
      (B) persons and organizations under contract with the Administrator engage in the review of the applications, and corresponding risk and benefits information and assessments; and
      (C) advisory committees and other accredited persons or organizations, on the request of the Administrator, engage in the peer review of risk or benefits information associated with covered pesticide applications;
      (2) costs of management of information, and the acquisition, maintenance, and repair of computer and telecommunication resources (including software), used to support review of pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; and
      (3) costs of collecting registration service fees under subsections (b) and (c) and reporting, auditing, and accounting under this section.
      7 U.S.C. §136w-8.
 Toxics Substances Control Act (TSCA).
      §2625. Administration.
      [...]
      (h) Scientific standards. In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science, and shall consider as applicable --
      (1) the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information;
      (2) the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture;
      (3) the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented;
      (4) the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and
      (5) the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models.
      15 U.S.C. §2625.
      §2617. Preemption.
      [...]
      (b) New statutes, criminal penalties, or administrative actions creating prohibitions or other restrictions.
      (1) In general. Except as provided in subsections (c), (d), (e), (f), and (g), beginning on the date on which the Administrator defines the scope of a risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title and ending on the date on which the deadline established pursuant to section 2605(b)(4)(G) of this title for completion of the risk evaluation expires, or on the date on which the Administrator publishes the risk evaluation under section 2605(b)(4)(C) of this title, whichever is earlier, no State or political subdivision of a State may establish a statute, criminal penalty, or administrative action prohibiting or otherwise restricting the manufacture, processing, distribution in commerce, or use of such chemical substance that is a high-priority substance designated under section 2605(b)(1)(B)(i) of this title.
      [...]
      (f) Waivers.
      [...]
      (2) Required exemptions. Upon application of a State or political subdivision of a State, the Administrator shall exempt from subsection (b) a statute or administrative action of a State or political subdivision of a State that relates to the effects of exposure to a chemical substance under the conditions of use if the Administrator determines that --
      (A)(i) compliance with the proposed requirement of the State or political subdivision of the State would not unduly burden interstate commerce in the manufacture, processing, distribution in commerce, or use of a chemical substance;
      (ii) compliance with the proposed requirement of the State or political subdivision of the State would not cause a violation of any applicable Federal law, rule, or order; and
      (iii) the State or political subdivision of the State has a concern about the chemical substance or use of the chemical substance based in peer-reviewed science; or
      (B) no later than the date that is 18 months after the date on which the Administrator has initiated the prioritization process for a chemical substance under the rule promulgated pursuant to section 2605(b)(1)(A) of this title, or the date on which the Administrator publishes the scope of the risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title, whichever is sooner, the State or political subdivision of the State has enacted a statute or proposed or finalized an administrative action intended to prohibit or otherwise restrict the manufacture, processing, distribution in commerce, or use of the chemical substance.
      15 U.S.C. §2617.
      §2605. Prioritization, risk evaluation, and regulation of chemical substances and mixtures.
      [...]
      (b) Risk evaluations.
      [...]
      (E) Metals and metal compounds. In identifying priorities for risk evaluation and conducting risk evaluations of metals and metal compounds, the Administrator shall use the Framework for Metals Risk Assessment of the Office of the Science Advisor, Risk Assessment Forum, and dated March 2007, or a successor document that addresses metals risk assessment and is peer reviewed by the Science Advisory Board.
      15 U.S.C. §2605.
Response: EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (1981) states that, where the issue of raw data collection has been raised, the courts have permitted the EPA to use studies like the fish surveys. The commenter notes that in Coal of Battery Recyclers, the United States Court of Appeals for the District of Columbia Circuit held that the EPA could use non-public data in order to support its regulation of the amount of lead in ambient air. The commenter adds that, in another case using non-public data, epidemiological studies protected elderly Americans from cardiopulmonary disease caused by a contaminated respiratory environment. The commenter provides that the Uinta Basin ozone concentrations in Colorado were found by the EPA to exceed the NAAQS on the basis of non-public data.
Commenter (1981) states that, occasionally, studies use data that is not publicly available, often because the research focuses on medical issues and patient privacy is protected under other federal laws. The commenter provides that this research is reviewed by experts in the field before it is ever considered for policy development. The commenter states that it is both pragmatic and efficient to recognize that the EPA delegates review of raw data to experts. The commenter states that it was for this reason that in Coal of Battery Recyclers, the court made a distinction between the EPA's "reliance on a study's results from its reliance on the raw data underlying such results." The commenter contends that the proposed rule unrealistically suggests that the EPA cannot and will not utilize the peer review that it has delegated in cases like Battery.
Response: The rule does not preclude EPA from considering studies with non-publicly available data, so it does not conflict with those decisions.
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. This rule does not require the EPA to disclose or host data, but to determine if data and models are available and to give greater consideration to those studies for which data and models are available. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Further, although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (6194) states that EPA's own SAB, which Congress created in 1978 to provide scientific advice to EPA, 42 U.S.C. § 4365(a), challenged EPA's failure to solicit SAB's input on the proposal -- pointing out that it was not even aware of the proposal until it was published in the Federal Register and discussed in news articles. According to the commenter, given that EPA should have provided the proposal to another federal agency to review, EPA was also required to make the proposal available to the SAB, along with any relevant scientific and technical information on which the proposal was based. 42 U.S.C. § 4365(c).
Commenter (6916) asserts that EPA failed to confer with its chartered SAB, and seek its input, as required under the Environmental Research Development and Demonstration Authorization Act (ERDDAA), 42 U.S.C. § 4365(c)(1), even though EPA claims authority under the CAA, and admits in the proposal that it is a "significant regulatory action" submitted to OMB for review. The commenter states that, as EPA's SAB recently wrote to the Administrator, a proposal like this, which "focus[es] on the EPA's foundational policies related to the use of science in rulemaking and policy development," is within the SAB's purview and should have been submitted to it for review." The commenter states that EPA's failure even to submit the proposal to SAB review, never mind take account of the SAB review results, is evidence of the arbitrary and capricious nature of this whole undertaking.
Response: The EPA received comments from the SAB on the 2018 NPRM and 2020 SNPRM on April 24, 2020. The EPA considered these comments (EPA-SAB-20-005) in development of the final rule. In brief, the SAB consensus was that it supported the precept of enhancing public access to scientific data and analytical methods to ensure scientific integrity, consistency, and robust analysis and their consensus recommendations were that the EPA develop additional policy and/or guidance documents; further describe how pivotal science would be selected, made available, and peer reviewed; consider establishing an office on data sharing with the support of a peer review panel; provide greater clarity in the definitions of key terms, including data, independent validation, and publicly available; develop specific criteria for the Administrator's exemptions under the rule; consider applying rule requirements only to information developed after the effective date of the rule; seeking input from experts to identify best practices to ensure efficiency, utility, and protection of data that are made available; and consider funding reanalysis work.
As noted in the preamble for the final rule, this rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information and technical considerations, including those noted by the SAB, could be addressed in subsequent implementation guidelines and/or statute-specific rulemakings. Further, in response to comments from SAB and the public, the EPA clarified and modified several key definitions, including data, independent validation, and publicly available, in the final rule. The EPA also provided criteria for the Administrator's exemption in 40 CFR 30.7 of the final rule. The EPA would like to clarify that this rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. Comments related to making data available and future funding opportunities are not within the scope of this rulemaking. Finally, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. However, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that the development of the dose-response data was completed or updated before the effective date of the rule -- to 40 CFR 30.7 to account for the concerns raised by the SAB and other public commenters.
Comment: Commenter (6916) asserts that the proposal is not consistent with OMB and EPA Guidelines under the Data Quality Act. The commenter states that the Data Quality Act, in force since the early 2000s, directs transparency in analysis and a process for questioning the results of science used in Agency decision making.
Commenter (6916) asserts that longstanding statutory authorities govern the quality, objectivity, utility, and integrity of scientific data relied on by federal agencies, as implemented by policies and formal guidelines in place for decades. The commenter states that these authorities have been effective at ensuring the quality of studies relied on by EPA, offering a process for questioning study results and for the controlled reanalysis or use in new studies of sometimes confidential or proprietary human health data and models. The commenter notes that adding additional regulatory requirements and constraints is both impractical and unnecessary.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency.
These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. As described in preamble discussion of costs and benefits, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. Furthermore, as the final rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenters (6119, 6916) assert that the FOIA also offers a pathway for the release of data that have been the basis of Agency decisions. Commenter (6916) states that the FOIA exempts from "public" disclosure are "medical files and similar files the disclosure of which would constitute a clearly unwarranted invasion of personal privacy."
Response: The rulemaking does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (0844) notes that the EPA proposal is consistent with policies on scientific integrity espoused by previous administrations. The commenter states that, for example, President Obama encouraged, "transparency in the preparation, identification, and use of scientific and technological information in policymaking" in a March 2009 Presidential Memorandum.
Response: The EPA agrees that the Agency and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The requirements in the final rule build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
<IP_ADDRESS> Other Existing Means to Determine the Validity of Studies
 <IP_ADDRESS>.1 Administrative Procedure Act (APA)
Comment: Commenters (4534, 6108, 6153, 6171, 6358, 6446, 6901, 6922) suggest that the current process is already transparent because EPA rulemaking provides opportunity for public review and comment.
Commenter (4534) asserts that the comment period provides opportunities for industry to validate or challenge the assumptions that have been made based on the data used to develop the regulation. The commenter states that regulations' quality should also improve as the authors will know that industry will be validating their conclusions from the data provided.
Commenter (6108) states that, if EPA has concerns about whether to use a study because it appears that the conclusions are based on faulty science, it is the EPA's duty to investigate the study further and make its concerns known in the rulemaking process. The commenter adds that, if the public has concerns on a study used by the EPA, those concerns and any counter studies may be raised in the rulemaking process.
Commenter (6446) asserts that the ultimate safeguard to assure that EPA regulatory action is based on high quality science is the fact that EPA's regulatory action is subject to the notice and comment requirements of the APA (APA, 6 1.J.S.C. § 553) and ultimately to judicial review. The commenter states that, to the extent that scientific information proposed as a basis for regulatory action falls short, the APA provides members of the public, regulated businesses and other scientists an opportunity to comment to the agency about those shortcomings and to submit contrary studies and information for agency consideration. The commenter notes that, following adoption of a regulation, interested parties may seek judicial review to determine whether it is arbitrary, capricious or lacking in evidentiary support. (5 1.1.S.C. § 706.) The commenter asserts that the APA requirements, and the opportunity for judicial review of EPAs regulatory action, protect against the adoption of regulations that lack high-quality scientific support.
Commenter (6901) notes that states can openly question the validity of studies during the comment period for the proposed rule. The commenter suggests that the rulemaking docket provides open access and transparency now for states to examine the basis for new rules. The commenter contends that states can review what scientific studies were used in the preparation of the proposed rule and enough detail is provided to judge whether these studies support EPA's conclusions. The commenter adds that, during this time, states can also recommend additional studies that they believe EPA should consider in developing the final regulations. The commenter notes that, in the future, EPA can enhance the opportunity for input by consistently allowing a minimum 90-day comment period for new/revised rules. According to the commenter, early involvement by states, as co-regulators, in the early stages of the regulatory development process (pre-proposal) will allow states even more opportunity to provide input on the science used to support the new rules. The commenter adds that, beyond the science, involving states as early as possible in the regulatory development process means the resulting regulations can be effectively implemented and public health protection enhanced.
Commenter (6922) states that the EPA follows a robust public notice and comment process for its significant rulemakings. Where the public has valid reason to doubt the assumptions or data underlying a particular study, or is aware of a study that EPA has not considered, the commenter notes that the public can share conflicting studies and other information with EPA during the public participation process of the rulemaking. The commenter asserts that the EPA is obliged to exercise independent reasoned judgment to disregard information the Agency deems unreliable or biased. The commenter states that this is the traditional, well-settled process of reasoned administrative rulemaking.
Response: The EPA agrees that existing mechanisms within the EPA, the federal government, and in the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
As described in the final rule, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. In compliance with the rule, the EPA shall give greater consideration to evidence, findings, and conclusions of pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation. When the dose-response data underlying pivotal science are not publicly available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request.
Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts, peer reviewers (including the SAB and other science advisory panels), and the courts generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA is confident that this approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
<IP_ADDRESS>.2 Bradford Hill Criteria (BHC)
Comment: Commenter (4541) asserts that, given its constrained mission, flawed paradigm, political pressures on it to chase the impossible goal of zero risk, and evidence of actual corruption, there can be no confidence in any science produced by EPA in justification of its regulations. The commenter states that much of the controversy over the impact of air pollution on humans is due to the assumption by policymakers, regulators, and advocates that evidence of an association between a chemical in air or water and a human health effect is evidence that the chemical causes that effect. The commenter states that, because distinguishing between coincidence and correlation, on the one hand, and causal relationships on the other can be very difficult in matters of public health, an English epidemiologist named Sir Austin Bradford Hill (1897 - 1991) established in 1965 what has become known as the BHC, nine minimal conditions necessary to provide evidence of a causal relationship between a physical cause (in this case an air pollutant) and a health effect (illness or mortality). The commenter suggests that a better way to determine the validity of studies is to consider the BHC and other requirements of the scientific method.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The EPA's attention to data transparency is consistent with federal initiatives on open data and responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
<IP_ADDRESS>.3 FOIA
Comment: Commenter (6955) asserts that the proposal does not mention current FOIA procedures for providing access to researcher data from federal research and from privately funded research submitted to EPA for consideration. The commenter states that, hence, there is no discussion of the differences, between the two approaches and no claim that the new proposal offers any added value to the current situation. The commenter contends that, by omitting mention of the OMB and IQA FOIA options, the proposed rule gives the impression it is a public-access rule, rather than a study exclusionary rule.
Response: FOIA processes are not the subject of this rulemaking however, in response to comments received, the EPA is modifying the definition in the final rule to add the following at the end of the "publicly available" definition: "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
<IP_ADDRESS>.4 Good Laboratory Practices (GLP)
Comment: Commenter (0044) states that, under its GLP inspection program, EPA already has a program "available to the public for validation in a manner that honors legal and ethical obligations to reduce the risks of unauthorized and re-identification." The commenter provides that the following is taken from EPA's Good Laboratory Practices Standards Compliance Program:
      EPA conducts inspections as part of its GLP program to monitor compliance with the regulations to assure that studies submitted to the Agency in support of a pesticide registration or under a testing consent agreement for an industrial chemical were done with integrity, are of good quality and valid.
Commenter (6161) recommends the Agency adopt a requirement that studies that warrant consideration as key or pivotal studies conform with GLP or "the spirit of GLP" in order to promote the goal of relying on reproducible data. The commenter states that, whether in the "pure" sciences or regulatory sciences, the principal reason for requiring transparency of scientific data is to enable reproducibility studies and validation. The commenter states that, for the Agency to require transparency but ignore findings of irreproducibility defeats the purpose. The commenter notes that the Agency requires that scientific testing data submitted by industry as the basis for product registrations, hazard evaluations, or other purposes comply with GLP. The commenter states that the reason for the requirement is to ensure that the Agency's evaluations are based on data of sufficient quality, rigor and reproducibility. The commenter states that, recognizing that not all provisions specified by formal GLP programs can be applied to all studies, the concept of the spirit of GLP is often adopted by regulatory agencies and study sponsors. The commenter provides the following additional information regarding GLP:
      The Organization of Economic Cooperation and Development (OECD) defines GLP as ...a quality system concerned with the organizational process and the conditions under which non-clinical health and environmental safety studies are planned, performed, monitored, recorded, archived and reported. The definition of GLP makes clear that the principles of GLP are applicable to all scientific studies, not just to "guideline studies." In the environmental sciences, the regulated community often adopts a requirement that a laboratory, contractor, or grantee follow "the spirit of GLP" as a condition of funding a study that may serve a regulatory purpose. In practice, conforming with the spirit of GLP involves preparation of a protocol that describes the details of a study. The description includes safety precautions, lists of required equipment, sampling procedures, chains of custody, analytical methods, sample archiving, QA and QC measures, calculating and reporting results, and predefining and documenting criteria for including or excluding data to help avoid bias. As a study progresses, changes may be required. A GLP-compliant protocol describes procedures for amending the protocol to record the changes. In this way, protocols promote successful study replication and assist in the types of detailed peer reviews that can (and should) be a feature of regulatory science.
      In the environmental sciences, adherence to GLP principles is not uniform. Scientific studies sponsored by the regulated community to meet regulatory requirements are likely to conform with GLP. However, it has been PCTC's experience that studies in the environmental sciences conducted in some academic institutions and non-regulatory government agencies show little or no evidence of conforming with GLP. We have found protocols prepared by government agencies that would be rejected out-of-hand by EPA if submitted by industry in support of product registration or evaluation. Scientist-to-scientist requests and FOIA responses sometimes demonstrate the complete absence of protocols or even something as simple as an outline of a study design. Quality control measures for analytical chemistry are generally in place, but quality management for other aspects of environmental studies seems to be the exception rather than the rule.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information, and does not prescribe conformity with GLP or other laboratory practices. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Also, the rulemaking does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (2907) states that the proposed regulation directs the EPA to use industry-sponsored studies by requiring it to use studies that follow "standardized test methods" and "good laboratory practices." According to the commenter, these are industry standard practices, not academic research practices which use state-of-the-art methods. The commenter notes that standardized test methods are industry-sponsored studies designed to gain regulatory approval of the test substance. The commenter also notes that industry test studies have been required to follow good laboratory practices since the 1970s after flagrant test violations and fraud were identified. The commenter contends that, in contrast, academic research is the cutting edge of methods development, and must meet IRB approval, which sets high scientific and ethical standards of conduct.
Commenter (6109) notes that the proposal states that where "available and appropriate, EPA will use peer-reviewed information, standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable and reproducible scientific assessments." The commenter states that, with this statement, EPA is displaying a bias toward industry-backed research over research from independent and university scientists. The commenter states that GLP, a protocol that federal regulators apply to industrial laboratories, is infrequently used outside of industry because GLP is expensive and does not reflect science quality. The commenter states that GLP regulations mandate methods for archiving data, record keeping, equipment calibration, etc., but they do not influence research design or methodology. The commenter states that academic researchers (who are frequently federally funded), on the other hand, follow scientific processes that are subject to the rigors of peer review and independent replication. The commenter states that, if EPA is to truly use the "best available science," it must focus on studies that use appropriate protocols and techniques, not merely favored record keeping processes.
Commenter (6122) asserts that adoption of "Good Laboratory Practices" would prevent the EPA from using the best available science. The commenter states that independent research that does not follow GLPs could be deemed "qualitative" by the EPA and therefore, by definition under EPA guidance, not valid for use in establishing thresholds. According to the commenter, this "qualitative" research can be fundamental in establishing cause and effect, result in seminal publications, and often funded by the United States government. The commenter asserts that EPA must use the most sensitive study the passes the rigor of independent peer-review in establishing protective thresholds. The commenter contends that GLPs are not an acceptable way of judging scientific quality and should only be used as an inclusion criterion for research where there appears to be some incentive for fraud.
Commenter (6122) asserts that, while GLPs became guidelines to prevent fraud in industry science by requiring high numbers of replicates and model test systems, public health studies from universities or other non-industry sources do not need  --  and would not benefit  --  from GLPs. The commenter notes that providing raw data or using extensive replicates is rarely a requirement for scientific publication, yet published research is how most scientists communicate their findings and it is the way science is able to progress in every single field. The commenter adds that the precarious funding of most academic and governmental research precludes the inclusion of many GLPs, since following them makes it cost prohibitive to conduct studies but in no way makes conclusions less scientifically valid. The commenter contends that there is no reasonable or rational basis for ignoring a peer-reviewed, published study that does not follow GLPs. The commenter provides that a 2012 Scientific Advisory Panel on atrazine stated that: "
      In the view of the Panel, the test design elements [GLPs] should not be applied so strictly to the published literature as to disqualify all studies that do not meet all of these criteria...In the Panel's analysis, the EPA's strict application of the test design elements to the published literature was flawed and many of the test design elements should be relaxed for review of the published literature.
      "A study could be considered a high-quality study and very useful in risk assessment (even quantitative assessment), even if some of these design elements are not met."
Commenter (6940) does not support the proposed requirements to use GLP. The commenter asserts that GLP does not ensure "good" science or use of optimal study design and analyses, while government-supported, peer-reviewed science had evolved to improve reproducibility. The commenter states that optimal regulatory science depends upon both scientifically valid data and credible data. GLP is only designed to generate and analyze data following specific protocols. The commenter notes that peer-reviewed science, on the other hand, can address both data needs required for regulatory science. The commenter adds that this is true now, more so than ever, given that the National Institutes of Health (NIH) has new requirements for outlining components contributing to rigor and reproducibility. The commenter provides the following are two examples of the lack of scientific (vs. procedural) rigor inherent in GLP:
      First, toxicological assessments of bisphenol A (BPA) by scientists following GLP and those conducting experimental toxicology studies came to disparate conclusions over the low dose toxicity of BPA. However, the GLP studies failed to use a sensitive strain of mouse and failed to show effects of a positive control, compromising the scientific validity of the studies. Further, adverse outcomes measured in GLP studies of BPA tend to be of low sensitivity. On the other hand, hundreds of independent studies by multiple experimental toxicologists have confirmed the low dose toxicity of BPA.
      Second, GLP is used as an approach in pharmaceutical safety testing, where its shortcomings also have been revealed. Guth and Pugsley present an example of a GLP-compliant study of cardiovascular effects of a therapeutic drug using instrumented dogs (telemetry-based data acquisition). Significant flaws in the experimental design (e.g. use of non-naïve dogs, failure to acclimate the dogs to the instrumentation or the study environment, insufficient power in the study design, insufficient demonstration of the signal quality from the telemetry system) which led to high variability in the data and difficulties in interpreting the data.
Response: The EPA would also like to clarify that the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information, and does not prescribe conformity with GLP or other standard practices. The EPA is also not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider the availability of underlying dose-response data when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. Including this review of dose-response data availability for pivotal science is critical to ensuring confidence in the EPA's decision-making because it supports the scientific reanalysis and scrutiny by independent third parties for pivotal science. This approach will result in final significant regulatory actions and influential scientific information based on the highest quality studies that maximize transparency, leading to human health and environmental protections consistent with the statutes the EPA administers.
<IP_ADDRESS>.5 Implement Existing Guidelines, Plans, Policies, and Programs
 General
Comment: Commenter (4595) states that there exist federal guidelines that provide access to and ensure the quality of scientific information used for federal policies and regulation which also emphasize the necessity to maintain needed privacy or assurance of suitable confidentiality. The commenter states that there are also procedures for providing protected, limited access to the full raw data to independent researchers for independent validation without making the full data publicly available. The commenter states that these procedures negate the need for this proposed rule, as they respect privacy laws and bona fide practices used by the scientific community.
Response: The EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule, which will occur in forthcoming statute-specific transparency regulations or programmatic regulations, will increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (4839) states the proposed transparency rule is not based on scientific best practices, and it is inconsistent with transparency processes administrated by major business organizations, academia and the federal government. The commenter states that, for example, EPA proposes that "data and models" become publicly available and be subject to additional, time-consuming and expensive validation and analysis; by contrast, the American Petroleum Institute (API) has maintained for decades a transparency policy that enables a university investigator receiving a grant to retain ownership of the scientific data generated in a study it has funded. The commenter states that, similarly, the American Chemistry Council (ACC) has a long-standing policy through its Long-Range Research Initiative that provides external scientific investigators to retain ownership of their data even while encouraging these researchers to publish their results in the open scientific literature. The commenter states that EPA, other federal agencies and universities have long abided by policies with similar principles and practices.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15 (Ref. 18), which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency (Ref. 18). The EPA's attention to data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions (Refs. 19, 20, 21, 22, 23). The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
Comment: Commenter (6899) states that EPA's goal to strengthen regulatory science transparency is attainable without adding this regulation to the federal collection. The commenter states that EPA must require enforcement of the existing federal rules that strengthen this kind of transparency. The commenter states that, for example, the IRB process and the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Privacy Rule requirements exist to protect privacy, confidentiality, safety and security when conducting scientific studies and reporting scientific results. The commenter states that EPA and other federal agencies already spend a significant amount of taxpayer dollars to enforce these federally required processes.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. This rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. This rule does not impact rules developed and overseen by other federal organizations, including rules governing the IRB process and privacy under HIPAA.
Comment: Commenter (6122) contends that, even where EPA may have some legitimate concern about underlying data, there are existing approaches that address those concerns without this rule, such as allowing EPA itself to examine underlying data in a secure manner. In short, the commenter suggests that the EPA can ameliorate any concerns about underlying data either by continuing to address these issues on a case-by-case basis, or by developing guidance for reviewing underlying data in a manner that both preserves confidentiality and any other legitimate interests.
Response: As discussed in the preamble of the final rule, the EPA, the federal government, and the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Food and Drug Administration (FDA)
Comment: Commenter (5041) suggests that, instead of requiring all scientific research used to make EPA policy, regulation or guidance be based on publicly available datasets, EPA can adopt the long established and proven processes and procedures used by FDA. The commenter states that this ensures that privacy laws and concerns are met while having the most robust numbers of studies to ensure sound scientific policy/regulation/guidance making where the public may bear the cost or harm of said policy/regulation/guidance. The commenter states that the FDA decision making process to either approve or clear new/changed medical devices has a proven track record in ensuring that good decisions are made using the current regulatory policies, regulations, and/or guidance, i.e., publicly available datasets are not required. The commenter notes that, like the FDA, EPA uses studies that may include participant privacy issues, including ensuring protected health information (PHI), is not disclosed as required by law. The commenter provides that every study using human participants includes participant consent ensuring their privacy is ensured while stating that domestic and/or international regulatory agencies may have controlled/private access to the data when ensuring that the conclusion(s) made by the study sponsor are correct and meet the study objective(s) and endpoint(s). According to the commenter, peer-reviewed and published scientific research has also never required studies to be supported by publicly available datasets. The commenter notes that, while there are examples of conclusions being revised based on further independent review of the data or of researchers dry-labbing (falsifying) the data, we have a substantive history of the peer-review process ensuring reliable evidence is presented to the public.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. This rule does not impact rules developed and overseen by other federal organizations (e.g., FDA), including rules governing the IRB process and privacy under HIPAA. The EPA is requiring that, when promulgating significant regulatory actions or developing influential scientific information, the Agency will determine which studies constitute pivotal science and give greater consideration to those studies determined to be pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation. Focusing the final rule requirements on the underlying dose-response data is intended to address public comments concerning clarity of the rule, potential unintended consequences, and the potential for far-reaching impacts. The requirements provide a workable framework for EPA to evaluate pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
IRIS
Comment: Commenter (6133) asserts that the EPA chemical assessment program, called the IRIS already uses credible transparent methods to provide the public with reliable, transparent, credible chemical hazard assessments and toxicity values. The commenter states the program received high praise from its last two reviews by the NAS (NAS 2014 and NAS 2018), as well as from the SAB (SAB 2017) for its continuous improvements and successes in its methods for evaluating and integrating scientific evidence from various streams including human studies, animal studies, and mechanistic studies. The commenter contends that the proposed rule would undermine decades of expert work to advance successful data evaluation methods described in the systematic review approach now underway in the EPA IRIS program.
Response: The EPA disagrees that implementation of the final rule requirements would undermine the Agency's work. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA may still use pivotal science when the underlying data are unavailable after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
2013 White House Office of Science and Technology Policy (OSTP) Memorandum/EPA's Plan to Increase Access to Results of EPA-Funded Scientific Research
Comment: Commenters (6191, 6924) support implementation of the 2013 White House OSTP memorandum and EPA's Plan to Increase Access to Results of EPA-Funded Scientific Research. Commenter (6924) states that, to increase public access to science funded and used by the government, EPA should ensure it is in full compliance with the provisions of the OSTP memo. Commenter (9224) asks what is the status of the OSTP plan is and what does the proposed rule cover that this document does not regarding data funded and produced by EPA? Commenter (6191) states that the OSTP plan reflects an understanding of the need for maintaining balance between the desire to public access to data and the responsibility to protect data providers and their subjects; however, the proposed rule extends the intent for public access beyond reasonable expectations.
Commenter (6191) provides that two sections of the OSTP plan that address current sharing of information are:
      1. "EPA has a long history of collaboration in scientific research and is a leader in providing access to environmental information to encourage better decisions and a more informed public. Transparency is a core EPA value. The Agency already makes publicly available much of the Agency's scientific and technical work, including information that supports regulatory decisions. For example, EPA provides all materials and scientific information supporting each regulation in public dockets, which are publicly available for comment at www.regulations.gov. In addition, the Agency maintains an enterprise dataset metadata catalog (the Environmental Dataset Gateway (EDG) at https://edg.epa.gov/metadata/catalog/main/home.page) through which thousands of EPA datasets are publicly available. EPA's Enterprise Information Management Policy (EIMP), adopted March 3, 2015, codifies the Agency's approach to facilitating access to data held in EPA information systems. Appendix D describes, in more detail, a small sample of EPA's extensive ongoing efforts to be transparent and to increase access to environmental information." (page 4)
      2. "While the Agency strives to increase access to its research results, it recognizes, consistent with the OSTP Memo, that Federal agencies have a responsibility to protect confidentiality and personal privacy, respect proprietary interests and property rights, and balance between the value of providing long-term access and its associated costs. It is important to recognize that some research data cannot be made fully available to the public but instead may need to be made available in more limited ways, e.g., establishing data use agreements with researchers that respect necessary protections. Whether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications."