The ultimate safeguard to assure that EPA regulatory action is based on high quality science is the fact that EPA's regulatory action is subject to the notice and comment requirements of the Administrative Procedures Act (APA) (APA, 6 1.J.S.C. § 553) (APA) and ultimately to judicial review. To the extent that scientific information proposed as a basis for regulatory action falls short, the commenter states that the APA provides members of the public, regulated businesses, and other scientists an opportunity to comment to the agency about those shortcomings and to submit contrary studies and information for agency consideration. Following adoption of a regulation, interested parties may seek judicial review to determine whether it is arbitrary, capricious, or lacking in evidentiary support. (5 1.1.S.C. § 706.) According to the commenter, the APA requirements, and the opportunity for judicial review of EPAs regulatory action, protect against the adoption of regulations that lack high-quality scientific support. (6446)
 The proposed rule ignores a host of existing methods and best practices already established -- and adhered to -- by the research community to ensure the transparency, reproducibility, replicability, objectivity, and validity of studies, analyses, models, and reports. (6111)
 The EPA SAB review process already provides assessment of the adequacy of the approaches taken in the studies considered in setting a standard. (1973, 6125, 6446, 6447, 6920, 8801, 9227)
 Congress created the SAB to provide the Administrator with scientific advice. (42 U.S.C. § 4365.) The SAB is required by its authorizing legislation to make every effort to maximize public participation and transparency, "including making the scientific and technical advice of the [SAB] and any investigative panels . . . publicly available in electronic form on the website of the Environmental Protection Agency." The commenter notes that the proposed rule does not mention the SAB, or its important function in providing expert assessments of whether scientific research offered as the basis for regulatory action is, in fact, the best available science. (6446)
 SAB points out multiple ways in which EPA and the scientific community already ensure data quality, obviating the need for any new restrictions. For example, "[t]he proposed rule fails to mention that there are various ways to assess the validity of prior epidemiologic studies without public access to data and analytic methods." The commenter adds that the SAB describes one specific example in which "an unusually rigorous form of peer review and independent reanalysis, coupled with many follow-up studies, has accomplished a measure of confidence in findings without public access to data and analytic methods." In addition, "[t]he proposed rule fails to mention that EPA has mechanisms for vetting science through several expert panels," and that review process "goes beyond the typical journal peer review procedures." (6447)
 As part of the process for the evaluation and publication of scientific research, the commenter notes that all of these studies invariably undergo rigorous peer review by independent experts. Peer review is the world-wide gold standard for evaluating scientific research. The commenter submits that no study is published in any reputable journal without undergoing peer review. Thus, the commenter asserts that any pivotal study used for regulatory decision making has already undergone such review. Moreover, the commenter adds that, in the event that EPA determines that further review is necessary, its own expert panels, such as the SAB, exist to serve such a purpose. Unfortunately, according to the commenter, EPAs intentional crippling of its own SAB, involving the replacement of qualified independent experts by industry 7 employees with obvious conflicts of interest, damages the credibility of the SAB to do its job in an impartial manner. (8801)
 Commenters (4845, 6110, 6174, 6446, 6872, 6920) infer that the EPA is already transparent in its use of peer-reviewed research in regulatory science.
 The proposed rule ignores that much of the scientific research considered in the regulatory context, and all research published in reputable journals, has already been subject to extensive peer review. The commenter adds that, while the proposed rule would place judgments regarding the science that EPA may use in the hands of political appointees -- with its suggestion of an ad hoc mechanism for making exceptions to the rule's limitations -- the method that is accepted in the scientific community for assessment of the strength of scientific research is peer review. According to the commenter, the fact that scientific studies are regularly peer reviewed, including peer review by expert panels such as the SAB, renders the proposed rule unnecessary. (6446)
 The commenter adds that situations in which personal data need to be protected are clear and do not compromise the validity of the scientific studies. (6174)
 It is a standard practice for EPA to conduct comprehensive reviews of the best available scientific research when evaluating air and water quality standards. The commenter states that the EPA relies on peer-reviewed studies that describe the underlying data, methods, assumptions, sensitivity, and uncertainty of the results. These studies are cited and are published and available for public review. The commenter adds that reproducibility and independent validation are critical aspects of the scientific method and have resulted in significant advancements in our understanding of the health effects of air pollutants at different exposures and thresholds. (6920)
 The EPA also relies on independent advisory panels comprised of nationally recognized experts, such as the SAB, and the CASAC, to review and evaluate the state of the research and to provide an additional layer of independent peer-review. (6920)
 EPA science assessments generally include an exhaustive and critical review of relevant studies and a full explanation of how they are being interpreted. Extensive information about each study is typically part of the public record, even if all underlying data may not be included. EPA assessments are normally subject to public comment and independent peer review. And members of the regulatory community are free at any time to replicate studies they deem flawed or to independently seek access to underlying data and reanalyze them. In short, the "problem" that the proposed rule seeks to fix is largely imaginary. (4845)
 EPA already develops and implements policies, guidance, risk assessments, and toxicology assessments using detailed and transparent approaches that include extensive peer review and public input. (6872)
 There are already Health Insurance Portability and Accountability Act (HIPAA)-compliant and Institutional Review Board (IRB)-approved methods to transfer data between researchers so that studies can be evaluated and verified, and to determine if findings can be replicated. (5012)
 There exist federal guidelines that provide access to and ensure the quality of scientific information used for federal policies and regulation which also emphasize the necessity to maintain needed privacy or assurance of suitable confidentiality. Furthermore, according to the commenter, there are credible procedures for testing results and verifying outcomes with methodologies that do not require access to raw data. There are also procedures for providing protected, limited access to the full raw data to independent researchers for independent validation without making the full data publicly available. These procedures negate the need for this proposed rule, as they respect privacy laws and bona fide practices used by the scientific community. (4595)
 Several aspects of the proposal are already standard Agency practice. For example, the commenter states that the proposed §30.4 would require EPA to "identify all studies (or other regulatory science) relied upon when it takes any final agency action." According to the commenter, the EPA offers no evidence to suggest that the Agency has ever failed to identify studies or science that it relies upon. Similarly, the commenter states that the proposed §30.6 would require the Agency to "describe and document any assumptions and methods used, and [] describe variability and uncertainty." Again, the commenter asserts that this is standard Agency practice, and EPA has not provided any evidence to suggest that its staff has ever failed to follow this practice. (6447)
 EPA already has significant data sharing requirements and does not need more. (0671)
 Key information about studies is already publicly available, including study protocols, recruitment criteria, measurement techniques, and statistical modeling methods including lists of the adjustment variables have all been made publicly available, and peer reviewed. According to the commenter, these protocols and guidelines, such as CONSORT, ARRIVE and STROBE, do not require public access to all study data but still improve the scientific basis of evaluating studies. (1973)
 The Proposal ignores, or presents no argument for rejecting, scientific practices embodied in the host of nongovernmental methods and best practices established and adhered to by the research community to ensure the transparency, objectivity, and validity of studies, analyses, models, and reports, including by reproduction and/or replication where appropriate. (6194)
 The EPA's "Scientific Integrity Policy" provides guidance to Agency employees to ensure that science is used and communicated "with honesty, integrity, and transparency." Part of the policy to enhance transparency includes direction that:
 The Agency will continue to expand and promote access to scientific information by making it available online in open formats in a timely manner, including access to data and non-proprietary models underlying Agency policy decisions. Further, the use of nonproprietary data and models are encouraged, when feasible, to increase transparency. (6903)
Commenter (6446) asserts that the lack of any explanation for a need for the proposed rule is doubtless because procedures are already in place that assure use of the best available science for EPA's regulatory activity. The commenter submits that these procedures include the SAB established by Congress in 1978 to advise on scientific matters (42 U.S.C. § 4365.), existing independent peer review of much of the scientific research relied on for regulatory action, and legal requirements that prohibit adoption of regulations that are arbitrary, capricious, or unsupported by substantial evidence (e.g. 5 U.S.C. §. 706). When consistently implemented, the commenter states that these procedures and requirements, among others, assure that EPA's regulatory actions are based on the best available science and cast substantial doubt that the true purpose of this rulemaking has anything to do with strengthening the validity of regulatory science.
Commenter (9227) suggests that the Proposal neither acknowledges the mechanisms EPA already uses to ensure the integrity of science in decision-making nor establishes that there is a problem that the Proposal is needed to solve. The commenter submits that the reality is that both Congress and EPA have established an array of mechanisms and safeguards over the last five decades to ensure that the Agency's decisions are grounded in best available science. The commenter notes that these mechanisms include review of agency science and decisions by EPA's scientific advisory boards, including the SAB, the CASAC, Board of Scientific Counselors, the Science Advisory Committee on Chemicals, and the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) Scientific Advisory Panel -- a process that a work group of the SAB recently described as a "rigorous review process that goes beyond the typical journal peer review procedures," and that the National Research Council (NRC) recognized as playing an "important role in helping EPA to ensure the credibility and quality of . . . science-based decisions." The commenter adds that the Proposal also ignores EPA's use of independent peer review processes to evaluate certain studies used in regulatory decisions; the use of transparent literature surveys that are themselves subject to peer review and public comment, such as the ISA that inform the NAAQS; and independent review of EPA science programs and risk assessment practices by authorities such as the NRC. Lastly, the commenter states that major regulatory decisions -- and the underlying scientific bases for those decisions -- are also subject to public comment and judicial review, which serves as an important check on agency decisions that fail to properly account for the best available science.
Commenter (9227) asserts that thanks to these multiple and overlapping safeguards, the quality of the science underlying EPA decisions is robust. In addition, the commenter provides that numerous independent reviews of EPA's science-based actions by the courts, as well as the consistency with which the Agency has solicited and relied on the advice and approval of its external SAB committees have added to the credibility of EPA's decisions.
Commenter (6125) states that the EPA's past practice reflects the governing authorities far better than the current proposal. Before 2017, the commenter notes that the EPA over the years had developed and refined its approach to these-case-by case assessments, using a variety of methods that include examination of the data underlying studies where it is available, but without exclusive reliance on that examination, much less making it the foundation of a per se rule. Such examinations typically begin with EPA expert staff identifying and assessing peer reviewed studies and studies published in reputable scientific journals. This includes examining the strengths and weaknesses of individual studies, including factors such as design, the reputation and past work of the researchers, and quality assurance methods and analyses. This is followed by a broader look at the consistency and coherence of the study at issue with similar study types across multiple studies, as well as a more integrated assessment of the weight-of-evidence that considers multiple lines of scientific evidence. The assessments are in turn peer reviewed by panels of EPA's SAB as well as the public.
In conducting such assessments, the commenter (6125) provides that greater weight is given to the results of studies that have been replicated by other investigators using different data sets than to individual studies with limited or no replication. The commenter notes that replication refers to additional studies testing the same hypotheses as the original study, typically conducted by other investigators using different study designs and data sets, to see if the same conclusions are reached. As one of the authors EPA cited in the proposal recognized, replication is "the cornerstone of the scientific method." "Designing and conducting a replication study does not require access to raw data from the original study; this would abrogate the concept of independence." In reviewing the epidemiology studies supporting air quality standards, the commenter states that the EPA has placed greater weight on studies that have been replicated. In cases where one or two new studies appear to break new ground, the commenter notes that reanalysis by EPA or competent third-party investigators can provide some additional credibility, but not as great as that provided by replication.
Response: The EPA agrees that existing mechanisms within the EPA, the federal government, and in the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
As described in the final rule, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. In compliance with the rule, the EPA shall give greater consideration to evidence, findings, and conclusions of pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation. When the dose-response data underlying pivotal science are not publicly available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request.
Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts, peer reviewers (including the SAB and other science advisory panels), and the courts generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA is confident that this approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenters (6137, 6446, 6920) provide that the process for establishing NAAQS illustrates the scientific review procedures that are already in place for EPA's regulatory actions.
Commenter (6446) notes that in the NAAQS process, research results that are given the most weight are from the peer-reviewed literature. After EPA staff and their contractors review the literature, the commenter states that the CASAC of the SAB, which consists of internationally recognized experts in their scientific disciplines, reviews the EPA staff reports. The commenter notes that the Committee provides advice to the Administrator regarding the adequacy of current standards and recommendations for revisions, if necessary, for the protection of public health. The commenter adds that EPA staff reports also receive public comment, including from independent and industry scientists, and the Committee review includes consideration of those public comments. The commenter states that EPA NAAQS documents typically receive multiple rounds of expert scientific review before they are finalized.
Similarly, commenter (6920) provides that the NAAQS review process is a model of scientific transparency. Its reviews are renowned for their high quality. In regard to the studies on which they are based, the commenter states that the costs of publication as well as requirements for the privacy of study subjects have prohibited making all the data obtained publicly available. The commenter strongly disagrees that this diminishes the value of such epidemiology and toxicology studies to regulatory science and states that exclusion of such studies would significantly reduce the amount of scientific data available to establish appropriate standards to protect public health and the environment.
Commenter (6137) by way of example provides that the EPA's NAAQS process begins with preparation of an ISA, an extensive review of available science relevant to the development of NAAQS. See EPA, Integrated Science Assessment for Ozone and Related Photochemical Oxidants, EPA 600/R-10/076F, at 1i-1ii (Feb. 2013). The commenter adds that the preparation of the ISA is preceded by a public workshop and call for information. Id. EPA then collects and screens studies (with a heavy focus on studies that have been peer reviewed), prepares an initial characterization of evidence, and then provides a peer review process of the initial draft materials for scientific quality of "building blocks" from scientists from both outside and within EPA. Id. at 1vii. The commenter states that there is then preparation of draft syntheses of the studies and draft conclusions and causal determinations, followed by CASAC input and an opportunity for public comment before preparation of the final ISA. After that, the commenter notes that EPA staff prepares a Policy Assessment (PA) based on integration and interpretation of the findings of the ISA and a separate risk and exposure assessment (REA). The commenter provides that both the PA and REA are themselves subject to separate rounds of public comment and there is yet another round of public comment after EPA proposes its action on the NAAQS.
The commenter (6137) states that all of the foregoing comprises one of the most open, publicly accessible processes ever devised for the development of health standards. The commenter notes that there are multiple layers of peer review and more than ample opportunity for the public to raise questions about the adequacy and accuracy of the studies and models presented. The commenter expresses that the EPA does not and cannot rationally explain why this system requires yet another layer of complexity to provide adequate transparency, integrity, and public understanding of the process.
Additionally, commenter (6137) notes that the EPA's process for revising ambient water criteria for the protection of human health provides another example of the transparency afforded to its rulemaking, as it provides open access to information for interested persons. In 1998, the commenter provides that the EPA "improved" this process "to provide expanded opportunities for public input, and to make the process more efficient." 63 Fed. Reg. 68,354, 68,355 (Dec. 10, 1998). To revise its ambient water criteria, the commenter states that the EPA must follow a multistep process. First, EPA must "undertake a comprehensive review of available data and information" before developing draft criteria. Id. Second, EPA must "publish a notice in the Federal Register and on the Internet announcing its assessment . . . of the pollutant" which "describe[s] the data available to the Agency," and solicits "scientific views as to the application of the relevant Agency methodology." Id. Third, EPA must "utilize information obtained from both the Agency's literature review and [from] the public [comments] to develop draft recommended water quality criteria." Id. Fourth, and concurrent with the development of the draft criteria, EPA must publish in the Federal Register a notice soliciting the public's "scientific views on the draft criteria," and must "initiate . . . a documented critical review by qualified independent experts." Id. Fifth, EPA must then evaluate and respond on the record to all "[m]ajor scientific issues" (if any) raised during the peer review or public comment period. Id. Finally, EPA must "revise the draft criteria as necessary, and announce the availability of the final water quality criteria in the Federal Register and on the Internet." Id. According to the commenter, like the NAAQS process, this process too is fully open and transparent.
Moreover, the commenter (6137) contends that it would be irrational and impracticable to require EPA to seek out the underlying data for, and demand separate peer review of, all data and models the Agency uses for purposes of characterizing the quantitative relationship between dose or exposure and magnitude of a predicted health or environmental impact. For instance, the commenter notes that in the last ozone NAAQS review, EPA reviewed more than 4,000 studies and references for the ISA, and cited more than 2,200 in the final ISA. See EPA, Health and Environmental Research Online, ISA Ozone (2013) (last updated July 2, 2018), https://hero.epa.gov/hero/index.cfm/project/page/project_id/1628. The commenter asserts that it is neither practicable nor necessary for EPA to demand production of underlying data from thousands or even hundreds of studies, and to require new peer reviews of each and that EPA cannot construe the statute in a way that would render it impossible to complete the review and revision of the NAAQS that Congress mandated.
Response: The EPA agrees that existing mechanisms within the EPA include meaningful opportunities for peer review and public comment. Although the process and justifications for scientific decision making is clear, the public, subject matter experts, and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. Where underlying dose-response data in pivotal science are available, subject matter experts would be able to reanalyze the data and validate the original research findings, check for errors, test alternative assumptions, and/or better understand and evaluate the implications of the uncertainty. As the EPA better understands the data, the Agency can make stronger, more informed decisions in future rulemakings or in revisions to existing rules or influential scientific information. The incremental progress made by this rule provides an important step towards prioritizing transparency in the EPA's most impactful rules and assessments and will inform future statute-specific rulemakings. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent subject matter experts to validate pivotal science, and as data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Further, the EPA agrees that determining data availability for all the studies EPA considers in significant regulatory actions and influential scientific information may be infeasible at this time. The EPA modified the scope and the definition of "pivotal science" in the final rule to clarify that the requirements apply to the dose-response data underlying pivotal science and that "pivotal science" is a subset of the studies used in the analysis. The EPA also modified language in 40 CFR 30.6 to clarify that the Agency is not required to peer review to pivotal science that has already undergone adequate peer review.
The EPA would also like to clarify that this rule does not demand the release of underlying data. As stated in the preamble, this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The EPA is only required to determine if such a mechanism was available, should a subject matter expert decide to use it.
Comment: Commenter (6160) provides that, when assessing scientific evidence for policy and rulemaking, EPA employs a weight‐of‐evidence method that "considers all relevant information and its quality, consistent with the level of effort and complexity of detail appropriate to a particular risk assessment," according to the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency ("EPA Guidelines"). To insure that the information used in policy and rulemaking is accurate, reliable, and unbiased, the commenter states that the EPA Guidelines require the use of "(i) the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies; and (ii) data collected by accepted methods or best available methods ..." Furthermore, the commenter adds that the EPA Guidelines already address the transparency of data and methodology, including "(1) the source of the data used, (2) various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed."
In the General Assessment Factors for Evaluating the Quality of Scientific and Technical Information ("EPA SPC Assessment Factors"), the commenter (6160) provides that the EPA Science Policy Council has established five additional criteria to support the EPA Guidelines. These include:
 Soundness ‐ The extent to which the scientific and technical procedures, measures, methods, or models employed to generate the information are reasonable for, and consistent with, the intended application;
 Applicability and Utility ‐ The extent to which the information is relevant for the Agency's intended use;
 Clarity and Completeness ‐ The degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented;
 Uncertainty and Variability ‐ The extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods, or models are evaluated and characterized; and
 Evaluation and Review ‐ The extent of independent verification, validation, and peer review of the information or of the procedures, measures, methods, or models.
The commenter (6160) notes that the EPA utilizes these assessment factors to evaluate the scientific quality of individual studies that will contribute to the weight‐of‐evidence approach. According to the commenter, this balanced approach that is already in use and is consistent with existing scientific practice is more than sufficient to assess the quality of science that is relied upon for policy and rulemaking.
Commenter (6160) states that existing EPA scientific processes and assessment factors are part of various statutory frameworks, including the guidelines issued by EPA. The statutory basis for this proposed rule is not presented in a manner to support upending the existing EPA scientific processes and assessment factors.
Response: The EPA agrees with this comment in part and has clarified the role of the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency and assessment factors in the final rule. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. The EPA will generally continue to use the following, established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA also clarified that it is relying exclusively on its housekeeping authority to promulgate this purely procedural rule.
Comment: Commenter (6194) asserts that existing laws, regulations and guidance at the federal level (i.e., the Information Quality Act, OMB and EPA's guidelines implementing the Information Quality Act, Circular A-110, and the 2009 Memorandum for the Heads of Executive Departments and Agencies on Scientific Integrity) advance transparency-related goals and make the Proposal redundant.
Response: The EPA agrees that existing mechanisms within the EPA and the federal government are increasingly pushing towards greater data sharing and increasing scientific rigor. The requirements in this final rule build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (6920) notes that EPA also has existing policies and procedures in place to prevent conflicts of interest and ensure that the boards and committees are well-balanced and comprised of independent members with the appropriate expertise. The commenter provides that processes for developing human health assessments and setting and reviewing standards by EPA have been routinely scrutinized by organizations such as the National Academy of Sciences (NAS) and the Government Accountability Office (GAO) and EPA has incorporated their recommendations and improved its approach over time.
As examples, commenter (6920) notes that several of the landmark studies on the health effects of air pollution, such as the Harvard Six Cities and ACS studies, have been peer-reviewed and re-analyzed by multi-disciplinary expert panels from the HEI. The commenter describes HEI as an independent non-profit research institute that receives funding from both EPA and industry to provide impartial credible science on the health effects of air pollution. In a testimony before a Congressional committee, the commenter notes that the President of HEI stated that "US EPA and other agencies have established procedures to produce and review science for decisions, and in many cases those procedures work to enhance the quality and credibility of the science."
Commenter (6920) expresses that the proposed rule uses the phrase "best available science," but calls into question established processes such as EPA's Integrated Risk Information System (IRIS) review program and NAAQS program. In fact, the commenter states that the EPA routinely makes the `best' use of scientific information in these programs. That includes, according to the commenter, study-by-study evaluation of strengths and weaknesses of all relevant research. Also, the commenter asserts that the EPA more-than-adequately explains its decisions and analyses both in recommending NAAQS revisions and in quantifying chemical toxicities in IRIS.
Commenter (6920) submits that IRIS is not a regulatory program, but it provides essential scientific information for decisions made by Washington State Department of Ecology. In a recent review of the IRIS program, the commenter states that NAS reported that EPA has made "substantial progress" in implementing the recommendations outlined in previous NAS reports, improving the program's overall scientific and technical performance.
Commenter (6920) submits that IRIS uses a rational weight-of-evidence method to assess available research and provide access to a comprehensive source of toxicity data. The commenter notes that it increases the capacity to evaluate chemical hazards, and to quantify risk magnitudes and uncertainties, but it does not inform on how to manage risks. The commenter contends that the proposed exclusion of research lacking all underlying data from IRIS would be an unnecessary waste of information.
Response: The EPA agrees that scientific studies should not be categorically excluded from consideration when promulgating significant regulatory actions and developing influential scientific information and has clarified this in the preamble of the final rule. As described in the preamble to the final rule, the EPA will consider data availability, among other factors, when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. When the dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request.
<IP_ADDRESS> Recommend that EPA Continue to Implement Existing Procedures/Policies
Comment: Commenters (2119, 5172, 5178, 5977, 6119, 6160, 6351, 6924, 6925) urge EPA to continue to implement longstanding EPA procedures that promote use of sound science. Comments include:
 It is assumed that EPA's regulations are already and always based on the best science, which regulates itself in proving facts and providing solutions. (5977)
 Commenter urges EPA to follow the current, effective measures in place to ensure the use of robust, uncensored scientific research to protect the health of our patients and our communities. (6361)
 Commenter suggests EPA focus on implementing existing initiatives and guidelines for improving data sharing and transparency at federal agencies. (6924)
 Commenter urges EPA to continue to rely on the existing EPA weight-of-evidence process and assessment factors that are based in scientific standard practices. The commenter does not support a rule that departs from these important EPA-recognized approaches for assessing the quality of science in policymaking. (6160)
 There is an existing and well-established framework in place governing the use of scientific information to ensure that the information informing regulatory actions is the best-available, peer-reviewed science. The commenter provides that this includes EPA's Scientific Integrity Policy, OMB's Guidelines Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information, and Final Information Quality Bulletin for Peer Review. While EPA alludes to instances where EPA's own policies regarding scientific integrity have been followed inconsistently, the commenter contends that the EPA fails to provide a single example where this occurred and was problematic. Even if this were the case, the commenter asserts that it is unclear why this proposed regulatory action would be needed to cure such problems. The commenter submits that the solution instead seems simple  -  enforce the current well-established and well-reasoned framework regarding the use of scientific information  -  either through EPA's own internal procedures, or by OMB during review of regulatory actions. (6925)
Response: The EPA agrees with this comment in part and has clarified the role of the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency and assessment factors in the final rule. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. The EPA will generally continue to use the following, established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.
The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
<IP_ADDRESS> Support for Proposal to Address Transparency Problem and to Reduce Costs to the Regulated Industry
Comment: Commenters (0050, 0555, 0556, 0560, 0563, 0567, 1273, 1302, 1307, 1784, 1938, 1939, 1972, 2121, 2421, 2426, 2978, 3981, 4009, 4430, 4534, 4541, 4932, 5169, 5419, 6045, 6102, 6112, 6115, 6124, 6140, 6141, 6150, 6151, 6182, 6356, 6362, 6364, 6365, 6366, 6369, 6376, 6377, 6448, 6855, 6857, 6878, 6906, 6911, 6913, 6918, 6921, 6927, 6936, 6942, 6948, 9243) express support for the proposed rule. Commenters generally assert that the proposed rule will help ensure that only scientifically sound regulatory initiatives are considered for implementation. Some specific comments include:
 If the taxpayers are funding the research, the data and results should be open access and available to all. (1939)
 Transparency is especially important given that courts provide deferential review of agency decisions involving scientific or technical matters. (1972)
 Commenter supports EPA's full consideration of making its regulations consistently more transparent, timely and objective by balancing both costs and benefits. (6182)
 Transparent data and scientific debate are critical elements for developing objective and legally defensible regulations, in contrast to the cloistered and opaque science that has been practiced in the past. The commenter states that, without such transparency, the scientific or legal bases for rules are not understood by all and industry is often forced to into appellate litigation. The commenter states that non-transparency usually slows down, rather than expedites, proposed regulations from becoming law. (6182)
 Commenter applauds EPA's willingness to interact with stakeholders while developing this critical regulation. The commenter understands that implementation and application of this regulation is not clear cut and commends EPA on proposing such a challenging regulatory program. The commenter states that they are optimistic that through continued involvement with the public and stakeholder community, EPA will be successful in implementing procedures to ensure sound, science-based decisions in a manner that is verifiable and transparent. (6906)
 Scientific data used by the EPA to make regulatory decisions must be fully available to public analysis and scrutiny so that EPA decision making can be fair, democratic, and responsive to those people who use and rely on the multiple use of federal lands. The commenter states that scientific data veiled in secrecy will lead to politicized decisions tailored to a particular agenda and not sound environmental policy. (6878)
 Commenter supports the proposed rule's underlying values which originate from EPA's statutory authority and principles of good government, including the following:
 The NRC concluded that transparent and effective public participation is essential for EPA to issue valid rules that can be effectively implemented.
 EPA recognizes several reasons why public participation in government actions is important, regardless of statutory basis:
 The dialogue can result in deeper and more practical insights into the issues than if the interested parties acted individually.
 Those affected are far more likely to understand and accept decisions when their concerns have been acknowledged and addressed.
 Citizen participation in government programs is a democratic ideal.
 Pursuant to Executive Order 13563, EPA must base its regulatory decisions on the "best available science," and this goal can be significantly aided by informed public comments.
 EPA's Public Involvement Policy recognizes the importance of public participation to the Agency's core mission of protecting human health and the environment. (6918)
 The Transparency proposal is a vital and very important policy for the EPA that will have beneficial effects in that it will identify the nature and magnitude of the EPA research misconduct that has been intentionally and viciously put forward as good science for purely partisan ideological purposes, not the pursuit of science that allows the EPA to identify real risks and mitigate the effects of those risks. (4009)
 Commenters (6112, 6115) assert that poor rulemaking, particularly when it is not based on sound science, can have disastrous impacts on the aggregates industry. For example, commenter (6115) notes that the "Waters of the United States" (WOTUS) Rule that was proposed in 2015 lacked sufficient science, research methods, and data to support the stated purpose and revisions. In fact, the commenter states that the proposed revisions, as introduced, would have dramatically impacted aggregate operations in the nation with an increase in costs associated with expanding their operations, with the potential to be barred from mining future reserves that will be needed as our economy grows and our population continues to increase. The commenter states that, while their company prides itself as being environmentally responsible, the broadened scope of this WOTUS Rule would directly impact their operations, with little environmental benefit. In turn, according to the commenter, these proposed changes to the "Waters of the US" will increase the costs of public works projects across our nation.
 Commenter supports adoption of the proposed rulemaking because many "significant" and "lesser-significant" regulations have imposed (and will continue to impose) costly requirements and choices on the public but have lacked transparency and replicability with regard to their scientific/technical foundation. According to the commenter, not only has data for these decisions been withheld from public review, but the EPA is conspicuous among federal agencies for its reliance on proprietary data and models. The commenter adds that, not only are these agency practices inconsistent with EPA's own Scientific Integrity Guidance, but they also are inconsistent with research and publishing guidelines adopted by hundreds of peer-reviewed scientific journals and universities. The commenter states that their (NEDA/CAP's) members support peer-reviewed research that invites scrutiny from researchers worldwide with respect to the foundation, causality, and conclusions of that research. Most important, the commenter notes that their members support public access to the scientific and technical underpinnings of EPA regulation. (6356)
      As a general matter, the commenter (6356) suggests that the lack of regulatory transparency on the scientific foundation of EPA regulations has led to fevered political debate on EPA regulations that has impugned their integrity and acceptability by politicians, the regulated community, and the public. As a matter of course, the commenter expresses that the EPA has come to rely improperly on judicial decisions such as Chevron USA v. Natural Resources Defense Council, 467 U.S. 837 (1984), which vest the agency with broad discretion in interpreting ambiguous statutory authority and should be distinguished from its obligation to use the best available science in its decision-making. As a consequence, the commenter notes that stakeholders in regulatory development have been asked to trust EPA's conclusions on important health-based initiatives and not given the opportunity to review data, models, and interpolations of dose-response relationships on which they are based. The commenter asserts that, as EPA's regulations become more and more costly in attempts to achieve incrementally smaller reductions in pollution, it is critically important for the agency to take steps to avoid criticism that it regulates in secret.
      In short, the commenter (6356) states that the proposed rule generally tracks components of the scientific method in use by scientists and researchers since the seventeenth century. The commenter submits that it is important that EPA codify these principles to ensure that the data and models underlying scientific studies that are pivotal to regulatory actions are made be available to the public.
Response: The EPA agrees with commenters on the importance of data transparency to scientific scrutiny and debate in creating sound environmental policy. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent subject matter experts to validate pivotal science, and as data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (0556) suggests that the EPA rule is likely to, according to them, finally establish specific standards for openness. Moreover, the commenter contends that these standards will set a potential precedent for other Federal Agencies, possibly even other Governments, or even for scientific journals.
Commenter (6936) believes that the proposed rule is a major advancement in terms of EPA's compliance with the expectations before regulatory agencies. The commenter states that, by not disclosing relevant data and methodology behind studies used to justify EPA rulemakings, the reviewing public is often left only with the conclusions or findings of the given studies. The commenter provides that a recent Congressional Research Service report regarding judicial review of agency actions notes the following:
      [C]ourts require agencies to provide the `essential facts upon which the administrative decision was based' and explain what justifies their determinations with actual evidence beyond a `conclusory statement.' An agency's failure to provide an adequate explanation for its decision will typically result in remand or invalidation of its decision.
Commenter (6936) suggests that given the importance of independent verification and replicability of results in terms of confirming scientific conclusions, it is their view that EPA reduces the risk of its future rules being invalidated by the courts by adopting the proposed transparency standards.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions. Although other federal agencies may adopt similar standards, the EPA would like to clarify that this rule only applies to EPA practices.
Comment: Commenter (6140) attached a paper by Dr. Logomasini and comments offered by Marlo Lewis, Jr. The commenter states that the paper and comments provide details that explain their support for the Transparency Rule. That paper and comments are summarized below:
   A summary of the paper:
 The EPA rule on transparency should be viewed as another tool in a larger effort to promote transparency and reproducibility within the scientific community, most of which is happening outside of government.
 The EPA's transparency rule will help bolster private pro-transparency efforts and should be the beginning of a government-wide effort.
 Transparency guidelines could also be applied to government research and government-funded research within all agencies, from the EPA to the National Institutes of Health (NIH), the Food and Drug Administration (FDA), and the Consumer Product Safety Commission.
 The rule contains provisions to protect privacy and provide exceptions that allow regulators to consider quality research that is not fully transparent because of privacy concerns.
 Improving transparency is an achievable goal.
 Transparency ensures that all aspects of scientific methods and results are available for critique, compliment, or reuse.
 Researchers often avoid transparency out of fear that their novel findings may eventually be proven wrong when someone tries to replicate their work.
 Many positive associations will occur by mere chance or unintentional biases.
 The EPA's proposed rule follows principles laid out in 2017 by the bipartisan Commission on Evidence-Based Policymaking --  humility, transparency, privacy, capacity, and rigor -- and moves us toward providing greater access to scientific data while protecting individual privacy. The commission report addresses privacy issues, finding that transparency of data can be achieved without compromising privacy concerns:
 EPA has plenty of funds within its $8 billion budget to pay for transparency.
 Transparency might reduce costs for businesses and consumers, who ultimately pay the high costs of misguided regulations.
   A summary of the comments:
 Independent validation is difficult if the study's authors refuse to share their data. That is a longstanding problem in air pollution epidemiology.
 EPA is not proposing to require researchers to divulge data in past studies used to support previous regulatory decisions.
 The rule includes a provision allowing EPA to exempt significant regulatory decisions on a case-by-case basis by the Administrator
 Nothing in the proposal states or implies that EPA will not consider studies or comments referencing studies unless the data are fully transparent or the results are independently validated.
 EPA is within its authority to determine that transparent and reproducible science is of higher quality than opaque and irreproducible science.
 The agency is within its authority to accord greater weight to higher-quality science than to lower-quality science.
 If a study is pivotal to significant regulatory action, preference should be given to studies that can be independently validated. Independent validation is not possible unless the public has access to the underlying data and models.
Response: The EPA agrees that this rulemaking is designed to build upon other important transparency initiatives occurring within the federal government and the scientific community. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings. Although other federal agencies may adopt similar standards, the EPA would like to clarify that this rule only applies to EPA practices.
Comment: Other comments providing specific support and reasons for addressing transparency include:
 Commenters (0555, 0844, 6150) suggest that greater transparency in the studies, models, assumptions, and risk assessment policy choices used in regulatory decisions could encourage more open, constructive debate on those choices.
 Commenters (1974, 4430, 6102, 6172) support peer review and transparency.
 Commenters (1784, 3971, 4009, 4430, 4541, 5185, 6161, 6176, 6189, 6365, 6376) support the proposed rule and assert that some past, flawed EPA air quality research has been used as the basis of various regulatory actions.
 Commenter (4541) states that, given the agency's own admission that this failure is widespread within the agency due to a culture of disrespect for the scientific method and independent peer review, a new set of rules is desperately needed. Commenter (6161) states that EPA ignores the absence of transparency and findings of irreproducibility when it suits the Agency's desire to regulate. Commenter (6176) asserts that EPA is not immune to seeking preferred policy outcomes and using questionable science to achieve those outcomes; transparency helps to minimize these problems.
 Commenters (4931, 6102, 6108, 6126, 6140, 6141, 6150, 6448, 6857, 6878) support the statement in the proposal that "the best available science must serve as the foundation of EPA's regulatory actions."
 Commenter (3971) states that the proposed regulation is responsive to concerns expressed for the past three or four decades by many within the toxicology community. The commenter states that there are numerous examples of EPA risk assessments that would have been much more readily accepted by the regulatory community had they been more transparent.
 Commenter (5185) states that, for many years EPA has relied upon non-public data to justify its aggressive regulatory agenda. The commenter states that the most egregious but certainly not the only example of this involves two controversial studies undertaken in the 1980s that suggest a linkage between PM2.5 and health outcomes. The commenter states that the data associated with these decades-old studies has never been made public, but EPA has nonetheless used them to monetize regulatory benefit claims that dominate the communications and regulatory marketing associated with nearly all of its major rules.
 Commenter (6189) attaches articles and letters regarding PM2.5 and states that these are a highly relevant example of the importance of independent access to underlying data and transparency in regulatory science.
Response: The EPA agrees that scientific openness benefits both scientific and government processes. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (6365) states that the Lanphear case study demonstrates the need for EPA to finalize its regulatory transparency rule in the following ways:
 First, it shows that the peer review process does not always identify errors in studies that would constitute "pivotal regulatory science" within the meaning of EPA's proposed rule.
 Second, the absence of this data prevented EPA and stakeholders from discovering errors in "pivotal regulatory science" at the time EPA relied on it.
 Third, it shows that absent a regulatory transparency rule, it could take stakeholders years of effort and untold expenditures to obtain data that may demonstrate errors in "pivotal regulatory science."
 Fourth, its shows that making this data available will facilitate the validation and replication of pivotal regulatory science by reducing barriers to obtaining these materials, furthering (not hindering) public health science.
Response: Although this rulemaking is not intended to remedy a particular instance in which the EPA relied on a poor-quality study or a study for which the conclusions could not be independently validated, it does provide important incremental progress toward the Agency's goal of greater transparency. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. In addition, as data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (6376) states that, under President Obama's Administration, the EPA pursued an aggressive regulatory agenda based on single sided scientific studies and modeling. The commenter states that, to provide a real-life example of this misapplication of scientific studies one can look at the Boiler Maximum Available Control Technology (MACT) rulemaking. The commenter states that, when setting the emission standards for the best available technology for boilers the Agency "cherry-picked" data from various boilers and used that data to inform the standard for all boilers. The commenter states that the problem is the final standard was achievable for a hypothetical boiler rather than real life boilers. The commenter states that, not only was the result an unattainable standard, but the difference in the cost estimate from EPA and the regulated community was drastic. The commenter states that EPA estimated it would cost $3 billion to come into compliance whereas industry estimates put the cost at $20 billion.
Response: Although this rulemaking is not intended to remedy a particular issue, the EPA agrees that the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods is great; therefore, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment.
Comment: Commenters (0563, 1486, 1939, 2142, 3134, 6350, 6374, 6878) support the proposal that the science used to inform the making of regulations be publicly available for independent verification. Commenter (6878) asserts that, as an agency of a democratically elected government, the EPA should ensure that the American public has full access to its decision-making processes. Commenters (1486, 2142, 6350) assert that evidence is the bedrock of the Scientific Method, and it must be observable and measurable and available to all who wish to verify, validate, or disprove the hypothesis which the evidence is claimed to support. Commenter (6350) states that scientific conclusions said to be made based on evidence that is neither public, repeatable, nor reproducible, are neither scientific, nor conclusive.
Response: The EPA agrees that scientific openness benefits both scientific and government processes. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment.
Comment: Commenters (1939, 0062, 4430, 6045, 6140) support the proposal and agree that, regarding medical information, there are methods available to protect the confidentiality of individuals whose data would be used in environmental studies.
Commenter (6045) states that, while the EPA might agree to redact these data before public posting, they could be given to anyone who signed a confidentiality agreement designed by the agency. The commenter states that all NIH clinical studies can only be approved for funding if the investigators "surrender personally identifiable information  --  as it pertains to statistical reevaluation of the data and corroboration of the methods employed to study such data, which in turn might lead to public regulatory policy." The commenter states that EPA clinical studies should be held to the same high standards as all NIH clinical studies.
Commenter (4430) states that EPA's proposed rule follows principles laid out in 2017 by the bipartisan Commission on Evidence-Based Policymaking  --  humility, transparency, privacy, capacity and rigor  --  and moves us toward providing greater access to scientific data while protecting individual privacy.
Commenter (6140) states that, while some claim that the rule will prevent EPA from using research that relies on confidential data collected from study participants, the rule affords the EPA Administrator considerable leeway to permit regulators to use research in cases where privacy or other concerns limit public availability.
Response: The EPA agrees that mechanisms exist to share data while maintaining protections for sensitive information. As discussed in the final rule, underlying data or models are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Further, the EPA would like to clarify that the rule does not prevent the Agency from using research with confidential data and does not rely on an Administrator's exemption to be able to consider such research. As explained in the preamble for the final rule, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenters (0019, 4430, 6102, 6150, 6161, 6910, 6938) support the proposal that scientific studies used by EPA be available in a format that allows other interested parties to replicate or reproduce the results. Commenter (0019) asserts that, if this is not done, EPA could make decisions on research that may be flawed.
Commenter (6102) states that increased transparency and access to underlying decision-making information encourages replication and validation, key steps in the scientific process. Commenter (6150) states that replication and validation are key tenets to scientific inquiry and promotion of data sharing will allow testing of new ideas on existing data sets. Commenter (6161) states that the reason transparency is paramount in the scientific enterprise is that clear exposition of materials and methods is necessary to allow attempts to replicate studies. Commenter (6910) states that, in order for science to be accepted and utilized in policy making and consumer choices, science has to be reputable, repeatable, and factual. Commenter (6938) states that too often in the past, EPA has relied upon public health studies with non-public data to justify regulatory policy and without access to the non-public data, it is impossible to replicate and validate the conclusions in those studies.
Response: The EPA agrees with commenters on the importance of data transparency to scientific scrutiny and debate in creating sound environmental policy. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenters (0039, 1974, 2114, 4537, 4882, 6112, 6115, 6140, 6162, 6172) support the proposal that regulatory decisions be based on reproducible and transparent research.
Commenters (2114, 4882, 6140) assert that the scientific method requires research to be transparent and reproducible. Commenter (0039) believes that these reforms will strengthen American science, by prompting researchers to incorporate and make routine in their practices the highest standards of reproducibility.
Commenter (4537) states that larger concerns about reproducibility suggest that this measure should be applied more generally within EPA and across the Federal Government.
Commenters (6112, 6115) state that because rulemaking and other regulatory actions by EPA and other agencies have enormous impacts, they should be able to clearly understand the rationale behind EPA actions.
Commenter (6162) states that, while EPA has not consistently followed the current EPA Scientific Integrity Guidance that encourages use of non-proprietary data and models, the proposed rule requires increased access for outside researchers to independently reproduce the underlying analyses on which environmental rulemakings are based.
Response: The EPA agrees with commenters on the importance of data transparency to scientific scrutiny and debate in creating sound environmental policy. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will also increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. Although other federal agencies may adopt similar standards, the EPA would like to clarify that this rule only applies to EPA practices.
Comment: Commenters (4541, 6150, 6878) support the proposal and assert it is consistent with statutes and/or Executive Orders. Commenter (6150) states that EPA's proposal to enhance transparency and to facilitate the use of scientific data that are publicly available in a manner sufficient for independent validation is in keeping with its obligations under federal statutes. Commenter (6878) states that the proposed rule will make EPA decision making more consistent with the requirements in the Administrative Procedures Act (APA) that ensure public participation in the rulemaking process.
Commenter (4541) states that governing statutes and Executive Orders authorize the proposed Rule and, indeed, have long required it. The commenter states that, in not following the proposed rule in the past, EPA has been flouting governing statutes and Executive Orders, particularly in matters of toxicology and epidemiology, and abusing its authority by ignoring and discarding valid objections to EPA adopted science and the derivative policy and regulatory actions.
Response: The Agency agrees that the EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The requirements in this rule build upon the open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Implementation of this rule will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenters (0563, 1784, 1939, 0844, 3971, 6102, 6182) support the proposal because it could reduce costs.
Commenters (0563, 0844, 3971, 6102) support the proposal because it could reduce costs of regulations that might be based on data that are not publicly available.
Commenter (1784) states that, with the cumulative economic impact of federal regulations reaching nearly $2 trillion per year, research reform is essential.
Commenter (3971) states that, if EPA can gain more support for its proposals, which this regulation will do, the number of legal challenges would be lessened and billions of dollars in legal fees and awards would be avoided.
Response: The EPA would like to clarify that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. Because this rule does not require the EPA or other parties to disclose or host data, this rule does not impose costs on the EPA or any other party to make such data available. As more fully discussed in the preamble to the final rule, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. However, the EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
<IP_ADDRESS> Proposed Rule Will Not Deliver Improvements in EPA's Current Regulatory Processes
Comment: Commenters (2119, 4589, 4589, 4590, 5170, 6111, 6157, 6160, 6168, 6872, 6924) assert that the proposed rule will not deliver improvements in EPA's current regulatory processes. Commenters (5176, 5179, 6158) assert that the goal of increasing transparency in science and the public's faith in the EPA is undermined by the proposal. Comments include the following:
 EPA calls for transparency, many features of the proposed rule themselves are not transparent. The commenter contends that ambiguity and lack of detail lend themselves to decision-making that is arbitrary and open to influence, i.e. the opposite of transparent or based on best available science. The commenter notes that the rule never addresses the wide range of measures that would be necessary to achieve true transparency for either scientists wishing to replicate results or for a broader public wishing to better understand and participate in the regulatory process. The rule proposes just a single measure -- making data and models available -- and proposes a false equivalence between "transparency" and (poorly defined) independent validation. The commenter adds that the proposed measures on data and model availability and use facilitate not transparency but the culture of science denialism: dismissing important evidence and dredging data to endlessly produce false uncertainty and delay scientific advances regarding the harmful effects of chemical exposures. The commenter notes that this false uncertainty is used to justify failure to protect environmental and public health. (6168)
 The proposed rule will do little to nothing to improve the transparency. (6872)
 The proposed rule would not improve data sharing, replicability, or transparency in decision-making. (6924)
 The approach advocated in the proposed rule is inconsistent with professional best practices in their respective disciplines for conducting, reviewing, and confirming the results/findings of studies, especially those based on confidential personal health data of study participants. (6111)
 The proposed rule runs counter to the recommendations of leading scientific organizations like the NAS. (6158)
 Commenter urges EPA to abandon its plans to advance this misguided proposal that would undermine sound scientific norms in the rulemaking process. (6157)
 The emphasis of the proposed rule on making data sets publicly available fails to demonstrate any improvement on the existing process and adds nothing to the evaluation of the scientific merit of any piece of evidence. (6160)
 Because of the peer-review process, EPA already has a firm scientific basis for using studies to protect public health. The commenter adds that making the raw data available provides no improvement to the robustness of the studies. (5170)
Response: In response to public comments, the EPA clarified and modified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and the final rule. As described more fully in the preamble to the final rule, the EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The provisions of this rule prioritize the transparency of dose-response data in pivotal science, while still ensuring that all quality, relevant literature is considered in the development process.
2.1.3 Proposed Rule is Biased
 <IP_ADDRESS> Takeover of EPA Leadership
Comment: Commenter (0844) cites The Washington Post quote of the Union of Concerned Scientist's (UCS's) Center for Science and Democracy Director Andrew Rosenberg, who said, "[f]irst, they came after the agency's independent science advisers, and now, they're going after the science itself...what is transparent is the unabashed takeover of EPA leadership by individuals who have demonstrated disinterest in helping communities combat pollution by using the best available science."
Response: The EPA disagrees with this commenter. As discussed more fully in the preamble of the final rule, the federal government and the scientific community have been pursuing greater transparency and data sharing for years. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in an assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA is very interested in public engagement and, as data are better understood as a result of the increased opportunity for independent experts to validate pivotal science afforded by this rule, the EPA will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
<IP_ADDRESS> Implementation Bias
Comment: Commenters (1006, 2787, 6877) note that proposal could lead to implementation bias. Comments include:
 The Proposal seems like a way for the EPA to have say over what gets implemented by what gets approved with any easy excuse of withholding of data. (1006)
 Contrary to its name, the proposed rule would implement an opaque process allowing EPA to selectively suppress scientific evidence without accountability and in the process undermine bedrock environmental laws. (2787)
 Discriminating what science to use for drafting regulation risks introducing potential bias in the rulemaking process. The commenter suggests that bias is likely to be introduced through the process of identifying what science can and cannot be used because there is not a bright line for what studies this proposed rule would include or exclude. Specifically, the commenter points out that the requirement to make a decision for what constitutes "high quality studies" is problematic because the term is not defined in the proposed rule nor are operational criteria for its definition specified. The commenter provides that recent work in systematic reviewing of research in many areas of science has demonstrated considerable variation in judgments about "quality" by competent scientists unless the criteria for quality are specified precisely. The commenter suggests that this variation in judgement risks arbitrary and capricious decisions as to what evidence to include or exclude. The commenter notes that several good resources for avoiding bias and gaming of the regulatory process in specifying the standard for "high quality" are available. The commenter concludes that the exclusion of otherwise valid and reliable research from consideration would also often result in regulatory analyses that suffer from biased selection and inadequate statistical power. (6877)
Response: The EPA considered these and other similar comments when developing the final rule. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. Where there is convincing and well-substantiated evidence to support a relationship between exposure and effect, the EPA will identify a subset of those studies -- based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- in either final significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of personally identifiable information (PII), confidential business information (CBI), or propriety information, available through restricted access in a manner sufficient for independent validation. Including this review of data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and scrutiny by independent third parties. This approach will result in final significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. The EPA does not believe that this final approach will lead to bias.
The EPA intends for the final rule to be implemented in a transparent manner. The final rule requires the EPA to document the rationale for exemptions granted by the Administrator to the requirements included in the final rule in the final significant regulatory action or final influential scientific information. Additionally, the EPA expects to identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment or other input to the EPA.
<IP_ADDRESS> Injects Politics into Science
Comment: Commenters (0040, 0055, 1301, 1395, 2427, 2492, 2907, 4881, 6122, 6152, 6158, 6895, 6903, 6909, 6914, 8274, 9225, 9227) suggest that the proposal is biased in ways that will facilitate political interference in science-based decision-making. Comments provided include:
 Asserts that billion-dollar industries make so much profit that they can spend unlimited amounts of money in lobbying to erode environmental protections that are in the civic best interest. The commenter states the proposal throws away merit-based, evidence-based policy decisions and opens up government policy for hire. (0040)
 Commenter (1301) asserts that the proposal is an attempt to use the concept of transparency to support a policy that provides the ability to reject almost any scientific study of human health effects that EPA would like to jettison to meet Administration goals of subverting environmental protection.
 Asserts the proposal is an abuse of the concept of scientific transparency in the service of political objectives that support increasing pollution limits to serve business interests but will harm people and result in higher long-term public health costs. (2492)
 The proposed regulation would substitute the judgement of political appointees for scientists and technologists, many from Americans leading universities and research organizations. (2907)
 Despite its title, the commenter believes that the EPA's proposal is not an effort strengthen transparency, but to suppress information that exposes corporations of wrongdoing and caters to special interests. (6914)
 Believes that the proposed rule undermines the essential role of science in the regulatory process and would open the door to partisan manipulation of EPA risk assessment science. (4881)
 If finalized, the rule would allow political appointees to reject the best-available science for virtually any reason and delay taking critical, life-saving actions in order to benefit special interests and corporate polluters. (6122)
 The proposed rule would ignore scientific best practice and inject politics into science. (6152)
 While academic science strives to be as disinterested as possible, parties with an interest in regulation could produce a self-serving but flawed analysis of raw data to sow doubt. The commenter contends that the EPA is clearly attempting to divorce itself from the use of real science in order to promote a political agenda. (6895)
 The proposed rule's attempt to exclude perfectly reliable scientific studies from EPA's consideration is a thinly veiled attempt to allow the EPA to ignore sound science at its discretion if that science does not conform with the current Administration's political agenda. (6909)
 Commenter expresses concern that shifting the decision making process for which scientific studies are used or exempted in EPA policy from experts in the field of the respective research to a politically appointed position creates the possibility of a disastrous situation where political ideals replace scientific fact and expert opinion. (8274)
 Rather than strengthen science, the proposal grants the Administrator authority to censor science--that by any scientific definition is the best--simply because it conflicts with this Administration's political goals. (9227)
Response: The EPA disagrees that the rule will facilitate political interference in regulatory decision-making. Transparency assumes not political ideology. The intent of the rule is to build upon existing federal government efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The EPA agrees with commenters that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will continue to evaluate and consider all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA does not agree with commenters regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.9 and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.9 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the final significant regulatory action or final influential scientific information.
<IP_ADDRESS> Industry Bias
Comment: Commenters (1905, 2171, 2787, 4840, 4884, 4933, 6121, 6123, 6125, 6195, 6879, 6892, 9227) suggest that the proposed rule provisions introduce a bias against peer-reviewed research in the use of science in regulatory decision making that favors industry. Comments provided include:
 The EPA is trying to discredit scientific findings that threaten powerful corporate interests. (1905)
 Commenter views the Proposal, and its requirement of making publicly available data for "independent validation", simply as a means of providing industries concerned with various studies an opportunity to reanalyze data to reshape or recast conclusions drawn by researchers and subsequent peer reviews, and to allow EPA to censor important studies without justification. (2171)
 The substance of the rule is concerning. The commenter notes that it appears to be targeted at excluding important public health studies while privileging industry-sponsored research and urges the EPA to withdraw the rule. (2787)
 There is every reason to believe this is the plan of the current leadership of the EPA and is a recipe for privileging mercenary studies by industry commissioned to influence regulatory policy, because they will be structured and released in a way acceptable under this regulation. (4840)
 The real purpose of these rules is to reverse regulations on industries who have been harmful to public health. The commenter suggests that science speaks the truth and the EPA should hear from all scientific studies, not just the ones the industry wants you to listen to, adding that "[s]cience is the cross before the corporate devil". (4933)
 The commenter states that business is most often about annual profit and loss while treating a clean environment as a free raw material. The commenter notes that the proposed rule insertion to Title 30 is meant to frustrate and debilitate the legislated missions of the EPA. Rather than the avowed transparent science, the commenter states that it will enable cherry picked industry data and flush valid scientific research. (4884)
 The EPA's proposed rule is not about transparency or strengthening science. Instead, the commenter suggests that it is a wolf of pro-industry bias hiding in the sheep's clothing of transparency in science and should be withdrawn. (5022)
 Deconstructing Regulatory Science," by Professors Rena Steinzor and Wendy Wagner (June 19, 2018) (cited by 6892)
 The only "transparency" in this proposed regulation is its clear intent to censor legitimate and sound science that may be inconvenient to powerful industries that are regulated by EPA. The commenter requests withdrawal of the rule. (6879)
 The only outcome of this rule will be to put industry profits before the mission of the EPA, which is to protect human health and the environment by reducing risks based on the best available science and urges EPA to withdraw the rule. (6195)
Commenter (4881) notes that, in the light of other recent EPA directives, the proposed rule can be fairly judged to favor the industry perspective at the expense of public health protection, in violation of the Court-mandated balance. The commenter provides that the EPA is currently operating under a directive that eliminates EPA-funded scientists from consideration for membership on EPA scientific advisory boards, such as the CASAC mandated by the Clean Air Act (CAA). Even if they are subject experts on research relevant to regulatory considerations, the commenter notes that scientists with some EPA funding are judged too conflicted to provide advice, while industry representatives are judged to have no such automatic conflicts. The commenter asks:
 Is the advice of such now unbalanced scientific advisory boards to be critical in judging which published research is acceptable for EPA to include in formulating standards?
 Are intellectual property claims of proprietary data by industry to be handled consistently with claims of confidentiality by independent researchers?
 Will internal industry memos establishing research programs and judging whether results are consistent with industry goals be made publicly accessible, as well as raw data, to strengthen transparency and enable independent judgments on the validity of that research?
Response: The EPA disagrees that the rule will introduce bias in regulatory decision-making. The intent of the rule is to build upon existing federal government efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public.
The EPA also disagrees that the rule is intended to benefit industry. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a final significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The rule requirements also direct the EPA to evaluate the availability of the underlying dose-response data for any subject matter expert, regardless of their affiliation. Further, the final rule dictates that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Restricted access in the context of this rule means that data must be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. CBI data are not given different or preferential treatment.
The EPA agrees with commenters that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is a very important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will continue to evaluate and consider all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA would like to clarify that this final rule will be implemented prospectively and rules that have already been finalized will not be subject to this rule's requirements. The EPA would also like to clarify that this rule is not related to membership processes for the EPA's scientific advisory boards.
Comment: Commenter (4877) states that their organization and its members support open access to research outputs to democratize access to information, accelerate scientific discovery, and stimulate innovation. However, they express concern about the rule's proposed selective use of "open" as an enabling strategy in the EPA's policy-making process.
Response: In response to public comments, the EPA clarified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and the final rule. The requirements in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenters (2909, 4596, 4840, 5022, 6046, 6109, 6121, 6123, 6133, 6144, 6157, 6336, 6446, 6895, 6920, 8281-PH34, 9227) point out the inequity/hypocrisy of the proposed rule subjecting health study privacy data to additional scrutiny while allowing continued confidentiality of equally critical and opaque business information. Comments provided include:
 Questions the "double standard" of allowing industry data to be confidential, which favors polluters over those impacted by their pollution. The commenter adds that the only thing transparent about the proposed rule is the outcome it would have compromising science to the detriment of human health and our environment for the benefit of big polluters. (2909)
 EPA's claims of increasing transparency are disingenuous because the Proposal may allow EPA to consider significant amounts of information submitted by industry to the agency, even though the information will be protected as CBI and shielded from the public. According to the commenter, because industry data are associated with CBI claims more so than public health researchers' work, the Proposal reveals its industry bias. (4596)
 The commenter notes that the proposed rule requirements imposed on independent researchers are onerous and burdensome. (4840)
 Proposal would have an effect opposite to its claimed purpose; it would suppress important and relevant science conducted in large part by the best minds in academia and government, thereby unduly restricting the evidence available to EPA and potentially favoring data developed by industry. (5022)
 The Proposal is biased against non-pesticide contaminants while ignoring the very large transparency issue with pesticide evaluations, could result in increased uncertainty in standards and health values, and ignores current measures that try to address scientific credibility of studies. (6046)
 CBI may be exempted from the transparency rule, but not necessarily confidential health information. The commenter provides that the EPA states that it seeks comment on "methodologies and technologies designed to provide protected access to identifiable and sensitive data, such as individual health data" which they believe illustrates EPA's commitment to making health data public. The commenter provides that the EPA sets a different standard for protecting CBI, suggesting CBI may be allowed to remain nonpublic: "EPA seeks comment on how to balance appropriate protections for copyrighted or CBI, including where protected by law ..." The commenter inquires what EPA means by the phrase "balance appropriate protections" for CBI and why is the same phrase not used for protecting private health data? The commenter suggests that the EPA does not appear to place private health data on the same footing as CBI. This, according to the commenter, suggests EPA will be more likely to allow industry to submit nonpublic data than it will health researchers. The commenter adds that it also suggests that if EPA applies the proposed rule retrospectively, the agency will still consider research that contains nonpublic CBI while discounting research that contains nonpublic private health data. (6109)
 "[P]ublic scrutiny" would only serve industry's interest either through eliminating the use of studies that contradict their claims of chemical safety or through attempting to discredit studies and researchers by "reinterpreting" the data in their favor. (6121)
 EPA indicates that the proposed rule is intended to strengthen transparency of EPA regulatory science, however the commenter finds this to be a duplicitous claim. The commenter states that the proposed rule would favor industry data protected as CBI over public peer-reviewed research. (6123)
 Under the Proposal, EPA would not be able to rely on the best available science for its ISAs of air pollution which inform the NAAQS-setting process, while industry-funded research calling into question the air pollution-health link, would not be subject to similar data release requirements, or even peer-review and independent reevaluation. The commenter notes that this approach is asymmetric and favors selective, opaque, and questionable research methods over the consensus of robust peer-reviewed scientific investigation. The rule is arbitrary in its selective application of data release requirements and disregard for the quantitative complexities of epidemiologic research. (6133)
 If EPA is going to implement any version of this proposed rule, it must not, and cannot include a loophole for industry data while requiring that underlying public health data be made available. (6144)
 The proposed rule would subject health studies alone to additional scrutiny by exposing the privacy of participants and illogically raise the standards by which they can be considered. Meanwhile, the commenter states that the rule would allow for the continued confidentiality of equally critical and often much more opaque business information such as economic cost modeling or industry labeled proprietary data, which is rarely subject to verification at all, let alone rigorous peer review. The commenter contends that this approach would only serve to eliminate the strong and well-founded basis for health and environmental regulations millions of Americans depend on while keeping information deemed confidential by for profit industry interests out of the public eye. The commenter adds that the dividing line drawn by this proposal creates two camps: the privileged, industry data kept out of the public eye and often used to avoid protections like pollution controls for heavily emitting sources such as coal-fired power plants and oil refineries; and the epidemiological data prejudiced by EPA's proposed transparency rule that is used to support protections for the benefit of public health and welfare. (6157)
 Industry research might still be protected as CBI under the proposed rule, while academic or public interest research would not be entitled to rely on intellectual property protections. (6446)
 EPA is looking to establish a rule that would require independent public and private institutions to release confidential data behind environmental health studies while at the same time seeking to exempt private industry from being required to do the same. Given that the confidential data held by independent institutions are also protected by law, the commenter states that the EPA should be seeking the same transparency privilege as that afforded the private sector. (6895)
 Commenter finds it hypocritical that EPA's proposal appears to not want to increase transparency for the thousands of chemicals in the millions of products we buy everyday citing the need for business to keep trade secrets confidential while pushing for unsubstantiated and unnecessary increases in transparency in science that could threaten public health. (6920)
 The EPA has excluded administrative proceedings where EPA and industry regularly rely on nondisclosed information and where EPA action in general, and particularly expeditious action, tends to favor industry. The commenter suggests that by limiting the proposal to "significant regulatory actions," the proposed rule would treat exactly the same study differently depending on whether it supports regulation or non-regulation in a particular context. The commenter adds that the proposed rule will tend to exclude evidence when it supports a health-protective regulation that is costly to industry, but the proposed rule will then allow the use of the exact same evidence when the ultimate agency decision avoids regulation or deregulates industry activities or otherwise has low compliance costs. Thus, according to the commenter, the Proposal is clearly shaped to favor industry interests, not to further transparency. For example, the commenter contends that the EPA's exclusion of the new chemicals program clearly favors industry, allowing industry to conceal information and evade regulation. In addition, the commenter asserts that the EPA cannot rationally impose stringent new disclosure requirements that exclude extensive peer-reviewed, high-quality studies in some contexts while simultaneously authorizing the commercial distribution of new chemicals with almost no disclosure and no peer-review. (9227)
Response: The EPA disagrees that the rule would be biased toward industry. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a final significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The rule requirements also direct the EPA to evaluate the availability of the underlying dose-response data for any subject matter expert, regardless of their affiliation. Further, the final rule dictates that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Restricted access in the context of this rule means that data must be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. CBI data are not given different or preferential treatment.
Based on a consideration of these and other similar comments, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will continue to evaluate and consider all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
According to the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence of a relationship between exposure and effect, the EPA will identify those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA shall give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. The Agency shall give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through a data sharing mechanism sufficient to protect sensitive data, such as through an agreement with the originating author or institution or restricted access in a manner sufficient for independent validation.
Comment: Commenters (6109, 6144, 6915) note that the EPA statement that where "available and appropriate, EPA will use peer-reviewed information, standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable and reproducible scientific assessments" displays a bias toward industry-backed research over research from independent and university scientists.
Commenter (6109) provides that GLP, a protocol that federal regulators apply to industrial laboratories, is infrequently used outside of industry. According to the commenter, this is because GLP is expensive and does not reflect science quality. The commenter notes that GLP regulations mandate methods for archiving data, record keeping, equipment calibration, etc., but they do not influence research design or methodology. Conversely, the commenter notes that academic researchers (who are frequently federally funded), on the other hand, follow scientific processes that are subject to the rigors of peer review and independent replication. According to the commenter, if EPA is to truly use the "best available science," it must focus on studies that use appropriate protocols and techniques, not merely favored recordkeeping processes.
Commenter (6915) adds that this would favor consideration of industry toxicology studies over equally valid peer reviewed studies from other institutions. Under current EPA risk assessment approaches, the commenter states that all relevant scientific data are considered. In contrast, the commenter asserts that this language indicates the proposed rule is significantly more restrictive than current EPA guidance as far as the types of valid peer-reviewed scientific data that can be considered.
According to the commenter (6915), it is critical to note that the phrase GLP referenced by EPA is not a value descriptor. Rather, the commenter explains that it is a technical term referring to a specific category of study conduct and reporting that is intended for specific regulatory purposes. The commenter provides that GLP/standardized test method studies are typically conducted by industrial or contract laboratories, and test for limited parameters in order to meet specific regulatory requirements, such as for registration of pesticides, drugs, and other products. The commenter adds that these protocols often have not been updated to incorporate recent approaches in toxicology, and they may not look at the most sensitive and relevant toxicological effects of the product being studied. In contrast, the commenter notes that other equally scientifically valid studies, typically conducted in research laboratories in academic, industrial, or government institutions, use specialized approaches to evaluate specific toxicological effects of the chemical under study, and may not follow the standardized protocols specified in regulatory requirements. The commenter asserts that the use of GLP protocols does not necessarily mean that the study is of higher quality, and there is no scientific reason that the data generated under the highly circumscribed regulatory requirements for product registration should receive greater weight than any other valid scientific data. Rather, the commenter suggests that all studies be evaluated on their own merits.
Similarly, commenter (6144) notes that the EPA's proposed rule alludes to EPA's use of GLP and standardized test methods where "available and appropriate." The commenter states that this would bias the literature supporting rules toward industry funded GLP studies, even though GLP practices are more about recordkeeping than upholding a high quality of research design. As an example, the commenter notes that a study reviewing the quality of GLP literature on the chemical bisphenol A (BPA) led the authors to conclude that "simply meeting GLP requirements is insufficient to guarantee scientific reliability and validity." While GLP studies should not be excluded from EPA's consideration, the commenter contends that the highest quality assessment that EPA can engage in would be a systematic review approach that considers all relevant data and establishes criteria for study quality evaluation, as recommended by the NAS.
Response: The EPA considered these and other similar comments when developing the final rule. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The Agency will also continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA disagrees that this rule will favor any stakeholder group. The EPA would also like to clarify that the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information, and does not prescribe conformity with GLP or other standard practices.
Comment: Commenters (5022, 6881) note that the proposal runs counter to advice provided to EPA by the NRC in a way that favors industry.
Commenter (5022) contends that the pro-industry orientation of the proposal is revealed in its assault on the linearity assumption in the dose/concentration-response function which runs counter to the advice provided to EPA by the NRC, which stated, "[t]he committee recommends that cancer and noncancer responses be assumed to be linear as a default."
Commenter (6881) notes that the "Science and Decisions" committee was clear that EPA's approach to dose response modeling needed to be changed to reflect growing scientific knowledge and the evolving needs of decision makers. The commenter notes that the proposed rule does not reflect the conclusions from the NRC committee (or other standard mechanisms by which changes in modeling and risk assessment practice are typically promulgated), and instead proposes prescriptive steps that, if taken in aggregate, would delay the risk assessment process and confuse risk managers, ultimately leading to paralysis by analysis.
Response: The EPA disagrees that this final rule favors industry. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a final significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The rule requirements also direct the EPA to evaluate the availability of the underlying dose-response data for any subject matter expert, regardless of their affiliation.
In response to public comments, the EPA clarified and modified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and final rule, including that the rule requirements are now focused on dose-response data and do not apply to dose-response models.
The EPA also disagrees with commenters that requirements of this rule will result in any meaningful delay to its processes. While this rule does require the Agency to evaluate the availability of the underlying dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., a small, though highly important, subset of all the studies EPA reviews).
Comment: Commenter (4881) notes that further evidence of an industry bias is reflected in the proposed rule itself by the complaint in footnote of the proposed rule preamble (footnote 3) about the Court decision in the case of Coalition of Battery Recyclers Association v. EPA, in which a central issue was precisely the "growing empirical evidence of non-linearity in the concentration-response function" that EPA now claims is its primary concern. The commenter provides that, in Battery Recyclers, the D.C. Circuit Court supported the EPA's current practice with respect to scientific evidence, rather than that proposed in the new rule. In particular, the commenter notes that the Court upheld EPA's use of evidence revealing important non-linearity in dose response to lead concentrations in the atmosphere, in the face of industry arguments opposing the claim of non-linearity. The commenter expresses that the EPA now appears to support the Battery Recyclers' opposition, because the evidence used suggested a non-linearity opposite to that preferred by industry. The commenter provides that the evidence comprised four independent research studies of correlations between blood lead levels and the intelligence quotient (IQ) of young U.S. children. Rather than supporting a threshold effect, the commenter notes that the data indicated that incremental IQ loss actually increased at lower blood lead levels. The Agency thus based its revised standard for atmospheric lead concentrations on analyses of children with blood lead levels closest to the low average levels of children in the United States today. The commenter contends that, having failed to persuade the Circuit Court of its position, industry, it appears, has now found a friendly audience in the current EPA administration. The commenter explains: