EPA has failed to provide the public with sufficient information on which to offer comment. In fact, the commenter contends that it is clear from the "Request for Public Comment" section that EPA does not have a full understanding of the very rule it is proposing. The commenter contends that the EPA clearly does not have a plan for how it will implement the rule or what its effects will be. The commenter states that the topics upon which EPA seeks comment should have been provided to the public as part of a fully thought-out rule proposal. The commenter adds that because EPA provides only questions and no proposed actions, EPA eliminates their ability to comment on many aspects of the rule. (6109)
 The Proposal suffers from what the commenter believes is unlawful vagueness. If EPA intends to finalize what they believe is an unlawful proposal, EPA first must withdraw the Proposal, then issue a supplemental proposal with the necessary definitions and explanations. Better yet, the commenter suggests that EPA abandon what they believe to be an illegal and harmful proposal altogether. (6133)
 Commenters are supportive of scientific transparency in regulatory development. Unfortunately, according to the commenters, the rule is vague in several areas and does not provide specific regulatory language for review and comment. Commenters (6104, 6878) request also that the EPA provide more details regarding the intent, scope, and implementation processes associated with this proposal. The commenter believes these discussions would improve the quality of the comments on the rule and will contribute to an enhanced and improved final rule, should this rulemaking go forward. Further, the commenter requests that EPA issue a supplemental notice of proposed rulemaking that includes actual regulatory language. As part of this supplemental notice, the commenter requests that EPA provide sufficient detail for an analysis of whether this new approach would achieve the results intended, while also continuing to support states efforts to implement environmental requirements in their states. (6104, 6878) . In the spirit of cooperative federalism, commenter (6104) suggests that before the rule is finalized, the EPA host coregulatory discussions.
 The proposed rule lacks clarity and it is difficult to ascertain what it seeks to achieve, whether EPA is interested in greater transparency across all of its programs and activities, and what the long-term implications could be of any changes made. The commenter welcomes additional dialogue with the Agency on what this rule aims to achieve, but even with an extended comment period, the commenter believes that the proposed rule is too ambiguous for the commenter to provide detailed comments. (6146)
 The rule is poorly written and in fact lacks transparency which it purports to support. The rule is vague, open-ended and without necessary safeguards to assure that scientific evidence is used without social, political, or economic modifying factors. (6147)
 The proposal is vague and unworkable and notes that it was developed without significant input from the scientific community, and therefore the definitions and processes in the proposed rule are not specific. The commenter contends that this leaves stakeholders unable to determine how the rule would be applied. (6152)
 The Proposal's vague language suggests unpredictable implementation. (6188)
 The current proposal is far too vague and missing key components that must be included in the final rule to ensure its understandability and appropriate implementation. For example, the commenter notes that currently the proposal lacks definitions for many key terms, omits critical protocols and methodologies necessary to put this rule into action, and does not fully explore the implications of implementing a rule of this nature. (6875)
 The EPA has not adequately considered the implications of this rule for the conduct of science as a whole, and those implications are very negative. According to the commenter, the proposed rule is so woefully incomplete and lacking in support as to deprive the public of a meaningful opportunity for comment. (6909)
 With respect to EPA's process, this proposal has been rushed, is vague, and creates more questions than it answers: it does not clearly state the actual parameters of the proposed rule, it is open-ended in terms of alternatives under consideration, and it fails to provide critical information such as projected costs. The commenter also notes that it is also completely unclear -- or worse, contradictory -- whether and how this proposed rule would apply to EPA's cost-benefit analyses. (6915)
 The Proposal's terms are so vague as to fail to afford an opportunity for meaningful comment. The commenter provides that significant issues are presented in the Proposal only in the broadest terms, despite an APA requirement that a proposed rule includes "the terms and substance of the proposed rule or a description of the subjects and issues involved," 5 U.S.C. § 553(b)(3), and with enough specificity "to permit interested parties to comment meaningfully." (6916)
 The process underlying the proposed rule is flawed due to its lack of adequate analysis and input from the start and the insufficiency of the notice and information provided to commenters. As such, the commenter suggests that the proposal be withdrawn until proper preliminary analysis has been completed. (6919)
 The language of the rule is confusing, contradictory, and poorly and incorrectly referenced with little science or policy foundation. According to the commenter, this suggests the rule authors lack understanding of the scientific process. (6924)
 The current proposal is far too vague and missing key components that must be included in the final rule to ensure its understandability and appropriate implementation. (9224)
 The ill-conceived, badly written, and unlawful proposal is flawed beyond repair and should be rescinded. Further, according to the commenter, this proposed rule runs counter to EPA's stated commitment to "robust and civil dialogue with the public." The commenter contends that any further time and money spent on this proposal would be a waste of valuable public resources and that EPA and OMB should focus their limited resources on protecting public health and the environment rather than continuing to consider such a flawed proposal. (9225)
Response: In response to public comments, the EPA clarified many aspects of the proposed rule in the supplemental notice of proposed rulemaking and final rule, including key definitions and an overview of the process that the EPA would follow. The final rule clarifies that the rule requirements set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. As further discussed in the preamble of the final rule, the EPA maintains that this rulemaking establishes internal Agency procedures and does not intend to change the behavior of external parties. Finally, in consideration of public comments, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. The final rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
<IP_ADDRESS> Insufficient Details Regarding the Handling of and Sensitive Personal Data
Comment: Commenters (0068, 2429, 2787, 4594, 4831, 6125, 6133, 6362, 6448, 6893, 6911, 6916, 6955, 8272, 9227) suggest that there is insufficient detail regarding the handling of CBI and sensitive personal data. Comments include:
 Before EPA imposes a requirement that all primary data developed in future studies be publicly available, the Agency should perform a systematic review of the relevant literature on the strengths and limitations of strategies to anonymize. (0068)
 Proposed rule does not state the Agency's plans for disseminating publicly available data that are under the jurisdiction of other federal agencies, such as Medicaid and Medicare. The commenter points out that a recent study that might be excluded from consideration showed that long-term exposure to particulate matter and ozone at levels below the annual standards was associated with an increase in mortality in the Medicare population. (2429)
 Key issues including how data would be made available to the public and how private information would be protected are not addressed. According to the commenter, this is a serious deficiency in a rule meant to increase access to data for the public. (2787)
 EPA has solicited input on measures to "provide protected access to identifiable and sensitive data." The commenter asserts that this is a significant issue and one that EPA should fully understand prior to moving forward with any new rule. The commenter recommends that EPA initiate a formal request for public comment on this issue alone and use what is learned to help inform and guide any potential future rule on transparency. (4594)
 The proposal acknowledges that data may be "controlled by third parties," but its entire explanation of what this may mean is a vague promise that "EPA shall work with those parties to endeavor to make the data available in a manner that complies with this section." The commenter notes that this language does not begin to address the variety of complexities involved with data controlled by a wide variety of third parties each with different policies of access, making it nearly impossible for commenters to know whether third party studies can ever be used without some type of interactive process between the study authors and EPA. Again, the commenter states that even if EPA does not intend to apply this provision to categorically bar the use of third party studies, its failure to point this out, or otherwise explain this provision makes the rule too vague to implement and is still another instance of failure to provide adequate notice. (6125)
 Commenter suggests that EPA should provide greater clarity regarding what it intends to do in circumstances where raw data cannot be made publicly available. (6362)
 Concerned with the lack of details around appropriate access to confidential information. While noting that "[n]othing in the proposed rule compels the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections," the lack of specificity and applicability is found wanting and the Agency should expand at great length should the rule go forward. (6448)
 The proposed rule states that EPA aims to honor "legal and ethical obligations to reduce the risks of unauthorized disclosure and reidentification" and that concerns about access to confidential or private information can "be addressed through the application of solutions commonly in use," but it does not describe criteria or processes for determining which data sets would be exempted from data release requirements, what alternative requirements might be applied to data sets containing identifiable information, or which activities the Administrator will exempt. (6893)
 Commenter suggests that the Proposal be revised to further clarify how EPA will protect private, sensitive and confidential information, as well as provide an exemption process that allows for clearly and objectively assessing studies without excluding them from the rulemaking and guidance process. (6911)
 The Proposal provides no discussion of how underlying data of relevant studies can be released consistent with the various laws and regulations protecting privacy and prohibiting disclosure. According to the commenter, the failure to "consider an important aspect of the problem" renders the Proposal arbitrary and capricious. (6916)
 No protections have been built into the proposal to prevent or reduce potential abuse of researcher data. (6955)
 The proposal states that EPA aims to honor "legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification" and that concerns about access to confidential or private information can "be addressed through the application of solutions commonly in use," but it does not describe criteria or processes for determining which data sets would be exempted from data release requirements, what alternative requirements might be applied to data sets containing identifiable information, or which activities the Administrator will exempt. (8272)
 In the proposal, EPA blithely and irrationally ignores or assumes away the real and significant issues, suggesting that existing mechanisms and techniques can be used to protect privacy and confidentiality while making underlying research data publicly available. In fact, the commenter provides that evidence (including several of the sources that EPA cites) indicates that the potential mechanisms alluded to by EPA would only have the potential to address some of the barriers cited above, have serious limitations even for those, and are actually becoming less effective as it becomes easier to combine and manipulate public data sets. The commenter adds that the "solutions" EPA might have in mind do not address the issues raised by the Proposal because no other agency has tried to implement a requirement such as the one EPA proposes. The commenter provides that other agencies provide guidance and techniques to protect privacy during data collection and disclosure to allow more use of data collected by the government, not to mandate that data collected by academic or industry researchers be publicly available for purposes of replicating analyses. The commenter adds that the EPA cursorily mentions a range of options for facilitating secure access to confidential data, including: "[r]equiring applications for access; restricting access to data for the purposes of replication, validation, and sensitivity evaluation; establishing physical controls on data storage; online training for researchers; and nondisclosure agreements." The commenter states that the EPA does not indicate whether it would deem providing access with these types of controls in place sufficient to meet EPA's proposed requirement "publicly available in a manner sufficient for independent validation." Comments provided by this commenter that are specific to de-identification methodologies/issues, implementation costs, and privacy data requirements are presented in other relevant sections of this document. (9227)
Commenter (6133) provides that the EPA concedes that concerns about access to confidential or private information cannot always be addressed but says nothing about these instances or how it intends to evaluate them. The commenter notes that, for the times that EPA believes concerns about access to confidential or private information can be addressed, the Proposal does not explain how it plans to do so nor address the costs. The commenter adds that the Proposal merely directs readers to general and vague statements from different contexts and fails to provide fair notice or justification of what EPA would do to address issues with confidential or private information.
Commenter (6133) states that the Proposal merely says to "[s]ee examples from the U.S. Department of Health and Human Services, National Institute of Standards and Technology, U.S. Department of Education, and the U.S. Census Bureau," id. n.16, and points generally to Health and Human Services "Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the HIPAA Privacy Rule," id. n.17. The commenter notes that the Proposal does not say what actions from these examples EPA proposes to use.
Commenter (6133) also states that the Proposal states that the National Academies have noted that in the past, restricted data products were created by relatively simple data masking, coding, and de-identification techniques, and notes that "[n]othing in the past suggests that increasing access to research data without damage to privacy and confidentiality rights is beyond scientific reach." 83 Fed. Reg. at 18,771 (citing Expanding Access to Research Data Reconciling Risks and Opportunities, The National Academies Press, 2005, https://www.nap.edu/catalog/11434/expanding-access-toresearch-data-reconciling-risks-and-opportunities at 27, 36). First, according to the commenter, this is not fully supported as experience shows increasing access to data can damage privacy and confidentiality rights. Again, the commenter notes that the Proposal does not say which, if any of these techniques the EPA will use, or how the EPA will use them. And while the National Academies may believe that increasing access to data without damage to privacy and confidentiality is not beyond scientific reach, the Proposal does not explain how this belief translates to past, present, and future scientific studies EPA considers in regulatory decision making. The commenter provides that this document does not explain how EPA will address concerns about confidential or private information and does not support EPA's Proposal to preclude consideration of those studies that do not make public underlying data for those, or other reasons.
The commenter (6133) provides that the Proposal next cites to two National Academies documents and a document from the Bipartisan Commission on Evidence Based Policy. 83 Fed. Reg. at 18,771 & n.19. But, according to the commenter, the Proposal fails to explain how these documents support its proposed actions or explain how EPA intends to protect confidential information. The commenter notes that the Proposal merely states that they "have discussed the challenges and opportunities for facilitating to secure access to confidential data for nongovernment analysts." 83 Fed. Reg. at 18,771. The Proposal does not address those challenges or describe the opportunities it intends the EPA to use. Again, the commenter asserts that these documents do not support the vague Proposal.
Commenter (6133) states that the Proposal states that "the requirements for availability may differ," and "may range from deposition in public data repositories, consistent with requirements for many scientific journals, to, for certain types of information, controlled access in federal research data centers that facilitate secondary research use by the public." Id. (footnotes omitted). The commenter provides that the Proposal again cites to journal policies or recommendations generally and the policies for access to data from NIH and Census Bureau. Id. nn.20 & 21. Importantly, the commenter notes, the Proposal does not say how the requirements would differ, what studies would be required to deposit what data into what repositories, and what studies would be required to allow controlled access to what data in what federal research data centers. Moreover, the commenter provides that the Proposal does not address the costs that these actions would entail. Again, according to the commenter, if EPA intends to use these different ways to provide data that meet concerns about confidential and private information, the agency must withdraw the rule and issue a new proposed rule that explains the methods it proposes to use.
The commenter (6133) expresses that the proposal generally wraps up this section with:
      EPA should collaborate with other federal agencies to identify strategies to protect confidential and private information in any circumstance in which it is making information publicly available. These strategies should be cost-effective and may also include: Requiring applications for access; restricting access to data for the purposes of replication, validation, and sensitivity evaluation; establishing physical controls on data storage; online training for researchers; and nondisclosure agreements.
83 Fed. Reg. at 18,771 (footnote omitted).
The commenter (6133) contends that the Proposal's many flaws are clear in these sentences. The commenter asserts that the EPA does not know what the Proposal entails. The commenter notes that the Proposal suggests that EPA should identify strategies in the future and that these strategies should be cost-effective. The commenter further notes that the Proposal does not say what cost-effective means, nor what EPA should do if it does not identify any cost-effective strategies, yet it still seeks to alter legal obligations and regulatory decision making in reliance on this unexplained suggestion. The commenter adds that the EPA also does not point to any authority for the proposition that the agency's consideration of peer reviewed scientific studies depends on the cost-effectiveness of some strategy the agency develops for publicizing and protecting the underlying data. The commenter argues that listing options EPA can use does not help. The commenter asserts that the Proposal fails to explain why EPA has not already identified the strategies or options and in what circumstances it would use them. The commenter adds that the Proposal suggests that it will exclude a large class of scientific studies from regulatory decision making but contains a vague assertion that it will look for "cost effective" ways in the future to exclude less them.
Commenter (6133) concludes that the EPA's belief that concerns about access to confidential or private information caused by the Proposal should be addressed in the future is problematic by itself. The commenter provides that the cited materials --  describing ways different organizations can address concerns in different contexts -- do not support this belief. The commenter adds that the Proposal does not propose or analyze any strategies it notes EPA should consider, even though it seeks to implement a binding legal change. While at least EPA recognized the merit to concerns about confidential or private information, the commenter states that in the four days since sending the version for review the agency clearly did not perform the analysis necessary to figure out how those concerns would be addressed. The commenter states that the impact and costs of the Proposal are dependent on such strategies and cannot be measured or analyzed without proposed regulatory text. The commenter contends that the EPA cannot publish a final rule without first proposing what it will do about confidential and private information and analyzing the option it proposes and that the EPA should withdraw the Proposal.
Response: Upon consideration of public comments, the EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation.. Therefore, the EPA is not finalizing the 2020 SNPRM proposed option that would have categorically required that for studies to be considered pivotal science the underlying data would need to be available for independent validation. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA would also like to emphasize that the EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule and the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
<IP_ADDRESS> Proposal Should be Revised to Include a Clearly Defined Process for Checks and Balances in the Implementation of the Rule
Comment: Commenter (0562) suggests that the EPA release a revised proposed rule with explicit language as to who would be advising the Administrator on which data or models are valid for decision making, and what form of independent review is needed if they have been exempted under §30.9. According to the commenter, advisors should have the authority or recourse to call into question decisions by the Administrator that are counter to good scientific practice and that result in excluding valid data or models from consideration in a regulatory decision based solely on lack of transparency. The commenter states that the "scientific" criteria used by the Administrator or his or her advisors to make such decisions should be explicitly stated (or referenced) in the revised rule. In other words, the commenter asserts that there should be a clearly defined process for checks and balances in the implementation of the proposed rule to instill trust both within the Agency and for stakeholders, and that decisions on what data or models could or could not be used in a regulatory decision are not the result of the Administrator's personal bias. In the absence of such explicit language, the commenter states that one can only assume that these decisions "could" be made solely at the Administrator's discretion.
Commenter (2169) states that there is no mechanism in the regulation for assuring that the science drives the decision. Therefore, according to the commenter, there is no means for judging the term pivotal. The commenter contends that, without transparent consideration of all the information available, and public comment on whether all pertinent information was considered, it is pointless to promulgate rules on the availability of the information.
Response: In consideration of public comments, the Agency is retaining the Administrator's exemption provision because there are conditions under which compliance with the requirements in 40 CFR part 30 might be impracticable. For example, the underlying dose-response data for some studies, particularly older studies, may not be readily publicly available because of the technological barriers to data sharing (e.g., differences in data storage devices or data retention practices) that existed when they were developed. As a result, the EPA is finalizing the Administrator's exemption provision as proposed in the 2020 SNPRM, with additional conditions described here.
The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation.
The EPA is also including criteria that may be used by the Administrator when he or she is determining whether greater consideration should be afforded to pivotal science for which the underlying dose-response data are not available in a manner sufficient for independent validation. As a result, the Administrator may determine that greater consideration is warranted when:
 Technological or other barriers render sharing of the dose-response data infeasible;
 The development of the dose-response data was completed or updated before the effective date of this final rule;
 Making the dose-response data would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security;
 Third-party independent validation of the study's underlying dose-response data through reanalysis has been conducted; or
 The factors in 40 CFR 30.5 used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Finally, in consideration of public comments, the EPA clarified the definition of pivotal science in the final rule. Rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate points of departure (PODs)). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
<IP_ADDRESS> Request for EPA to Clarify Why Additional Peer Review of Data is Necessary When it Has Already Been Reviewed or Adequately Describes the Process
Comment: Commenters (5165, 5419, 6125, 6373, 6915, 9227) suggest that EPA clarify why the proposed rule is requiring additional peer review of data and/or present an adequate description of what the peer review process would entail.
Commenter (9227) notes that the EPA's proposed peer review regulations do not even mention EPA's Peer Review Handbook, let alone explain how the new proposed regulations would impact EPA's compliance with the Handbook. For example, the commenter states that the EPA's Handbook specifies "exemption criteria" in Section 3.3. The commenter requests that EPA clarify whether anything in the proposed peer review regulation would supplant instructions in the Peer Review Handbook, and if so, provide a reasoned explanation for the change. Likewise, the commenter suggests that EPA explain the role of the Peer Review Handbook going forward in administering peer review requirements. Additionally, the commenter notes, the preamble to the proposed rulemaking lacks any explanation whatsoever for why EPA is proposing this new peer review requirement or what its impact might be. The commenter adds that the EPA has additionally not provided any information to suggest that EPA is not already following OMB's Peer Review Bulletin. According to the commenter, EPA's lack of any supporting rationale or analysis frustrates the public's ability to provide meaningful comment on this provision, and is itself a sign that this requirement is fundamentally arbitrary.
Commenter (5165) notes that the proposed rule would require EPA to conduct "independent peer review" of scientific studies underlying its significant regulatory decisions, such as the establishment of health-based air quality standards and that EPA's in-house peer reviewers would also be tasked with articulating "the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results." The commenter notes that it is difficult to provide meaningful comments on this aspect of the proposal because EPA has included no details about how the "independent peer review" requirement would be implemented. According to the commenter, the fact that EPA has requested comment on "which parts of the Agency should be responsible for carrying out these requirements" suggests that it has not worked out a plan for this fundamental provision. The commenter inquires: Would EPA have to hire new experts, and if so, how many and in what fields? How much would this cost? More fundamentally, the commenter asks why should scientific literature that has already undergone peer review and been vetted by EPA's science advisory panels be subjected to an additional layer of government peer review? The commenter asserts that these key questions should have been considered, and the answers made public, prior to the rule's proposal.
Commenter (5419) states that §30.7 of the proposed rule states that EPA shall conduct independent peer review of pivotal regulatory science but lacks any detail of this process, including selection criteria for peer reviewers, decision criteria, and how this compares to existing processes through the SAB and third‐party contractor‐managed peer reviews. The commenter requests clarification and additional detail related to this key provision of the proposed rule.
Commenter (6915) notes that the EPA apparently proposes to require that EPA itself conduct an additional "independent" review. See 83 Fed. Reg. at 18774. Yet the commenter notes that the proposal nowhere discusses how EPA would vet reviewers to identify persons who are purportedly more competent than those already used in past or current peer review processes, or the level of EPA staffing and associated costs that would be needed for additional review -- only stating that EPA will implement the proposed rule in a manner "that minimizes costs." Id. at 18774. But, according to the commenter, any requirement for EPA to conduct additional review would entail additional significant costs, contrary to the proposal's assertion. Id. at 18772. The commenter contends that the practical outcome of the proposal is that EPA may end up relying on a much smaller number of studies and/or on a less robust subset of relevant available studies, thus undermining the regulatory decision-making process. In sum, the commenter asserts that the EPA fails to acknowledge the rigor of existing processes in statutes, policies and federal procedures, or to explain how its proposal would provide any added value and minimize costs. EPA should abandon this unnecessary and counterproductive exercise.
Commenter (6125) states that the meaning of proposed §30.7 suggests that EPA will be required to re-peer review every study upon which it relies. The commenter notes that the regulatory text is accompanied by no explanation, leaving commenters to speculate about how it would apply, or what problems its implementation might present. The commenter contends that the proposal's complete failure to explain this far-from-self-explanatory provision is yet another instance of failure to give adequate notice of the substance and basis of the rule.
Commenter (6373) provides that the Rule does not describe how EPA intends on conducting the assessment of models and assumptions described in §30.6 or the independent peer-review process described in §30.7 or identify which individuals would be a part of these reviews. The commenter notes that it is unclear if the peer-reviewers will consist of EPA staff, a group of independent individuals, a single reviewer, or if the peer-reviewers will be required to have varying backgrounds free from personal and professional conflicts of interests to promote a complete and adequate review of the science. According to the commenter, the Rule provides no insight into if, or how, the "independent" analyses for validation would be incorporated into the federal decision-making process, or if such analyses would be required before the findings from peer-review studies could be considered.
Response: As discussed in the preamble to the final rule, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule.
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will conduct independent peer review, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
<IP_ADDRESS> Request that EPA Provide Greater Detail on the Kinds of Research/Data that Could be Used/Excluded from the Rulemaking Process
Comment: Commenters (0556, 2335, 2876, 6357, 6873) request clarity on the kinds of research/data that could be used/excluded from the rulemaking process. Commenter (6357) recommends the EPA work with the scientific community to develop clear processes and inclusion and exclusion criteria regarding studies that can inform health impact, but for which ethical and practical considerations may preclude public release of data.
Commenter (6873) expresses concerns regarding what the proposed rule will actually do (albeit unintentionally) is create a new category of "EPA usable research." The commenter asks the following:
 What does it mean to make research "usable" or "accessible"?
 In what format?
 At what stage of research?
 Will data standardization be required (otherwise, usability can be quite limited)?
 On what timetables, with what funding and on whose authority?
 What protections will be applied for the use and reuse of data?
 And what might be the potential broader effects on research in general if we don't know exactly which forthcoming studies will be potentially of interest to the EPA?
 Will this cause all science to shift to an EPA-standard of openness?
The commenter (6873) notes that the answers to these questions are not simple and will require a significant and lengthy undertaking.
Commenter (6357) encourages the EPA to clarify the process by which the Agency will make determinations about the inclusion of studies that cannot ethically or legally release certain types of data. The commenter notes that the proposed Rule states that "concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions" already in use by the federal government. The commenter asserts that the document does not articulate how current EPA practices differ from other federal agencies' processes, what gaps the proposed EPA Rule would address, and what precisely would be updated or changed at EPA to address those gaps.
Commenter (6358) provides that the proposal states that EPA aims to honor "legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification" and that concerns about access to confidential or private information can "be addressed through the application of solutions commonly in use," but it does not describe criteria or processes for determining which data sets would be exempted from data release requirements, what alternative requirements might be applied to data sets containing identifiable information, or which activities the Administrator will exempt.
Commenter (0556) provides that EPA needs to address the following:
      First, they have to figure out what "using" a given research result even means. For example, proposed major rules are accompanied by a voluminous Technical Support Document. It may cite hundreds, or even thousands, of research journal articles. Does each of these have to meet the availability and replicability standard? Or is regulatory usage confined to just a few key studies?
      Second, what does availability mean? For example, does the researcher have to document their data, or just provide it? What about the decisions made as the research progressed, which can be numerous? Does each of these have to be explained? How documented does the software have to be, etc. Here the danger is that the availability requirement might become too burdensome.
The commenter adds that, assuming that these deep challenges finally get worked out, the EPA needs to consider what the regulation does. The commenter notes that the rule, in effect, creates a new category of "EPA usable research". According to the commenter, researchers (or their institutions) will want their research to be usable, even if EPA does not fund it, but that it will be a big step forward with regards to open science and a big job for the EPA and research community to work out the extension of Public Access.
Commenter (2335) states that there is a need for more details about the kinds of research that could be excluded from the rulemaking process and the kinds of decisions that could be affected.
Commenter (2876) requests that the proposed rule explicitly identify some number of publications with standards that would fail to meet the proposed transparency rules. Further, the commenter suggests that the proposed regulation clearly state how those publications could bring their practices into compliance with the new standards while still meeting legal and ethical requirements for protecting the privacy of research subjects.
Response: In response to public comments, the EPA clarified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and the final rule, including definitions of key terms and the stage of data required under the rule. As discussed in the preamble to the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
The EPA also clarified the definition of pivotal science in the final rule. Rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
As also detailed in the preamble to the final rule, underlying data or models are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA also modified the definition of "publicly available" in the final rule to add the following at the end of the definition: "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
Finally, the EPA will not be categorically excluding studies based on the availability of their underlying data. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. The EPA believes that the implementation of this final rule will increase the transparency of pivotal science for significant regulatory decisions and influential scientific information, while still ensuring that all quality, relevant literature is considered in the development process. The final rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
<IP_ADDRESS> Rule Needs to be Clarified  -  Specific Requests
 <IP_ADDRESS>.1 How will the Rule be Applied to Past Regulatory Actions
Comment: Commenter (6447) states that the EPA has not explained whether or how this new policy will be applied to past regulatory actions. If EPA is planning to use this new policy to go back and revisit past regulatory decisions, it should say so explicitly. Commenter (6909) inquires whether the rule can and will be applied retroactively. The commenter states that the EPA has specifically asked for comment from the public as to how it should approach this issue. According to the commenter, this is hugely significant in terms of what the rule's effects will be.
Commenter (6915) states that the proposal says the rule is intended to apply prospectively, but also states that EPA "should be guided by this policy to the maximum extent practicable during ongoing regulatory action." Id. at 18,771. Yet it never explains how or why ongoing EPA actions would be subject to the proposed rule and which existing scientific studies are implicated by the proposed rule.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. As described in 40 CFR 30.3, the rule provisions apply to significant regulatory actions for which a proposed rule was published in the Federal Register after the effective date of this rule and influential scientific information submitted for peer review after the effective date of this rule. This final rule is effective upon publication in the Federal Register.
<IP_ADDRESS>.2 Specific Language of Proposed Rule that is Unclear
Comment: Commenter (6120) states that the language in the proposed rule is not clear. For example, the commenter notes that the proposed rule states that the "EPA should give appropriate consideration to high quality studies that explore: a broad class of parametric concentration-response models with a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the exposure range; and spatial heterogeneity." The commenter states that this type of statement needs to be described more explicitly. The commenter inquires: How does the EPA define "high quality studies"? The commenter requests that the language that describes the models should be written in language that is clear to the public and gives examples of what is truly being referred to here. The commenter contends that this type of language is found throughout the proposed rule. The commenter encourages the EPA to clarify the text throughout so that each proposed action is clear and easy to understand by everyone.
Commenter (6194) asserts that key issues beyond definitions also raise issues in such a general manner that it fails to provide the public with adequate notice of what EPA intends. For instance:
 The Proposal requires that certain data and models be made "publicly available in a manner sufficient for independent validation," and indicates that information will satisfy this standard when "it includes the information necessary for the public to understand, assess, and replicate findings." 83 Fed. Reg. at 18,773-74. But it is not clear what degree of availability or explanation would satisfy the Proposal. For example, the "public" referenced in the Proposal could be scientists trained in the relevant field or members of the general public.
 The Proposal provides that, "where necessary, data would be made available subject to access and use restrictions." Id. What this process would look like is not addressed in the Proposal or discussed at all in the preamble. This leaves significant open questions, such as who will pay for and implement these restrictions and who will decide which people are allowed to access the data and under what standards.
 The Proposal appears to require EPA to conduct independent peer review of all "pivotal regulatory science," but provides no detail as to what this means or how it will be implemented. Id. For example, it is unclear who within the agency would perform this peer review, when it would occur, and whether (and when) prior peer review in the journal publication process would serve as a substitute for EPA peer review.
 The Proposal includes an exemption mechanism but does not provide information about the process or conditions under which exemptions shall or can be sought and granted.
Response: The preamble to the final rule clarifies that the requirements in this final rule set the overarching structure and principles for transparency in the EPA's final significant regulatory actions and influential scientific information. The EPA intends to develop internal implementing guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes that the EPA administers.
In the final rule, the EPA clarifies how the Agency will evaluate the quality of scientific studies when promulgating significant regulatory actions and developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. The EPA will generally continue to use the following, established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.
The rule also clarifies other general aspects noted by the commenters. For example, criteria that the Administrator would consider in determining whether to grant an exemption to the provisions in this rule are provided in 40 CFR 30.7. As described in the preamble, the rule also does not direct or require any outside entity or the EPA to establish data sharing mechanisms, nor does it require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The EPA is only required to determine if such a mechanism was available, should a subject matter expert decide to use it. The EPA also modified the definition of independent validation to now mean the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced. The preamble to the final rule also provides clarity regarding peer review of pivotal science.
<IP_ADDRESS>.3 Need for Greater Clarification on Data Refinement Issues
Comment: Commenter (6362) notes that an important aspect relevant to "public availability" is the level of data refinement EPA will require. The commenter notes that the NAS held a workshop in 2016 to discuss obstacles for sharing data and they defined several key terms to ensure clarity at the workshop. The commenter recommends that the EPA consider adopting a similar lexicon to increase the clarity of its regulation (and refers to Table 1 in Appendix B). Definitions in NAS Principles and obstacles for sharing data from environmental health research: Workshop summary) of their comment letter). In addition, the commenter provides that the NAS Report suggests a "cleaned dataset" would be acceptable to use for all routine analyses and verification (and refers to Table 2 in Appendix B (Data flow from NAS Report) of their comment letter). EPA should establish clear standards on the acceptability of "cleaned datasets." According to the commenter, this will help to standardize data reporting and formatting. It will also prevent over- and under-reporting.
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information. The EPA does not agree with establishing standards for the acceptability of data independent of the definition provided in the final rule because additional standards would be overly prescriptive and not informative.
<IP_ADDRESS>.4 Implementation Requirements Vague/Insufficient
Comment: In addition to comments on the overall general vagueness of the proposal, several commenters specifically request/point out the deficiencies in requirements set out in the Proposal that would allow them to properly comment on and implement the rule. Comments include:
 The proposed rule lacks clarity as to how the rule would apply and how EPA intends to implement the rule. (2876, 5165, 6351, 6920)
 For those researchers who do attempt to comply with the proposed rule's requirements, the extent and nature of the data that must be maintained and made publicly available is vague and unclear, making compliance virtually impossible. (6915)
 The proposal lacks details on how its provisions would be implemented, specifically when it comes to privacy issues, reproducibility, and possible exemptions. (6110)
 The rule as written is too vague with regards to implementation, which leaves far too much editorial discretion up to the politically appointed administrative roles on what constitutes "transparent" study data and model design. (6127)
 The Proposal offers no process for public hearing or even consultation with the SAB over implementation. As written, the commenter notes that the Proposal would require review and assessment of volumes of existing research and revisions to internal processes yet to be determined and seems to give arbitrary decision-making authority to the Administrator to determine the fate of such research. (6351)
 The lack of clarity regarding how the EPA will implement the proposed Rule risks exclusion of scientifically valid and relevant studies during the Agency's decision-making process. The commenter asserts that the rulemaking process itself benefits from the same transparency, consistency, and objectivity. As such, they suggest that clear guidelines around how rules are implemented are as important as data transparency in ensuring public trust and an objective approach that prioritizes the EPA's mission of protecting human and environmental health. (6357)
 Commenter (6916) assumes (as they say they must, given no other information from the Agency), that the Proposal is aimed at precluding EPA's reliance on studies for which the data are not made public. The commenter notes that is the natural reading of the proposed regulatory language, which is the most choate information offered in the Proposal. But even that reading is not certain, and the commenter states given the very broad discretionary nature of the exemptions provided for in the rule text (see, e.g., 83 Fed. Reg. 18774, proposed 40 C.F.R. §30.9, giving the Administrator the ability to grant an exemption to the data publication requirement if that is "impracticable," which is undefined in the Proposal). As a result, the commenter states that it is unclear how this rule would apply to EPA decision making, and how it would be different than the Agency's current protocol for considering studies relevant to specific rulemaking proposals. (6916)
 While asserting an interest in "ensur[ing] that the data and models underlying science is [sic] publicly available in a manner sufficient for validation and analysis," the commenter notes that there is no more specific statement explaining how that would be implemented. (6916)
 The proposal also omits critical protocols and methodologies necessary to put this rule into action and does not fully explore the implications of implementing a rule of this nature. (9224)
Commenter (6046) contends that several of the requests for comment topics should have been evaluated and decided upon by EPA and then incorporated into the proposed rule. The commenter adds that just throwing out a rule without tools to implement the rule is inefficient and a waste of time. For example, the commenter notes that the EPA requested comment on the following: "EPA solicits comment on implementation of the proposed regulation, including which parts of the Agency should be responsible for carrying out these requirements." The commenter states that the EPA best knows which part or if multiple parts of the Agency should be responsible. Further, the commenter notes that a request for comments on implementing the regulation is analogous to saying that our agency's goal is to fly every person to the moon and then ask how the agency should do it.
Commenter (6104) raises questions that states believe should be considered before the rule is finalized. These questions include:
      1. How would the $100 million economic threshold analysis be implemented? Would new rules be assessed differently than updates to current rules?
      2. Where data masking, coding, or de-identification is not technically feasible, will EPA still consider using high quality scientific research?
      3. While it appears this rule would only apply prospectively to regulations, would there be any impact to science historically used to inform regulations that have existed for years/decades? How will EPA ensure the science remains timely?
      4. If EPA were to phase in the requirements or prioritize certain specific actions, will states have any role in helping identify those priorities?
      5. What impact, if any, would this rule have on institutions of higher education, hospitals, and other nonprofit organizations that received EPA funding through grants and cooperative agreements?
Commenter (6109) states that the EPA should provide the public with the answers to the very questions it poses, including:
 When or how it will develop procedures for protecting private information such as patient health records of the sort currently kept confidential by researchers;
 How the proposed rule will be applied to its regulatory actions and under what circumstances;
 Whether other regulations might be necessary to implement the proposal, or whether statutory changes would be needed;
 Whether and how the agency will establish exceptions, including whether it will provide case-by-case exceptions;
 Whether the proposal will apply to a broader or narrower set of regulatory proceedings than it would as currently drafted, or whether certain categories will be exempt; and
 Whether the agency will add stronger data access requirements to the terms of research grants
Commenter (6133) notes that proposed §30.7 states that "EPA shall conduct independent peer review" without providing any coherent explanation or accompanying regulatory text about what that means: how will that peer review be conducted? By whom? Who will select the peer reviewers? How many will there be? Who will assure their independence and expertise? Will peer reviewers be subject to federal conflict of interest rules and policies? Will peer reviewers be anonymous? Where will the funds come from to conduct EPA peer reviews for "all pivotal regulatory science"? Has EPA estimated how many instances of "pivotal regulatory science" it anticipates conducting peer review for in one year? In prior years? Will the peer review be conducted openly and publicly? Will it be conducted in accordance with the Federal Advisory Committee Act? What will the duration of any peer review be? What purpose will that peer review serve? How will it affect future regulatory decisions? Or will it? Will there be an administrative docket? Will any product of the peer review be included in the administrative dockets for rulemaking? Will peer reviewer comments be part of the certified record for judicial review? Will the agency seek deference from future reviewing courts for the views expressed by peer reviewers? Does EPA not believe that peer review conducted by professional journals and societies is valid? Or sufficient? On what basis does EPA think professional peer review is invalid or insufficient, considering there is not one iota of evidence or support for that belief in the Proposal or the accompanying docket? What is the basic justification for proposed §30.7? The commenter asserts that the Proposal provides no answers to these questions.
Commenter (4831) suggests that the potential impacts of the proposed rule on the quality of regulatory science will depend on many aspects of the rule's implementation that are not described in detail in the Federal Register notice, including the following:
 Criteria and processes to make objective and transparent decisions about which studies will be included in scientific analyses used to inform federal regulations;
 Approaches for evaluating the data and models used to characterize the dose-response relationships underlying federal regulations; and
 Approaches for protecting the confidentiality of certain kinds of data while balancing the need to make data publicly available.
Response: As discussed in the preamble of the final rule, the EPA intends to develop internal implementing guidance and promulgate either statute-specific science transparency regulations or programmatic guidance implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers. The final rule also provides more detail regarding selection of pivotal science, consideration of restricted access to underlying data to protect privacy, criteria for the Administrator's exemption, and peer review.
In brief, the EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. In compliance with the rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. The EPA provides these factors in 40 CFR 30.5(d) and they are discussed more fully in the preamble of the final rule. The EPA will comply with OMB guidance on peer review for pivotal science and will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. Criteria for the Administrator's exemption are provided in 40 CFR 30.7.
The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (5181) states that they find it difficult to predict the effect of the proposed transparency rule given the broad and general nature of the described intent and applicability, and lack of information regarding how it would be implemented in practice. According to the commenter, it is impossible based on the information provided in the Federal Register notice to fully evaluate the potential demand on time and agency resources, and to ascertain the benefits or impediments that might result from the proposed regulation. The commenter expresses concern that the notice fails to provide sufficient detail for an analysis of whether the new approach will achieve the stated purpose without creating unintended consequences that could make it difficult for states to implement clean water programs.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule.
As detailed in the preamble to the final rule, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to pivotal science for which the underlying dose-response data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
The EPA also agrees that the benefits of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. The EPA emphasizes, however, that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. As discussed in Section III.A.1, the main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
<IP_ADDRESS>.5 Definitions
Comment: Commenters (2426, 3971, 5170, 6125, 6126, 6133, 6137, 6194, 6351, 6360, 6361, 6362, 6373, 6447, 6449, 6875, 6910, 6915, 6924, 6955, 8281-PH20, 9224) express that definitions are not defined or ill-defined. Comments include:
 Commenter (6373) asserts that the impacts of the proposed changes will be difficult to predict unless EPA better defines what data the Rule applied to, and terms such as "independent validation." Commenter (6126) suggests that after EPA clarifies definitions, a new round of public comment will likely be needed to address the basic questions raised by EPA in the proposed rule on other matters.
 The EPA solicits comment on the definition of "pivotal regulatory science" and the "dose-response data and models" and how to implement such definitions" (page 13 of 27; second para). The commenter responds to this solicitation by stating that this question is not trivial, but it can be resolved by assembling small groups of dedicated scientists who will invest the time to share information with the Agency. The commenter notes that EPA did this on many occasions in the 1980s and 1990s when they were working on the cancer risk assessment and exposure assessment guidelines. According to the commenter, they participated on several highly productive SABs that saved the Agency tens of thousands of wasted hours. (3971)
 There are multiple terms that will need to be defined in the final rule. In particular, they suggest that "transparency"; "data", singular versus set; and "reasonable effort/endeavor", in relation to how much work the agency must put in before justifying the use of data that cannot be made available to the public. The commenter suggests that if the agency wants a rule focused on the raw data then EPA must better define what "data" would be included within this rule. The commenter inquires: Is it just data produced by EPA? If not, is it the methods and protocols or the actual raw data? (6875)
 The terms "pivotal regulatory science," "replication," "reproducible," and "research data," are not defined or are problematic. (6924, 8281-PH20)
 Neither the proposal, nor the legislation it is based on, define "reproducible," or "replicable" as used in the rule and are in fact not clear on how broadly this goal might be applied. §30.5 (information must be sufficient to "replicate" findings). The commenter notes that under standard usage, a study can be replicated without access to the underlying data. (6125)
 The proposed §30.2 refers to "pivotal regulatory science as the studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." The commenter adds that this definition is distinguished from "regulatory science," defined as "scientific information, including assessments, models, criteria documents, and regulatory impact analyses, that provide the basis for EPA final significant regulatory decisions." According to the commenter, these two definitions can be interpreted as simultaneously referencing something identical as well as one being a subset of the other. Therefore, the commenter contends that the definitions are vague and need clarification. (6362)
 Commenter suggests that the Agency clearly communicate its expectations regarding "replicability", "validation" and availability of "data" to scientists designing and publishing their studies. (6449)
 The Proposal argues that research must be "replicable" without defining what that means. The commenter provides that many studies cannot be specifically repeated, especially those that examine the impacts of historic events, such as the exposure of a half-million Americans to no longer-existing levels of air pollution, or the health effects stemming from a massive oil spill. The commenter notes that subsequent, similar studies from around the world have echoed their findings on health impacts. The commenter asks "[w]hich concept would EPA consider as replication?" (6351)
 Commenter expresses that the definition of the goal of study validation, is missing and no indication is given as to how public access is actually expected to produce validation. The commenter states that, based on the top-down nature of the proposal and the views of the former EPA Administrator who promoted it, the commenter notes that it is possible to guess at the unspoken definition: a study is validated if no stakeholder or true believer in alternate models can find anything to criticize about it after a (post-hoc) analysis of a study's data, which the commenter states is an impossible standard to meet. (6955)
 Although the proposed rule includes a definition for "pivotal regulatory science," the definition is useless. As defined, the commenter states that "pivotal regulatory science" could be as narrow as the study or studies from which a specific point of departure was selected, or as broad as all information considered by EPA in a risk assessment or rulemaking. The commenter notes that the public has no way of knowing how narrow or broad EPA intends this proposal to be. (6447)
 The proposed rule would require the Agency to make certain data and models are "publicly available in a manner sufficient for independent validation." Proposed §30.5. The commenter provides that, as the SAB notes, the concept of "validation" is not clearly defined in the rule. Generally speaking, the commenter contends that due in large part to the vague language used, the proposed rule gives EPA broad discretion to suppress science. The commenter expresses that this appears to be the whole point of the proposal. (6447)
 Where possible, the EPA should be explicit about important definitions, including "publicly available," "pivotal regulatory science," "dose response data and models," and "validation." (2426)
 Definitions that would be helpful in framing the proposed rule include "pivotal regulatory science," "dose response data and models," and "validation." According to the commenter, although "pivotal regulatory science" is likely intended to mean just the key study or studies underlying a regulatory analysis, it could be argued that supporting studies (i.e., those that help to build the weight of evidence or describe the mode of action) are also pivotal to the analysis. In addition, the commenter notes that it is unclear what the term "dose response data and models" includes. The commenter suggests that this term could mean anything from summary data to specific calculations and code, which matters for the potential sensitivity of the information. Finally, the commenter recommends that the EPA define what it means to "validate" study results. The commenter contends that this could be interpreted to mean that the study must be repeated (i.e., a new group of human subjects are exposed in a controlled experiment or are assessed in an observational setting) or simply that the statistics and models are recalculated from the existing data. Further, the commenter suggests that the EPA determine what level of validation is necessary for pivotal regulatory science so that it can be consistently applied. (2426)
 The term "independent validation" is insufficiently defined in the Rule. The commenter provides that validation typically refers to the ability to replicate a study's results, based on the published methods and using the original data. The commenter notes that, to be independent, validation must be performed by groups without conflicts of interest related to the exposures and outcomes of the original research and its findings. According to the commenter, independence is not clearly required by the Rule, and therefore "independent" could be interpreted to require only that validators be external to the original research study team. The commenter contends that, by not clearly defining independence, the Rule creates the opportunity for groups with strong economic conflicts of interest related to environmental regulations to try to undermine scientific processes related to validation. The commenter asserts that, unless the Rule clarifies the process for the "independent validation" and who would conduct it, such "validation" may lead to an analysis of poorer scientific quality than the original study. (6373)
 The proposal does not indicate how "validation" will occur. The commenter notes that scientists familiar with these types of studies are also familiar with the protocols under which these studies are carried out and the studies are peer-reviewed, providing an independent, assessment of the studies. EPA has not identified any deficiencies with the current, time-tested approach, and thus, EPA's desire for "independent validation" is unclear how EPA believes it would be best suited to provide this validation and how this independent review is completed. (6361)
 The commenter provides that outside the context of this proposed rule, "publicly available" generally means that information is available without restriction to anyone who wants it. According to the commenter, while this level of accessibility is acceptable for most information, it becomes problematic for sensitive personal information, personal health information, or CBI. The commenter recommends that the EPA strongly consider implementing a formal request and approval process for qualified researchers to obtain access to data that is protected by confidentiality laws, rather than aiming for all data to have unrestricted access. According to the commenter, this process would still allow identifiable or fairly refined de-identified data to be re-evaluated by experts external to the EPA or the original study authors, while protecting the privacy of the study participants or businesses who created the model or dataset. (2426)
 The proposed rule is lacking in defining the boundaries of the terms used in section I. General Information, including, " strengthen ," " transparency ," " publicly available ," and " inform ." The commenter encourages the EPA to revisit these terms and discuss how the public will benefit from this new proposed rule. The commenter raises the following questions regarding the ambiguous nature of the listed terms:
 How would this new transparency initiative change things? Is the proposed rule referring to research, general science, or regulation outreach?
 What specific things are happening now that are not considered to be transparent in nature?
 How will resources become more publicly available? Will there be more online outreach regarding policy & regulations? Will there be additional supporting documents and resources for pivotal regulatory science research? How will this be more publicly accessible? (6910)
 The proposed rule lacks definitions as well as key implementation and operational information, including critical information about what it would cover and how it would be implemented. The commenter states the following terms need to be explained and operationalized:
 replication
 to the extent practicable
 minimizes costs
 not feasible. (5170)
 The proposal lacks definitions for many key terms. The commenter suggests, in particular, "transparency"; "data", "singular" versus "set"; and "reasonable effort/endeavor", be defined in relation to how much work the agency must put in before justifying the use of data that cannot be made available to the public. The commenter adds that EPA should also clarify what it considers to be "publicly available". (9224)
 The purported need and objective of the Proposal is couched in references to "transparency" and "validation" but that these terms are not defined in the Proposal, nor do they have a single definition amongst professionals in the fields impacted by the Proposal. The commenter adds that the Proposal also improperly uses distinct concepts, such as "reproducibility" and "replicability," as though they are interchangeable. (6194)
 The proposal defines "pivotal regulatory science" as "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." 83 Fed. Reg. 18773 (April 30, 2018). The commenter notes that the proposal does not specify to what extent studies must support regulatory decisions to be considered a "driver," who will determine what qualifies as pivotal regulatory science, or at what stage of the rulemaking process such determinations will be made. The commenter adds that the proposal is also unduly vague in its use of undefined terms that are subject to interpretation, such as the use of the term "uncertainty" in §30.6 of the proposed rule. 83 Fed. Reg. 18774. (6915)
 Commenter (6137) notes that the Proposed Rule defines "regulatory science" as "scientific information . . . that provide the basis for EPA final significant regulatory decisions." 83 Fed. Reg. at 18,773. The commenter states that this definition lacks any discernible meaning. According to the commenter, there is no standard or definition for determining when scientific information does or does not "provide the basis" for a regulatory decision, nor are there any standards or definitions for what subset of "regulatory decisions" should be deemed "significant." Likewise, the commenter notes that the Proposed Rule defines "pivotal regulatory science" as studies that "drive the requirements and/or quantitative analysis" of EPA's action, id., but the commenter notes that this too lacks any understandable meaning. The commenter states that there is no standard governing what science does or does not "drive" the regulatory action. The commenter contends that the lack of regulatory standards here means that EPA staff will impermissibly determine applicability of the rule "based upon their own unwritten personal standards." White v. Roughton, 530 F.2d 750, 754 (7th Cir. 1976).
 Commenter states that, while proposed §30.2 specifies that "all terms not defined herein shall have the meaning given them in the Act or in subpart A," the proposal nowhere says to what "Act" it is referring. The commenter states the proposal purports to implement multiple Acts administered by EPA, with different terms and definitions and court interpretations that may contradict one another. The commenter states that nowhere does the proposal square this factual and legal reality with structure of its unlawful approach, and the language in proposed §30.2. The commenter states that, in the Proposal at issue here, it is incoherent and internally inconsistent across the different statutes that EPA administers. The commenter states it also is not clear to what "subpart A" EPA is referring, because there is no citation to the Code of Federal Regulations. If this is intended to reference 40 C.F.R. Part 30, Subpart A, that Subpart was removed from the C.F.R. in 2014. See 79 Fed. Reg. 75,871; see also 80 Fed. Reg. 61,087. (6133) Similarly, one commenter states that §30.2, addressing definitions, unhelpfully provides that all undefined terms "shall have the meaning given them in the Act or in subpart A," without identifying either the Act or subpart A. (The commenter notes that these are yet other examples of failure to notify the public of the substance and basis of the rule). (6125)
Response: Based on a consideration of this and other similar comments, the EPA clarified and modified key definitions in the supplemental notice of proposed rulemaking and final rule. The EPA considered comments received on the 2018 NPRM and 2020 SNPRM when finalizing the definitions included in the final rule.
Comment: Commenter (6360) expresses that the EPA could fundamentally improve the rulemaking through more precise definitions in both the preamble and regulatory text states that the rule's definitions should start with grounding EPA's rulemaking in fundamental terms. The commenter provides suggestion for definitions for what they believe are the most foundationally important terms here:
 Science. The study of nature as it is. Science can also refer to the collection of scientific facts and relationships between these facts. For example, biology is a branch of the study of nature that is concerned with living organisms.
 Scientific fact. A clam about nature that is objective and reproducible. A scientific fact must be transparent to be objective  -  no matter who the viewer or experimenter is, the same result of nature is observed. Objectivity is a necessary foundation for reproducibility. If a claim is not transparent enough to be objective or reproducible, it is not a scientific fact.
 Applied Science. The use of scientific facts to create a tool. Fields like engineering, toxicology, economics use scientific facts to construct models, cell phones, bridges, dose-response relationships, markets, and many other things that give us longer, happier, and safer lives.
According to the commenter (6360), the EPA's rulemaking's scope is applied science. The commenter notes that the EPA must use tools developed by the applied sciences to achieve its statutory responsibilities. Traditionally, according to the commenter, terms important to EPA's purview such as "best available science" encompass both scientific facts and applied science constructions. The commenter asserts that choosing among different models, dose-response relationships, or mousetraps involves judgement -- e.g., what is "available" and what is "best?" The commenter states that this judgement is independent from the question of whether a claim is a scientific fact. The commenter asserts that error is the choice to apply it where it is not useful and that the practical question is EPA's judgement. The commenter states that using the word "judgement" for EPA's application of science carries with it a minimum expectation of transparency in our society. We expect almost all our government decisions of judgement to be open and to be able to be observed by all citizens. The commenter asserts that trials, notice-and-comment rulemakings, permits, records of decision, and many government judgements contain opportunities for public comment and levels of transparency.
Response: Based on a consideration of this and other similar comments, the EPA clarified and modified key definitions in the supplemental notice of proposed rulemaking and final rule. The EPA considered comments received on the 2018 NPRM and 2020 SNPRM when finalizing the definitions included in the final rule.
The EPA does not find it necessary to include definitions for the terms identified by the commenter but agrees with the commenter's general observation of the need for transparency in government decisions. The EPA believes the final rule will result in final significant regulatory actions and influential scientific information based on the highest quality studies that maximize transparency.
<IP_ADDRESS> Proposed Rule Requirements are Based on a Fundamentally Invalid Assumption
Comment: Commenter (4589) notes that the proposed rule makes a fundamentally invalid assumption that the creation of public policy typically has been and should be based on specific studies targeted to policy application. According to the commenter, this is an unwarranted assumption because it suggests that individual studies must be policy-relevant and therefore subjected to a standard of evaluation laid down by a risk manager that is different from the study's authors or other scientists. The commenter contends that the proposed rule appears to ignore the practical problems of conducting human epidemiologic studies, which are often of value only in retrospect, especially toxicological studies dealing with exposure-response relationships and low-dose extrapolation.
Response: As described in the preamble to the final rule, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
<IP_ADDRESS> The Process EPA Used to Develop the Notice of Proposed Regulation Lacked Transparency, and Thoroughness
Comment: Commenters (0055, 0057, 1742, 2907, 3644, 3971, 4594, 4831, 4894, 5022, 5165, 6115, 6119, 6125, 6126, 6144, 6194, 6373, 6879, 6894, 6897, 6904, 6924, 8272, 8281-PH39, 9227) express concern that the process EPA used to develop the NPRM lacked transparency and thoroughness.
Commenter (6125) asserts that the substantive concerns about inadequate justification can only be magnified by the process EPA followed in developing the proposed rule, which was anything but transparent and indicates that the proposal was not the result of reasoned consideration. The commenter notes that, even their imperfect knowledge of the speed with which this proposal was put together, and the specific steps taken to develop it, indicates a complete departure from approaches that have been used by all past EPA Administrations for developing rules, policies, and procedures that involve the assessment and use of scientific information.
Commenter (6126) provides that the Commission on Evidence-Based Policymaking, established through bipartisan legislation in 2016, recently completed a lengthy fact-finding and deliberative process, followed by the release of unanimous recommendations relevant to EPA's proposed rule. The commenter states that the commission's fact-finding process included seven public meetings, three public hearings, a Federal Register request for public comment, a survey of federal offices, and meetings with representatives of more than 40 organizations. According to the commenter, this thorough effort to understand the implications of using data along with the necessary protections for confidential data -- which included participation from researchers, former program administrators, privacy experts, and dozens of federal agencies -- demonstrates the importance of giving this regulation due consideration with extensive stakeholder feedback. The commenter contends that the commission's research process itself could serve as a valuable model for EPA in taking additional steps to develop this proposal.
Commenters (0046, 1689, 5178, 6155, 6918, 8272) suggest EPA treat the proposed rule as an ANPRM. Commenters (6155, 6918) recommend that EPA review the comments it receives on the current proposal and then issue a new or supplemental proposal with revised proposed regulatory language that reflects the full range of issues on which the Agency has sought comment.
Commenter (1689) joins others in asking EPA to withdraw this proposal and re-propose the rule after considering the many comments received, rather than proceeding to a final rule. The commenter states that EPA's request for comment "on whether the disclosure requirements applicable to dose response data and models in the proposed rule should be expanded to cover other types of data and information, such as for example economic and environmental impact data and models that are designed to predict the costs, benefits, market impacts and/or environmental effects of specific regulatory interventions on complex economic or environmental systems "is more like an ANPRM than a proposed rule. The commenter suggests that EPA treat the Proposed Rule as an ANPRM, and that EPA review the comments it receives on the current proposal, then issue a new or supplemental proposal with revised proposed regulatory language that reflects the full range of issues on which the Agency has sought comment.
Commenters (5178, 8272) urge EPA to undertake a more thorough process to identify if problems with data transparency exist that impede health protections, including by issuing an ANPRM and requesting a study by the National Academies. The commenters state that, if a problem is identified, EPA should then issue a proposed rule that includes a full analysis of costs and benefits.
Commenter (6918) suggests that, because the Proposed Rule identifies a number of open questions, many of which address core issues, EPA should evaluate and reach decisions on before issuing a new or supplemental proposed rule. The commenter states that the ANPRM approach would allow the Agency to address some lack of clarity in the language of the proposal.
Commenter (6918) expresses that, given the breadth of the request for comments in the preamble as compared to the proposed regulatory language, they suggest EPA consider treating the Proposed Rule as an ANPRM. The commenter recommends that EPA review the comments it receives on the current proposal and then issue a new or supplemental proposal with revised proposed regulatory language that reflects the full range of issues on which the Agency has sought comment. The commenter notes that the Proposed Rule identifies a number of open questions, many of which address core issues that EPA should evaluate and reach decisions on before issuing a new or supplemental proposed rule.
Commenter (6918) asserts that the ANPRM approach would allow the Agency to address some lack of clarity in the language of the Proposed Rule. For example, the commenter states that the heading of proposed 40 C.F.R. §30.7, "What role does independent peer review in this section?" seems to be missing a verb. See 83 Fed. Reg. at 18774. In addition, the commenter provides that some of the citations in the authority section of the proposed rule text, id. at 18773, appear incorrect. The commenter expresses that Section 115 of CERCLA is codified at 42 U.S.C. § 9615, not at 42 U.S.C. § 9616, as indicated in the proposed regulatory language; Section 7009 of the RCRA (or the Solid Waste Disposal Act), 42 U.S.C. § 6979, concerns labor standards; Section 8001 of that act, 42 U.S.C. § 6981, which concerns "Research, demonstration, training and other activities," would appear to be a more appropriate citation as providing authority for the Proposed Rule.
Response: Based on a consideration of these and other similar comments, the EPA issued a SNPRM in 2020, which included clarifications, modifications, and additions to certain provisions proposed in the 2018 NPRM and allowing for additional public comment. Public comments received on both the 2018 NPRM and the 2020 SNPRM were considered in the development of the final rule. In addition, as detailed in the preamble to the final rule, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements. In addition, the EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
Commenter (9227) provides that Executive Order 12866 requires "an assessment of the potential costs and benefits of the regulatory action" in addition to other analyses. As indicated above, the commenter states that the Proposal gives scant consideration to the costs of the proposed action.
Commenter (6126) notes that reports that EPA did not follow the agency's own internal guidance for development of regulations through the Action Development Process (ADP) are alarming. As the ADP states, the commenter provides that the guidance helps to "ensure that Agency actions are of consistently high quality, involve senior managers early in the development process, are supported with strong analysis, and are developed via an open process." According to the commenter, failure to engage in thoughtful policy discussion with EPA's career senior leaders likely resulted in a less well-developed and lower-quality proposed regulation than the agency could have otherwise put forward for public debate. Altogether, the commenter states that there is an irony in EPA's rationale that the proposed rule is intended to promote transparency while the very process used to develop the regulation lacked transparency and engagement.
Commenter (8272) asserts that the EPA's process for promulgating this rule represents a gross deviation from the agency's typical action development process. The commenter expresses that the EPA failed to perform an appropriate regulatory impact analysis, despite indications that implementation costs could be substantial. The commenter adds that the EPA does not appear to have met with outside groups during the development process, or with its SAB, which states that the rule "deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board."Taken together, the commenter states that these deviations from typical process leave EPA unable to consider its proposal accurately, and constitute an arbitrary and capricious approach to rulemaking.
Response: Public input was an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018, and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental notice of proposed rulemaking and took public comment from March 18, 2020 to May 18, 2020. All of these comments were considered in developing the final rule.
The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits of the proposed action. As discussed in the preamble to the final rule, this is a rule of internal procedure promulgated under the EPA's housekeeping authority and a benefit-cost analysis is not required. However, the EPA has included a discussion of costs and benefits in the preamble of the final rule.
Briefly, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
The EPA also agrees that the benefits of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. The EPA emphasizes, however, that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. As discussed in Section III.A.1, the main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
This action did not trigger the PRA; because it does not contain any information collection activities, it does not impose an information collection burden. Comments related to the deliberative interagency review process are outside the scope of this rulemaking.
Comment: Commenter (6879) cites Pruitt's Utterly Opaque `Transparency' Proposal," by Professor Daniel Farber (April 26, 2018):
      This proposal raises a host of legal and policy issues ... I wanted to kick off the conversation with a basic legal point: the document fails to meet the fundamental legal requirements for a valid rulemaking proposal. It is vague as to the actual parameters of the proposal, open-ended in terms of the alternatives under consideration, and utterly lacks the data transparency it purports to advocate. Any rule issued on the basis of this notice will be invalid, quite apart from its merits. ...
      Really, the agency might just as well have said: `We want to do something about data transparency. Just spit balling, folks, but here's one idea. We're also open to doing something completely different. Let's just bat this around and see what we come up with.' That might be fine for a brainstorming session over a few beers. But it's not what the courts expect from a notice of proposed rulemaking.
Response: Based on a consideration of these and other similar comments, the EPA issued a SNPRM in 2020, which included clarifications, modifications, and additions to certain provisions proposed in the 2018 NPRM and allowing for additional public comment. Public comments received on both the 2018 NPRM and the 2020 SNPRM were considered in the development of the final rule. As detailed in the preamble to the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
<IP_ADDRESS> EPA's Proposed Data Transparency Regulation Can Be Substantially Improved to Promote the Rule's Stated Goal, But Substantial Modifications Will Be Needed
Comment: Commenter (6126) states that the preamble to EPA's proposed data transparency regulation indicates that the purpose is to "strengthen the transparency of EPA regulatory science." The commenter notes that their assessment of the proposed regulatory text is that this goal cannot currently be achieved based on the information provided in the Notice of Proposed Rulemaking in conjunction with the current EPA organizational infrastructure. The commenter contends that the proposed rule is vague in ways that make some details of its likely implementation difficult to understand in the absence of further elaboration or subsequent implementation guidance. Based on the issues EPA solicits comment on, the commenter (BPC) offers 10 areas for consideration about how to improve the proposed rule consistent with the principles of evidence-based policymaking, while also setting an expectation about responsible and reasonable implementation of an effort that encourages science transparency while protecting the confidentiality of data subjects. The commenter states that their comments on the proposal identify select areas for improvement or concern and are not intended to be comprehensive of all aspects of the rulemaking that may need modification.
Response: Based on a consideration of these and other similar comments, the EPA issued a SNPRM in 2020, which included clarifications, modifications, and additions to certain provisions proposed in the 2018 NPRM and allowing for additional public comment. Public comments received on both the 2018 NPRM and the 2020 SNPRM were considered in the development of the final rule. As detailed in the preamble to the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. The EPA also intends to develop internal implementing guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
2.1.8 Other Effects/Impacts
 <IP_ADDRESS> Openness and Transparence Improvements Will Take Time and Needs to Be Developed to Benefit and Not Harm Research
Comment: Commenter (6873) states that everyone in science acknowledges there is room for improvement when it comes to openness and transparency, which is why the systems for handling transparency and fact-checking communication (peer review, journal publishing, conferences, replicability, statistical analysis, etc.) are slowly changing for the better. The commenter notes that perhaps where the conflation comes in about secrecy is that, at the study level, there is a tolerance of and even need for certain research practices that are not entirely open (including but not limited to the privacy concerns mentioned above. The commenter asserts that this will continue to be a reality even as science becomes more open -- there will be a continued need to preserve the research and development (R&D) incentives of drug companies by keeping certain data private, for instance, and to recognize that researchers want to avoid being scooped (to preserve their job security and future funding) by releasing their data too soon.
Commenter (6873) adds that, overall, the global research community is working hard to make the transformation to more openness and transparency but it's a long way from getting there and they are only at the starting edge of discussing how some of these changes will be able to happen (especially with regard to open data). The commenter notes that animating concern in this global push for transparency and openness is to benefit science and society. Along the way, though, the commenter states that there is a need to be careful to ensure that the openness and transparency improvements we make end up benefitting research and not harming it.
Response: The EPA agrees that transparency and openness are benefits to science and society. The EPA also agrees that this rulemaking augments other important transparency initiatives occurring both inside and outside of the federal government.
<IP_ADDRESS> Limits on the Use of Scientific Evidence by EPA
Comment: Commenters (0040, 0043, 0055, 0057, 0061, 0064, 0566, 0569, 0670, 1011, 1012, 1170, 1358, 1433, 1796, 1822, 1943, 1994, 1997, 2170, 2171, 2259, 2336, 2353, 2361, 2428, 2429, 2431, 2479, 2495, 2548, 2907, 2908, 3131, 3132, 4410, 4542, 4588, 4590, 4591, 4595, 4596, 4597, 4840, 4842, 4844, 4877, 4878, 5011, 5129, 5165, 5170, 5172, 5173, 5176, 5178, 5184, 5977, 6046, 6103, 6108, 6110, 6111, 6114, 6119, 6121, 6123, 6125, 6127, 6130, 6131, 6133, 6137, 6142, 6144, 6145, 6152, 6153, 6156, 6157, 6158, 6159, 6166, 6168, 6170, 6171, 6173, 6188, 6194, 6351, 6355, 6357, 6373, 6378, 6426, 6447, 6450, 6464, 6872, 6874, 6877, 6879, 6880, 6882, 6884, 6893, 6894, 6896, 6897, 6904, 6909, 6915, 6919, 6920, 6924, 6932, 6941, 6955, 8272, 8275, 8317, 9227) oppose the limits on the use of scientific information/best available science. Commenter (6125) asserts that, without the information provided in excluded research studies it would no longer be possible to assess the full weight of the available scientific evidence  -  a key guiding principle for judging the scientific integrity of the decision-making process. Comments include:
 The proposal could result in unprecedented limits on the use of scientific evidence by EPA. The commenter notes that these limits in turn will shape the development of evidence-based public health policies, including air pollution standards, drinking water regulations, pesticide tolerances, worker protections, and more. (0566)
 The proposed rulemaking jeopardizes the integrity of the science that the EPA relies upon. (1012)
 The promulgation of this proposed rule would set a dangerous and potentially life-threatening precedent regarding the use of health-based data, modeling, and research in regulatory decision-making. (4410)
 Studies will have difficulty being approved by the IRB if raw patient data must be made public. (4844)
 If EPA's rule takes effect, it could introduce selection bias that may slow studies and alter results and thereby affect regulatory decisions. (4878)
 Commenter asserts that they depend on the science the EPA uses to protect their health and the health of their family and community. The commenter notes that the proposed rule, which would place unprecedented limits on the use of science at the EPA, would prevent the agency from using the most strongly scrutinized peer-reviewed science. (5129)
 Commenter states that multiple aspects of the proposed rule will be costly, including its demand to make data publicly available and its metrics of high-quality studies that cover a broad range of models and doses. The commenter states that costs such as these not only eat up scarce resources, but they also lead to selection biases regarding which studies EPA will include in risk analysis. The commenter states that smaller organizations will likely lack the financial resources both to support such comprehensive studies and to make their data publicly available, so their studies may be excluded. The commenter states that, alternatively, if EPA is covering part of the cost of making data public, it may choose fewer, more general studies to include in its analyses. (6168)
 Denying the EPA the ability to use studies without making all the data public would seem to cripple the standard-setting process with no actual gain. (5170)
 The proposal would dramatically limit the human health studies the EPA can use to support development of health-protective regulations, eliminating critically important research from the policy-making process. (5172)
 By disallowing EPA researchers and analysts the use of study data which are not published, the non-published study data, regardless of their quality and applicability, will not be used or discussed by EPA, thus hiding their existence from the public. Thus, according to the commenter, the public is less informed about the spectrum of data and a thorough critical weight of evidence evaluation cannot be conducted. (6046)
 The Proposal may have the unintended consequence of excluding sound, peer-reviewed science in the rulemaking process. (6108)
 Although the proposed rule may be striving to improve regulatory science, they fear that it could be a case in which the "best is the enemy of the good. The commenter contends that the EPA cannot let their pursuit of "perfect science' lead them to disregard good scientific studies. The commenter suggests that, just as scientists cannot pick and choose the data they use for analysis, it is imperative that EPA use all available scientific studies to formulate its decisions. The EPA should not ignore existing data. (6110)
 State and tribal environmental agencies rely on the EPA to provide accurate scientific information. The commenters contend that the proposed rule would slow the EPA's work to a near standstill or leave it open to criticism in terms of credibility, leaving states with data gaps that they cannot fill. (6114, 6131, 6361)
 The implementation of the proposed rule would undermine the ability of the Agency to use the best available science to protect public health and the environment. The commenter adds that the proposal will not improve the use of science at EPA, but instead would restrict the types of science the Agency may use in regulatory decision-making. (6166)
 Many academic scientists whose research is relevant to EPA regulations may not conduct and report their studies in a way that satisfies the requirements of the proposed rule. The commenter adds that the proposed rule's provisions would require significant additional resources and could impose unreasonable and impractical requirements beyond those included in current protocols. The commenter notes that academic researchers, who often study sensitive and relevant health effects that are not evaluated in industry-sponsored GLP studies, typically focus on publishing their studies in peer-reviewed journals and obtaining research funding; they may not be concerned about or even consider whether their studies would qualify for use in establishing EPA regulations, and/or may not have the resources to reshape their approach to maintaining data. Additionally, according to the commenter, many researchers, particularly in other nations but also in the United States, may not even be aware of EPA requirements for a study's use in regulations. (6915)
Commenters (0043, 5176, 6880) object to the proposed rule because it would weaken science by rejecting a century of scientific and medical knowledge and progress. Comments include:
 The proposed rule will weaken science by drawing a line in the sand that effectively disregards scientific discoveries prior to the first part of the 21st century when the importance of open data started to be increasingly recognized. The commenter states that the EPA proposal would throw out the mass of accumulated scientific evidence from the past, when open practices were not customary  -  and indeed often not practical before computers for big data were available. The commenter adds that most studies that would be relevant to EPA take years to do, and even longer to filter through to affect policy. The commenter states that recognition, for instance, of the dangers of asbestos, took years because the impacts on health were not immediate. The commenter states that work demonstrating the connection occurred many years ago and the data are not openly available, yet the EPA's proposed rule would imply that it could be disregarded. The commenter states that, similarly, open data demonstrating the impact of lead in paint or exhaust fumes, or of pesticides such as DDT are not available: does this mean that manufacturers would be free to reintroduce these? (0043)
 The proposed rule, if applied retroactively, would prohibit the EPA from considering studies completed prior to the effective date of the rule for which the data are not publicly available. The commenter states that this entails rejecting the contributions of thousands of individuals who -- for the benefit of future generations and to advance the public good -- agreed to take part in research. The commenter states that many important public health and environmental studies have taken place after one-time events and natural disasters and replicating these studies would be ethically impossible, as doing so would require intentionally exposing people to harmful contaminants or recreating one-time public health disasters. The commenter states that disqualifying such studies, as the rule proposes to do, means losing valuable public health knowledge, for instance, on the effects of environmental pollutants on human health. (5176)
 Commenter (6880) suggests the rule would apply its restrictions to any scientific study conducted before or after the rule is promulgated. The commenter states that, since most science conducted prior to the Internet age would likely fall short of the rule's transparency standard, the proposed rule would essentially throw out a century (and more) of scientific and medical knowledge and progress. The commenter states that this certainly contradicts the stated purpose of the proposal in the preamble, which is that the proposal will "help ensure that EPA is pursuing its mission of protecting human health and the environment."