Relatedly, the commenter (12727) contends that the proposal could impede any EPA rulemaking, or joint rulemaking, for which the National Environmental Policy Act (NEPA) requires an environmental impact statement (EIS). The commenter contends that the Safer Affordable Fuel-Efficient (SAFE) Vehicles Rule is a case in point. The commenter asserts that there, NHTSA prepared an EIS examining the emissions, air quality, and health impacts of weakening fuel economy standards, with EPA as a cooperating Agency in the preparation of the EIS, where NHTSA averred that it prepared the EIS "[t]o inform its development of the final CAFE standards." Thus, to the extent that EPA's Greenhouse Gas (GHG) standards in the SAFE Vehicles Rule conform to NHTSA's CAFE standards in the same joint rule, the commenter notes that the studies considered in the EIS also qualify as "pivotal regulatory science" that "drive[s] the requirements and/or quantitative analysis of [an] EPA final significant regulatory decision." The commenter states that NEPA requires the Agency to take a "hard look" at the relevant data and, in some cases, gather more data, and at the very least, the Agency must explain why data it omitted "would not alter its conclusions in the EIS or the approval of [the action]." According to the commenter, the SNPRM here would summarily rule out data that NEPA requires the Agency to include and consider in an EIS.
Response: The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to any applicable procedural and analytical requirements under those statutes and judicial review.
14.2.2 The EPA's Readiness to Implement the Rule
Comment: Commenter (10974) assert that there is no evidence that EPA is prepared to implement this rule. The commenter states that the SNPRM continues to leave a void around the process for implementation, thereby leaving their comments original 2018 unaddressed.
Response: This final rule is effective upon publication in the Federal Register. As outlined in 40 CFR 30.3, the provisions of the rule apply to significant regulatory actions for which a proposed rule was published in the Federal Register after the effective date of the rule and influential scientific information submitted for peer review after the effective date of the rule. The implementation guidelines are forthcoming. The comments on the 2018 were considered in the development of the SNPRM and all the comments were considered in the development of the final rule.
14.2.3 Implementation Timing/Process
Comment: Commenter (12399) contends that Section 30.3's exclusions of drafts and preliminary analyses are too vague and may exclude exceptionally important research. The commenter provides that it might make sense to exclude preliminary analyses that are not publicly available, however, anything the Agency releases or discusses publicly as a possible justification for a regulatory standard or for educational purposes should meet the transparency rule. The commenter notes that draft risk assessments, for example, may have profound market impacts, including impacts on litigation, and hence, should not be exempt from the rule.
Response: EPA notes that this final rule does not categorically exclude studies. Section 30.3 of the final rule states, "The provisions of this part do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review. In the event the procedures outlined in this part conflict with statutes the EPA administers, or their implementing regulations, the statutes and regulations will control."
Implementation guidance and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (11499, 12397) requests EPA clarify that the statement "at the time [a rule] ... is developed" means that EPA will apply the transparency rule to the proposed rulemaking stage, if not sooner, that is, data and models underlying any pivotal regulatory science or pivotal science that EPA might use in promulgating a significant regulatory decision or making influential scientific information final will be made available for analysis and comment in connection with the notice of proposed rulemaking. The commenter asserts that if this interpretation is incorrect, EPA should revise the rule to make clear that the proposed 40 C.F.R. Section 30.5 applies at the proposed rule stage as well as to the final Agency action.
The commenter (11499) states that, as noted in their comments on the 2018 proposal, the proposed rulemaking stage is when public stakeholders will need to weigh in on potential concerns related to the scientific basis for EPA's proposal and when efforts at independent validation of the relevant data might occur. The commenter contends that deferring public access to the "pivotal regulatory science" or "pivotal science" until the final rulemaking is issued is inappropriate, as it will be too late for active public engagement. The commenter also provides their concern that the statement could be interpreted as excluding subsequent reanalysis of the data used to develop a rule during the period between the proposed rule and the final rule. Therefore, the commenter requests that EPA include language explicitly addressing data reanalysis in its final rule. As currently proposed, the commenter provides that EPA has not specified how or when the public will be able to engage in the reanalysis of data or independent validation. The commenter notes that obtaining and reanalyzing data that are made public through this rule could take more time than has traditionally been provided to submit comments on a proposed rule and EPA must account for this step in the public comment process to allow meaningful engagement.
Response: The final rule states that the EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Also, to assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information.
This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
14.2.4 Implementation Costs and Benefits
Comment: Commenters (11192, 11354) contend that there are no analyses regarding costs, the types of research affected, or the existing rules that may be called into question when their underlying scientific basis is dismissed.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This internal procedural rule does not interpret or apply provisions of a particular statute or statutes that EPA administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling.
<IP_ADDRESS> Assessing Costs and Benefits
Comment: Commenters (11176, 11894) state that this rule should not be finalized without a full cost-benefit analysis.
Comment (9372) notes that they do not yet know all of the costs of pollution and accompanying benefits of regulation. The commenter states that pollution is actually far more costly in lost years of life, health-related costs, human capital costs, lost wages (Borgschulte, Molitor and Zou, 2017; Isen, Rossin-Slater and Walker, 2017), lost tax revenue, and crime (see Feigenbaum and Muller, 2016; Aizer and Currie, 2017; Billings and Schnepel, 2017) than they ever guessed. The commenter contends that ignoring the indirect costs and benefits means that new research on the costs and benefits of pollution would not be taken into consideration, which could have disastrous effects on public policy.
Commenter (11911) notes that purporting to comply with executive orders that require economic impact analysis of regulations, EPA claimed that "the benefits of [the Strengthening Transparency in Regulatory Science (STRS) Rule] justify the costs." The commenter contends that in reaching this conclusion, however, the Agency both drastically underestimated the costs of compliance with the Rule, and more importantly, neglects to consider how the "[e]xclusion of relevant studies... will adversely affect decision-making processes," imposing serious costs on the health and well-being of the American people.
The commenter (11911) states that for support of its conclusion, EPA cites to a single working paper issued by the Mercatus Center at George Mason University. The commenter notes that this paper estimates the economic impacts of a policy with a very different scope: it assumes that while 80 percent of studies will fail to meet transparency criteria, these studies will nevertheless continue to be relied upon in Agency rulemaking. According to the commenter, under the proposed STRS Rule, however, such studies would be excluded entirely from Agency consideration. According to the commenter, the Mercatus paper, therefore, fails to capture the STRS Rule's far-reaching costs.
The commenter (12430) provides that the purpose of regulatory impacts analyses -- to help the public and the Agency itself understand the costs and benefits of a significant rule -- is thwarted when impacts are not critically assessed, yet EPA did not estimate the increased resources necessary for the Agency to implement the SNPRM, even though, complying with the rule would considerably burden Agency resources. The commenter notes that EPA did not attempt to quantify the purported benefits of providing tiered access to restricted data and models or the profound impacts to Agency protection of public health and the environment. The commenter contends that not only does this failure violate Executive Order 12866, it precludes fully-informed public comment on this highly-consequential and far-reaching set of proposals.
Commenter (12727) asserts that the SNPRM would effect a major change in how EPA establishes public health protections without any consideration of the impacts or burdens that the policy would inflict upon the Agency, the research community, vulnerable populations, or the public at large. The commenter states that in comments on the 2018 proposal, they and others demonstrated that EPA failed to provide any assessment of the costs and benefits -- either quantitatively or qualitatively -- and thus fell gravely short of basic requirements for reasoned decision-making. The commenter contends that far from correcting that fatal shortcoming, the Supplemental Notice exacerbates the problem by conjuring an even more expansive policy from the same deficient record; the word "cost" appears nowhere in the Supplemental Notice, except in titles of two documents cited for other purposes.
According to the commenter (12727), it is arbitrary and capricious to "`entirely fai[l] to consider an important aspect of the problem' when deciding whether regulation is appropriate." The commenter provides that as in Michigan, EPA's present failure to consider the costs and benefits of a regulation where there is no statutory bar to doing so is arbitrary and capricious.
Furthermore, the commenter (12727) states that EPA's failure to characterize costs and benefits necessarily precludes the Agency from assessing whether the costs are disproportionate to the benefits, a finding that could render the proposal arbitrary and capricious. The commenter notes that this failure extends to harms that the proposal would inflict upon the public at large, vulnerable populations, the research community, and the Agency. The commenter asserts that EPA does not identify any statutory authority that could justify its failure to assess the costs and benefits of the proposed rule, and it is certain that the Agency could not rely on a radical use of a generally applicable statute like the Housekeeping Act in order to evade well-established requirements of administrative law.
Commenter (12731) states that an underlying assumption of the tiered access approach is that researchers involved in the development of pivotal science or pivotal regulatory science will participate. The commenter states that, however, the EPA has provided no analysis indicating the extent to which the Agency believes compliance could or would occur. The commenter states that, instead, it is reasonable to assume that many researchers would not engage in a tiered access approach for a variety of reasons, the significant anticipated costs being just one for which EPA has utterly failed to provide any analysis.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See III.J of the preamble for further discussion of benefits and costs of the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
<IP_ADDRESS>.1 Conducting Assessment at a Later Date
Comment: Commenter (12727) provides that EPA might analyze costs and benefits of future rulemakings that implement this policy is no substitute for analyzing costs now. The commenter contends that moreover, in future rulemakings, the methodology of this policy would already be baked into the process, and disentangling its specific impacts would be virtually impossible. The commenter notes that to do so, EPA would have to identify which studies it would have relied on (or weighted differently) but for this rule, the difference in the standards or other outcome that would have resulted, and the costs and benefits the counterfactual standards would have yielded, and even then, this rule would have been long since finalized, so any assessment of costs and benefits would come far too late to influence EPA's decision-making. The commenter also asserts that analyzing the costs of this rule only as-applied in future rulemakings would be unlikely to capture costs imposed on the research community or resulting from the public's reluctance to participate in studies as a result of this policy. The commenter provides that, as explained in their previous comments, the influential scientific information to which this proposal would apply may be utilized in processes other than EPA rulemakings -- such as to inform decisions by state and local health agencies -- that would not provide even a theoretical opportunity for EPA to assess the costs and benefits of this proposal.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. The EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS>.2 Assessment is Necessary for Meaningful Public Participation
Comment: Commenter (11480) states that the SNPRM does not include any feasibility or cost analysis for either approach to data or the implementation of any portion of the proposal. The commenter provides that the proposal does not specify the means by which it will establish a system for tiered access, instead citing only the Research Data Center at the National Center for Health Statistics as a model.
According to the commenter (11480), the policies contained in this proposal, particularly if they include making massive amounts of raw data publicly available, may be very costly to the American people via the Agency's budget. The commenter contends that EPA may require those costs to be borne by researchers who (1) would be forced to account for these costs in budgeting for future research projects or (2) would be forced to find funding for this disclosure for current or completed studies. The commenter provides that EPA cannot and should not move forward with implementing a broad reaching policy in which roles and responsibilities are unclear and the costs and benefits have not been analyzed, disclosed, and available for public comment.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. The EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
Comment: Commenter (11894) states that EPA fails to provide any information on such basic matters as potential impacts of the expanded proposal under each of EPA's substantive statutes; the potential social costs in the form of decreased protection to public health; the economic costs (the tiered access option could be very costly); and any benefits of the supplemental proposal. The commenter contends that EPA does not discuss the need for and implications of fundamental changes in position from such things as EPA's own peer review guidance. According to the commenter, commenters can only guess as to each of these matters; it is not their responsibility to have to guess. The commenter maintains that EPA thus has failed to provide any notice of issues of fundamental import, and failed to provide adequate notice and opportunity for comment. According to the commenter, the Agency has also completely failed to meet the rulemaking standards required under FCC v. Fox Television, 556 U.S. 502 (2009). The commenter notes that these errors deprive commenters of the most basic information needed to formulate reasoned comments and so are prejudicial.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
Comment: Commenter (14397) contends that EPA does not appear to have conducted a cost-benefit analysis on either the original proposal or the supplement, nor has a Regulatory Impact Assessment been done. According to the commenter, the lack of proper analysis leaves commenters, and the EPA itself, in the dark as to what the results of this action will be.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
<IP_ADDRESS> Costs to the Agency
Comment: Several commenters (11350, 11395, 11408, 11478, 11484, 11492, 12715, 12405, 12422, 12720, 12727) state that analyzing similar house legislation, notably the 2015 Secret Science Reform Act and the 2017 HONEST Act, that would impose similar obligations on EPA reveals that the cost of implementation would be between $100-250 million annually, resulting in a significant burden on the Agency's current activities.,,,,,,,  Specific comments on the overall cost of the rulemaking include:
 Commenter (11408) contends that expanding the scope to include "influential scientific information" will only further direct resources toward unnecessarily scrutinizing nearly every study the Agency considers due to the extensive review process this SNPRM would entail.
 Commenter (11484) states that the SNPRM comes at a potentially significant cost without contributing to any serious effort to improve the transparency and validity of scientific information.
 Commenter (11492) states that the expansion will impede both the speed and accuracy of EPA decision-making, something that is particularly important during a pandemic.
 Commenter (12720) notes that it is not practical for EPA to arbitrarily screen specific studies, and EPA has not articulated in the supplemental proposal how such requirements will be implemented without significant interference in the Agency's work. According to the commenter, the supplemental proposal does not engage with the complexity and cascading ramifications of the requirements it spells out and should be withdrawn.
 Commenter (11479) contends it is clear that the current proposal would create a tremendous time and resource burden on EPA and on scientists outside of the Agency, who would be forced to track down, collect, compile, and post (including any statistical methods needed to conceal private information) all the data, models, code, and other analytical tools for each of the hundreds of studies EPA cites on any given decision.
 Commenter (12720) maintains that despite this significant cost estimate, EPA does not confront the financial dimensions or the need for financial incentives to support the unprecedented data release requirements in the rule. The commenter asserts that it also does not consider the fact that scientists do not typically receive funding to make the data underlying peer-reviewed studies available for public inspection.
 Commenter (12727) notes that alternatively to the $100-250 million cost estimate, Congressional Budget Office (CBO) estimated that EPA might reduce those costs to $5 million over a five-year period, but only by "significantly reduc[ing] the number of studies that the Agency relies on." The commenter asserts that EPA failed to address these costs -- including the effects of potentially relying on fewer studies -- in the 2018 proposal, and it has failed to do so again in the Supplemental Notice, even though commenters provided these figures in their 2018 comments. The commenter contends that EPA's present failure to address the CBO estimates is even more arbitrary than in 2018 -- and not only because the estimates are now clearly in the record. The commenter provides that in addition, the Supplemental Notice more closely parallels the bills that CBO reviewed due to the expanded scope of Agency actions and studies it would encompass.
Response: While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
Comment: Commenter (11408) provides that the rulemaking would do little, if anything, to improve the quality of EPA's decision-making and enhance the scientific integrity of its program implementation, as documented in detail above. The commenter notes  -  without necessarily endorsing  -  that the Supreme Court has suggested that a hallmark of rational regulatory decision-making is that the costs created by a regulation not grossly outweigh the benefits it produces.
Response: EPA disagrees that this rule would do little to improve the quality of its decision-making and the scientific integrity of its programs. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11403) states that a study using epidemiological data would require independent validation by EPA scientists even though it has already undergone the peer-review process. The commenter states that this is duplicative and wastes Agency resources.
Commenter (14397) states that this expanded proposal would mean that the EPA would either have to spend hundreds of millions of dollars to have private information redacted or move ahead without the benefit of the dozens of valid studies that cannot legally, ethically or efficiently publicly release all of their data.
Commenter (12715) states that, although data may technically be "available," it is likely to be costly to obtain.
Response: As was noted in the SNPRM, "Although the ability to independently validate pivotal regulatory science or pivotal science is a key component of this rulemaking, EPA would like to clarify that neither the proposed nor the alternative 40 CFR 30.5 would require that EPA, a member of the public or other entity must independently validate a study before it can be considered to be pivotal regulatory science or pivotal science."
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA to disclose, redact, or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12720) asserts that if the final rule applies only prospectively to new Safe Drinking Water Act (SDWA) advisories and benchmarks and toxicity values used in Superfund cleanup, EPA will still need to replace the current methodologies for these recommended values and revise the hierarchy of human health toxicity values in order to incorporate the new requirements, incurring significant costs and time lost to replacing these methodologies.
Response: The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to judicial review.
Comment: Commenter (12715) asserts that EPA ignores the large costs that would be associated with the complex process and fails to identify who would pay for these procedures. The commenter notes that as scientists from the Union of Concerned Scientists has discussed, redacting confidential data from large studies "`isn't just blocking out a line' It's a huge job that can occupy entire offices for thousands of hours," at commensurately high cost. Furthermore, the commenter notes that in a 2018 article discussing this same issue with respect to the HONEST Act, the Union of Concerned Scientists also said that:
      EPA career staff argue that the HONEST act would incur additional costs and time to implement, would limit research that gets conducted, and would deter industry and academics from working with the agency. In their comments to the CBO, [EPA] staff said that, "In addition to spending dollars and staff time on requesting and getting data from study authors, creating [information technology] infrastructure and a data management system to manage, store, and archive large volumes of data, and making the data available in a format that is useful and accessible to the public, EPA would also have to spend dollars and staff time combing through these extensive datasets to find and redact Personally Identifiable Information and Confidential Business Information.
Response: While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12727) states that EPA has not estimated the proportion of valid, relevant research that it would disregard under the supplemental proposal. The SAB, according to the commenter also noted that "[i]t is not clear: (1) how many of the studies EPA currently relies upon to take important regulatory actions would meet the public disclosure standards stated in the Proposed Rule." According to the commenter, the CBO has estimated the number of studies that legislative proposals similar to EPA's rule would affect, both overall and at the level of individual actions, such as reviews of the NAAQS. The commenter asserts that EPA presumably has better information than CBO does about the volume of scientific information it relies on across its programs as CBO relied on information from EPA on how the Agency would implement the legislation, as well as the cost of complying with it, in developing the CBO estimate. Using updated information, the commenter contends that EPA must conservatively estimate both the absolute numbers and the proportion of studies it would rule out through this SNPRM -- overall and in each foreseeable individual action -- reasonably assuming that tiered access and de-identification are not feasible. The commenter states that, similarly, the Agency must conservatively estimate the number of studies that would be given lesser consideration under the alternative approach described in the supplementary proposal. The commenter provides that the omission of any such analyses renders the proposed rule arbitrary.
Response: The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to judicial review.
<IP_ADDRESS> Costs to the Research Community
Comment: Commenter (11361) states that publicly available datasets, especially those created under federal research grants, enable more researchers to utilize data that has already been collected and broadly increases research efficiency. The commenter provides that primary data collection is often extremely expensive and time-consuming, while secondary data analysis can often be accomplished more rapidly and on a much leaner budget.
Commenter (11395) provides that the proposed regulation would impose significant costs on research scientists. The commenter notes that formatting, preparing and de-identifying datasets and associated materials to be publicly available or to conform with EPA's proposed tiers of data availability would be extremely time and resource intensive. Further, the commenter states that according to a 2002 report from the National Academy of Sciences (NAS), re-analyses of studies almost always require the participation of the original researchers to provide additional information and support, at an additional cost of personnel time and resources.
Commenter (11482) notes that EPA fails to define or consider the financial costs required to implement the proposed rule and queries who will fund the collection, de-identification, storage, and dissemination of such an enormous amount of sensitive data? The commenter asserts that the Agency has not formally announced who would pay for rule implementation, and no new federal money for the Agency or outside scientists has been proposed. The commenter maintains that in suggesting that the potential costs of the rule would be minimal, the EPA ignores the costs to researchers (who would have to pay to set up and maintain data sharing for their previously published studies to be considered), to the EPA (for presumably conducting the multiple re-analyses required in section 30.6 of the rule), and to public's health (for the very real costs in dollars and human lives of undermining existing regulations by limiting access to science).
The commenter (11482) contends that had EPA adequately considered costs, it would have determined that the collection, deidentification, storage, and dissemination of sensitive data in a manner sufficient for independent validation are likely to cost millions of dollars. The commenter states that the proposed rule fails to offer any guidance on how these strategies can be achieved with zero funding and upon whose shoulders this cost burden should fall.
The commenter (11482) provides that commenters must rely on the only applicable estimates that currently exist  --  two CBO reports on (failed) federal legislation that was substantially similar to the proposed rule. The commenter notes that in 2015 and 2017, CBO estimated that a similar science program would cost $1 million dollars per year to implement; however, a 2017 cost estimate from the EPA itself placed that number at as much as $250 million. The commenter asserts that one thing is certain: the "public availability" requirement in this proposed rule would cost a great deal of money to carry out. The commenter contends that if EPA is not covering this cost, scientists and scientific institutions will bear the lion's share of this burden at a time of shrinking budgets, resulting in less science  --  not better science  --  and weaker public and environmental health decision making.
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
If researchers want to increase the likelihood that their studies receive greater consideration by the EPA, they may take steps to ensure that the underlying dose-response data are available to the greatest extent possible. But any such response to this final rule would be purely voluntary. It is not required by this rule.
Comment: Commenter (11487) provides that because Research Data Center (RDC) contains data collected by Centers for Disease Control and Prevention (CDC) and other federal agencies, not by independent academic researchers, a great deal of additional work and infrastructure would be necessary before it could host the full range of data that should be informing EPA work. The commenter notes that fact that RDC charges researchers $3,000 to access a single year's data hints at the substantial costs associated with the enterprise. The commenter states that EPA does not appear to have calculated the costs that the Agency and outside researchers would incur in order to create a system that evaluates researchers' proposals and provides them with access to the appropriate level of data. The commenter contends that EPA also has not explained who would be in charge of making those determinations -- a crucial issue given that communities disproportionately harmed by pollution might not trust EPA to make them.
Response: The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12546) notes that the CBO's analyses were underestimates, as they did not include the costs to research/ academic scientists. The commenter provides that making datasets publicly available along with "associated protocols...and detailed descriptions of how to access and use such information" would entail significant time and costs to format, prepare and, in the case of human data, attempt to de-identify individual results from the men, women and children who participated in the study. The commenter notes that such re-analyses almost always require the participation of the original researchers to provide additional information and support, which costs personnel time and resources. The commenter provides that a 2013 memo from the Office of Science Technology and Policy on increasing data access acknowledges these costs and directs agencies to "Allow the inclusion of appropriate costs for data management and access in proposals for Federal funding for scientific research."
The commenter (12546) asserts that in contrast, the supplemental proposal contains no provisions that address the funding needed for academic scientists to make their datasets publicly available and support re-analyses --  and if the data are not publicly available, EPA may downgrade or fail to incorporate it in relevant policymaking.
Response: This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
Comment: Commenter (12291) states that creation of both the workflow implemented to create the analytical data set used and the statistical software used to analyze the data set in a manner that enables even highly-qualified investigators to reproduce the same results is difficult and time consuming. The commenter notes that that this is likely to be costly to the researchers and their institutions  -  both in the creation of the data and methods hubs, and in the continuous requests for further explication from those seeking the data.
Response: This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
14.2.5 Implementation Bias
 <IP_ADDRESS> Rule Favors Certain Stakeholders
Comment: Commenters (10055, 11330, 12522, 12727) contend that the SNPRM provides for implementation in favor of industry stakeholders and/or special interests. These concerns include:
Commenter (10055) asserts that the SNPRM would open up the process of standard-setting to lengthy debates arising from the "reanalyses" and "alternative" models funded by the regulated industries. Further, the commenter states that the proposed supplement to the Transparency Rule gives immense power to regulated industries in the development of policy and regulation. According to the commenter, there are no provisions for controlling conflicts of interest in the "reanalysis" of data, the "alternative models", or the "independent validation".
Commenter (11330) contends that the rulemaking is not about transparency, nor about protecting human health or protecting our environment. The commenter states that this emperor simply has no clothes and must again ask EPA to put science and public interest ahead of political and special interests and withdraw this rule based on its negative impacts on science, and its negative impacts on our public health.
Commenter (12422) notes that polluting corporations, attempting to defend their products and emissions, will no doubt be able to provide whatever data and materials supporting their positions in any format the EPA might demand. The commenter states that this rule could impose an onerous and burdensome requirement on independent researchers so that much environmental science, particularly epidemiologic studies, would be excluded from the evidence base and become irrelevant to efforts by EPA to protect the public and the environment.
Commenter (12727) provides that not all funders would be equally disinclined to support research meeting EPA's requirements. The commenter notes that for instance, research backed by industry that profits from pollutants or toxicants might be adequately resourced to adapt to EPA's new requirements, since such funders would have a financial interest in which studies the Agency considers. The commenter states that the proposal could thereby skew the research landscape in favor of studies whose funders hope to reap a monetary benefit from the outcome of the research.
Conversely, commenter (12692) states that conflicts of interest in the rulemaking process are ubiquitous. The commenter notes that although participants may be honest people, few are honest brokers -- persons without an interest or stake in the outcome. The commenter provides that stakeholders include not only the industries subject to the Agency's regulations but also the advocacy groups that gain influence and contributions by supporting or opposing the Agency's agenda, the researchers who receive Agency grants to study issues of regulatory interest, and the Agency itself, whose scientific assessments drive the scale and scope of its regulatory activity.
According to the commenter (12692), although every stakeholder is an interested party, disclosure requirements that bring conflicts of interest to light can help make the process more competitive, nothing in the Transparency Rule would change any conflict of interest disclosure regulations or any part of EPA's scientific integrity policy.
Response: The EPA does not agree that its approach will lead to systematic bias towards certain types of stakeholder goals.
The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
14.2.6 Rule Includes Insufficient Implementation Detail/Clarity
 <IP_ADDRESS> Need for Implementation Clarity
Comment: Commenter (10042) provides that there are two distinct ways transparency is used, and understanding the distinction between the two conditions of transparency gives clarity for the EPA in their direction and guidance for the respondents to make substantive and meaningful comments to the EPA' s proposed rulemaking.
The commenter (10042) states that the first way transparency is used is to understand that it is woven into the fabric of science and how scientific processes are carried out. The commenter asserts that the key attribute of this transparency is there is no choice. According to the commenter, one cannot choose to eliminate transparency from a scientific endeavor and still have a scientific activity; if there is no transparency, there is no science.
Although, the commenter (10042) provides that there are conditions and circumstances where transparency is a choice. The commenter notes that situations where individuals or groups are allowed to see a decision-making process across a spectrum of levels from non-participative observation to active and participative contribution in the final decision; this is transparency by choice.
Response: This rulemaking is one incremental step to increased transparency. The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Comment: Commenter (11495) indicates several specific areas where they assert there is a lack of clarity on the proposed balance of the new shared responsibilities, including:
 How will an investigator know if their study is being considered as part of an Agency action that would fall under this rule? Will the EPA directly contact individual researchers or is it expected that individuals must preemptively be aware of these requirements and act accordingly?
 Who will determine what comprises a complete data set sufficient to meet the requirements of this new rule?
 Who will determine what is adequate with regards to the tiered access proposed in the supplement?
 Who will be responsible for hosting the data, and providing funding for data hosting, so that it can be made available to the public?
 Who is responsible for screening those that request the data to determine which tiered level of data should be made available to them?
 Who is responsible for interactions between those requesting data and those hosting the data in order to ensure those requesting the data can access only those tiers that they are eligible to receive?
 Who is responsible for mitigating any dispute between those requesting data and those hosting the data with regards to data access based on the proposed tiered system as well as the sufficiency of the provided data to enable the proposed reanalysis in the new rule?
 When will data be required to be available in order to be included in informing EPA actions?
The commenter (10042) contends that the reality is that the answer to all these questions is the same: the EPA proposes to wield capricious authority to determine if researchers have complied with these new requirements without providing any resources, clear guidance, or accountability for how this new rule will be applied as part of future Agency actions.
Response: This procedural rule does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. Rather, it exclusively pertains to the internal practices of the EPA. The rule does not require the EPA to disclose or host data, but to determine if data and models are available and to give greater consideration to those studies for which data and models are available. Hence, this rule does not impose costs on the EPA or any other party to make data and models available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the rule.
The EPA will not identify a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying data or models are publicly available or available through restricted access. Restricted or tiered access in this rule means that the underlying data or models are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This rulemaking applies prospectively to final significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future final significant regulatory actions and influential scientific information, it applies equally to all the data and models underlying studies used as pivotal science.
Implementation guidance and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (11576) states that EPA has failed to articulate clearly what the proposed rule would even require, obscuring the implications of the regulation with vague language and broad grants of discretion. According to the commenter, the SAB repeatedly emphasized this problem in its comments on the proposal, noting, for example, that:
 "[C]ertain key terms and implementation issues have not been adequately defined or described" in the proposed rule;
 "Given the relatively skeletal nature of the Proposed Rule, it is not possible to define the implications of the rule with confidence";
 "To ensure that the rule is evidence-based EPA must provide greater clarity regarding details of the rule and how it will be implemented, as well as example analyses of how it would be deployed";
 "The lack of specific criteria for what might satisfy the [proposal's availability] requirement makes it difficult for the SAB to understand the implications";
 "Given the lack of clarity in the Proposed Rule, it is difficult to understand how this regulatory action could be accomplished in a standardized and consistent manner";
 "[T]he lack of criteria for what data might satisfy the requirements of the Proposed Rule makes it difficult to understand the implications for protection of [personally identifiable information]";"[G]reater clarity is needed in definitions of `data and models' and `pivotal regulatory
science[,]'" as "[t]he definitions provided in the Proposed Rule are not adequate";
 "Case-by-case exceptions [to the proposal's requirements] without criteria may create public concerns about inappropriate exclusion of scientifically important studies";
 "The requirement in the Proposed Rule that `data' be made publicly available is
vague and, as a result, can be interpreted in different ways"; and
 "More clarity is ... needed to define the nature of the `data' that must be publicly available."
The commenter maintains that under the Administrative Procedures Act (APA), a rulemaking proposal "must describe the range of alternatives being considered with reasonable specificity. The commenter states that otherwise, interested parties will not know what to comment on, and notice will not lead to better-informed Agency decision-making."
Response: The definitions in section 30.2 of the final rule are data, dose-response data, independent validation, influential scientific information, pivotal science, publicly available, reanalyze, science that serves as the basis for informing a significant regulatory action, and significant regulatory actions.
"Data" is defined as, "the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party."
"Pivotal regulatory science" has been replaced with "pivotal science" in the final rule and is defined as, "the specific dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information."
Section 30.7 of the final rule states, "The Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration is warranted because:
       technological or other barriers render sharing of the dose-response data infeasible;
       the development of the dose-response data was completed or updated before the effective date of the final rule;
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified."
"Publicly available" is defined as, "lawfully available to the general public from federal, state, or local government records; the internet; widely distributed media; or disclosures to the general public that are required to be made by federal, state, or local law. The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
The EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Implementation guidance and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12464) notes that the SAB highlighted the multiple ambiguities in the Proposal in its recent report. The commenter provides that, as the SAB put it, "key considerations that could inform the Proposed Rule are not present in the proposal, or presented without analysis, and certain key terms and implementation issues have not been adequately defined or described." The commenter maintains that moreover, the SAB concluded that, "[g]iven the relatively skeletal nature of the Proposed Rule, it is not possible to define the implications of the rule with confidence." According to the commenter, if even EPA's expert science advisors cannot determine the implications of the Proposal, it is impossible for the public to understand and provide meaningful comment on the Supplemental Notice.
Response: Section III. A. Purpose and Effect of the Action in the preamble of the final rule attempts to address these concerns. This final rule is effective at publication in the Federal Register. The implementation guidelines are forthcoming.
Comment: Commenter (11496) contends that the rule and SNPRM do not address how the rule may work in practice. In addition, the commenter objects that the SNPRM deems comments that are not directly "addressed, altered, or replaced by the SNPRM" as out of scope when the rule has been issued in pieces and without a full analysis of its impacts. The commenter provides that the SNPRM significantly modifies the effect of the rule and creates an untenable standard where research scientists, clinicians, and peer reviewers must collect and store protected personal data so their work can be utilized by EPA.
More broadly, the commenter (11496) notes that EPA's own SAB described the rule as "skeletal" and stated that the full implications of the rule remain undefined. The commenter concurs with the SAB's conclusions. The commenter adds that EPA's supplement provides additional definitions and exemptions that only serve to add ambiguity about how the rule will be implemented.
Additionally, the commenter (11496) provides that the proposed changes in the SNPRM revise many critically important definitions such as "reanalyze," "independent validation," and even such basic terms as "data" and "models," along with a new use and interpretation of the term "substantially reproduced." The commenter asserts that these new definitions do not resolve the numerous conflicts between this rule and other existing regulations. The commenter is also concerned that the rule language places the financial costs of action above human and environmental health costs. The commenter states for example, the definition of a "highly influential scientific assessment" puts a significant amount of weight on whether an impact of regulation is greater than $500 million in any one year, but does not describe how those costs may be outweighed by health benefits to people and the environment through regulatory action.
Response: Section III. A. Purpose and Effect of the Action in the final rule preamble of the final rule attempts to address these concerns. This final rule is effective at publication in the Federal Register. The implementation guidelines are forthcoming.
The definitions in section 30.2 of the final rule are data, dose-response data, independent validation, influential scientific information, pivotal science, publicly available, reanalyze, science that serves as the basis for informing a significant regulatory action, and significant regulatory actions.
Comment: Commenter (11894) notes that EPA sponsors some studies through grants, some through contracts for various purposes, conducts some studies itself, and yet other studies are submitted to the Agency in response to regulatory requirements. The commenter provides that all these activities take place on a decentralized basis in many different offices. The commenter queries: At what point in the process would studies be selected for the data bar? At what stage in their preparation would the bar apply? Who would enforce it, and under what standards? What rights of appeal would exist? And what would happen to disapproved studies? Would they be withdrawn from the public and burned? If not, what would be their fate?
The commenter (11894) states that the proposal's only approach to addressing these questions states that the data bar would apply before "finalizing" of affected studies by EPA, an undefined term that is too general to be any answer at all.
Response: The Agency will not be removing studies funded by EPA through grants, contracts, and intramural research.
This rule governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
Comment: Commenter (12457) contends that EPA should not exclude studies from policy considerations, but rather include all available literature to obtain a better understanding of relevant science. The commenter continues in stating that while transparency and data sharing are important in science, it should be considered whether the proposed rule is the best method for advancing these initiatives. The commenter notes that other methods for increasing transparency and the availability of data are already underway in the scientific community, including work currently being done by academic institutions, local government, and federal institutes including the CDC, the Committee on National Statistics, and the National Research Council. According to the commenter, EPA should consider the proposed rule in light of these efforts and decide whether there may be better ways to encourage transparency in the science considered in policy decisions. The commenter maintains that if the EPA believes this proposed rule does represent the best methods to accomplish increased transparency, any proposed revisions to the rule should include a detailed explanation of how the EPA reached that conclusion, as well as a description of how this rule will operate in the context of ongoing transparency and privacy protection efforts.
Response: EPA notes that the final rule does not categorically exclude studies. The purpose of this rule is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (12467) urges the Agency to clarify that the proposed rule applies to all stages of the notice and comment rulemaking process. According to the commenter, this would ensure the consistency, transparency and predictability called for by the proposal extends to those stages of the regulatory process at which impacted stakeholders are afforded the opportunity to evaluate and respond to not only the data presented, but also the regulatory and policy actions they influence.
The commenter (12467) states that when impacted stakeholders and the public are provided an opportunity to comment on federal proposals, they must have access to all relevant data to ensure they fully understand the Agency's justification for the proposed rule, and their comments are clearly in response to the selected regulatory option(s) being proposed. The commenter contends that if the Agency does not make clear a full range of options and the data that informed them, including cost benefit data, those seeking to offer public comment cannot hope to be able to respond to the range of actions being considered. The commenter notes that if EPA only applies the provisions of this proposal to final regulatory actions, it is possible that impacted stakeholders will be cut off from responding to critical data that may significantly alter the outcome of regulatory policy decisions.
Response: Section 30.3 of the final rules states, "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science."
The final rule states that the EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (12546) notes that EPA's SAB issued a report earlier this year outlining that it was: "concerned that key considerations that should inform the rule appear to be omitted or presented with no analysis - for example, it is not clear how many of the studies EPA currently relies upon to take important regulatory actions would meet the public disclosure standards in the proposal, whether EPA has assessed how many of those studies would be feasible to provide underlying information, or what the impact of precluding those studies would be on EPA's decision making and its ability to protect public health / environment."
The commenter (12546) also states that the SAB had concerns about the scientific and technical challenges and feasibility of implementing some of the rule's requirements, recommending that the Agency provide more detail on how it can meet the requirement to clearly identify and make publicly available all studies relied upon when it takes "significant final agency action," among other recommendations. According to the commenter, the SAB has also not been additionally consulted, even though they have identified important flaws in the proposal.
Response: The preamble of the final rule addresses some of the concerns from the SAB report on the proposed rule. See sections III.A.1., III.C.6. and III.G. of the preamble of the final rule.
The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
There may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenter (12645) provides that the increasing number of co-authors in published studies raises the question of unanimous consent to data sharing. The commenter questions would all authors need to agree to data-sharing and only one refusal would prevent the study's data from being provided? The commenter states that EPA could find out by sampling the scientific community represented in cited publications underlying its regulatory decisions. According to the commenter, like so many other aspects of the initial transparency rule and this Supplement, EPA has not bothered to do any preliminary analyses or to have such analyses done by the NAS or the SAB.
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
Comment: Commenter (12720) contends that Agency staff would need to conduct large information gathering assessments to determine the degree to which any study previously considered by the Agency in science reviews met the new requirements-- itself a huge task, given that each study would have to be assessed individually. The commenter states that it would be unreasonable at this stage in the process for the Agency not to contact the study authors and inform them of the new data requirements in place at EPA.
According to the commenter (12720), such communications will need to be comprehensively documented and coordinated for Agency use and public reference. The commenter notes that in many cases, it will be difficult to contact the primary study author, since in some cases years (or decades) have elapsed since study publication. The commenter queries: Would additional authors be contacted in this case? What if, upon communication of the new requirements, there arises disagreement amongst the study authors, study funders, or academic institutions about how and whether to make all underlying data available for public inspection?
Response: While the final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12720) states that EPA has not explained in the proposal how the requirement to make the studies publicly available could be practically achieved. The commenter asserts that EPA does not identify any phasing-in period for the requirement, which would significantly complicate EPA's science review process. Additionally, the commenter provides no criteria have even been established by EPA to understand. The commenter notes that the lack of criteria at this stage in the process is problematic because it does not allow researchers--whose work can take many years to plan, fund, conduct, and publish-- adequate opportunity to understand the proposed requirements and submit comments to EPA.
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
Implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12720) states that in EPA's memo to the House Committee on Science, Space, and Technology, "EPA's envisioned implementation  -  absent from the text of the Supplemental Proposal  -  would be as follows:
 The researchers would be responsible for managing the logistics of making the data and models publicly available in a manner that complies with the rule, in consultation with EPA staff.
 The researchers would be responsible for judging the sensitivity of the study's data and models and what information can or cannot be made publicly available through tiered access.
 The researchers would decide what tier of access should be designated for different types of information.
First, the commenter (12720) maintains that EPA burdens researchers with the obligation for managing complicated and costly logistics of making underlying data and models available to the public. The commenter notes that it is not clear how EPA would assure that these researchers actually carried out these logistical details, and no hint of this major burden to researchers is mentioned in the proposal; the logistics themselves are nowhere spelled out in the Supplemental Proposal.
Second, the commenter (12720) states that EPA evidently expects researchers to assess the sensitivity of study data but provides no criteria for doing so. The commenter provides that the expectation that researchers evaluate their own data and models conflicts with the Agency's plan to independently assess whether the research has indeed been made publicly available or whether EPA should grant a waiver from the requirements identified in the Supplemental Proposal. Similarly, the commenter notes that in the third bullet, EPA now says that it expects researchers to ascertain what data access tier is important for the underlying data and models used in their studies. The commenter states that no details are provided to assist researchers in making such important determinations. Further, the commenter contends that EPA's inability "to say how conflicts between the researcher's judgment and the Agency's judgment would be handled" is further indication that the rule is arbitrary and capricious and fails to provide fair notice or any reasoned explanation about this key implementation concern.
Commenter (12720) provides that, under the current iteration of the Supplemental Proposal, the public is left with gaping questions about how the Supplemental Proposal will be implemented. The commenter queries: How would EPA handle a reanalysis conducted by a third party? What procedures will EPA follow for assessing a reanalysis during a rulemaking? How might the absence of peer-review for a reanalysis influence the incorporation of those findings into influential scientific information or a rulemaking? How would EPA balance the timeline of a rulemaking with the timeline of a reanalysis, given that a reanalysis of a major study can take years and may not be completed before a regulatory deadline? The commenter contends that all of these critical questions have not be raised, let alone addressed, by the Agency.
The commenter (12720) notes that EPA seemed to acknowledge as much in its meeting with House staff, where the Agency "was largely unable to address these issues and frequently declined to answer questions, admitting that the Agency had simply not considered the issue this far into the rulemaking process." The commenter contends that not only are the substantive aspects of the rulemaking arbitrary and capricious, but also EPA's glaring failure to notify parties and include information on the content of the rulemaking in the Supplemental Proposal text renders the Proposal arbitrary and capricious and a truly egregious abuse of discretion. The commenter provides that as the memo notes, "the public comment period is no substitute for a well-reasoned, deliberative policymaking process. EPA's inability to answer basic questions about the rule's operation and implementation reflects the ill-conceived nature" of the Supplemental Proposal.
Commenter (12720) considers the situation where a single study examining several distinct health endpoints is consulted in different ways. The commenter questions whether EPA would make a single determination on such a study, regardless of its application in specific regulatory decisions. If so, the commenter contends the supplemental proposal would arbitrarily prevent a study from being considered in different ways for different contexts and regulatory decisions (e.g., in the case where a study examines both commonly and less-commonly studied health endpoints).
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
As was noted in the SNPRM, "Although the ability to independently validate pivotal regulatory science or pivotal science is a key component of this rulemaking, EPA would like to clarify that neither the proposed nor the alternative 40 CFR 30.5 would require that EPA, a member of the public or other entity must independently validate a study before it can be considered to be pivotal regulatory science or pivotal science."
Comment: Commenter (12727) argues that the vagueness of the proposed definition creates problems regardless of how widely or narrowly the rule is applied. Attempting to apply the rule to all studies involved in the development of influential scientific information "would be entirely infeasible, effectively paralyzing the development of influential scientific information and with it the work of the agency." On the other hand, a narrow interpretation "introduces incredible risk for selective and biased application with no accountability."
Response: The definition of "influential scientific information" at proposed 40 CFR 30.2 in the 2020 SNPRM is the same definition as in OMB Bulletin for Peer Review. The EPA proposed to adopt this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. Given that the definition is both established and has been routinely applied by the EPA, the EPA disagrees that the term is inherently too narrow or too broad. Rather than modify the proposed 40 CFR 30.2 definition of "influential scientific information," the EPA is modifying 40 CFR 30.3 in the final rule to clarify the Agency's intent that the requirements in 40 CFR 30.3 apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Final Information Quality Bulletin for Peer Review.
Comment: Commenter (13788) notes that the SAB warned EPA that changing transparency requirements could have negative consequences:
      It is plausible that in some situations, the Proposed Rule will decrease efficiency and reduce scientific integrity, determining if in fact that will be the case requires a thorough and thoughtful examination that is currently absent in the Proposed Rule. Moving forward with altered transparency requirements beyond those already in use, in the absence of such a robust analysis, risks serious and perverse outcomes.
According to the commenter (13788), the SNPRM does not include the additional "robust analysis" of the need for the changed policy called for by the SAB, and instead continues to leave important details regarding implementation of the rule undecided. The commenter maintains that at the very least, EPA must provide a response to the problems raised by the SAB, including some explanation and justification for this rule.
Additionally, the commenter (13788) provides that EPA in the SNPRM proposes to apply its data release requirements across the board, without distinguishing between types of studies and the data they rely on, which may or may not include confidential information. Additionally, the commenter notes that while the 2018 Proposal was limited to dose-response data and models, the SNPRM expands that to all data and models broadly and without justification. The commenter references the SAB's report on the 2018 Proposal that stated:
      the requirement to make data available for public inspection will be more easily implemented for some datasets than others (e.g., [certain] studies include individual sample data as part of standard reporting). The expectation that data and methods will be available for all endpoints may be unrealistic.
The commenter (13788) contends that the SAB was clearly concerned that the rule's requirements could not even be put into practice, never mind whether they are lawful or rational or adequately justified. Additionally, the commenter states that although the SNPRM allows the Administrator to grant an exception to the requirements, the SNPRM provides no reason to believe that provision will sufficiently address the concern identified by the SAB, nor is there any safeguard included to ensure that the EPA Administrator, in exercising his discretion, will not bring extra-statutory political considerations into the decision-making process.
Response: Section III. A Purpose and Effect of the Action in the preamble of the final rule attempts to address these concerns. This final rule is effective at publication in the Federal Register. The implementation guidelines are forthcoming.
Comment: Commenter (11176) states that the supplement introduces a new process that lacks description and that uses language requiring judgement calls for which no protocols are provided. The commenter provides that the supplement calls on EPA staff to determine, for example, what studies are "sufficient for independent validation" and what studies should be given "greater consideration."
The commenter (11176) notes that, starting with the first phrase, "sufficient for independent validation", the EPA's supplement does introduce a definition for the term, independent evaluation: "the reanalysis of study data by subject matter experts who have not contributed to the development of the original study to demonstrate that the same analytic results are capable of being substantially reproduced." The commenter contends that the concern is that there is insufficient guidance or standards for what the terms, "sufficient" and "substantially", mean or who will make the determination (e.g., scientific staff with oversight of an EPA scientific advisory panel). The commenter states that to its credit, EPA is using the 2002 OMB definition for "Capable of being substantially reproduced". The commenter notes that the definition however uses the words, "similar" and "acceptable," which are subject to the same concerns as "sufficient" and "substantially". The commenter asserts that standards for these terms and determination should be included before this proposed rule is finalized and should include sufficient guidance to, for example, take into account how unlikely it is for two studies to exactly replicate. The commenter provides that in 2015 Ralph Cicerone, then president of the NAS addressed this point in a speech, noting that a NAS panel had grappled with the very question: "Because variability across studies is to be expected, how can we assess the acceptable degree of variability and when should we be concerned about reproducibility?" According to the commenter, there should also be guidance to require explanation of whether research has or has not met the sufficient-for-independent-validation criterion.
Additionally, the commenter (11176) highlights the importance of meta-analyses as a research standard and states the fact that the supplement does not reference this well-known and valued scientific practice, which EPA staff regularly uses, reinforces the point that EPA has much work to do in laying the foundation for accomplishing the stated goal of the proposed rule.
According to the commenter (11176), these two issues -- ambiguity of terms and lack of process and transparency -- risk the introduction of bias, if not improper influence, into the EPA process. The commenter states that specifically, the two issues risk "arbitrary and capricious decisions as to what evidence to include or exclude" and "would also often result in regulatory analyses that suffer from biased selection and inadequate statistical power," among other issues.
The commenter (11176) contends that before the EPA implements the rule, it should design and explain its process for considering different studies. Additionally, the commenter provides EPA should also acknowledge the risk of bias in the process and explain how it will work to minimize such risk. The commenter notes that a highly regarded site is that of the Cochrane Collaboration's revised risk of bias framework available at the following url: https://www.riskofbias.info/.
Response: The EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given."
The EPA does not agree that its approach will lead to systematic bias towards certain types of stakeholder goals.
Comment: Commenter (11894) asserts that EPA's alternative proposal on study and model acceptability does not set out any truly binding standards for making decisions. The commenter provides that the factors it mentions are so numerous and poorly articulated that the Agency would remain free to do whatever it wants.
Response: Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given."
<IP_ADDRESS> Added Definitions
Comment: Commenter (12715) The commenter provides that the supplemental proposal both fails to address many of the technical problems identified in the 2018 Comments on the initial proposal and creates several new problems. The commenter states that for example, the clarifications, modifications, and additions to the proposed rule raise significant additional concerns regarding the intent, applicability, and implementation of the rule, as discussed above. The commenter notes that the expansion of the scope of studies that would now be excluded or given less weight is particularly problematic not only because the manner in which EPA proposes to evaluate data and studies for consideration defies basic scientific principles, but also because there is a complete lack of transparency regarding how the Agency will apply the rule.
Additionally, the commenter (12715) notes that the narrative section of the supplemental proposal states that EPA is "modifying, deleting and proposing new regulatory text" in the definitions section, including "deleting the first paragraph of the 2018 proposed rulemaking regulatory text" and the definition of research data, but in the proposed rule section, EPA states that it is "[r]evis[ing] § 30.2 by adding the [listed] definitions . . . to read as follows." The commenter provides that unlike other sections of the supplemental proposal where EPA appears to have included the full revised provision, the proposed rule text for section 30.2 appears to include only the added definitions. The commenter asserts that it is thus unclear which, if any, portions of the original proposed rule text are being "modif[ied]" or "delet[ed]" as the narrative description indicates. The commenter states that EPA's unclear presentation of the changes it is making to this section of the supplemental proposal deprives the public of the opportunity to provide meaningful feedback on this portion of the rule.
Further, the commenter (12715) contends that the definitions are also problematic because the proposed rule repeatedly refers to "significant regulatory decisions," but the proposal fails to provide any definition of the terms "significant regulatory decisions" or "significant." The commenter notes that even more confusingly, "regulatory decisions" are defined in the original proposal as "final regulations determined to be `significant regulatory actions' by the Office of Management and Budget." According to the commenter, the confusion throughout the definitions section infects the entire proposal, because understanding whether and how EPA has defined key terms is critical to understanding how the other substantive provisions of the rule would operate.
Response: The definitions in section 30.2 of the final rule are data, dose-response data, independent validation, influential scientific information, pivotal science, publicly available, reanalyze, science that serves as the basis for informing a significant regulatory action, and significant regulatory actions.
The Agency is changing the term from "regulatory decisions" to "significant regulatory actions" in the final rule. "Significant regulatory actions" is defined as, "final regulations determined to be `significant regulatory actions' by the OMB pursuant to Executive Order 12866."
<IP_ADDRESS> Section 30.5 Alternatives
Comment: Commenter (11192) notes that EPA proposes two alternatives to outright dismissal of studies that can't make their underlying data public for legal or ethical reasons: tiered access to the data (which doesn't ameliorate privacy considerations) and a weighting scheme to devalue those studies. According to the commenter, the EPA provides no accompanying details about how these measures would be implemented, leaving either or both open to manipulation for political ends.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12427) provides that the proposed rule and SNPRM have a:
 Lack of development, demonstration, and evaluation of alternative approaches prior to arriving at proposals for a regulation
 Lack of a detailed evaluation of whether, and if so how, the proposed rule could conflict with existing statutes
 Lack of development of experience with proposed measures by testing them prior to attempting
Response: As discussed in the final rule preamble at I. C. "What is the Agency's authority for taking this action?," this internal procedural rule does not interpret or apply provisions of a statute that EPA administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Alternatives approaches were discussed in the March 2020 Supplemental Notice of Proposed Rulemaking, and commenters were specifically asked to comment on the alternative approaches proposed.
This final rule is effective at publication in the Federal Register. The implementation guidelines are under development and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12715) contends that in the words of EPA's own SAB, "[g]iven the relatively skeletal nature of the Proposed Rule, it is not possible to define the implications of the rule with confidence." The commenter states that in particular, it is unclear which definitions are included in the proposed rule, EPA fails to define key features of the two "options" proposed for determining which data and models may be used, and the proposed rule's exemptions provision contains insufficient standards to guide the Administrator's discretion. According to the commenter, the supplemental proposal thus runs afoul of the APA's substantive and procedural requirements due to its vagueness as well as its arbitrariness, thus depriving the public of a meaningful opportunity to comment. Accordingly, the commenter asserts that if EPA does not abandon the proposal altogether, it should reclassify the proposal as, at most, an Advance Notice of Proposed Rulemaking.
Response: The definitions in section 30.2 of the final rule are data, dose-response data, independent validation, influential scientific information, pivotal science, publicly available, reanalyze, science that serves as the basis for informing a significant regulatory action, and significant regulatory actions.
Section 30.5 of the final rules states, "The EPA shall give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. The Agency shall also give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation."
Section 30.7 of the final rule provides the aspects of a study the Administrator can consider when determining whether to grant an exemption.
Comment: Commenter (10972) asks how the weighting of studies will be determined. Commenters (9362, 10972, 11484, 12715) question who is responsible for determining that "other things are equal" when deciding which studies should be given greater consideration. Commenter (12715) states that neither approach in section 30.5 address concerns regarding the arbitrary exclusion or down-weighting of relevant scientific studies. According to the commenter, Option 1 unlawfully and arbitrarily excludes studies from consideration while Option 2's weighting approach is also arbitrary due to no inclusion of criteria for discretion. Additionally, the commenter points out that the clause "other things equal" is vague and fails to provide criteria to bound the Agency's discretion.
Furthermore, the commenter (12715) contends that the arbitrariness of both alternatives for section 30.5 raises troubling conflict of interest concerns regarding the types of studies that EPA would be able to consider. The commenter provides that studies for which the underlying data are or could likely be made available for independent validation are typically Good Laboratory Practice ("GLP") studies performed by industry. The commenter notes that while EPA has requested comment on "how to ensure that, over time, more of the data and models underlying the science ... are available to the public," as well as "how to provide sufficient incentives and support to researchers to increase access to the data," there is no indication of how this would be accomplished. The commenter states that accordingly, either version of section 30.5 would strongly favor the Agency's consideration of scientific studies conducted by or on behalf of the very industries and special interest groups that will ultimately be regulated by EPA's standards. The commenter maintains that these are the entities whose studies are more likely to have underlying data that is currently available for public validation or who can, going forward, afford to comply with the proposed rule's public availability requirements.
Response: The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given." EPA did not finalized the "other things being equal" concept from the 2020 SNPRM.
The EPA agrees that additional clarification is needed regarding the other factors that the EPA will consider. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available;
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
These factors are intended to assist the EPA in determining the consideration to afford to pivotal science with underlying dose-response data that are not available for independent validation. The final rule requirements and the consideration of these factors apply to any data used in characterizing the relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, regardless of the direction of that effect. Because study quality factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) would have already been evaluated at an earlier stage in the assessment process, the studies evaluated with these additional factors are presumed to be of the highest quality available.
The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12720) contends that EPA does not adequately describe the process by which "greater consideration" will be given to specific studies, and whether the characteristics of specific studies (e.g., sample size, confidence intervals of results, or overall methods validity) may compensate for any lack of full data transparency. The commenter provides that for example, the Agency typically considers thousands of studies in compiling integrated science assessments as part of the NAAQS. The commenter notes that each of these studies may satisfy the new data requirements proposed in the rule to differing degrees, and it is not clear whether or not any Agency-perceived shortcomings in meeting these data requirements may be compensated by other criteria that help EPA to determine the overall degree of study quality.
Response: The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given."
<IP_ADDRESS> Agency Programs
Comment: Commenter (11894) provides that the proposal is thoroughly ambiguous on what Agency activities will be covered. The commenter states that on the one hand, it states that the new provisions would "not apply to any type of agency action" other than "significant" rules, however, it does not define the term "agency action." According to the commenter, "action" could mean literally everything an Agency does in the course of its work; it could mean "agency action" as defined in the APA, but it does not say that. In any event, the commenter maintains that even that term is notably ambiguous and regularly generates difficult administrative law questions. On the other hand, the rule's requirements would extend to all "influential scientific information," which is defined as:
      scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions.
The commenter (11894) contends that given the scope of the expressed concerns, it would seem unreasonable to exempt the majority of EPA activity from its scope by a broad definition of "agency action." The commenter states that a broad definition of "agency action" would make the new restrictions applicable to studies in their less important uses, and inapplicable to studies used for the more consequential task of supporting Agency decisions, seemingly inconsistent with the policy behind the SNPRM.
One such program lacking clarity according the commenter (11894) are the "advisories" and "benchmarks" for state and local water supply systems detailing the potential hazards of pesticides and other chemicals in drinking water. The commenter states that this information has no binding legal effect, but state agencies rely on it to shape their efforts to protect public health. The commenter contends that EPA's proposal, however, does not discuss whether the issuance of these advisories and benchmarks would be regarded as "agency action," exempt from the new requirements, or the impact on the program and how EPA would address that impact, if they are not exempt.
Additionally, the commenter (11894) provides a review of the process of assessment in the Existing Chemicals program that would be impacted should the rule apply retroactively. With only one study used in the most recent trichloroethylene (TCE) assessment in which its raw data was available, the process would essentially shut down, according to the commenter.
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities. Implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12399) states that 40 CFR 30.3 of the proposed rule excludes anything not considered a "significant regulatory action," such as adjudications, enforcement actions, and permit proceedings. The commenter contends that some provision may be necessary to ensure the Agency does not needlessly delay permits and the like, but the rule should apply to Agency-produced science, including risk assessments and research that could have impacts in the marketplace.
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (12720) provides that in briefings to the House Committee on Space, Science, and Technology on April 2, 2020, EPA staff have stated that the proposed rule "would open up existing regulations to being reworked" by confirming to staff that the rule's application to "prospective rulemaking" includes statutorily mandated reviews of existing standards. The commenter states that in the April 2 briefing, EPA noted that such "prospective rulemaking" includes statutorily mandated reviews of existing standards, such as the NAAQS.
The commenter (12720) contends that is highly inappropriate for EPA to make this clarification in a private setting, rather than engaging with the issue within the SNPRM itself, a setting that would provide members of the public an opportunity to comment on the issue. The commenter provides that beyond making this clarification in the briefing, EPA has not explained how the ongoing NAAQS reviews for fine particulate matter and ozone air pollution would proceed in the drastically altered scientific review context envisioned by the SNPRM.
Additionally, the commenter (12720) states that according to the memo, "EPA's envisioned implementation  -  absent from the text of the Supplemental Proposal  -  would be as follows:
      EPA staff would reach out to the researchers involved in a study that the Agency wants to consider during the development of significant regulatory actions or Influential Scientific Information.
The commenter (12720) asserts that these implementation details (none of which are included in the SNPRM), raise several glaring problems. The commenter first notes that EPA regularly consults thousands of scientific studies for influential scientific information like the Integrated Science Assessments for criteria air pollutants under the CAA NAAQS program. The commenter contends that it is not feasible for EPA staff to contact research authors in such a way, particularly because many studies involve multiple authors and conceivably permission would need to be obtained from all co-authors for any new plan to share data with EPA or the public.
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (11495) notes that it is not specified which specific Agency actions will actually be covered under this rule, which individual(s) will be responsible for deciding if an action will be covered, and at which point in the process can this decision be made or revised. To this end, the commenter states that this lack of clarity is likely to lead to inconsistent application of these new requirements.
Commenter (12453) asserts this exclusion has long been recognized by EPA. As such, the commenter wishes to confirm that the supplemental proposal, and any promulgated final rule, does not include individual adjudications or permit proceedings, such as FIFRA or Federal Food, Drug, and Cosmetic Act (FFDCA) decisions on individual pesticide registrations or tolerances. The commenter notes that the scope would more broadly be applicable to rulemakings under FIFRA or FFDCA that do meet the definition of "significant regulatory action."
Conversely, Commenter (12399) commends the definition for "influential information" because it indicates that the rule could also apply to research that impacts both policy decisions and private sector decisions. The commenter asserts that this is important due the marketplace impacts Agency science can have. According to the commenter, it only makes sense that Agency science  -  including "nonregulatory activities" such as research studies or risk assessments  -  complies with the best available scientific standards and transparency requirements.
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. Influential scientific information is defined as the "scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions," consistent with the definition of "influential scientific information" provided in the OMB Peer Review Bulletin. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (12467) states that the inclusion of influential scientific information is limited when compared to the full extent of regulatory and policy statements issued by the Agency and does not ensure impacted stakeholders will be afforded the full benefit of the rulemaking. As such, the commenter recommends that the requirements also be applied to Agency guidance documents due to their potential for impact on the regulated community at the same level as "significant regulatory actions".
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities. The rule also does not apply to guidance documents.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (11892) asks whether individual permits would be considered significant regulatory actions, as attempting to apply this rule would place a large burden on state agencies. The commenter notes that "the statement in Section §30.5 `except where explicitly stated otherwise that provisions of this subpart do not apply to any other type of Agency action, including individual party adjudications, enforcement activities or permit proceedings' should be interpreted to mean that that individual permits are not covered by this rule" because there are already "adequate appeal processes and public involvement steps in place for challenging the validity of an agency's decision without adding the additional steps and criteria in the proposed rule."
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.