Response: EPA has described how it will identify pivotal science in the preamble in Section III.C of the final .rule preamble. Specifically, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate points of departure (PODs)). EPA has finalized the definition of pivotal science using "drive" instead of "underly". Pivotal science are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information.
EPA will identify the pivotal science at the proposed rule stage for significant regulatory actions and for influential scientific information when it is submitted for peer review. In both instances, there will be opportunity for public comment on the designation of a study as pivotal science.
EPA will continue to identify influential scientific information consistent with the OMB Bulletin for Peer Review. To align this rule with the definition of influential scientific information in the OMB Bulletin for Peer Review, the requirements in 40 CFR Part 30 apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Bulletin for Peer Review.
Comment: Commenters suggest the following revisions to the rule in order to address perceived issues with clarity and/or interpretation of pivotal science:
1) Related to 40 CFR 30.3.
          Clarifying the language in § 30.3, as the comma after "data and models" could be interpreted to mean that the proposed rule would only apply to data and models, whereas commenter (12699) prefers the more expanded interpretation that includes all pivotal science and pivotal regulatory science.
2) Related to the definitions of pivotal science and pivotal regulatory science.
 Using the citation of research within an EPA document on which a regulatory decision is based as an indicator of "pivotal science" or "influential scientific information" (12645).
 Designating information as "pivotal" if it comes from a peer-reviewed scientific publication (9362).
 Defining "pivotal regulatory science" as "the study (or studies if necessary) on which the regulatory limit is based," and requiring EPA to provide a transparent explanation of how and why the selected study or studies were chosen over others (12720).
3) Related to 40 CFR 30.5
 Giving more weight to publicly funded studies than to private studies which use proprietary data and giving peer-reviewed information the highest weighting to avoid favoring privately funded research (9362).
 Establishing criteria for selecting studies for decision-making based on (1) representativeness of the broader body of evidence; (2) usefulness; (3) past efforts to reanalyze, reproduce, and confirm the findings, and (4) public availability of data and methods (11921)
 Deleting the phrase "other things equal" in § 30.5 due to its subjectivity (9362).
 Clearly stating in § 30.5 that the rule applies to EPA's IRIS program (11499, 12397).
 Deleting the statement "nonparametric models that incorporate fewer assumptions" in § 30.6 on the grounds that parametric models may be more appropriate in many cases (9362).
 Adding language in § 30.6 "strengthening the concepts that EPA decisions should be evidence-based, pertinent to the issue being addressed and fully transparent, including disclosure of areas of uncertainty and potential confounding of the data" (12397).
Response: Responses to these comments follow.
1) Related to 40 CFR 30.3:
EPA has revised 40 CFR 30.3 to make it clear that the rule applies to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science .
2) Related to the definitions of pivotal science and pivotal regulatory science:
EPA disagrees that pivotal science should be defined simply based on whether a study has been peer-reviewed or whether it is cited in an EPA document used to support a significant regulatory action. Pivotal science is the subset of the science supporting a significant regulatory action or influential scientific information that drives the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information.
See the description in Section III.E of the preamble of how EPA will identify studies as pivotal science.
3) Related to 40 CFR 30.5:
The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence of a relationship between exposure and effect, the EPA will identify those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both of an EPA significant regulatory action or influential scientific information will be identified as pivotal science.
In 40 CFR 30.5 EPA has identified the factors it will consider in determining the degree of consideration to afford pivotal science for which the dose-response data are not available for independent validation. EPA considered the comments and included factors based on usefulness and quality.
EPA has incorporated the language from proposed 40 CFR 30.6 into final 40 CFR 30.5. EPA notes that the language in 40 CFR 30.5 requires a consideration of both parametric and non-parametric models so that EPA can consider both in determining the consideration to give pivotal science for which the dose-response data are not available for independent validation. The commenter has not provided convincing rational on why non-parametric models should not be considered, particularly given that this factor is only one of several considerations in 40 CFR 30.5.
"Other things equal" has been deleted from 40 CFR 30.5.
All IRIS assessments are influential scientific information and thus subject to this rule. Adding a statement that IRIS assessments are subject to the rule would be superfluous.
6.2.2 Scope of Pivotal Science
<IP_ADDRESS> Scope is Too Broad
Comment: Commenters (11390, 11576, 12435, 12546, 12688, 12702, 12727) note that because of the expanded scope of the rule, the lack of clear criteria contained in the proposed rule, and the fact that properly conducted science considers "the entire body of evidence," nearly any study would meet the definition of "pivotal science" or "pivotal regulatory science."
Response: EPA narrowed the scope of the rule from data and models to dose-response data. As EPA has explained in the preamble rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect.
<IP_ADDRESS> Support for Expanded Scope
Comment: Commenters (11499, 11500, 11915, 12399) support the expanded scope of the rule to include all "influential scientific information" and "pivotal science."
Response: EPA notes the commenters support of EPA's approach.
Comment: Commenter (11499) supports retroactive application in stating that existing data and models should be covered if they underlie "pivotal regulatory science" in a rulemaking or "pivotal science" in the development of influential scientific information.
Response: This rule applies to dose-response data, regardless of when the dose-response data were generated, that are considered to be pivotal science in a significant regulatory action proposed after the effective date of publication of this rule or in influential scientific information submitted for peer review after the effective date of this rule.

 Chapter 7: Publicly Available
In the notice of proposed rulemaking (NPRM), the EPA used the term "publicly available" but did not define it at 40 Code of Federal Regulations (CFR) 30.2 or in the preamble to the 2018 NPRM. Given its use at 40 CFR 30.1, 30.5 and 30.9, the supplemental notice of proposed rulemaking (SNPRM) proposes a new definition of "publicly available."
As noted in SNPRM preamble, the EPA received a large number of comments stating that the approach in the 2018 NPRM would likely preclude the use of valid data and models from consideration as pivotal regulatory science, because the proposed requirement to make data and models publicly available in a manner sufficient for independent validation would prevent the use of data and models that include ccconfidential bbusiness iinformation (CBI), proprietary data, personally identifiable information (PII) that cannot be sufficiently deidentified to protect the data subjects, as well as many older studies. Based on a consideration of these comments, the EPA is proposing a modified version of the 2018 NPRM regulatory text at 40 CFR 30.5 called Option 1. Option 1 would allow Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects.
The supplemental proposal also proposes an alternative approach to 40 CFR 30.5 called Option 2. Under Option 2, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency will, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII; and appropriate techniques have been used to reduce the risk of re-identification.
Several comments were received that address Options 1 and 2 together as well as separately. Comments that encompass both Options 1 and 2 are covered under the topic of Broad Issues. Thus, these comments are organized under the following topics:
Section 7.1: Publicly Available, Tiered Access
 40 CFR 30.2 Definition of Publicly Available
 40 CFR 30.5: Broad Issues
 40 CFR 30.5: Option 1
 40 CFR 30.5: Option 2
 Ensure Over Time More Availability of Data
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
7.1 Publicly Available, Tiered Access
7.1.1 40 CFR 30.2 Definition of Publicly Available
The supplemental notice of proposed rulemaking (SNPRM, or supplemental proposal) rule adds the following definition to section 30.2:
      Publicly available means "lawfully available to the general public from federal, state, or local government records; the internet; widely distributed media; or disclosures to the general public that are required to be made by federal, state, or local law. The public must be able to access the information on the date of publication of the proposed rule for the significant regulatory action or dissemination of the draft influential scientific information for public review and comment."
7.1.1 Clarification
Comment: Commenter (11401) supports the clarity achieved by defining publicly available.
Conversely, commenters (9587, 11381, 12720) suggest the EPA clarify the definition of publicly available. Commenter (9587) notes that the EPA would be well-served by a systematic reform to provide clear definitions of all terms relevant to policymaking and suggests that the EPA, following government best practices in the Department of Defense, institute a formal ontology to establish terminological exactitude in all policymaking.
Commenter (11381) states that the definition of publicly available in the supplemental notice of proposed rulemaking (SNPRM) is unclear. The commenter notes that it is different from the Information Quality Guidelines (IQG) term dissemination in potentially confusing ways, apparently because the Agency is trying to make the term perform dual purposes. The commenter states that, under the IQGs, dissemination is triggered only by agency action. Information an agency does not disseminate is not covered, and thus need not meet information quality standards. The commenter states that the term does not apply to what information an agency might consider, evaluate, or analyze. The commenter states that, in contrast, publicly available appears to act as both a substitute for dissemination and a filter governing what information the EPA will consider, evaluate, or analyze. The commenter states that the SNPRM uses the term publicly available in a manner resembling dissemination, but it also uses publicly available as a screen on what the Agency will consider to be pivotal regulatory science and/or pivotal science.
Commenter (12720) states that the SNPRM's definition of "publicly available" data is vague in not making it clear whether the information itself would be actively made available to members of the public by data holders in government sources, media sources, or other online sources, or whether data subject to public distribution from government sources through legal approaches, including data requests submitted to EPA via the Freedom Of Information Act (FOIA) regardless of actual distribution, would also be subject to this definition.
Response: The term "publicly available" is not meant to convey that the EPA is responsible for disseminating dose-response data considered pivotal science under the final rule. Therefore, the definition of "disseminate" in the IQG is not relevant.
The final rule is limited to dose-response data and does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. EPA does not intend for the term "publicly available" to resemble "dissemination."
Comment: Commenter (11479) states that, in this definition [publicly available], the EPA wrongly applies a catch-all approach to transparency, as a recent paper in Environmental Health Perspectives points out:
      "Strategies designed to serve some goals of the open science movement may be inadequate for achieving other goals. For example, efforts to provide access to all the data underlying a study may be helpful for enabling other scientists to reanalyze the results and identify weaknesses that could undermine its reproducibility. However, providing all that data may do little to help citizens or policymakers understand the overall significance of the study or the crucial limitations or weaknesses associated with it...Additional strategies -- such as effective two-way communication between the scientists who designed the study and those using the information -- might be needed to clarify the information needed by decision makers."
Response: The EPA agrees that subject matter experts are key to effectively determining if the results of a study can be produced when the study data are reanalyzed. For that reason, "independent validation" was defined in 40 CFR 30.2 as "the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced" (emphasis added). However, the EPA maintains that underlying dose-response data should be available to a wide audience. Therefore, the EPA is also maintaining the definition of "publicly available."
The final rule is limited to dose-response data and does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12731) states that, in general scientific usage, "publicly available" means the data, methods, results and summary evaluation are available in a usual form (e.g., a peer-reviewed scientific publication or a technical report). The commenter states that the definition proposed by EPA goes well beyond the generally accepted scientific definition and is likely to mislead the public into believing that the Agency's definition is the scientific norm.
Response: As discussed in the preamble to the SNPRM, the EPA relied upon the definition of "publicly available" found in 16 CFR 313.3. There does not appear to be a published standard definition of "publicly available" in the scientific community so the definition from 16 CFR 313.3 was utilized.
Further, in this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (10020) states that the proposed definition is ambiguous, misleading and unworkable. The commenter asserts that the definition must incorporate contrasting definitions to clarify its meaning and make the proposed new definition workable. The commenter suggests the proposed new definition of "publicly available" should include a contrasting definition similar to the definition of "nonpublic personal information" found in 16 CFR section 313.3(n). The commenter states that, without a contrasting definition or definitions as are found in section 313.3, the proposed new definition is devoid of meaning.
Commenter (10020) does not agree with the proposal's suggestion that its definition of "publicly available" is "similar" to the definition of "publicly available" at 16 CFR section 313.3. The commenter states that 16 CFR section 313.3 was carefully crafted to include a contrasting definition of "nonpublic personal information" which your new proposed definition does not. The commenter states that the contrasting definition gives clarity and meaning to the definition of "publicly available".
Commenter (10020) states that the proposal's stated reason for 16 CFR section 313.3 as the basis for your new proposed definition is that "the meaning of information that is available to the general public should not vary." The commenter states that proposed new definition varies "the meaning of information that is available to the general public" contrary to your announced intention, which is quite correct, that the meaning in the Code of Federal Regulations "should not vary." The commenter states that the new proposed definition of "publicly available" ought to simply and expressly incorporate 16 CFR section 313.3(p) by reference. The commenter states that the proposed new rule defining "publicly available" would serve no purpose since there is already a perfectly workable definition of "publicly available" found in 16 CFR section 313.3(p).
Response: The EPA disagrees that a contrasting definition is needed or that incorporation of 16 CFR 313(p) by reference is needed. Section 30.2 of the final rule provides the following definition:
Publicly available means "lawfully available to the general public from federal, state, or local government records; the internet; widely distributed media; or disclosures to the general public that are required to be made by federal, state, or local law. The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
The final rule is limited to dose-response data and does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA has finalized the approach that will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Restricted or tiered access in this final rule means that the underlying dose-response data are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
<IP_ADDRESS> FOIA
Comment: Commenters (11192, 11361, 12720, 12464, 12715) express concern that the proposal's definition of "publicly available" in the SNPRM includes information available through the FOIA, which means the studies and data may not be immediately or easily available to the public. The commenters assert that the FOIA is not a reasonable tool for making data publicly available because the use of such a data release mechanism cannot ensure equitable access, could involve significant costs, would complicate the existing system due to an anticipated surge in FOIA requests for underlying data and models used in pivotal regulatory science, and/or would involve long processing times with comparatively short public comment periods for EPA regulatory proposals.
Commenters (11192, 11361) state that, while most government records are legally available through the FOIA, anyone who has filed a FOIA request knows that the actual availability of information can be extraordinarily laborious and expensive to obtain. Commenter (12720) states that the FOIA process could involve significant costs. Commenter (12715) states that, although the data may technically be "available," it is likely to be difficult and costly to obtain.
Commenter (12720) states that the FOIA system at EPA and other federal agencies is widely known to be dysfunctional and slow, with thousands of pending requests in the backlog each year. Commenter (12464) states that once someone files a FOIA request, it may take months or years for an agency to provide all requested documents, and frequently, an agency will deny a request or miss statutory deadlines to respond, meaning that the requestor must resort to the courts to obtain the desired documents.
Commenters (12720, 12715) express concern that the FOIA process involves long processing times and comparatively short public comment periods for EPA regulatory proposals.
Commenter (12715) states that the data often may not be received by the requestor until after the comment period ends, if at all, in which case, it is unlikely that interested parties would be able to obtain the information in a manner timely enough to allow for their own review and comment within any timeframe designated by EPA.
Response: EPA agrees that FOIA mechanisms do not achieve the goals of this rule. Availability via FOIA mechanisms does not mean that these data are immediately or easily available to the public. Some commenters cited the EPA's Freedom of Information Act Annual Report Fiscal Year 2019 (2020), which lists a median response time for "expedited processing" of FOIA requests by the EPA as 493 days (Ref. 42). Also, FOIA applies to federal agencies and cannot be used to compel other entities to release information.
The concern regarding the long processing time for FOIA requests has been addressed in the final rule by the addition of the following sentence to the definition of publicly available: "The public must be able to access the information on the date of publication of the proposed rule for the significant regulatory action or dissemination of the draft influential scientific information for public review and comment."
Comment: Commenter (12720) states that the EPA has not explained how its SNPRM would affect the existing FOIA activities at the Agency, despite the potential for major disruption of the process if members of the public are expected to use FOIA requests in order to obtain "information legally available from government sources" as it pertains to this proposal. The commenter states that the EPA has not described any modification of the online FOIA request and response system to accommodate a significant volume of requests for all "data and models" underlying specific scientific studies.
Response: The final rule will not affect the Agency's existing FOIA activities. The final rule does not require EPA to obtain, store or publish dose-response data. The rule merely provides that "[t]he public must be able to access the information on the date of publication of the proposed rule for the significant regulatory action or dissemination of the draft influential scientific information for public review and comment." If the dose-response data underlying pivotal science is not available, those studies will receive lesser consideration under section 30.5 of the final rule.
Comment: Commenter (12720) asserts that there are major flaws which arise from EPA's proposed definition of "publicly available" in the context of FOIA. The commenter states that it is not clear that the EPA would itself be the custodian of "data and models" subject to this rule, nor how the EPA would ensure sufficient data security, integrity, and sharing mechanisms. The commenter states that it is also not clear how members of the public could access "information legally available from government sources" given the volume and complexity of the underlying "data and models" that fall under the scope of this proposal. The commenter asks if members of the public will be expected to identify specific studies for release under FOIA and how would the EPA evaluate such requests. The commenter states that it is likely that, upon receipt of such a request, the EPA would need to contact and potentially negotiate a public data release with study researchers and actual study participants, in the case of research analyzing human exposures and health responses. The commenter states that it is far from clear that such a FOIA request could be fulfilled by EPA, given existing legal restrictions on data sharing that are characteristic of many studies involving human subjects. The commenter notes that the EPA's own interpretation states that FOIA "does not require agencies responding to requests to conduct research or analyze data."
Response: The final rule will not affect the Agency's existing FOIA activities. The rule merely provides that "[t]he public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment." The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information.
Section 30.4 of the final rule provides that the EPA "shall clearly identify all science that serves as the basis for informing a significant regulatory action. The EPA shall make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law." Members of the public will not need to submit FOIA requests to gain access to the studies EPA relies on.
The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
Comment: Commenters (11192, 11361) state that studies originating outside of the government would not be subject to FOIA, so there would need to be another system for creating public access. The commenters state that there is currently no central place for academics to publish their data, let alone to publish data with tiered access.
Response: The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which data and models are available. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, when making their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more dose-response data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate. Further, the EPA recognizes that there are currently many options related to data sharing mechanisms and repositories for researchers. The rule does not prescribe a type of mechanism, but just requires the EPA to determine whether dose-response data is available in a manner sufficient for independent validation.
Comment: Commenters (12720, 12464, 12715) express concern that the proposal's definition of "publicly available" in the SNPRM could systematically favor the EPA's use of industry-submitted data.
Commenter (12720) states that, in 2019, the EPA finalized an update to its FOIA regulations. The commenter states that, under this rule, the Administrator and other officials are allowed to review all materials that fit a FOIA request criteria and can decide whether to release or withhold a record or a portion of a record on the basis of responsiveness or under one or more exemptions under the FOIA, and to issue no records responses. The commenter states that, given the vague language in this proposal and existing FOIA regulations at EPA, there is a significant potential for political interference and arbitrary determinations within EPA for public release of data and models.
Commenter (12464) notes that the definition provides that "publicly available" means, among other things, "lawfully available to the general public from federal, state, or local government records." The commenter states that this means that data submitted by industry when applying to state or federal agencies for a permit or approval will be treated as "publicly available" for purposes of the proposal, even if members of the public could obtain access to those data only by filing a FOIA or state public records law request.
Commenter (12715) states that the limitations on the actual ability for requestors to obtain data and documents upon which EPA relies increase the likelihood that when EPA adopts regulations or standards, it will consider studies or data that are not in any practical manner available to the public, while at the same time failing or refusing to consider peer-reviewed studies that have been published in the open literature but which have private underlying data.
Response: These comments are addressed in the preamble of the final rule:
      Several commenters contended that the proposed definition of "publicly available" would introduce a bias favoring industry data submitted to the EPA. They asserted that industry-generated studies submitted to the EPA pursuant to FIFRA would be considered publicly available because they could be obtained by the public in response to a Freedom of Information Act (FOIA) request. However, this does not mean that these are immediately or easily available to the public. Some commenters cited the EPA's Freedom of Information Act Annual Report Fiscal Year 2019 (2020), which lists a median response time for "expedited processing" of FOIA requests by the EPA as 493 days (Ref. 42). The EPA finds that such comments have merit and is modifying the definition in the final rule to add the following at the end of the definition: "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
<IP_ADDRESS> Other
Comment: Commenters (11576, 12720) share the concerns that the Science Advisory Board (SAB) outlined with respect to what "available" means in the context of the supplemental notice:
      A question to be answered is whether making the scientific papers reporting these studies available without charge makes the studies "available," or whether all data from every measurement taken as part of the study need to be available to anyone to analyze. At one
end of this range of interpretation the requirement is easily implementable. On the other end of the spectrum, meeting the requirement would be enormously expensive and time consuming at best and could be expected to result in the exclusion of much of the scientific literature from consideration (the machine data may no longer be available and/or the researchers may no longer be alive or in a position to assemble the data).
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize potential unintended consequences
Comment: Commenter (12712) asserts that the default rule for all work funded by the EPA must be that relevant code and data are available online or on request. The commenter states that the EPA must be clear that making data available means that the actual final dataset used in statistical analysis is made available.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees, and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, the rule requires the EPA to evaluate the availability of the analyzable dataset, not the more limited dataset that accompanied the publication of the study or model. For this rulemaking EPA will not identify a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (12406) states that the proposed definition regarding "publicly available" does not include any reference to the concept of tiered access to CBI and PII as a permissible and appropriate means by which information could be deemed within the definition of publicly available. The commenter suggests this definition should be revised as follows:
      Publicly available means lawfully available to the general public from federal, state, or local government records; the internet; widely distributed media; in designated reading rooms; available through tiered access when the data includes CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects; or disclosures to the general public that are required to be made by federal, state, or local law.
Response: The EPA does not agree that restricted access should be a part of the definition of "publicly available" and therefore, EPA has not made the suggested edits to that definition. Restricted access is covered in other provisions of the rule. The EPA has finalized the approach that will give greater consideration to pivotal science whose for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Restricted or tiered access in this final rule means that the underlying dose-response data or models are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12720) notes that the SAB report finds that requiring the identification of all studies and regulatory science supporting regulatory actions and making them available for reanalysis will be a complex process. The commenter contends that identifying and making "pivotal science data or studies" available could present challenges if some studies were considered in the regulatory decisions but were not used to determine the point of departure or reference dose or other regulatory/technical level. The commenter states that it is not clear how much information and which studies should be included in the requirement to identify and make studies available.
Response: In the final rule, pivotal science "includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information." The final rule does not require EPA or any other entity to conduct independent validation before EPA can use pivotal science. Nor does the final rule require EPA to obtain, store or publish dose-response data.
Comment: Commenter (12720) suggests that the rule could benefit from use of the term "analysis dataset" to define data that should be made publicly available. The commenter states that this term refers to data that have been collected and processed (e.g., cleaned and transformed) for analysis. The commenter states that, in this way, the public could understand which studies/data were considered and used without expending the resources to gather and disseminate all available data. The commenter states that the level of detail required to allow the public to transparently reach the same conclusion(s) as the Agency will differ among individuals who seek the data. The commenter states that some people may wish to verify that the EPA has selected the right "pivotal studies." The commenter states that, however, a more expansive scope would increase the reporting burden in a way that may make the proposed rule untenable.
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
7.1.2 40 CFR 30.5: Broad Issues
<IP_ADDRESS>. Retroactive Application of the Rule
<IP_ADDRESS>.1 General
Comment: Commenters (11393, 11395, 11407, 11895, 12412, 12474, 13549) generally oppose the proposal that the rule would apply regardless of when the data and models were generated.
Commenters (11393,11395, 12412, 12435, 12474, 13549) state that many environmental statutes require the EPA to promulgate standards and protections based on the best available science, and then reanalyze those decisions at set points in the future. The commenters state that the SNPRM, and its proposal to apply regardless of when the data and models were generated, suggests that as the EPA reexamines standards and protections, the Agency would potentially exclude from consideration studies that led it to enact those standards and protections in the first place. The commenters state that it would be nonsensical and unlawful for EPA to overturn or weaken a standard or protection not based on any new science, or on a concern that the results were invalid, but rather because that study has not publicly released all its underlying data.
Commenter (11401) states that, while it is reasonable not to apply the new standard to existing rules, it would be a mistake to perpetuate outdated, substandard scientific studies into the future. The commenter states that these standards should apply to any science that a new rulemaking is using, regardless of when the science was generated; perpetuating non-transparent, irreproducible studies potentially in perpetuity should not be the goal of this rulemaking.
Commenter (11894) states that the supplemental proposal attempts to have it both ways by maintaining that the rule would apply retroactively while requesting comments on whether this should apply only to data and models that are generated after the effective date of the rulemaking.
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
<IP_ADDRESS>.2 Planning
Comment: Commenters (10339, 11890, 12430, 12720, 12715) express concern that the rule and SNPRM fail to explain how historical data, which may have been collected under different policies and procedures to ensure confidentiality, will be treated. Commenter (11890) states that older records may have lost underlying data due to record retention schedules.
Commenter (12430) states that the EPA is in the process of adopting, and has recently adopted, multiple rules that will have a significant negative impact on air quality and human health, and it supports these rulemakings with information that may not meet the proposed standards for public availability. The commenter states that, for example, in the Safer Affordable Fuel-Efficient Vehicles Rule Part One: One National Program, the EPA notes that some information relied upon in the rulemaking is not publicly available, e.g., CBI. The commenter states that the EPA has similarly relied upon information not publicly available in the "Safer Affordable Fuel-Efficient Vehicles Rule for Model Years 2021-2026 Passenger Cars and Light Trucks." The commenter states that the EPA does not explain the extent to which the SNPRM would prevent the Agency from relying upon non-public information in other concurrent or near-term rulemakings. The commenter states that it is an open question, then, whether the EPA would need to amend its open rulemakings to eliminate the non-public information that it is proposing to limit itself from considering if the SNPRM is finalized  -  or whether the EPA would simply find reasons to exempt its preferred studies from the SNPRM's requirements under its arbitrary exemption provision. The commenter states that retroactive application is even more problematic than prospective-only application, but either approach would abrogate EPA's duties to protect public health and the environment.
Commenter (12720) asserts that the retrospective application of data transparency standards is a monumental challenge. The commenter states that a large amount of work would be required to locate, curate and retrospectively make datasets available for public access. The commenter states that this requirement could adversely affect the ability to move this program forward in a meaningful capacity. The commenter states that, for example, the EPA could decide not to apply the proposed rule and its specific requirements retrospectively given the potential difficulty accessing, reviewing and making data available that were not originally intended to be disseminated in such a manner as defined in a future rule. The commenter states that the EPA could consider designating a "start date" and begin collecting and releasing pivotal study data at that time. The commenter states that, when the EPA updates an existing risk assessment after the start date, the EPA could collect and release pivotal data. The commenter states that it is true that a substantial amount of work would be required in order to make any data and models underlying studies available to the public, but the EPA has not identified a responsible party for this work, nor has it made clear the timelines, electronic data sharing mechanisms, or public reporting of such availability would be achieved, archived, and maintained over time. The commenter states that it is not clear how such a "start date" for future applicability of data requirements could be determined or fairly applied, and in many cases such as the national ambient air quality standards (NAAQS) science reviews, the assessments rely on a foundation of scientific evidence that stretches back many years, if not decades. The commenter states that, as a result, determination of any "starting date" would be arbitrary and deprive the EPA of the best available science.
Further, the commenter (12720) contends that EPA has not acknowledged that there are regular review cycles for different environmental rules, and that new requirements for all "data and models" to be considered by the Agency in development of "influential scientific information and/or pivotal regulatory science" will substantially complicate existing science review processes at the Agency. The commenter contends that the supplemental proposal makes no reference to the fact that EPA is constantly re-reviewing scientific evidence, and that the body of science remains relevant regardless of when it was generated. Additionally, the commenter states that EPA has not explained why retroactive application is warranted, even as the Agency proceeds with a number of influential science products that rely on studies that do not meet the requirements of the supplemental proposal.
Commenter (12715) states that the proposal at 15,399 and 15,403 is unclear whether studies with data and models generated before the effective date of this proposed rule would be subject to the 40 CFR 30.5 criteria.
Commenter (11392) asserts that the supplemental proposal's retroactive application could have radical consequences and that EPA must provide greater clarity on what the implications would be for past studies that underpin the Agency's regulatory standards.
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
<IP_ADDRESS>.3 Costs
Comment: Commenters (11890, 12731) express concern regarding the costs associated with any retrospective application of the rule.
Commenter (11890) states that scientists who volunteer to provide data would need funding to respond to requests from EPA. The commenter states that this rule should not apply retroactively. The commenter states that archived materials were likely collected under grants that have long expired or are funded by grants that can't be used to fund such activities.
Commenter (11892) states it is difficult to envision how the supplemental proposal could be applied retroactively to the science developed through long term experience in the implementation of various regulations and standards. The commenter asserts that the exclusion of research that was accepted as scientifically valid in the past could only result in the need to duplicate such research, adding needless cost and delay to the process of decision making. Furthermore, according to the commenter, there are already EPA mechanisms for revising regulations and standards based on new data and analyses.
Commenter (12731) refers to the SAB report which concluded, "the retrospective application of modern transparency standards is a challenge. A large amount of work would be required to locate, curate and retrospectively make datasets available for public access. This requirement could adversely affect the ability to move this program forward in a meaningful capacity."
Response: The EPA has identified some incremental costs that the Agency may incur as a result of this final rule. As stated in Section III.A.2, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
<IP_ADDRESS>.4 Consent Needed
Comment: Commenters (11361, 12430, 12720, 12731, 12727) express concern that it may not be possible to obtain consent from study authors and/or data owners of historical studies.
Commenter (11361) states that any study that includes primary collection of human health data would need to have obtained informed consent from subjects to participate in the study. The commenter states that consent agreements are created to be very specific, scope- and time-limited agreements. The commenter states that, for any study that did not originally make its data public--which describes the vast majority of studies conducted--participants haven't consented to their data being made public, even if a de-identification process has taken place. The commenter states that it could be an enormous undertaking to try to track down participants years after a study concludes to attempt to gain consent in order to comply with this rule. The commenter states that, since many consent agreements are time-limited, researchers may not have the consent of participants to even attempt to contact them after the conclusion of the study. The commenter states that the EPA is leading researchers into a legal, ethical, and financial quagmire by retroactively dismissing or devaluing existing studies that do not comply with these unrealistic requirements.
Commenter (12430) states that participation in epidemiological studies is often contingent on keeping subject data confidential, consequently, studies that have already been completed may contain data that was collected contingent on confidentiality. The commenter states that researchers cannot now make that data publicly available, in response to the SNPRM, without violating state and/or federal laws (e.g., California Information Practices Act, the Health Insurance Portability and Accountability Act) or locating research participants and requesting their consent to the release (an expensive and time-consuming process that may frequently result in participants declining to give their consent).
Commenter (12720) expresses concern that the proposal does not explain how permissions will be obtained from study authors and research participants. The commenter states that this shortcoming raises numerous questions, including the length of time that will be allowed for study participants to consider and respond to any requests for data release.
Commenters (12731, 12727) refer to Dr. Janice Chambers' comment in response to EPA's charge questions that even a "tiered access" approach to disclosing data would not be possible to apply retroactively to studies carried out in the past. The commenters state that, based on her personal experience, Dr. Chambers opined that "many of the older epidemiology studies would not have considered this Option in their Informed Consent Forms" and that the researchers who conducted those studies (assuming they are even available) would therefore not have consent to share the data for reanalysis. The commenters note that participants in past health studies may either have died or be unable to provide consent due to age or illness -- and that even participants who are reachable may be unwilling to release subsets of their data even to "safe harbor data archives."
Commenter (11900) states that a participant's right to privacy is established during informed consent to participate in a study. The commenter states that informed consent is an ethical and legal requirement of human subject research. The commenter states that consent documents describe the nature of the research, the risks and benefits associated with participation, and how the researcher will protect the confidentiality of the research records. The commenter states that, up to the present, nearly all human environmental health studies have guaranteed in their informed consent not to share identifiable study data outside the research team. The commenter states that all human subjects data is potentially identifiable, even after overt identifiers are removed; thus, researchers will be put in the position of having to decide whether placing data in a restricted-access enclave -- subject to access requirements that are not yet specified -- violates their pre-existing pledge to participants.
Response: The final rule does not require EPA or any other entity to obtain, store or publish dose-response data. As provided in 30.5(f), "[w]here the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBICBI, and is sensitive to national security." Dose-response data is considered "publicly available in a manner sufficient for independent validation" when it includes the information necessary for the public to understand, assess, and reanalyze findings and may include, for example:
 Data (data would be made available subject to access and use restrictions);
 Associated protocols necessary to understand, assess, and extend conclusions;
 Computer codes and models involved in the creation and analysis of such information;
 Recorded factual materials; and
 Detailed descriptions of how to access and use such information.
The provisions of this section apply to dose-response data underlying studies that are pivotal science, regardless of who funded or conducted the studies. The Agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of privacy, confidentiality, national security is not possible. Further, an exemption under 30.7 may be granted so that pivotal science where the underlying data are not available for independent validation may be used and granted greater consideration under 30.5.
Comment: Commenters (11890, 12418) argue that, to protect privacy, the rule should not be retroactive. Commenter (11890) states that new technology has enabled re-identification of study participants and studies considered de-identified 20 years ago would be vulnerable to breaches in privacy today.
Commenter (12418) agrees with the modification in the supplemental proposal that all studies completed before the effective date of this proposed rule are considered eligible for EPA rulemaking regardless of whether they contain data or models that are publicly available. The commenter states that the EPA must acknowledge that protecting the privacy of human participants in studies which have been used to support regulations in the past is an essential justification for prohibiting application prior to the effective date of this rulemaking. The commenter states that confidentiality of participant information is a scientific imperative of human health research that this rule must address more directly.
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
<IP_ADDRESS>.5 High Quality Research
Comment: Commenters (10973, 11361, 12617, 12731, 13788, 14397, 11900) object to the exclusion of older research simply because it does not have full data sets available. Commenter (11361) states that, by retroactively applying this rule, the EPA nearly guarantees that most existing public health studies would be stricken from consideration.
Commenter (12731) asserts that the Agency's proposed solutions do nothing to allow rigorous consideration of the vast body of historical studies that document the harm that pollutants cause to human health and welfare. The commenter states that, while the EPA acknowledges objections to the original proposal's exclusion of "many older studies" that include valid data and models (85 FR 15401-02), the SNPRM does not respond to these concerns. The commenter states that, as the EPA admits, there are compelling reasons why it may be impossible or infeasible to provide access to data and models underlying older studies.
Commenter (14397) states that this proposal could allow the EPA to retroactively dismiss studies that have been used for decades when setting or renewing NAAQS and related regulations. The commenter states that these studies could include seminal publications regarding lead and mercury, and particulate matter., The commenter states that it should be noted that the EPA is not suggesting that past studies are invalid or reached erroneous conclusions, and indeed many of these studies have been reviewed, replicated, and validated by further independent studies. The commenter states that the EPA simply wants these studies to be dismissed because properly confidential data cannot be released. The commenter states that this approach would force the EPA to arbitrarily make public health and environmental decisions based on incomplete or improperly weighted information.
Response: The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies -- based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. Further, because study quality factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) would have already been evaluated at an earlier stage in the assessment process, the studies evaluated with these additional factors are presumed to be of the highest quality available.
Comment: Commenter (12423) understands that the EPA interprets the proposed rule as not retroactive, but asserts that interpretation is misleading. The commenter states that any future regulations or policies would need to adhere to this proposed rule, including the review of existing regulations and policies. The commenter states that, without the effective ability to tier access these data under the proposed rule, an important study would be either discounted or deprioritized, depending on whether the modified proposal (tiered access) or alternative proposal (weighted studies) for public data is applied. The commenter states that actions and decisions made by EPA then could not make use of this scientifically valid and critical study without an exemption by the Administrator.
Commenters (12403, 11900) express concern that retroactive application of the proposed rule would be detrimental to public health.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies -- based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. Further, because study quality factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) would have already been evaluated at an earlier stage in the assessment process, the studies evaluated with these additional factors are presumed to be of the highest quality available.
Comment: Commenter (12455) provides that the supplemental proposal needlessly complicates their use of pesticide reviews, which are traditionally based on CBI studies. According to the commenter, it is unclear whether older studies, when they come up for review, will be completely excluded from consideration as they would not qualify under the supplemental proposal. Commenter (12715) states that Option 1 is fatally flawed in that it continues to unlawfully and arbitrarily exclude from consideration valid scientific studies where the underlying data is unavailable, regardless of whether the studies have been peer reviewed or would be considered part of the "best available science" the Agency is commanded to consider.
Response: The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. With this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes that the EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
<IP_ADDRESS> Best Available Science
<IP_ADDRESS>.1 Legal Concerns
Statutes
Comment: Commenters (11360, 11390, 11393, 11396, 11403, 11407, 11489, 11491, 11895, 11911, 12412, 12474, 12702, 12731, 13549, 13788, 12715) assert that the proposal violates statutes that require the EPA to base its decisions on the best available science.
Commenter (11360) states that the proposed rule and the SNPRM in effect bar the EPA from utilizing scientific information that rests on difficult-to-release information. The commenter states that the best available science must serve as the foundation for EPA's regulatory and policy actions. The commenter states that Congress intentionally and wisely embedded peer-reviewed research in the foundation of the Clean Air Act (CAA), including requiring regular reviews of the science, explicitly recognizing that the EPA needs the most current, peer-reviewed data to protect the public health. The commenter states that these expectations also are reflected in other public health laws that support the EPA's mission, including the Toxic Substances Control Act (TSCA).
Commenters (11393, 11407, 11895, 12412, 12474, 13549) assert that the EPA must base its decisions on the best available science and that anything less is nonsensical, arbitrary, and in violation of the environmental laws the EPA is charged with implementing. The commenters state that the SNPRM instead provides for ways the EPA would exclude and lessen consideration of the best available science.
Commenter (11390) asserts that weighting, discounting, or excluding relevant studies from consideration based not on the quality of the science but on the public availability of underlying data directly contradicts the EPA's statutory mandates and core mission. The commenter states that the EPA itself has defined "best available science" and similar statutory standards without reference to the public availability of data; for example, the EPA has explained that:
      Use of best available science involves the use of supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies and data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
Commenter (11489) asserts that science does not depend on the public availability of underlying data to indicate quality and reliability of evidence and public availability of research data is not a proxy for the reproducibility of science. The commenter states that, if this rule is made final, the EPA will fail to meet a key component in its enabling legislation that requires the agency use the "best available science" in its regulatory decisions.
Commenter (11491) states that the EPA is often legally obligated to base its rules on the best available science, and the Agency does not have the authority to modify statutes requiring the Agency to use the best available science and health information.
Commenter (11911) asserts that ignoring valid and potentially valuable information in rulemaking with profound public health and economic consequences is a steep cost to pay  -  and it is at odds with the Agency's mandate to use the "best available" science.
Commenter (12731) asserts that, like the original proposal, the SNPRM lacks any demonstration that the public unavailability of a study's underlying data or models necessarily -- or even likely -- renders the study invalid or unreliable. The commenter notes that comments on the original proposal demonstrate that mechanisms are already in place to validate studies for which disclosure of underlying data and models is either illegal or impracticable. The commenter contends that the EPA's automatic exclusion from consideration of (or, at least, diminished reliance upon) studies based solely on the public unavailability of underlying data or models would result in the Agency failing to consider all available science -- and even the best available science in some cases -- in direct contravention of statutory requirements. The commenter states that, under the differential weighting approach, the EPA would not be able to use the best available science and consider all available information, in violation of the EPA's statutory obligations.
Commenter (13788) states that, unlike the 2018 proposal, the SNPRM does not include any acknowledgement that the EPA must use the "best available science" for regulatory actions -- but it must.. The commenter notes that, under the Administrative Procedure Act (APA) and CAA, agency actions that are arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law are to be held unlawful and set aside. The commenter states that agencies are required to "articulate a satisfactory explanation for [the] action including a `rational connection between the facts found and the choice made.'" The commenter states that, here however, the EPA is proposing to exclude or devalue scientific studies based on criteria that are unrelated to scientific value or merit, and that would eliminate the Agency's ability to rely on research that may be the best available science.
Commenter (11576) expresses concern with respect to the practicability of requiring all information to be made available, given unequivocal statutory deadlines embedded within environmental statues.
Response: The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. The requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Case Law
Comment: Commenters (11390, 11403, 11479, 11495, 12720) refer to court cases and assert that the EPA must use the best available science regardless of whether the data are publicly available.
Commenter (11390) asserts that the "best available science" and similar standards set out in environmental statutes require the use of scientific information and analysis that follow standards accepted by the scientific community, not information and analysis based upon the public availability of data. The commenter states that the proposed requirements ignore well-established indicia of reliable scientific research, such as peer review. The commenter states that proposal would force the EPA to ignore high quality science and that this controverts the EPA's statutory obligations and will impair the EPA's ability to protect public health and the environment. The commenter contends that public availability of data is a criterion often irrelevant to scientists' standards for determining whether research represents the best science. The commenter asserts that the proposal's emphasis on public availability contradicts the EPA's own statutory mandates and is inappropriate, given the courts' agreement that publicizing the data underlying studies upon which EPA relies "would be impractical and unnecessary."
Commenter (11403) states that the EPA has long relied on peer-reviewed studies in its rulemaking, even where the underlying data may not be publicly available. The commenter states that, as the proposed rule notes, the EPA has successfully defended that exact approach before the D.C. Circuit, which has recognized that "[i]f EPA and other governmental agencies could not rely on published studies without conducting an independent analysis on the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment...."
Commenter (11495) states that, in at least two instances, the D.C. Circuit Court of Appeals has recognized that studies for which the underlying data are not publicly available may constitute the "best available science." The commenter states that, in American Trucking Associations, Inc. v. EPA, the Court held that the CAA did not require the EPA to make underlying data public where EPA relied on the study itself and not the raw underlying data. The commenter states that the Court agreed with EPA's argument that requiring the Agency to obtain and publicize the relevant data "would be impractical and unnecessary." The commenter states that in Coalition of Battery Recyclers Association v. EPA, the court again upheld the EPA's reliance on non-publicly available data, reasoning that the EPA relies on studies and their results rather than the raw data underlying those results.
Commenter (11479) states that records released through a FOIA request confirm that EPA staff are aware that the Agency does not need to have access to the underlying raw data, codes, etc. to promulgate regulations. The commenter states that during the development of this proposed rule, Dr. Nancy Beck, the Deputy Assistant Administrator for the Office of Chemical Safety and Pollution Prevention, emailed colleagues "for awareness" that in the 2002 court decision American Trucking Association v. EPA, it became evident that, if the EPA were required to have access the raw data of peer-reviewed scientific studies, the Agency would not be able to use a significant amount of important scientific literature. The commenter states that the courts, through that decision and through Coalition of Battery Recyclers Association v. EPA, have made it clear that the EPA is not required to obtain or analyze raw data in order to use independent scientific studies that are published in peer-reviewed journals to regulate.
Commenter (11495) states that the EPA has already established an array of rigorous review techniques that go beyond the typical scientific journal peer review process to ensure that the best available science is used. The commenter states that imposing this new public availability of data obligation negates the purpose of these "rigorous reviews" and the already-strict review imposed by the scientific community. The commenter states that courts have held that agencies "cannot ignore available...information." The commenter states that plaintiffs and petitioners can establish a violation of a statutory "best available science" requirement imposed on the Agency by "point[ing] to any scientific evidence that the agency failed to consider." The commenter states that the Ninth Circuit Court of Appeals has also held that "the best available data requirement . . . prohibits [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on." The commenter states that, under the SNPRM, the EPA would not consider some scientific studies that are potentially more accurate and robust than the studies it does consider based upon the arbitrary criterion of the public availability of the study's underlying data.
Commenter (12720) states that, in its SNPRM, the EPA proposed to categorically ignore and exclude all peer-reviewed research with non-public underlying data, without individually considering each study or offering specific reasons for not relying on that study. The commenter states that the SNPRM, by barring consideration of foundational scientific research premised upon non-public data, would result in EPA "fail[ing] to consider an important aspect of the problem." The commenter states that there is no evidence of a court supporting an agency's decision to exclude entire categories of evidence, or studies or information based on categorical prohibitions like the ones in the SNPRM, without considering the source and offering specific reasons for not relying on the study. The commenter states that, both the EPA and the courts have indicated in API and Coalition of Battery Recyclers, that a rule like the one the EPA is currently proposing is not required by the CAA and would be both impractical and unnecessary. The commenter states that this SNPRM runs counter to the D.C. Circuit's decision in API and would render the EPA's regulatory actions based on the SNPRM arbitrary and capricious and an abuse of EPA's discretion. The commenter states that SNPRM's blanket rule would represent a significant and unlawful departure from D.C. Circuit rulings on agencies' limited discretion to choose the sources it will consider and ignore.
Response: The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. The requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information and do not interpret provisions of the statutes referenced by commenters. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
<IP_ADDRESS>.2 Informed Decision-Making
Comment: Commenters (11360, 11393, 11399, 11473, 11576, 11895, 12403, 12412, 12427, 12434, 12443, 12455, 12474, 12546, 12731, 13549) object to the SNPRM because the rules will lead the EPA to make decisions based on incomplete and/or improperly weighted information, rather than the best available information.
Commenters (11393, 11895, 12412, 12474, 13549) contend that the SNPRM does not address the fact that many past and ongoing public health studies simply cannot lawfully, ethically, or efficiently publicly release the underlying data. The commenters state that the SNPRM does not claim that such studies are invalid or reached incorrect conclusions. The commenters state that some of these past studies have been reviewed, replicated, and validated by additional independent studies. The commenters state that, even if applied only to future research, the SNPRM would limit participation, limit the information that can be collected, and waste limited resources.
Commenter (11399) states that additional "independent validation" that goes above and beyond the peer review process and the EPA existing science advisory infrastructure is a new procedural hurdle that will limit the universe of scientific studies that the EPA may consider. The commenter states that creating such an impediment undermines the EPA's legal obligation to use the best-available science to guide its decision-making processes and portends a weakening of the scientific underpinnings of health-based environmental regulations.
Commenter (11473) expresses concern that discounting evidence from the decision-making process on the basis that some data are confidential runs counter to the EPA stated mission to reduce environmental risks...based on the best available scientific information. The commenter states that the EPA 2016 document on transparency stated that "Classified or otherwise protected EPA-funded scientific research will not be made publicly available."
Commenters (11576, 12405, 12427) refer to a joint statement by a group of scientists and editors-in-chief at leading scientific journals in response to the proposed rule: "It does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making. Excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes."
Commenter (12403) asserts the rule would compromise the EPA's decision-making by excluding from consideration or devaluing well-conducted peer reviewed studies that cannot comport with data availability requirements. The commenter states that it also runs the risk of regarding small, poorly designed studies while penalizing larger and more representative research studies.
Commenter (12434) states that the proposal would exclude validated and qualified science that has undergone rigorous peer review if the data were not publicly available. The commenter states that, consequently, the EPA would necessarily rely on scientific studies with publicly available data regardless of whether these studies were the most robust or best suited to inform decision making. The commenter states that decisions could be based on science that does not represent the latest scientific consensus, but by science arbitrarily selected by the Administrator, a political appointee.
Commenter (12443) asserts that, as written, this rule would result in excluding many well-designed studies that contain private medical information that cannot ethically be made publicly available, because to do so would jeopardize the privacy of study participants' medical data and possibly other information. The commenter states that researchers should not be forced to choose whether to protect the privacy of participants' data or to make their findings applicable to the EPA. The commenter states that, if the EPA were to ignore findings from studies with private data, many high-quality research findings could not be considered, which would weaken the basis of EPA decision-making. The commenter states that there is no scientific basis for discounting such findings, and valuable research would be suppressed, harming the public health.
Commenter (12455) states that the EPA's intent to have all decisions based on publicly available data as stated in the SNPRM could harm federal decision-making. The commenter states that there are many qualified research entities that conduct studies where the report may not be available to the public. The commenter states that, although the concept of having studies be publicly available so peer review can validate data has the veneer of transparency, it clearly subverts the ability of the researcher using the data to determine the validity of data, study, or methods used.
Commenter (12546) notes that the EPA is proposing to downgrade the consideration of studies that do not make underlying data and models publicly available. The commenter states that the EPA itself acknowledges that potential studies for downgrade or elimination are "high-quality" and that scientific findings and research are valid whether or not data are publicly available, a reiteration of the conclusion put forth in the EPA's 2016 Plan. The commenter states that, if this model is pursued, the EPA can base new and renewed standards on lower-quality science, rather than the "best available science," and "adequate information." The commenter states that the omission of high-quality studies because they do not have available the underlying data could greatly impact the strength of systematic reviews and meta-analyses needed to show the harms of an exposure on a health outcome, and this could lead to conclusions that the evidence-base lacked sufficient weight.
Commenter (12731) refers to the SAB report which noted that "[i]t is not clear: (1) how many of the studies the EPA currently relies upon to take important regulatory actions would meet the public disclosure standards stated in the proposed rule, and (2) whether EPA has assessed the feasibility of making underlying information from the studies publicly available, or what the impact of precluding those studies would be on EPA's decision making and its ability to protect public health/environment." The commenter states that the EPA thus continues to disregard the likelihood that its rule would severely erode the best available science available to the agency, which it recognizes "must serve as the foundation of [its] regulatory actions." The proposal remains arbitrary in ignoring this obvious -- and now expanded -- counterproductive effect.
Response: The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The final rule does not require EPA or any other entity to conduct independent validation before EPA can use pivotal science. Nor does the final rule require EPA to obtain, store or publish dose-response data.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenters (10040, 11479, 11491, 11907, 11926, 12457, 12458, 12716, 12720, 12715) object to the rules because they would eliminate important health evidence collected under assurance of confidentiality and undermine the EPA regulatory decisions.
Commenter (10040) states that, by not allowing the use of science grounded in datasets that use confidential, protected health data from individuals who have consented on the condition that their data remain confidential, this proposal stands to severely undermine the scope of research available to inform the EPA as it works to make scientifically informed regulatory decisions.
Commenter (11479) notes that one Option offered in the new notice would allow the EPA, if considering a study whose underlying data are not made available, to place less weight on the results "to the point of entirely disregarding them," even if the research and results were strong and compelling, met the highest scientific standards, and had undergone extensive scientific peer review. The commenter states that, in other words, the EPA would give the most weight to studies with publicly available data rather than to studies that provide the strongest scientifically supportable evidence. The commenter states that, because there are many environmental health studies relying on medical data and other private information, the rule's restrictions would mean that the best available science could be excluded.
Commenter (11491) states that neither of the Options provided in the SNPRM, regarding the need to both use and maintain the confidentiality of certain information relied upon by the Agency in its decision-making process, adequately addresses the constraints that the rule would put on EPA's ability to rely on the best available science. The commenter states that the rule would constrain the Agency's ability to use the best available science and thereby undermine the quality and defensibility of its final actions. The commenter refers to the EPA's SAB report which states:
      There is minimal justification provided in the Proposed Rule for why existing procedures and norms utilized across the U.S. scientific community, including the federal government, are inadequate, and how the Proposed Rule will improve transparency and the scientific integrity of the regulatory outcomes in an effective and efficient manner. It is plausible that in some situations, the Proposed Rule will decrease efficiency and reduce scientific integrity, determining if in fact that will be the case requires a thorough and thoughtful examination that is currently absent in the Proposed Rule. Moving forward with altered transparency requirements beyond those already in use, in the absence of such a robust analysis, risks serious and perverse outcomes.
Commenter (11362) contends that the EPA must prioritize public health in its decision making, therefore data from public health and epidemiological studies must be taken into consideration for influential scientific information and regulatory actions and should be exempt from this rule if passed into law.
Commenter (11907) states that this rule will likely reduce or severely restrict the EPA's use of epidemiology studies that are based on anonymous or confidential PII that can contribute to identifying sources of air, water, or ground pollution that can lead to chronic human health conditions and terminal diseases, not only within Native American communities, but within all communities throughout the United States.
Commenter (11926) asserts that the proposal and SNPRM would seriously impact the EPA's ability to utilize the best available science in formulating new regulations or renewing existing ones. The commenter states that, by mainly considering studies with publicly available data or those posted to a special database that allowed public access to sensitive data by special arrangement, the rule would mean little weight would be given to studies with strictly confidential non-public data. The commenter states that this would eliminate important health evidence collected under assurance of confidentiality because publication of the data would be a breach of confidentiality. Ignoring the results of such studies, many in the most exposed and most affected populations, would mean that regulations and policies essential to protect public health would never come into existence.
Commenter (12457) contends that the proposed revisions could result in the failure to properly consider important pivotal science and research findings during the policy making process. The commenter states that limiting information that is utilized for policy setting is not the most prudent course of action as policy makers often need to consider a range of information and data in order to set sound policy.
Commenter (12716) asserts that, to have access to the best available science, studies must protect the privacy of those affected by environmental risks. The commenter states that the EPA's proposal requires access to underlying data and, consequently, will limit the types of scientific studies that will be conducted and used to inform regulatory decision-making. The commenter notes that the EPA recognizes the need to balance the interests of making information available to the public and protecting other interests, like privacy.
Commenter (12720) expresses concern that the EPA's proposal does not clarify how fundamental patient privacy and confidentiality issues can be overcome. The commenter notes that the SAB points out that such hurdles will prove insurmountable in some cases, depriving the EPA of the best available science to inform critical safeguards to limit pollution and protect public health.
Commenter (13788) asserts that the EPA's real intention in the SNPRM, as in the proposal, is to limit the Agency's access to public health science, not to ensure the quality of the studies the Agency relies on. The commenter states that it is made clear by the Agency's statement that if "multiple high-quality studies exist but only two studies have data and models that are available for independent validation and reanalysis, EPA would only include those two studies as pivotal regulatory science and/or pivotal science." The commenter states that, by acknowledging that the highest quality studies will be ignored if confidential data and models are not made available for validation and reanalysis, the EPA is admitting that this rule is not concerned with improving the quality of studies relied on by the Agency. The commenter states that the SNPRM is instead focused on information availability, even if it comes at the expense of the quality of scientific work. The commenter states that the best studies may not rely on available underlying data -- precisely because they are human health studies that requires the maintenance of the privacy of the study participants  -- and in those situations this rule would clearly prioritize data availability over quality.
Commenter (12715) states that the issue of protecting confidentiality of data has been well recognized. The commenter states that this process demonstrates that, even after going through an arduous, time consuming process, the removal of personal identifying information would be difficult to achieve. The commenter states that, therefore, many valid, peer reviewed epidemiological studies would be excluded from consideration, or arbitrarily given less weight, because the data would not be publicly available.
Commenter (12458) asserts that giving greater weight to studies where the underlying data and models are available or made available through tiered access would disproportionately exclude research that relies on personal health data, including research that draws crucial connections between environmental pollution and human health. The commenter states that such research could include the kind of critical work on the human health impacts of water and air pollution that has contributed to regulations that have saved thousands of lives and trillions of dollars. The commenter states that giving greater weight to studies where the underlying data is available will increase the relative weight of industry-funded research and research using non-human subjects, which are less likely to foreground questions about human health. The commenter states that, for these reasons, the SNPRM and rule would actively interfere with EPA's mission to protect public health both in rulemaking and in other Agency activities.
Response: The final rule does not require EPA or any other entity to conduct independent validation before EPA can use pivotal science. Nor does the final rule require EPA to obtain, store or publish dose-response data. The EPA shall give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. The Agency shall also give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation. For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9. The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given.
<IP_ADDRESS>.3 Public Health
Comment: Commenters (10040, 11377, 11378, 11390, 11491, 11926, 12443, 12458) object to the rule because it would undermine the Agency's ability to execute its mission to protect public health, and its obligation to use best available science. Commenter (10040) states that these regulatory decisions have profound implications for the lives, health and well-being of communities nationwide. Commenter (11377) asserts that systematically and arbitrarily favoring fully-public data for even more of the Agency's work only further imperils citizens whose health depends on the EPA's using the best science, not just the most public. Commenter (11926) asserts that ignoring the results of such studies, many in the most exposed and most affected populations, would mean that regulations and policies essential to protect public health would never come into existence. Commenter (12731) states that, in November 2019, the editors of the nation's leading science journals issued a statement underscoring that the notion of discounting studies based on the availability of underlying data is contrary to good scientific practice and would be a "catastrophe" for public health.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
<IP_ADDRESS> Study Quality, Validity and Accepted Practices
<IP_ADDRESS>.1 Quality and Validity of Studies
Public Availability and Quality/Validity of Studies
Comment: Commenters (11392, 11399, 11408, 11473, 11480, 11487, 11489, 11576, 12455, 12457, 12546, 12720, 13788, 14397, 12715) do not agree with the rule's premise that studies are "better" or more "valid" than others, depending on whether or the degree to which their underlying data and models are available to the public.
Commenter (11392) states that all studies go through a rigorous peer review process prior to publication, and as a result, the scientific community does not use the so-called potential for "independent validation" as a metric for quality of a study.
Commenter (11399) states that both rule Options rest on an incorrect premise that scientific studies should be "graded" by EPA based on the transparency of their underlying data and models, and that more "transparent" science is more worthy of EPA's consideration than studies based on confidential data that cannot be made publicly available. The commenter states that this assumption is fundamentally untrue and conflates two separate issues: transparency and scientific rigor. The commenter states that this issue is distinct and separate from EPA's obligation to consider the complete range of published, peer-reviewed scientific research that is relevant to its regulatory decisions.
Commenter (11408) states that the EPA's response to concerns regarding data availability is to weigh more heavily studies where the data is publicly available. The commenter states that this "compromise" runs counter to principles of transparency, as the Agency would rely more heavily on a study with publicly available data over one that meets the highest scientific standards and yields compelling results.
Commenters (11473, 12546, 13788) assert that the quality of the research is not dependent on data availability and refer to the EPA's 2016 Plan to increase access to results of EPA-funded scientific research which states: "The validity of scientific conclusions drawn from research publications or their associated research data, or EPA's ability to consider those conclusions and data in its actions, does not depend on compliance with this Plan." Commenters (12546, 13788) point out that the EPA's 2016 Plan emphasizes that data availability does not affect the validity or usability of science: "Whether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications."
Commenters (11487, 11479) assert that using non-scientific criteria to evaluate scientific studies is thoroughly inappropriate and at odds with established practices for evaluating study quality. The commenters state that, for instance, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist does not include public availability of data. Commenter (11479) states that the STROBE initiative deems transparency crucial but does not include any criteria that would require the underlying data to be made publicly available. The commenter states that, as external researchers conduct systematic reviews or meta-analyses on particular health outcomes, public availability of data is not among the criteria for study appraisal in the widely accepted Cochrane Systematic Review method. The commenter states that the five criteria used to appraise and down-weight studies are related to study quality, not the availability of data, and include "limitations in the design and implementation of available studies suggesting high risk of bias; indirectness of evidence; unexplained heterogeneity or inconsistency of results; imprecision of results; and high probability of publication bias." The commenter states that imposing a requirement that is disconnected from the realities of the research world would put the quality of EPA's assessments and the Agency's reputation in the scientific community at risk.
Commenters (11480, 12464) assert that both approaches outlined in the proposal remove from consideration or significantly reduce the weight of studies when raw data is not publicly available for independent validation and reanalysis. Commenter (11480) states that public availability of data and models is not a criterion of best available science and should not be considered in determining whether to consider a study in an Agency decision. Commenter (12464) states that the rule relies on the flawed assumption that the scientific reliability of a study can be best determined by a reanalysis of the original data.
Commenter (11489) contends that the value or merit of peer-reviewed science is not determined by the public availability of the underlying research data. Commenter (12455) states that the lack of public availability does not automatically mean the research is flawed. Commenter (12457) states that, whether a study has data that is publicly available is not a factor that reflects the quality of the study, nor the relevance of the study's results to EPA policy decisions.
Commenter (11576) states that, throughout the proposed rulemaking, the EPA erroneously conflates the public availability of underlying data with the accuracy of a study. The commenter states that this conflation is contrary to well-established standards of scientific practice, in which the process of peer-review ensures that studies are vetted for accuracy. The commenter states that, while many scientific journals are starting to make more data publicly available, it is important to note that the availability of data underlying published studies has no bearing on evaluating the accuracy of the studies. The commenter states that access to unprocessed data at the stage described in the SNPRM is typically not a requirement even for peer-review, as scientists can evaluate the robustness and accuracy of findings based on the presentation of processed data and the articulation of the methods and analysis used.
Commenter (12546) states that the EPA is proposing to downgrade or exclude science that is not made publicly available or "cannot be sufficiently de-identified to protect the data subjects." The commenter states that this would codify that study quality is based on availability of underlying data, which is not the measure of study quality. The commenter states that this language is both vague and dangerous in implying that studies that do not make their data available are less informative or of less quality  -  both of which have no empirical basis. The commenter states that availability of underlying data is not a measure of study quality or validity; rather, study quality is dependent on the methods used in the study.
Commenter (12720) notes that the EPA in section 30.5 proposes to consider studies differently based on the availability of the study's data. The commenter states that this proposal has no scientific basis as availability of data has not been empirically shown to affect the quality or validity of a study. The commenter states that, as public availability of data or access to data have not been identified in the scientific literature as factors that have any bearing on the quality or validity of science, these criteria should not be used by EPA to give greater consideration to particular studies.
Commenter (14397) states that this SNPRM approach improperly makes openness the primary factor, or one of the primary factors, in weighing which studies to use, when robust scientific analysis calls for priority being given to studies that fit the accepted hierarchy of evidence and that follow basic scientific recommendations as to control, randomization, sample size, peer review, etc. The commenter states that studies should be considered or rejected based on their adherence to the appropriate scientific standards and the importance of their findings, not on their manner of dealing with CBI or PII, when presumably they have already been properly peer reviewed.
Commenter (12715) asserts that the use of Good Laboratory Practice protocols or other protocols where data are publicly available does not necessarily mean that the study is of higher quality or that it has evaluated the effects that are most relevant for decision-making, and there is no scientific reason that the data generated under the highly circumscribed regulatory requirements for product registration should receive greater weight than any other valid scientific data.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.