Comment: In addition to the requirements provided in §30.6 of the Proposal, commenter (6921) recommends that EPA evaluate quantitatively the impact of key uncertainties and assumptions where possible in an integrated fashion, so that the public can understand the cumulative impact of these assumptions and uncertainties in assessing potential risks and hazards.
Response: The EPA disagrees with the commenter that how uncertainty is treated in dose-response assessments should be part of this final rule. The scope of the uncertainty analyses that the EPA conducts necessarily varies across assessments and actions. The intent of this regulation is not to force uncertainty analyses into a one-size-fits-all approach, as that is not practical, good policy, or good science.
<IP_ADDRESS>.11 Revise §30.6 to Emphasize the Use of National and International Guidance on the Measurement of Uncertainties
Comment: Commenter (0560) requests that the EPA emphasize the use of national and international guidance on the measurement of uncertainties, giving special attention to Type B errors. The commenter suggests that EPA amend §30.6 as follows (underlined text reflects suggested new regulatory text and crossed out text reflects suggested regulatory text deletions):
      EPA shall describe and document any all assumptions and methods used, . and should EPA shall describe variability and address replicability,. and EPA shall systematically discuss uncertainty in measurement per national and international guidelines, particularly addressing Type B errors." Taylor NIST 1994, JCGM 100:2008.
The commenter also requests that EPA add the following language to §30.6 (underlined text reflects suggested new regulatory text):
      EPA shall review other relevant data and models found or brought to its attention, and discuss why those were not included.
Response: The EPA disagrees with the commenter that how uncertainty is treated in dose-response assessments should be part of this final rule. The characterization of uncertainty is a key factor in the assessments that the EPA conducts. It is a component of various EPA guidelines (e.g., Framework for Human Health Risk Assessment to Inform Decision Making) that the EPA relies upon in conducting its assessments. The scope of the uncertainty analyses that the EPA conducts necessarily varies across assessments and actions. The intent of this regulation is not to force uncertainty analyses into a one-size-fits-all approach, as that is not practical, good policy, or good science.
<IP_ADDRESS>.12 Supports Use of Policy or Guidance Documents for Requirements Specified in §30.6
Comment: Commenter (6920) agrees with the first sentence of §30.6 ("EPA shall describe and document any assumptions and methods used, and should describe variability and uncertainty"), however, the commenter states that the remainder of the language in this section, if still desired after further evaluation, would be better placed in policy or guidance documents.
Commenter (6125) suggests that, if EPA has issues with its current risk assessment and benefit analysis procedures that currently follow NAS recommendations and its own existing guidelines, a more straightforward and fruitful approach would be to update the guidelines as needed, showing in far more detail than provided here the details of their reasoning, the conditions in which they should be employed, and the scientific bases for requiring consideration of alternative models in all risk assessments. The commenter adds that the draft guidelines should then undergo peer review by expert panels of the NAS and/or its own SAB and SAP. According to the commenter, while guideline development is arduous and time consuming, it has several advantages over rulemaking, including 1) guidance is developed by experienced risk assessors; 2) it is a science-based process that involves consensus building across the EPA as well as with the broader science community; and, 3) critically, it maintains the separation between risk assessment and risk management so that EPA's decision makers benefit from the availability of science that is produced in an independent and objective fashion. This has been the practice at EPA since 1983 when the NAS published Risk Assessment in the Federal Government: Managing the Process.
Response: The EPA disagrees that the requirements of this rule are in conflict with or indicate issues with current Agency guidelines and guidance. Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review., When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
<IP_ADDRESS>.13 Supports §30.6 Proposed Regulatory Language
Comment: Commenters (6150, 6918, 6921) support the principles and "additional requirements" reflected in the proposed §30.6 and believe that the proposed §30.6 language reasonably reflects what ought to be EPA's consistent practice in all rulemakings.
Response: The EPA agrees with the commenter. The Agency has removed proposed 40 CFR 30.6, and placed some of the proposed requirements in 40 CFR 30.5(d)-(e).
<IP_ADDRESS>.14 Clarify Provisions of §30.6 to Apply at an Interim Phase Prior to Issuance of the Final Rule
Comment: Commenter (6906) suggests that EPA amend §30.6 (documentation and evaluation of studies) to require transparency steps at an interim phase prior to issuance of the final rule.
Response: The EPA agrees with the commenter and has altered the definition of "publicly available" at 40 CFR 30.2 to include "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
<IP_ADDRESS>.15 EPA Should Not Base Any Regulatory Determination on a Single Study
Comment: Commenter (9224) encourages EPA to not base any regulatory determination on a single study, regardless of whether or not the data is publicly available. The commenter suggests that relying on a single study, no matter how robust it may be, can bring bias to a model or regulatory decision. If EPA must base a decision off of a single study, the commenter states that it should be imperative that the data be publicly available as well as the reasoning and methodology behind why the study was chosen, how it is being used and why no other studies were deemed sufficient to be included.
Response: The EPA agrees with the commenter and clarifies in this final rule that, consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review., When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
<IP_ADDRESS> Study Exceptions/Exemptions from Public Disclosure Requirements
<IP_ADDRESS>.1 EPA Should Minimize Study Exceptions to Public Disclosure Requirements
Comment: Commenter (6143) states that EPA should ensure that any final action it takes on the proposed rule encourages transparency while making limited exceptions to accommodate legitimate, protected privacy interests such as CBI.
Commenter (6375) requests that EPA minimize the use of exceptions to the extent feasible within this rulemaking, but they understand that there will be some situations in which exceptions are needed. Regardless of the criteria used, the commenter contends that it is critical that EPA provide a robust and transparent explanation in the Federal Register that explains in detail why the data and models in question cannot be made publicly available.
Response: The EPA generally agrees with the commenters and has finalized requirements that protect privacy, confidentiality, CBICBI, and national security interests. As described in the final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
Additionally, under 40 CFR 30.7, the Administrator may grant an exemption to the requirements in 40 CFR part 30 for a study on a case-by-case basis if he or she determines that greater consideration is warranted because making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, CBICBI, or national security.
Comment: Commenter (6867) states that, in view of the widespread problems with replicating even peer-reviewed studies published in prominent journals, the EPA should minimize exceptions to the disclosure requirements set forth in the Proposed Rule. The commenter notes that Courts have frequently rejected attempts to withhold information based on claims to privacy, etc., unless protections are specifically provided by law. See, e.g., Johnson v. Dovey, 2011 U.S. Dist. LEXIS 128577, at *6-7 (E.D. Cal. Nov. 7, 2011) (rejecting vague assertion of privacy rights: "CDCR's objections are not specific in any way, and are thus insufficient to assert a privilege."); see also Donovan v. Nat'l Bank of Alaska, 696 F.2d 678 (9th Cir. 1983) ("The bank cannot refuse to comply with the subpoena as a whole on the basis of its vague allegations that it might be required . . . to produce records in violation of the Financial Privacy Act."). The commenter requests that EPA similarly reject such claims to "privacy, confidentiality, [and] confidential business information" vis-à-vis data and models, unless those claims are clearly supported by law or the Executive Branch's inherent powers. Accordingly:
      (a) EPA should amend the second sentence of Section 30.5 to provide that
      Where the Agency is making data or models publicly available, it shall do so in a fashion that is consistent with law and is sensitive to national and homeland security.
      (b) EPA should amend the penultimate sentence of Section 30.5 to provide that
      The agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of national and homeland security is not possible.
Further, the commenter states that, in view of the evidence that even studies published in prominent journals are often false or overstate results, the EPA should carefully consider the weight given to studies, data, and models that are not disclosed for review by the public and other researchers, even where non-disclosure is required by law.
Response: The EPA agrees that the Agency should carefully consider the weight given to pivotal science where the underlying dose-response data are not available for independent validation. When determining the degree of consideration to afford pivotal science for which the underlying dose-response data are not available for independent validation, the EPA will consider the factors included in 40 CFR 30.5(d). These factors include, for example, the extent to which there are other studies for which the dose-response data are available and the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study.
The EPA does not agree with the commenter's suggested changes to the relevant language in proposed 40 CFR 30.5. Given public comments expressing concerns related to the potential release of PII or confidential information, the Agency believes that it is important to acknowledge the need to protect privacy, confidentiality, and national security.
<IP_ADDRESS>.2 Criteria for Excepting/Exempting Underlying Data from Public Disclosure Requirements
Comment: Commenters (6125, 6446) state that the notice of the proposed rule solicits comments on which criteria it should base exceptions to the rule, including whether case-by-case exceptions to the rule may be appropriate but has provided no suggestions of its own on that subject.
Commenter (6446) argues that, because the underlying premise of the rule is flawed, and the rule is unnecessary, unlawful, arbitrary, and harmful, they decline to provide suggestions for methods to determine exceptions to the rule. The commenter argues that there are no criteria or ad hoc methods for making exceptions to the rule that would not pose an unnecessary risk of constraining use of the best available science to make regulatory decisions, particularly where political appointees rather than expert scientific panels, such as the SAB, would apply the criteria or make the ad hoc determinations.
Additionally, commenter (6446) asserts that there are no criteria or ad hoc methods that would prevent stalling vital decisions to protect public health and the environment by "analysis paralysis." The commenter states that the proposed rule, if adopted, would inevitably delay setting protective standards through prolonged evaluation of the sufficiency of the public availability of research data and methods, rather than evaluation of the actual quality and import of scientific research. The commenter adds that, a related likely consequence of the proposed rule, even if it contains ad hoc mechanisms for exceptions, is a reduction in EPA's ability to respond quickly to emerging challenges, when data and models take time to be made publicly available and/or redacted and otherwise prepared in a format appropriate for public review.
Response: In response to similar comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements.
The EPA has finalized the Administrator's exemption provision at 40 CFR 30.7. As described in the preamble to the final rule, the Agency retained the provision because there are conditions under which the Administrator may determine that greater consideration of pivotal science for which the dose-response data are not available in a manner sufficient for independent validation is warranted (e.g., age of the study). To ensure that the Administrator's decision is appropriately transparent, the EPA will document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the Administrator's exemption provision provides sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
The EPA finds that the requirements of the final rule will result in any meaningful delay in promulgating regulations. While the final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the EPA to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
Comment: Commenter (6046) concludes that it would be a waste of resources to identify criteria for exceptions which would be much more objective than determining exceptions on a case-by-case basis. The commenter contends that it is likely that exceptions would be complex (e.g. an exception is to be made if 4 of 5 criteria are met if 3 of the 4 are considered strictly confidential info and the other two are highly confidential).
Response: The EPA disagrees with this comment and finds that it is appropriate to identify conditions under which the Administrator may determine that greater consideration of pivotal science for which the dose-response data are not available in a manner sufficient for independent validation is warranted (e.g., age of the study). To ensure that the Administrator's decision is appropriately transparent, the EPA will document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the Administrator's exemption provision provides sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Comment: Commenter (6921) recommends that specific research study exemption decisions should be based on the nature of the data, including whether it is protected from release under current law, the existence of alternative publicly accessible studies of similar quality, and the need to act, based on statutory or judicial deadlines. The commenter suggests that, instead of waiting to apply enhanced transparency requirements during the rulemaking process when it is often too late to acquire and release the data for public comment, EPA should begin now to identify research studies and models that are likely to become pivotal regulatory science in future rulemakings. The commenter suggests that, once EPA has developed a list, it can begin to contact study authors to determine the optimal path forward to increase transparency while assuring continued protection of the confidentiality of research participants and CBICBI. According to the commenter, this effort will reduce the number of exemptions EPA might give in the future.
Response: The EPA agrees with the commenter, in part, and has finalized the Administrator's exemption provision with additional conditions under which the Administrator may determine that greater consideration of pivotal science for which the dose-response data are not available in a manner sufficient for independent validation is warranted. Under 40 CFR 30.7, the Administrator may determine, on a case-by-case basis, that greater consideration is warranted when:
    Technological or other barriers render sharing of the dose-response data infeasible;
    The development of the dose-response data was completed or updated before the effective date of the final rule;
    Making the dose-response data would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security;
    Third-party independent validation of the study's underlying dose-response data through reanalysis has been conducted; or
    The factors in 40 CFR 30.5 used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
The EPA would like to emphasize that the final rule does not require the Agency (or other parties) to disclose or host data, but to determine if dose-response data underlying pivotal science are available and to give greater consideration to those studies for which such data are available in a manner sufficient for independent validation. The EPA may opt, at its discretion, to make dose-response data underlying pivotal science available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
<IP_ADDRESS>.3 Study Exceptions Should be Recommended by Non-Partisan, Unaffiliated Expert Scientists
Comment: Commenter (4878) states that the proposal grants the EPA Administrator broad authority to exclude individual studies. According to the commenter, this could have broad-reaching impact depending on the preference of the Administrator at the time and allows the Administrator to overrule scientists regarding their own science. The commenter asserts that allowing politically appointed officials to make decisions about whether a study qualifies for an exception is dangerous. According to the commenter, the Administrator already has broad authority to decide what action to take on an item, it should not have the power to hide evidence that does not support the action. The commenter states that the EPA should consider all relevant, peer-reviewed data when making decisions that impact American's health, and the proposed rule's exceptions process clearly undermines this goal. The commenter states that, if the proposed rule takes effect, EPA should at least require that exceptions decisions are made by an expert in the area of research. For example, the commenter recommends that a panel of non-partisan, unaffiliated expert scientists could be used to make recommendations on exceptions.
Response: The EPA does not agree with the commenter regarding the role of the Administrator in determining whether to grant an exemption and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Additionally, the EPA program or region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption.
The EPA would like to emphasize that, under the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information.
<IP_ADDRESS>.4 Request for Specific Study Exceptions
Major Scientific Studies Already Completed or Underway
Comment: Commenters (1012, 5165, 6141) request that EPA develop exemptions for scientific studies already completed or underway.
Commenter (6141) suggests that the Agency attempt to accommodate major scientific studies already completed or well underway. If EPA elects to apply the transparency rules retrospectively, the commenter suggests that EPA consider developing exemptions for those past studies for which it is not possible to release a dataset publicly for legitimate reasons pertaining to the use, for example, of human subject data. According to the commenter, in so doing, EPA should strive to establish clear and objective criteria for granting exemptions based on the inability to provide the underlying data while respecting appropriate data access limits.
Commenter (1012) requests that exceptions to the proposed rule include studies that were previously conducted or already underway prior to the implementation of this regulation as the protocols of these studies were set before the rule was finalized. Studies that were conducted and utilized by the Agency to make regulatory decisions prior to the implementation of this rule should be exempted as these studies had met the requirements previously set for inclusion in regulatory decision-making at the time, and the protocols used in these studies were already determined before the rule was finalized. Case-by-case exceptions shrouded in secrecy are not appropriate and violate the nominal title of this proposed rule. If case-by-case exceptions are sought, every step in the process of determining the exception should be made publicly available from the initial request, to the decision-making steps taken by the Administrator, in addition to the final decision.
Response: The final rule applies equally to all dose-response data underlying pivotal science, regardless of when the study or the data was created. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of the final rule under 40 CFR 30.7.
To ensure that the Administrator's decision to grant an exemption under 40 CFR 30.7 is appropriately transparent, the EPA has included a provision that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Additionally, the EPA program or region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption.
Well-Established Effects of Particle Pollution on Mortality Risk
Comment: Commenter (6135) specifically opposes any proposed or future action that would disregard the well-established effects of particle air pollution on mortality risk in the United States, either as part of future regulatory decisions or in cost benefit analyses included in mandated regulatory impact assessments. According to the commenter, these adverse risks have been confirmed and verified both through extensive reanalysis of individual studies and more importantly through additional studies that have been completed using numerous independent study populations in the US, Canada, and many other parts of the world. Regardless of whether these effects are reported by the EPA as ranges or as combined estimates of mortality risk, the commenter asserts that the continued inclusion of mortality impacts in regulatory analysis is critical for federal regulatory decisions that impact air quality in Utah. The commenter requests that, if the proposed transparency rule is promulgated, the associations of particle pollution and mortality risk are specifically and preemptively excluded from the judgment of the EPA Administrator in adhering to data availability requirements.
Response: As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science, regardless of when the study or data was created. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption, on a case-by-case basis, from the requirements of the final rule under 40 CFR 30.7.
Study Data from Authors Outside the Agency
Comment: Commenter (6449) notes that underlying study data may be difficult to obtain from authors outside the Agency. The commenter requests that EPA allow exceptions for such instances.
Response: As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption, on a case-by-case basis, from the requirements of the final rule under 40 CFR 30.7.
Studies that Meet a Minimal Threshold of Study Quality
Comment: Commenter (6449) requests that, in order to limit the potential for exempting pivotal studies that do not meet the data transparency requirements the studies in question meet a minimal threshold of study quality.
Response: The EPA finds that establishing a minimal threshold of study quality is not practical in this rulemaking. As described in the preamble to the final rule, the EPA will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. Additionally, under 40 CFR 30.5, the EPA is required to rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Studies that Represent "Best Available Science"
Comment: Commenter (6906) recommends that the EPA amend 40 CFR §30.9 regarding exemptions, to include an exemption from publication of dose response and methods, when not available, for studies EPA concludes are the "best available science," using factors EPA has articulated for "best available science."
Response: As described in the preamble to the final rule, this is a rule of internal Agency procedure and, therefore, does not interpret or apply any statutory provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
Studies Involving Human Subjects/Health Data
Comment: Commenter (0671) states that if EPA goes ahead with this counterproductive rulemaking, studies involving human subjects should be excepted. In particular, the commenter requests that clinical and population-based studies should be excepted, which they assert produce data that is almost guaranteed to be identifiable and no IRB would allow such data to become public. The commenter states that if this proposed rule disallows EPA from using the results of such studies in formulating regulations then the EPA will have prevented itself from using the best available science, counter to requirements of the CAA and amendments.
Commenter (1272) asserts that any regulation that may affect the health and wellness of people, should and must be allowed to use research that cannot publish its data. The commenter provides that health data is generally not allowed to be freely published and thus must have an exception.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The requirements included in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the EPA maintains a strong scientific basis for its decision-making. The incremental progress made possible by the final rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments.
Pesticide Registration and Tolerance Data
Comment: Commenter (6374) supports the exclusion of data submitted under FIFRA or FFDCA to support specific registration or tolerance decisions, which they say is consistent both with that threshold and with the statutory protections for such data. When issuing its final rule, the commenter requests that EPA expressly clarify that the scope of its final rule does not include such pesticide registration and tolerance data.
Response: In response to this comment and other similar comments, the EPA clarified in the 2020 SNPRM the relationship between this rulemaking, the environmental statutes and their implementing regulations by adding language to proposed 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations would control in the event of any conflicts.
With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Confidential Business Information
Comment: Commenter (6148) recognizes that in some circumstances full transparency may not be achievable. For example, some data submissions are protected as CBI and may present transparency challenges to the Agency. In such cases, the commenter provides that it may have to be considered whether to exclude such data from the analysis or determine whether a generic format may be adequate to conceal site or company-specific proprietary information.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR <IP_ADDRESS>.4.9.
Third-Party Laboratory and Research Organization Studies Should Not Be an Excepted/Exempted from Public Disclosure Requirements
Comment: Commenter (6108) asserts that, for non-peer reviewed commissioned studies such as third-party laboratory or other research organizations that may be commissioned, the underlying data should be made available to the public, along with the conclusions, assumptions, methods used, and a discussion of limits of extrapolation. The commenter notes that such third-party scientific studies may draw conclusions, but do not solicit other peers (a "fourth" party) to review the findings. The commenter contends that, because such studies lack the layers of review that accompany peer-reviewed science, they should be held to a higher standard of transparency if relied upon in rulemaking decisions.
Commenter (6375) recommends that, in cases where EPA does not control the data and models for studies, the EPA should not provide an exemption simply because a third-party does not agree to make the data and models publicly available where it does not have a valid reason (e.g. CBI, privacy, etc.) to prevent their release. In those cases, the commenter recommends that the EPA make every attempt to make as much of the data and models available for independent review as possible. According to the commenter, even if the dose-response data and models per se cannot be made publicly available, the commenter suggests that it may be possible for the results and analyses of this information to be made publicly available. According to the commenter, any such available results should be transparent as to the identification and selection of the key data, and the interpretation of those key data, in supporting the Agency's decision.
Response: As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not require the Agency (or other parties) to disclose or host data, but to determine if dose-response data underlying pivotal science are available and to give greater consideration to those studies for which such data are available in a manner sufficient for independent validation.
Study Exception Requests Would be Burdensome
Comment: Commenter (6373) asserts that, because the Rule's default application is to bar research where underlying data is not publicly released, EPA must separately consider exemption requests for each study, at considerable and likely prohibitive time and expense.
Response: Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The additional procedures required by the final rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. For similar reasons, the EPA does not find that the requirements of the final rule will result in any meaningful delay in promulgating regulations.
4.1.2 Definitions of Research Data (§30.2) and Data
<IP_ADDRESS> Definition of "Research Data"
Comment: Commenter (6880) asserts that the proposal disregards the cited definition of "research data" and the rule applies to a much broader range of data. The commenter states that this presents serious consequences for EPA, the people whose mission it is to protect, and the regulated community because the proposal would allow EPA to ignore research findings if it were not possible to make public all the underlying data, including data that is excluded from the definition of "research data." The commenter states that the proposed definition of "research data" excludes types of data that are nonetheless discussed throughout the proposal to strengthen transparency. The commenter states that EPA disregards this definition throughout the actual proposal, as evidenced by the following:
 Though EPA references this definition in §30.2 ("What definitions apply to this subpart?") of the proposed rule, the term "research data" is absent from the rest of the proposed regulation (except in §30.10). EPA instead refers only to "data" without ever requiring that "data" be limited by the definition (and exclusions) of "research data."
 In describing the data to which the proposal applies, EPA states in §30.3 "The provisions of this section do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses." As these are already excluded from the definition of "research data," it would be redundant for EPA to exclude them if they were using said definition. Also, by mentioning these exclusions while remaining silent on trade secrets and privacy information that are excluded from the definition of "research data," EPA clearly signals that those data are not excluded from the "data" to which EPA refers in the proposal.
 Given that the cited definition of "research data" excludes privacy data, the following concern about privacy in §30.5 of the proposal would be a non-issue if EPA were to actually adhere to that definition: "The agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of privacy, confidentiality, national and homeland security is not possible."
Commenter (6924) states that there are many different kinds of data generated in a research study. The commenter states that the rule treats all these data as if they were the same when that is not the case, indicating the lack of scientific input into this rulemaking. The commenter states that a 2013 memo from Office of Science Technology and Policy recognizes different data types and specifies that study meta-data, not individual-level data, should be made publicly available.
Commenter (6880) notes that, in §30.10, EPA states "EPA shall implement the provisions of this section consistent with the definition of `research data'". The commenter states that, however, such definition applies only to §30.10. The commenter states that, if EPA meant for it to apply to the entire proposed regulation (as stated in the preamble), they would have stated that they "shall implement the provisions of this part consistent with" the cited definition.
Commenter (6126) asserts that the term "research data" in the proposed rule is ambiguous and needs clarity to assess the rule's potential impacts and for commenters to understand how EPA intends to implement the proposed rule. The commenter states that EPA's existing "Plan to Increase Access to Results of EPA-Funded Scientific Research" provides for exemptions to public release requirements when "the research data cannot be released due to one or more of constraints (sic), such as requirements to protect confidentiality, personal privacy, proprietary interest, or property rights." The commenter asserts that it is not clear if these same exemptions would apply under the proposed rule.
Commenters (6137, 6449, 6924) express concern that the definition of "research data" is not defined or impermissibly vague/has problems.
Response: Proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of "research data." In the 2020 SNPRM, the EPA deleted the proposed definition of "research data." While one commenter on the 2020 SNPRM noted that the exclusions in the proposed definition of "research data" of trade secrets and personal and medical information were not incorporated into the proposed definition of "data," commenters did not request that the EPA maintain a definition of "research data." The EPA is not including a definition of "research data" in this final rule because the Agency is finalizing the definition of "data" in section 30.2 as follows:
      Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party.
In addition, the EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The rule allows the EPA Administrator to grant an exemption to the transparency requirements if the Administrator determines that greater consideration is warranted when:
       Technological or other barriers render sharing of the dose-response data infeasible;
       The development of the dose-response data was completed or updated before the effective date of this final rule;
       Making the dose-response data would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security;
       Third-party independent validation of the study's underlying dose-response data through reanalysis has been conducted; or
       The factors in 40 CFR 30.5 used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
<IP_ADDRESS> Definition of "Data"
Comment: Commenters (0671, 5419, 6373, 9224) express the need for EPA to better define/clarify what "data" the rule applies to.
Commenter (6373) states that the impacts of the proposed changes will be difficult to predict unless EPA better defines what "data" the rule applies to.
Commenter (9224) suggests that if the agency wants a rule focused on the raw data then EPA must better define what "data" would be included within this rule. The commenter asks whether it is just data produced by EPA, the methods and protocols, or the actual raw data?
Similarly, commenter (0671) notes that there is a problem with the definition of "data... underlying pivotal regulatory science". The commenter asks whether this means the raw data or the data used to make the final figures and tables in a paper or the data as it exists at some processing step in between? The commenter further asks, if the raw data, how raw? The commenter notes that some raw genomic data is so large it can't be efficiently stored. According to the commenter, there is no one unique thing called "the data" for a study.
Commenter (5419) asserts that the interpretation of the term "data" has critical implications. The commenter notes that, for complete independent validation and analysis, as stated in the proposed rule, the primary data record would be required. According to the commenter, this could include, for example, raw death certificates and raw chemical monitoring data, both of which are subject to transcription error during creation of an analytical file, as well as, in the case of death certificates, challenges related to mortality coding. The commenter states that, notwithstanding privacy concerns, which is the subject of other portions of the proposed rule, the public posting of these large quantities of data would be unwieldly. The commenter requests clarification of this issue.
Response: As described in the 2020 SNPRM, the EPA agrees that there are different stages of data. The EPA presented the different stages as described in the NAS Workshop Report. Since the purpose of 40 CFR 30.5 is to determine the consideration to afford to studies based on, among other factors, the availability of the underlying dose-response data that would support independent validation via reanalysis of the data underlying pivotal science, the appropriate stage of data would not be the processed data (data that have been computed and analyzed to extract relevant information) or the final clean data set that is provided with a publication. At these two stages of data, the analysis has already been conducted, and the results have already been determined. In order to determine if these results are valid, data that had not already been computed and analyzed are needed. Given concerns about potentially skewed analyses, the final definition of "data" maintains the stage of data in which obvious errors have been removed.
In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
The definition of "data" in section 30.2 of the final rule is as follows:
      Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party.
4.2 SNPRM Models, Variability/Uncertainty, and Assumptions/Defaults
Overarching Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. Because this rule only applies to dose-response data, the Agency is not finalizing the definition of "model."
4.2.1 Modeling Definitions
Comment: Commenters (11401, 12406) express support for the clarity achieved by defining "model." Commenters (12406, 12463) also express support for the clarification of the terms "models" and "model assumptions."
Conversely, several commenters (11894, 12435, 12720) express criticism of modeling definitions as being unclear or too broad.
Commenter (12435) indicates the supplemental notice of proposed rulemaking (SNPRM or supplemental proposal) definition of "model" is too broad to be useful. Commenter (12727) states that the definition, among other definitions, fails to adequately comprehensively describe the scope of its proposal. As a result the commenter contends that the EPA has denied the public the information it needs to comment in an informed way.
Commenter (11894) asks whether the definition of "model" applies only to "computational models". The commenter notes that previous guidance documents use the same definition of "model" to focus on the subset of all models termed "computational models" by the National Research Council (NRC). The commenter provides that these are models that use measurable variables, numerical inputs, and mathematical relationships to produce quantitative outputs. The commenter requests clarification as to whether the Agency envisions the same scope of applicability in the promulgation of the proposed rule.
Commenter (12720) states the definition of "model" is overly general and vague does not address how EPA engages with models and modeling output in conducting its scientific activities. The commenter adds that the definition does not distinguish between physical models, conceptual models, and software modeling packages, nor does it explicitly account for models applicable within the human health context, even though that type of tool is commonly deployed by EPA in its scientific work. According to the commenter, in failing to define models and distinguish between modeling inputs and outputs, EPA's definition is flawed and insufficient.
Response: EPA appreciates these comments. As discussed in the overarching response to this chapter, the Agency is not finalizing the definition of "model." Because this rule only applies to dose-response data, the rule does not apply to computational models.
4.2.2 Models (General) and Best Science Practices
Comment: Commenter (11484) indicates that by expanding the proposed rule to include all data and models rather than just dose-response data and models, it restricts the use of scientific information for arbitrary and non-scientific reasons. The commenter states that the newest version of the proposed rule expands the range of decisions that will be hampered by lack of access to the best available science.
Commenter (11347) suggests a connection between the impact of this rule and the present coronavirus pandemic. They note that policy and action should be increasingly based upon scientific results coming from powerful and reliable models in a constant state of evolution as the models themselves improve and as the inputs change. The commenter contends that the supplemental proposal would place such work in limbo. In other words, the commenter states that it doesn't take advantage -- in fact, it denies the efficacy of advances in computational science and other ways of creating reliable and complex models to help us solve these countless problems.
Commenter (11894) indicates that it is imperative to understand that empirical data do not exist for every element of a risk evaluation and that models, many of which have already been peer-reviewed and validated, are employed to fill the data gaps and allow the Agency to reach conclusions about risk.
Commenter (12688) states that EPA's proposed rule will require the Agency to ignore both foundational and cutting-edge research in core areas of public health and other scientific inquiry. According to the commenter, these could include, for example, models key to understanding earth and atmospheric systems and a limitless range of other research. In short, the commenter contends that the proposal would undermine EPA's ability to use science to protect the health of US residents across all facets of the agency's work.
Response: See response to prior comment. The EPA disagrees that implementation of the rule would undermine public health protections. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the public's understanding of the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. As further described in the preamble to the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be hampered by lack of access to the best available science or prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Finally, this rule will not impact the Agency's use or reliance on models.
Comment: Commenter (11393) expresses concern regarding the rule's potential impacts on health risk assessments. The commenter specifically expresses opposition to language that removes or dilutes the linear no-threshold (LNT) dose response in the evaluation or establishment of radiation health risk assessments, rule-making or establishment of radioactive recommended action levels or benchmarks (influential scientific information) made available to the public.
Response: After considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model. Given the specificity of 40 CFR 30.6 to dose-response data and models, and in particular dose-response models, the EPA is not finalizing 40 CFR 30.6. The EPA is adapting one provision of 40 CFR 30.6 as a factor in 40 CFR 30.5 in determining the consideration to afford pivotal science for which the dose-response data are not available for independent validation. Specifically, the EPA is finalizing as a factor in 40 CFR 30.5 the consideration that the EPA would give to high quality studies that explore a broad class of parametric dose-response models, non-parametric models that incorporate fewer assumptions, various threshold models, and models that investigate factors that might account for spatial heterogeneity.
Comment: Commenter (11492) indicates that the EPA continues to assign greater value to dose-response studies, which by definition reduce the use/value of epidemiological studies. Section 30.6 of the supplemental proposal has been revised but continues to give "explicit consideration to parametric dose-response or concentration-response models." As the commenter suggests they stated in its 2018 comments, the EPA itself admits that, "there is frequently a lack of dose-response data available for human subjects. When data are available, they often cover only a portion of the possible range of the dose-response relationship."1The commenter states that this gives more weight to toxicological studies as opposed to epidemiological studies, which form the basis of many human health exposure studies. The commenter asserts that the EPA must continue to use epidemiological studies; to do otherwise is to compromise the health of all Americans. Commenter (12403) concurs, indicating that the SNPRM's identification of specific dose-response modeling techniques that would be given "explicit consideration" is inappropriate and ignores the improvements and refinements that are the hallmark of good science.
Response: Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7.
The EPA disagrees that the rule implies a bias against the use of epidemiology studies or data. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences. The EPA will continue to consider and evaluate all relevant and appropriate science, including epidemiology studies, in its significant regulatory actions and influential scientific information.
In addition, after considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model.
Comment: In support of existing model availability, commenter (11894) provides the specific example of the trichloroethylene (TCE) case study. The commenter notes that the TCE assessment relied on approximately 15 models. The commenter notes that this is likely to be the norm with the same ones being used repeatedly. The commenter adds that most of the models EPA employs in the TSCA program have been developed by the Agency and are publicly-available. The commenter provides that access can be gained through a number of EPA webpages and/or searches on the internet, and others are described in the peer-reviewed literature, access to which is controlled by the authors.
Response: After considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model.
Comment: Commenter (12465) states that the EPA relies upon many types of data and models to support its regulatory decisions and it is unclear that the current proposal has adequately considered the numerous types. The commenter expresses concern that the treatment of existing data or models under the proposals disincentivize the generation of new data under the new requirements while reducing the importance of consideration of existing data, regardless of the scientific validity or merit of that data.
The commenter (12465) provides the example of the Westat Survey. The commenter notes that the report entitled "Household Solvent Products: A National Usage Survey" (or the Westat Survey) is a survey of consumer usage for a wide variety of products to determine various attributes and inform exposure assessments. According to the commenter, while the report was completed in 1987, it remains in use today informing the EPA's Consumer Exposure Model and the Westat Survey informed several of the first 10 chemicals currently being evaluated under reformed TSCA. The commenter states that it is unclear in the proposal how the Westat Survey would be treated, let alone the multitude of exposure models that rely upon it.
Response: As further described in the preamble to the final rule and the overarching response to this chapter, the EPA is not finalizing the definition of "model." The rule only applies to dose-response data and will not impact the Agency's use or reliance on models, including exposure models. Since the Westat Survey is a consumer behavior survey for assessing exposure, it would also not be applicable to this rule.
The EPA disagrees that the rule disincentivize the generation of data. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences. The EPA will continue to consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information.
4.2.3 Public Availability of Models/Data
Comment: Several commenters (11392, 11393, 11407, 11576, 11894, 12430, 12472, 12474, 12720, 12727) express opposition to requiring models and data be publicly available. Commenter (11393) opposes these expansions; and contends that the SNPRM takes a bad proposal and makes it worse.
Commenter (12727) indicates that EPA has not demonstrated -- and cannot demonstrate -- that a study cannot be scientifically valid unless the underlying data and models are publicly disclosed. The commenter states that mechanisms are already in place to validate studies for which disclosure of underlying data and models is either illegal or impracticable.
Commenter (12472) indicates that expanding the scope to include all models would place a particular burden on air quality models and potentially have an adverse impact on public health. According to the commenter, if air quality models are included, the data access requirements described in the SNPRM may pose difficulties in some instances, particularly for emissions inventories used in modeling.
Commenter (11576) highlights the impact on environmental-fate studies and models if these types of studies are included in its new, broader scope. The commenter provides that these data and models are critical for the EPA's assessments of the environmental risks of pesticides, for example, and the Agency has relied on well-established studies that were conducted before mechanisms existed for sharing data publicly. The commenter adds that the EPA also mentions bioaccumulation data as an example of the types of studies included in the new expansion, which they contend could unjustifiably undermine established science on the accumulation of mercury, lead, and other metals in aquatic systems and wildlife, as well as models used by the EPA for decades to describe accumulation.2 The commenter contends that the requirement for publicly available data underlying all model parameters would be particularly problematic and potentially impossible for models developed iteratively over decades, in which model parameters and structure evolve with growing science without the need for publicly available documentation of all the underlying parameters' source data.
Commenter (11894) points out that databases of publicly available models already exist, such as the Registry of EPA Applications, Models and Data Warehouses (READ) (available at https://ofmpub.epa.gov/sor_internet/registry/systmreg/searchandretrieve/basic/search.do), which catalogs publicly-available models that the agency uses. The commenter provides that there are currently 2,197 entries in this library of tools, a subset of which are models. And, specific to EPA's Existing and New Chemicals programs, Office of Chemical Safety and Pollution Prevention (OCSPP) hosts a web page (Predictive Models and Tools for Assessing Chemicals under the TSCA, available at https://www.epa.gov/tsca-screening-tools). The commenter adds that using these two sources and others available by searching the internet, one can access the publicly-available, Agency-developed models that it employs. The commenter notes that if the Agency has to resort to the use of a proprietary model to conduct some aspect of its work, it is not within its purview to release the code to it. Under that circumstance, the commenter notes that one must arrange that with the owner of the model. This, according to the commenter, however, has become an increasingly rare circumstance.
Commenter (11894) notes that securing the release of proprietary modeling information is difficult, and places an undue burden on EPA. The commenter adds that if the Agency has to resort to the use of a proprietary model to conduct some aspect of its work, it is not within its purview to release the code to it.
Commenter (12430) states that models are critical to help humans assess and analyze large quantities of data, but they are nearly always proprietary. The commenter provides that most scientists and agencies purchase access use these models for data. The commenter contends that restricting models to those that are publicly available would severely limit the science that U.S. EPA may consider or reference.
Commenter (12720) indicates that for many studies, proprietary software and/or proprietary code is necessary in order to effectively work with underlying study data. However, the commenter states that these models are generally not freely available to the public, and many require advanced technical training and considerable technical resources to operate. The commenter notes that the EPA has not discussed this issue in the proposal, even though it presents a major obstacle to the goals of the proposal. According to the commenter, for already burdened researchers, it is not feasible to expect them to provide underlying code (which may be subject to other release requirements and advance permissions) to the Agency and places an undue burden on the EPA.
Conversely, commenter (12712) expressed support for making models/data publicly available wherever possible. The commenter suggests that the default rule for all work funded by the EPA must be that relevant code and data are available online or on request. The commenter adds that the EPA must be clear that making data available means that the actual final dataset used in statistical analysis is made available.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to other types of data and models, including bioaccumulation data and environmental fate and transport models. The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees, and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public.
4.2.4 Weighting of Models
Comment: Commenter (9362) supports the wording of section 30.6 "When available, EPA shall give explicit consideration to high quality studies, including but not limited to those that explore: A broad class of parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity." However, the commenter states that "nonparametric models that incorporate fewer assumptions" should be explicitly stated in this sentence. The commenter notes that parametric models may be more appropriate in many cases.
Commenter (11395) suggests that the "explicit consideration" language of section 30.6 is inappropriate because the scientific information, techniques and models used in risk assessments are developing rapidly. Therefore, the commenter suggests that risk assessment methodology should be addressed in guidelines, which can be adjusted as more information becomes available, rather than with fixed regulations. Instead of establishing inflexible regulatory requirements, the commenter adds that the Agency should focus on continuing to improve its existing guidelines, in concert with other federal agencies. The commenter adds that the specific identification of modeling considerations in a regulation is inappropriate because multiple analyses using a variety of models do not necessarily make a study more credible, there is no scientific justification for this stipulation. The commenter adds that model choices consider such factors as biological plausibility and the degree of protection to human health. According to the commenter, an evaluation of a study that uses modeling should assess model assumptions, the fit of the model to the dataset, the goal of the analysis, and other issues. The commenter contends that the use of a greater number of models does not in itself improve results or make a study more plausible. Therefore, the commenter asserts that giving priority to studies based on the number or range of models used is scientifically inappropriate and may create an incentive for using inappropriate models without scientific justification, which may generate inaccurate outputs.
Commenter (10740) states that, under the supplemental proposal, EPA would be required not only to prepare its own models with multiple assumptions, but also to "give explicit consideration" to a long list of models that could be prepared by outside stakeholders. The commenter provides that the proposal presents a list of these models that EPA must explicitly consider, such as a "broad class of parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity."
The commenter (10740) suggests that perhaps it is the intent of this proposed rule to preclude scientific evidence based on data and models informed by raw data, including medical records, that is required to be held confidential. The commenter asserts that this is not transparency and nor is it science.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models. Therefore, the rule does not include discussion of any particular dose-response model. The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
4.2.5 Scope of Models Included
Comment: Commenter (11894) contends that the EPA has not adequately explained why the scope should be expanded to include far more models. The commenter asserts that when a federal Agency proposes to make such far ranging changes, it is obligated to explain why and to support that with data and information.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models.
Comment: Commenter (12472) expresses that the expansion could potentially impact on air quality models, and requests that EPA make explicit whether these models are included under the rule. The commenter specifically requests that the EPA clarify whether air quality and associated models such as Comprehensive Air Quality Model with Extensions (CAMx) are included as models subject to the rule.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models, including air quality models.
Comment: Commenter (12699) quotes § 30.3 "The provisions of this part apply to data and models, underlying pivotal science supporting influential scientific information and/or underlying pivotal regulatory science used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the science." The commenter notes that the comma after "data and models" appears to be communicating that the provisions apply to data and models, underlying pivotal science, and/or underlying pivotal regulatory science. Unfortunately, the commenter contends that this language could create confusion by giving the impression that the provisions only apply to data and models. According to the commenter, since this is unlikely what the EPA intends, they recommend that EPA make § 30.3 clearer.
Commenter (12699) adds that, if in fact the EPA does only intend to cover data and models, then this would make little sense and beg the question why it would not cover pivotal science and pivotal regulatory science. The commenter notes that both are critical in practice and for purposes of the rule. The EPA would be creating a transparency rule to help the public evaluate the science, and then arbitrarily only allow the public to evaluate a subset of what informs influential scientific information and significant regulatory actions.
Response: As further described in the preamble to the final rule and the overarching response to this chapter, the Agency is not finalizing the term "model." Therefore, the rule does not apply to dose-response models or other types of data and models. The text from 40 CFR 30.3 quoted by the commenter has also been clarified to now read: "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science."
The EPA acknowledges confusion over the definition of "science that serves as the basis for informing a significant regulatory action" and is altering the definition of the term in 40 CFR 30.2 to mean "studies, analyses, models, and assessments of a body of evidence that provide the basis for EPA significant regulatory actions." Examples of models include those used in regulatory impact analyses. Examples of assessments of a body of evidence include risk assessments, hazard identifications, IRIS assessments, and criteria documents. The EPA also clarifies in the final rule that the scope of science that serves as the basis for informing a significant regulatory action is equivalent to the science included in the public docket as part of a rulemaking, but not all of that body of science would typically be considered "pivotal science."
4.2.6 Assumptions/Defaults-Variability/Uncertainty
Comment: Commenter (11403) expresses that in peer-reviewed scientific publications, scientists state their assumptions as well as all model assumptions in the method sections (or else cite a publication that discusses those assumptions), so the peer-review process already provides this level of transparency. In addition, the commenter states that parametric models should not be favored over non-parametric ones. The commenter asserts that favoring one type of modeling over another does not support good science.
Response: As detailed in OMB's Final Information Quality Bulletin for Peer Review, "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. Where underlying dose-response data in pivotal science are available, subject matter experts could do more than evaluate the validity of an assumption; experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA is confident that this approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Finally, the final rule does not apply to dose-response models and does not prioritize types of dose-response models.
Comment: Commenter (12430) states that requiring justification for all default assumptions ignores the establishment of defaults through prior public processes and external public peer reviews. For example, the commenter notes that requiring justification for the use of default assumptions for every toxicity health factor derivation ignores the detailed development process of those default assumptions. The commenter states that those default assumptions were developed in several U.S. EPA documents (including the 2005 Guidelines for Carcinogen Risk Assessment3), which received public comment and extensive external peer-review. Requiring EPA to re-justify those default assumptions every time they are used, as the SNPRM seems to require, would serve no purpose other than to add unnecessary delay to risk assessments, regulatory action, and publication of influential scientific information.
Conversely, commenter (9362) expresses support for language that states that the "EPA shall describe and document any assumptions and methods used and shall describe variability and uncertainty". The commenter further supports language that states "EPA shall clearly explain the scientific basis for critical assumptions used in the analysis that drove the analytical results and subsequent decisions and shall present analyses showing the sensitivity of the modeled results to alternative assumptions." The commenter notes that it is good if all source information is disclosed. The commenter further supports language that states "[b]ecause transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results," noting that it is great as long as all pertinent scientific information is included.
Response: The EPA clarified in the SNPRM that the use of the terms "model assumptions," "assumptions" and "models" in the proposed regulatory text at 40 CFR 30.6 apply to the critical assumptions that drive the model's analytic results, not to each assumption used in the model. In the final rule, the EPA maintained the use of "critical assumptions" rather than "all assumptions" in what is now 40 CFR 30.5(e). The EPA also made edits to the text that the commenter quoted regarding peer review in what is now 40 CFR 30.6 to conform with the applicability of the rule to dose-response data. The EPA agrees with the commenter that greater data availability will allow peer reviewers to better evaluate and articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results.
Comment: Commenter (11361) provides that the EPA describes its intent for the proposed public data requirements to "allow stakeholders to reanalyze the data and models and explore the sensitivity of the conclusions to alternative assumptions." The commenter contends that this begs the questions of who the assumed "stakeholders" are and what the purposes are behind developing alternative assumptions.
The commenter (11361) adds that the EPA goes on to say the rule would "allow assessment of the robustness of the original analysis and conclusions by, for instance, showing the variability that can occur when a previously omitted variable is added to the statistical model, different functional form assumptions are made (e.g., a linear marginal effect of treatment), or different assumptions are made when estimating standard errors and drawing statistical inferences (e.g., allowing for spatial correlation in error terms)." According to the commenter, this is highly problematic, however, and is an overt way to attack credible science with unfounded critiques. The commenter notes that all statisticians know that including too many variables is at least as flawed, and often more so, than including too few variables, and one can artificially increase or decrease model fit without any theoretical grounding. Likewise, the commenter adds that decisions about functional forms are typically grounded in theory, and even if they are not, clear relationships can be obfuscated by limiting or overwriting functional forms. According to the commenter, applying a different functional form (e.g. linear rather than log-transformed) would always change the results and could obscure meaningful relationships. The commenter contends that the EPA is not providing an opportunity to improve science through this proposed rule. The commenter states that, for research that has gone through peer review (as the EPA, thankfully, clarifies it expects for research that is included in significant agency decisions ), altering model inputs, structure, and assumptions are easy4mechanisms for unsophisticated and inappropriate muddling of credible and substantiated models.
Commenter (11495) notes that the proposed rule suggests that data reanalysis will be completed by "stakeholders", who will have the opportunity to "reanalyze the data and models and explore the sensitivity of the conclusions to alternative assumptions while accessing only the data and aspects of the models that they need". The commenter states that there is a lack of clarity over who the "stakeholders" involved in this analysis may be, and that the suggestion that they may cherry pick data to force into models that support "alternative assumptions" goes directly against the scientific process and certainly looks at least superficially like exactly the sort of bias the EPA is hoping to avoid with their rule. According to the commenter, while good faith use of alternative methodologies to test sensitivity of various inputs is good scientific practice, expecting the same research data to yield the same results when put through alternative methodologies, potentially employed by undefined stakeholders who have the potential to intentionally introduce bias, is not a credible deployment of the scientific process.
Response: The EPA agrees that the word "stakeholders" in the SNPRM preamble was not defined. In this context, stakeholders meant any subject matter expert who has not contributed to the development of the study. The EPA fixed this inconsistency in the final rule and only uses the term "subject matter expert." Researchers generally use a number of assumptions in their development of a scientific assessment. The impact of the use of the assumptions is not always clear, particularly when the underlying data are not available. The intent of the language was to explain a benefit of greater transparency in the underlying data -- that subject matter experts could validate the study conclusions using the same assumptions, but also reanalyze the data using alternative assumptions to better understand the sensitivity of the results to the assumptions.
The EPA disagrees that the rule does not offer an opportunity to improve science. The benefits of greater data transparency and the significance of reanalyzing and validating study results are well-documented in scientific literature. McNutt (2014) noted, "reproducibility, rigor, transparency, and independent verification are cornerstones of the scientific method." The National Academies of Sciences, Engineering, and Medicine (NAS) workshop on Reproducibility and Replicability in Science also noted that "certainly, reproducibility and replicability play an important role in achieving rigor and transparency." Munafò et al. (2017) state, "the credibility of scientific claims is rooted in the evidence supporting them, which includes the methodology applied, the data acquired, and the process of methodology implementation, data analysis and outcome interpretation. Claims become credible by the community reviewing, critiquing, extending and reproducing the supporting evidence. However, without transparency, claims only achieve credibility based on trust in the confidence or authority of the originator. Transparency is superior to trust." The 2019 NAS workshop on Reproducibility and Replicability in Science also concluded, "the scientific enterprise depends on the ability of the scientific community to scrutinize scientific claims and to gain confidence over time in results and inferences that have stood up to repeated testing." The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Comment: Commenter (12472) provides that non-public data or tools are often used in EPA modeling, replication and verification of data and models, therefore, the commenter suggests that the use of these data and tools should be addressed in the SNPRM. For example, the commenter notes that the Integrated Planning Model (IPM) is used to generate future year Electricity Generating Units (EGU) emissions. The commenter states that the IPM is a proprietary model which is used extensively by EPA and cannot be verified or replicated by states. The commenter requests that the EPA provide additional details on the suitability of using EPA's modeling results in specific regulatory settings based on these variability, uncertainty, and verification issues.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. Because this rule only applies to dose-response data, the Agency is not finalizing the definition of "model."
4.2.7 Impacts on Older/Existing Models
Comment: The supplemental proposal maintains its original position in section 30.5 that the rule would apply to studies, data, and models regardless of when they were published, but requests comments on whether this should apply only to data and models that are generated after the effective date of the rule making.
Several commenters (11386, 11407, 11490, 11491, 12474, 12546, 12702, 12727) express opposition to applying the rule to existing/older models. Commenter (11576) provides that the EPA's own SAB has warned that "[f]or studies published many years ago, it may not be feasible to deliver public access to data and analytic methods[,]" and "[t]here are also sensitive situations where public access may infringe on legitimate confidentiality and privacy interests, and where exceptions from complete public access may be appropriate."
Commenter (11894) states that the Agency is again asking for commenters to weigh options for addressing older studies without providing any analysis of the relative costs, benefits and disruption not only to the Agency and the processes it manages, but for those scientists who produced the studies in question.
Commenter (12710) states that, in 2018, they stated their concern that most studies prior to the digital age would not meet the proposed requirements for data transparency; this would lead EPA to dismiss most of the science conducted in the 20th century, including seminal studies on the health and environmental effects of pesticides. The commenter provides that the preamble of the SNPRM clearly confirmed this concern by stating "[p]roposed 40 CFR §30.5 would maintain the temporal approach to data and models taken in the regulatory text of 40 CFR §30.5 of the 2018 proposed rulemaking, and thus would apply to data and models evaluated at the time a significant regulatory action or influential scientific information is developed, regardless of when the data and models were generated." The commenter asserts that if EPA were to go forward with the proposal, we strongly recommend that the new transparency guidelines apply only to studies conducted after the proposal goes into effect.
Commenter (11490) opposes applying the rules to older models and studies. The commenter states that high quality studies published to date must not be excluded on the grounds that authors did not follow data sharing practices which were not required nor expected at the time of publication.
Commenter (11386) states that if this rule were to be put into practice, it is clear that this should only apply to data and models that are generated after the effective date. According to the commenter, even if the transparency of data and ability to be reanalyzed did indicate quality of science, then this information was available to EPA when considering past rules. The commenter adds that nothing about the science used to make previous rules will be changed as a result of this new rule. Thus, the commenter asserts that to suddenly treat previous studies in a way different from how they were previously treated by the EPA would be arbitrary and capricious, as nothing about the studies themselves had changed. The commenter notes that if new information came to light, such as new studies or a reanalysis of previous studies, then it would be quite reasonable for this to potentially change how older studies are considered. However, the commenter concludes that to apply different consideration to previous studies due to factors that have nothing to do with the science itself would be completely arbitrary.
Commenter (12727) states that there are compelling reasons why it may be impossible or infeasible to provide access to data and models underlying older studies. The revised and alternative approaches therefore fail to address one of the most substantial shortcomings of the original proposal -- one that EPA has explicitly acknowledged in its supplemental proposal.
Commenter (11921) asserts that older studies that did not and are unable to comply after a prolonged interval still have substantial scientific merit and value for informing regulations. Some such studies, e.g. the NIOSH/NCI Diesel Exposed Miners Study (DEMS) have made their data available through federal Research Data Centers. However, the data for other critical studies, e.g. the North American Rubber Workers Study, which has played and continues to play a key role in assessment of risk from 1,3-butadiene and styrene, was designed and implemented at a much earlier time and would not be readily available. Although the proposed supplemental proposal does suggest that the age of a study and the difficulty of gaining access to the underlying data could be a basis for the Administrator finding it impractical to make such data and methods available, and therefore exempting the study from this rule, such after-the-fact and potentially arbitrary determination would obstruct the continued use of a larger body of evidence  -  especially occupational studies that are critical to consider the effects of the toxicity of a number of specific chemicals.
Conversely, several commenters (11499, 11922, 12454) support applying the rule to older/existing models generated/developed before the effective date of the rule. Commenter (11499) supports including all existing models and studies in the scope of the proposed rule and giving the Administration discretion over the case-by-case exemptions.
Commenter (12454) states that given the rule's goal of enhancing transparency in science, the support existing data and models being covered if they underlie "pivotal regulatory science" in a rulemaking or "pivotal science" in the development of influential scientific information.
Commenter (11922) states that the rule should apply to all models and studies when evaluated for pivotal or influential utilization in decision-making as opposed to only those developed after this rulemaking.
Commenter (11491) asserts that a case-by-case exemption granted by the Administrator for the consideration of older data and models to be unacceptable. If the proposed rule is finalized, the commenter states that it should prevent all retroactive application by placing older data and models outside the scope of the rule. The commenter notes that case-by-case exemptions are not an adequate substitute for removing this older category of science from the rule's scope; not only does the use of exemptions for this purpose increase uncertainty and internal process, but it introduces the possibility of bias by this or a future EPA.
Response: The EPA considered these comments in the development of the final rule. The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA also maintains that the Administrator is the appropriate decision maker in the context of granting exemptions under this rule. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will also provide clear documentation.
4.2.8 Linear vs Nonlinear Modeling
Comment: Commenter (11479) objects to EPA's recommendation to use non-linear modeling over linear modeling. The commenter states that force-fitting non-linear models, without scientific justification, could bias the data to such a degree that low doses of a pollutant may appear less harmful  -  or even beneficial  -  to the health of person. The commenter adds that if scientifically unjustified weight is given to non-linear models to assess relationships between pollutants and health in EPA policy decisions, these decisions are likely to disproportionately affect the sensitive populations that EPA is charged with protecting.
Commenter (11499) states that the EPA should not be so prescriptive in the use of linear vs non-linear modeling. The commenter recommends that EPA tweak the language in proposed 40 C.F.R. Section 30.6 to say: "EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis, including assumptions of a linear, no-threshold dose response and other default assumptions as appropriate." The commenter notes that the public needs to understand clearly why EPA selects particular models to evaluate dose-response effects of exposure to different substances and what uncertainty factors are included for the model chosen in regulatory decision-making for those substances. The commenter adds that public access to information on model selection and uncertainty factors is as important as transparency of underlying data. According to the commenter, if the final rule fails to include this discussion, the Agency has lost an important opportunity to guide regulators and the public on evaluating these underlying default models, particularly when they involve metals risk assessments.
Commenter (12454) supports the use of a non-linear model relationship because many metals are essential nutrients that result in non-linear dose-response toxicity relationships, but EPA often fails to consider non-linear dose-response approaches in assessing metals and metal compounds. The commenter posits that the Agency can include a specific reference to "assumptions of a linear, no-threshold dose response" without implying that the Agency should not evaluate other default assumptions by tweaking the language in proposed 40 C.F.R. section 30.6 as recommended by the North American Metal's Council (NAMC) in their comments (references commenter 11499).
Response: In the 2020 SNPRM, in response to public comments, the EPA proposed a variation of the regulatory text which did not include the phrase "including assumptions of a linear, no threshold dose-response," because this could imply that the regulation is specific to those particular assumptions. After considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Given other changes to the regulatory text, the text quoted or referenced by the commenters is now in 40 CFR 30.5(e) and reads: "The EPA shall also describe critical assumptions and methods used in its dose-response assessment and shall characterize the variability and uncertainty of the assessment. The EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis. The EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information."
4.2.9 Impacts on Specific Models
Comment: Commenters (11894, 12727) question the impacts of the supplemental proposal on Greenhouse Gases, Regulated Emissions, and Energy Use in Transportation (GREET)[9], Motor Vehicle Emission Simulator (MOVES)[10], Benefits Mapping and Analysis Program (BENMAP)[11] CE, Integrated Planning Model (IPM)[12], Model for the Assessment of Greenhouse Gas Induced Climate Change (MAGICC)[13] and Global Change Assessment Model (GCAM)[14] models, as well as various Community Multiscale Air Quality Modelling System (CMAQ)[15] models. The commenters suggest that it is incumbent on EPA to evaluate potential impacts of the proposal on all of its standard modeling tools and then provide the public with notice and opportunity for comment on these findings.
Commenter (12720) provides comments on the impacts of the proposed rule on the PBPK model. The commenter states that the EPA convened a  FIFRA Scientific Advisory Panel in October 2017 to assess its use of a Physiologically Based Pharmacokinetic Model (PBPK) used in its risk assessment for the pyrethroid pesticides. The commenter notes that the PBPK model was sponsored and submitted to EPA by the Council for the Advancement of Pyrethroid Human Risk Assessment, L.L.C. (CAPHRA). CAPHRA identifies its participating parties as chemical and agrochemical manufacturers. CAPHRA describes its intentions as follows: "The general area of CAPHRA's planned activity is to generate and submit to the [U.S. EPA] studies necessary to address EPA's concerns for the potential for age-dependent sensitivity to Pyrethroids."
Despite the central role of the pyrethroid PBPK model in EPA's regulatory approval for pyrethroid pesticides, the commenter states that it appears that scientific peer reviewers on the FIFRA Scientific Advisory Panel (SAP) were unable to obtain the raw data necessary to provide a robust peer review of the model. According to the commenter, SAP Panelist Dr. Dale Hattis requested these data from EPA on September 6 and September 12 without receiving them, including "key data" for "evaluating the uncertainty in the modeling" and "data needed for assessment of the calibration of the PBPK models."
Commenter (12720) provides comments on the impacts on the BenMAP, CO-Benefits Risk Assessment (COBRA) Health Impacts Screening and Mapping Tool,16 and other PM25 models. The commenter notes that a key component of those models is the application of exposure-response epidemiology to estimate the health implications of specific energy, climate, and air quality scenarios. The commenter states that the proposed rule does not explain how significant and burdensome data transparency requirements imposed on data and models used by the Agency would affect the existing models that EPA currently hosts and distributes to the international research community.
The commenter (12720) adds that it is not clear that the dose-response studies incorporated into these models would remain and, if not, the detrimental impact that change would have on efforts to describe health implications of future proposed rules. Importantly, the commenter notes that the EPA itself uses the BenMAP model to conduct health impact assessments as part of proposed rules; and the supplemental proposal does not include any discussion of the implications of data transparency requirements on the future implementation and use of EPA-endorsed software packages.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models, including the models noted by the commenters.
Also as described in the preamble to the final rule, based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). The EPA considered commenters' assertions that the scope of the 2018 proposed rule would be so broad as to make implementation infeasible. The 2018 proposed definition of "dose-response data and models" would apply to dose-response data [and models] "used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact." This relationship of the dose-response data to the magnitude of a predicted health or environmental impact would require the consideration of an array of studies beyond those that characterize dose-response relationships, including, for example, studies that inform the dose-response modeling (e.g., benchmark response selection); studies that identify data for toxicokinetic adjustments that inform calculation of a human-equivalent point of departure (POD); and studies that inform the selection of uncertainty factors. The number of studies that are used to establish the relationship between dose-response data and models and the magnitude of a predicted health or environmental impact can potentially be very large. This may make implementing the rule as proposed infeasible for at least some significant regulatory actions and influential scientific information. While transparency in EPA decision making is the purpose of this action, transparency cannot be effectively accomplished if the means to that transparency are not feasible. Rather than having this final rule apply to all the studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact, the EPA is balancing transparency and feasibility by focusing on those studies that describe the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Specifically, the scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. Therefore, the rule does not apply to PBPK models.
4.3 SNPRM Data, Variability/Uncertainty, and Assumptions/Defaults
4.3.1. Clarity of Proposed Definitions
Comment: Commenter (11391) quotes the SAB's review of the proposal that in the new requirement that all "data be made available is vague and, as a result, can be interpreted in different ways" and carries that sentiment to the supplemental proposal definition of data.
Response: This final rule does not require that all data be made available. Instead, the EPA may give greater consideration to pivotal science where the underlying dose-response data is available for independent validation.
Comment: Commenter (11894) asks whether the definition of "model" applies only to "computational models". The commenter notes that previous guidance documents use the same definition of "model" to focus on the subset of all models termed "computational models" by the National Research Council (NRC). The commenter provides that these are models that use measurable variables, numerical inputs, and mathematical relationships to produce quantitative outputs. The commenter requests clarification as to whether the Agency envisions the same scope of applicability in the promulgation of the proposed rule.
Comment: Commenter (11919) also encourages EPA to better define what data must be publicly available. The commenter asserts that without a clear definition, the requirements of the rule will be open to a variety of interpretations leading to confusion and potential misuse.
Commenter (12435) indicates the supplemental notice of proposed rulemaking (SNPRM or supplemental proposal) definition of "model" is too broad to be useful. Commenter (12727) states that the definition, among other definitions, fails to adequately comprehensively describe the scope of its proposal. As a result the commenter contends that the EPA has denied the public the information it needs to comment in an informed way.
Commenter (12720) The commenter states the definition of "model" is overly general and vague does not address how EPA engages with models and modeling output in conducting its scientific activities. The commenter adds that the definition does not distinguish between physical models, conceptual models, and software modeling packages, nor does it explicitly account for models applicable within the human health context, even though that type of tool is commonly deployed by EPA in its scientific work. According to the commenter, in failing to define models and distinguish between modeling inputs and outputs, EPA's definition is flawed and insufficient.
Conversely, commenters (9587, 11922, 11390, 11401, 12406, 12463) assert that the definitions of data and models provided in the supplemental proposal are clear. Commenter (11390) notes that it is hard to imagine a scientific study, dataset, or model that would not be subject to the supplemental proposal.
Response: The EPA has removed requirements pertaining to model because this regulation applies only to dose-response data, and therefore has removed the definition for model from 40 CFR 30.2.
In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by some commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
<IP_ADDRESS> Stage of Data being Requested
Comment: Commenters (10740, 11495, 12435, 12471, 12546, 12712, 12720) note that there is a lack of clarity in whether the data being requested is the raw data collected throughout the experiment or only the finalized data set used to generate the study results. Specific examples of comments include:
Commenter (10740) states that, while the rule proposes an exclusionary test that eliminates individual studies based solely on whether the data is transparent, there is, however, no clear mandate that the raw data underlying these models be publicly available in a manner sufficient for independent evaluation.
Commenter (11495) discusses the level of data necessary for validation or replication. They highlight that epidemiological studies utilize several individual, but related, data sets to formulate conclusions. The commenter notes that only after the evaluation of these data sets by experts of different fields is a "final data set" established from which the study findings are finalized. The commenter notes that the supplemental proposal toggles between the need for all data from a study and only requiring the final data set that includes the interactions of all underlying data, and also indicates that the EPA's discussion of data is alternating and confusing.
Commenter (12435) asserts that the definition of data is too broad to clarify the application of the rule and leaves room for the coverage of potentially any information. As published scientific results are often the final steps in a process involving several processing and analysis steps, the commenter contends that the definition provided does not clarify what step of data processing would be subject to this rule.
Commenter (12471) recommends that an additional definition for "Validated Data" be provided to mean "data with a proper level of quality assurance and has all obvious errors removed and that is capable of being analyzed by either the original researcher or an independent party" and the definition of "Data" in the supplemental proposal be edited to read "the set of observed and recorded material in a format that is commonly accepted in the scientific community as necessary to validate research findings.
Commenter (12546) indicates that "Cleaned-up" data is not the same as a data set in which "obvious errors" have been removed. This cleaned data set may also include harmonizing or transforming data to ensure consistency. The commenter notes that when comparing the definition of data used in the supplemental proposal and the 2016 EPA Plan to increase access to results of EPA-funded research (2016 EPA Plan), the 2016 EPA Plan's definition is several steps after the raw data collection, indicating the EPA is inconsistent in its use of "raw" and "clean" data.
Commenter (12712) states the EPA must be clear that the actual final dataset used in statistical analysis is what needs to be made available, referencing an unacceptable form of data presentation in "Assessing PM2.5 Exposures with High Spatiotemporal Resolution across the Continental United States."
Commenter (12720) asserts that the definition of "data" is inadequate and an additional definition for "raw" data must be published by EPA.
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required for this rule would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information. However, given concerns about potentially skewed analyses, the final definition of "data" maintains the stage of data in which obvious errors have been removed. The EPA acknowledges that this definition of data is different the 2016 EPA Plan to increase access to results of EPA-funded research, but emphasizes that the definition for this final rule has been chosen in order to best meet the goal of reanalysis.
<IP_ADDRESS>.1 Accessibility of Raw Data
Comment: Commenters (11408, 11576, 10742) express concern that this rule may exclude older studies where the raw data and/or models may no longer exist and/or are difficult to access due to outdated methods of storage.
Commenter (11576) states that the EPA's own SAB has warned that "[f]or studies published many years ago, it may not be feasible to deliver public access to data and analytic methods[,]" and "[t]here are also sensitive situations where public access may infringe on legitimate confidentiality and privacy interests, and where exceptions from complete public access may be appropriate." The commenter states that the rule could have the effect of excluding studies where it is not possible to make the underlying data publicly accessible would unnecessarily and irrationally exclude reliable science for no legitimate reason. The commenter states the requirement for publicly available data underlying all model parameters would be particularly problematic and potentially impossible for models developed iteratively over decades, in which model parameters and structure evolve with growing science without the need for publicly available documentation of all the underlying parameters' source data. The commenter states that, while the size and storage practices for many large datasets and model outputs may not be suited for open sharing, this should not exclude them from consideration.
Commenter (10742) provides that the rulemaking is not clear if this rulemaking will allow EPA to cite and use studies where the data is no longer available due to antiquated technology or shifting norms in science (i.e. studies published before posting data and code publicly was widely encouraged and expected).