Commenter (11576) notes that raw data is less likely to be available over time compared to more processed data.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
<IP_ADDRESS>.2 Definition Fails to Acknowledge Common Errors
Comment: Commenter (12430) states that unscrubbed "raw" data can inappropriately skew results and cause significant issues in modeling and analysis for third parties who were not familiar with the data collection process due to various errors in the data set. The commenter notes that these errors can include outliers or incomplete data, which are not covered by the SNPRM's definition. The commenter recommends scrubbed data that removes these errors remain valid under the proposed definition.
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required for this rule would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information. However, given concerns about potentially skewed analyses, the final definition of "data" maintains the stage of data in which obvious errors have been removed.
<IP_ADDRESS>.3 Definition Oversimplifies Data
Comment: Commenter (12720) references the SAB's suggestion that "it might be reasonable to consider the initial compilation of data (original data) in a spreadsheet as "raw" data," and points out that the Agency uses data from researchers that are not working from simple spreadsheets.
Response: The EPA is not finalizing its definition of "data" as specifically being the "initial compilation of data in a spreadsheet."
<IP_ADDRESS> Definition Is Not Corroborated With Scientific Community
Comment: Commenters (12720, 12727) provide that the definition of data used within the supplemental proposal is not in line with the definition generally used among the scientific community.
Commenter (12720) first asserts that the general conception of data does not include analysis by "an independent party" and that EPA does not provide rationale for the phrase's inclusion in the definition. Additionally, the commenter considers the definition overly broad and that it is the scientific method that allows researchers to arrive at the study result. Furthermore, the commenter asserts that "obvious errors" are not "data," even though some errors may be part of raw data sets not included in the analytical data sets used for the investigation and validation. Lastly, the commenter notes that the "original researcher" in question is not defined nor is there any consideration for typical scientific practice where teams of researchers and others are involved in the collection, analysis, and interpretation of data.
Commenter (12727) contends that EPA is attributing a flawed notion of validation to the scientific community through the definition and has failed to provide any analysis to suggest what the scientific community believes to be involved in study validation.
Response: The EPA disagrees that the definition of data provided conflicts with that generally accepted by the scientific community. The EPA has included "capable of being analyzed by either the original researcher or an independent party" in its definition of data because this rule is a reflection of the value the EPA places in the ability of subject matter experts being able to independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
<IP_ADDRESS> Definition of Data Excludes Information
Comment: Commenter (11361) raises a concern that metadata is explicitly omitted from the requirement that clean data be made public. According to the commenter, metadata is used to orient users to the data, including its design, purpose, collection protocols, measurement precision, data architecture, etc. The commenter contends that, without this information, users may misuse or misinterpret the data, incorrectly calling data and analyses into question.
Response: The EPA notes that in 40 CFR 30.5(f), the EPA clarifies that "dose-response data is considered `publicly available in a manner sufficient for independent validation' when it includes the information necessary for the public to understand, assess, and reanalyze findings," and provides a list of information that may compose the relevant metadata.
Comment: Commenter (11892) asserts that due to tiering exemptions, the data that would be available for analysis by a third party would be limited. The commenter suggests that "supporting QA/QC [Quality Assurance/Quality Control] information" be included in addition to the "set of recorded factual material" to address this limitation.
Response: The EPA disagrees with the commenter, as data is factual material beyond simply information supporting QA/QC, and has therefore not made the suggested change in the final rule.
<IP_ADDRESS> Expansion from Dose-Response Data and Models to All Data and Models
<IP_ADDRESS>.1 Opposition/Support for Expansion
Comment: Commenters (9493, 11103, 11362, 11390, 11392, 11393, 11396, 11399, 11400, 11404, 11479, 11484, 11492, 11576, 11895, 11920, 12412, 12430, 12466, 12474, 12540, 12546, 12688, 12715, 12719, 12727, 13549, 14397) oppose expanding the applicability from "dose-response" data and "dose-response" models to all types of data and models.. Specific impacts include:
Commenter (9493) contends that this was a significant expansion. As a result of this, members of the public will be required to address the implications of limiting the Agency's reliance on "environmental fate studies, bioaccumulation data, water-solubility studies, environmental fate models, engineering models, data on environmental releases, exposure estimates, quantitative structure activity relationship data, and environmental studies".
Commenter (11103) notes that the data included in the new definition would include those that inform decisions such as the 2011 Mercury and Air Toxics Standards (MAT), which saves up to 11,000 premature deaths in the United Sates according to the EPA's own projections. The SNPRM, according to the commenter, would undermine EPA's fundamental mission to develop and enforce regulations that protect human health and the environment.
Commenter (11362) states that internal scientific assessments as well as policies under the CAA, Safe Drinking Water Act (SDWA), and TSCA require information that would be covered by the new definition, thus requiring them to disregard crucial and accurate data based on accessibility.
Commenter (11390) indicates that the proposed expansion is virtually all inclusive - given the parameters, it is hard to imagine a scientific study, dataset, or model that would not be subject to the proposed rule.
Commenters (11392, 11484, 11895, 11920, 12466, 11492) contend that the proposed expansion of the rule to all models would hinder EPA's use of the best available science in regulatory decisions.
Commenter (11392) provides that the expansion would impact the application of several data types (bioaccumulation data, data on environmental releases, and exposure estimates) used in EPA's regulation of toxic chemicals. Also, the commenter states there is no rationale for how this expansion would improve the Agency's ability to consider the best available science nor any evidence provided in support of the expanded scope.
Commenters (11393, 14397) note that the expansion would create an even higher burden on scientists or the Agency to maintain the increased amount of information that this definition provides for.
Commenter (11404) contends that the calculation of economic benefits from environmental regulation would not be possible without the data now covered by the expanded definition. The commenter provides the EPA's calculation that a Particulate Matter of 2.5 um or smaller (PM2.5) standard of 11 micrograms per cubic meter would contribute to health gains and lives saved worth as much as $20 billion a year and the Bay Area's estimated cost savings of $456,400 per ton of PM2.5 reduced in the area.[,] According to the commenter, epidemiological data is critical to these analyses and should not be overlooked. Furthermore, the commenter states that covering all data would be harmful from the national level down to the local level and undermine the ability of environmental health agencies to carry out their mission.
Commenters (11479, 12430) indicate that the expanded rule places more burden on modelers, and does not necessarily support best available science. Commenter (12430) states that the expanded scope would limit many studies on public health, and provides the example of fecal indicator bacteria concentrations at beaches, EPA lifetime health advisories for Perfluorooctanoic Acid, The Harvard Six Cities study and the American Cancer Society studies that informed the NAAQS.
Commenter (11576) asserts that the expansion introduces new types of privacy and sensitivity concerns for a broader set of public-health and epidemiological studies and a new set of concerns for many types of environmental data.
Commenter (12540) notes that the move from only dose-response data to all environmental and health data would eliminate studies from consideration by EPA, including those that underly EPA's rules under the SDWA, NAAQS, CAA, and Federal, Insecticide, Fungicide, and Rodenticide Act (FIFRA).
Commenter (12727) states that the expanded SNPRM would conflict with EPA's ability to regulate emissions of hazardous pollutants under the CAA. The commenter references section 112 of the CAA, providing that EPA's definition of "data and models" to include "data on environmental releases" would eliminate data underlying emission standards under the section.
Commenter (12715) contends that while risk assessment evaluations performed under EPA's IRIS are not regulatory in nature, the toxicity factors provided by IRIS are used as the basis for many regulatory standards developed by EPA and numerous states, as well as federal and state guidance values and cleanup decisions. The commenter notes that some states' regulations require that IRIS toxicity factors be used as the basis for human health standards for environmental contaminants. According to the commenter, the SNPRM would significantly limit IRIS's consideration of and reliance on many key studies that are crucial for the assessment of human health effects and development of toxicity factors for environmental contaminants.
Commenter (12719) states that the expansion of the definition would have likely lead to the exclusion of data that contributed to the 1990 amendments to the CAA, which EPA estimated as preventing 230,000 early deaths in 2020.
Conversely, commenters (9587, 11397, 11401, 11499, 11500, 11922, 12406, 12467, 12612) promote the expansion of data coverage. Support for this expansion is due to:
Commenters (11397, 12467, 12612) state that limiting the scope of the proposal to the narrow definition of dose-response data and dose-response models would limit the efficacy of the proposal. Additionally, the commenter (11397) provides that the original limitation to dose-response data would have failed to carry out the directives in Executive Order 13777.
Commenter (11500) specifically outlines the necessity to apply the transparency rules to NAAQS proceedings, which would require the expansion as there are several types of data in addition to dose-response materials that are key considerations.
Commenter (11922) asserts that the public and regulated entities will have greater confidence in EPA's implementation of its policies and regulatory programs with the expansion of the supplemental proposal to cover all data and models.
Response: The EPA has considered these comments and determined to limit the scope of this final rule to dose-response data. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions.
Comment: Commenter (12472) expresses that the expansion could potentially impact on air quality models, and requests that EPA make explicit whether these models are included under the rule. The commenter specifically requests that the EPA clarify whether air quality and associated models such as Comprehensive Air Quality Model with Extensions (CAMx) are included as models subject to the rule.
Response: The EPA has removed requirements pertaining to model because this regulation applies only to dose-response data, and therefore has removed the definition for model from 40 CFR 30.2.
Comment: Commenter (12699) quotes § 30.3 "The provisions of this part apply to data and models, underlying pivotal science supporting influential scientific information and/or underlying pivotal regulatory science used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the science." The commenter notes that the comma after "data and models" appears to be communicating that the provisions apply to data and models, underlying pivotal science, and/or underlying pivotal regulatory science. Unfortunately, the commenter contends that this language could create confusion by giving the impression that the provisions only apply to data and models. According to the commenter, since this is unlikely what the EPA intends, they recommend that EPA make § 30.3 clearer.
Commenter (12699) adds that, if in fact the EPA does only intend to cover data and models, then this would make little sense and beg the question why it would not cover pivotal science and pivotal regulatory science. The commenter notes that both are critical in practice and for purposes of the rule. The EPA would be creating a transparency rule to help the public evaluate the science, and then arbitrarily only allow the public to evaluate a subset of what informs influential scientific information and significant regulatory actions.
Response: The EPA acknowledges the lack of clarity and has altered the portion of 40 CFR 30.3 relevant to this comment to read, "to dose-response data underlying pivotal science."
<IP_ADDRESS>.2 Necessity for Expansion
Comment: Commenter (10972) contends that the rulemaking effort proceeds on the premise that you can have the same set of requirements for models and data across different fields. The commenter states that no factual basis is presented for the choice to shoehorn all models and all data in all sectors under the same standard; it is an arbitrary approach. The commenter provides that for example, the preference for non-parametric models in section 30.6 may have been understandable in the context of dose-response models, but it seems irrational to simply extend that attitude to all other models and data in all sciences with no meaningful consideration of the propriety of this approach to those other circumstances. According to the commenter, thoughtful assessments are required to create clear-cut requirements in each sector based on the terms of statute concerned, the goals of the statute concerned, sectoral privacy and other legal requirements and challenges, sectoral ethical issues, the nature of the data predominantly used in a specific sector, and other concerns. The commenter also notes that sectoral sensitivity is the route counselled by the statutory considerations stated above and the nature of the EPA as an Agency with expertise. The commenter states that EPA highlights the former when it notes that there is some conflict between this proposed regulation and some of the laws which it is meant to effectuate and that the generalist approach in this rulemaking ignores relevant factors and considerations.
Response: The EPA has limited the scope of this final rule to dose-response data, thereby addressing the commenter's concern that the Agency was implementing a "one-size-fits-all" approach to models and data across different fields.
Comment: Commenter (11490) states that their previous testimony's question of "What is the problem this proposed rule seeks to fix? Where is the study for which the lack of access to raw data resulted in misinterpretation or in the promulgation of an inappropriate regulatory standard?" According to the commenter, these questions remain unanswered with respect to dose-response data in regulatory science, and no new information has been provided that rationalizes extending the rule to other data and scientific assessments at EPA.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11894) asserts that the proposal overwhelmingly focused on health effects data, and specifically epidemiology, yet the supplemental proposal greatly expands the range of studies while not providing discussion or investigation that the same "problems" asserted in the originally covered studies are also present among the much greater range covered in the supplemental proposal. The commenter notes that such far ranging changes require explanation and supporting data and information.
Response: The EPA has limited the scope of this final rule to dose-response data, thereby addressing the commenter's concern.
4.3.2 Application of the Rule in Respect to Data
<IP_ADDRESS> Variable Application to Types of Data
<IP_ADDRESS>.1 Application of Rule to Variety of Data Types
Comment: Commenter (12720) asserts that there is an enormous diversity of data used in EPA decisions and they reference the SAB's urging of EPA to consider how it could tailor approaches to varied data sets, including a trusted third party like the Health Effects Institute (HEI), Data Use Agreements, certification processes, a data archive accessed with confidentiality agreements and others. The commenter notes that the supplemental proposal does however propose a single approach for all types of data and no analysis of how the likely exclusion of specific data types due to this would affect its decisions.
Commenter (9587) states that the widened scope of the supplemental proposal will require detailed follow-up rulemaking, to apply the rule appropriately to every subcategory of data and models used by EPA. The commenter suggests the rulemaking include provisions for committees deputed to provide detailed guidance for how it should be implemented in every relevant subcategory.
Response: The EPA has limited the scope of this final rule to dose-response data, thereby addressing the commenter's concern that the Agency was implementing a "one-size-fits-all" approach to models and data across different fields.
Comment: Commenter (11892) notes that the example assumptions of "parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity," do not typically apply to the data used by wetland managers. Instead, the commenter provides, wetland data is mostly obtained from the field (e.g. plant species occurrence and distribution, and surface and groundwater levels and movement), although laboratory analyses may be incorporated if results are found useful. The commenter also states that wetland managers rely on expertise in the literature and their own expertise in order to determine whether or not their data or other reports yield usable information with implications for wetland management. Furthermore, the commenter provides several reasons the additional transparency requirements do not neatly fit wetland data:
      :: Wetland data is frequently collected according to peer-reviewed procedures approved by EPA through State Wetland Program Development Grants and 106 programs.
      :: While not unique to the topic of wetlands, states and tribes typically have limited independent resources for wetland data collection and rely on peer-reviewed publications for science applicable to their management concern.
      :: Wetland data, when published in peer-reviewed journals, doctoral dissertations or masters' theses represent very specific situations and often limited geographic areas making independent validation difficult to conduct or inappropriate for informing managers in other areas or for similar, but related water resources. Some data are collected on private land and the landowner(s) may not allow others to access these sites.
      :: More current data is often unavailable. However, many older, but still critical data and analyses remain useful in making technical and scientific assessments that can guide management decisions.
      :: Formal literature reviews which achieve the rigor of research best practices and peer review should be included in available science used by EPA when used in concert with the data from the original studies. The in-depth comparative analysis of a formal literature review can add insights from the body of science about a specific topic.
       :: Wetland management is highly influenced by stream management. Stream management often relies on use of engineering models, which typically are in the public domain for use by applicants for regulated activities.
Response: The EPA has limited the scope of this final rule to dose-response data. Wetland data will only fall under this rule insofar as they are integral to characterizing ecological dose-response relationships.
<IP_ADDRESS> Logistics of Handling Data
<IP_ADDRESS>.1 Data Sharing of Large Datasets is Logistically Challenging
Comment: Commenters (9362, 11386, 11392, 11495, 11576, 12617, 12720) note that the rule will be burdensome due to the extremely large nature of the raw data being worked with. Specifics:
Commenter (9362) agrees that the rule would require scientists provide their data upon request so that their analyses can be repeated, but does not believe all scientific papers could be applicable due to the volume of data being analyzed.
Commenter (12617) states that the broad application of public-data requirements would eliminate many instructive and valid studies from consideration because of the burden of finding ways to publicly share all code and data. The commenter states that many studies crucial for EPA to consider when developing rules related to these chemicals, may not be considered due to the proposed data access requirements. The commenter states that, since any such studies are based upon very large data sets analyzed with chemometrics, data reduction, and commercial or customized computer programs, it would be no small task to host all data and code in a public manner.
Commenter (11386) uses the example of Tropospheric Monitoring Instrument (TROPOMI), a satellite instrument that provides daily, global observations of column methane in 5.5 x 7km[2] pixels at the highest available resolution. As such, the commenter states the data are not easily stored, let alone manipulated, without significant computing resources that are unavailable on a typical personal computer.
Commenter (11392) asserts that the data required by the supplemental proposal would encompass almost all of the raw data generated in a study. The commenter notes that EPA would be obligated to ensure many different types of data, including meteorological data, pollution values, and data that serves as the basis for epidemiological studies and other research, were publicly availability in an organized format. According to the commenter, this could mean hundreds of files for any given study and require its own unique code simply to maintain the necessary organization. Currently, the commenter states that the EPA does not possess any data sharing system capable of performing this task, nor does the supplemental proposal address how the Agency would develop the additional capacity necessary to comply with the rule. The commenter provides that the rule does not explain the logistical methods that EPA would use to process and manage this enormous amount of data, nor how the Agency would make it accessible to the public. Additionally, the commenter notes that the rule does not clarify whether the Agency intends to develop an internal data storage system, or whether the Agency would utilize external parties.
Commenter (11576) states that studies involving continuous monitoring of air or water pollutants, greenhouse-gas emissions, or any other type of time-series data require sizable infrastructure and data-handling considerations to accommodate the vast size of these datasets. The commenter contends that without advance notice and creation of the infrastructure required to securely support the storage and transfer of large volumes of data, the EPA cannot require researchers to categorically accommodate publicly available data and models for all types of pivotal regulatory science and pivotal science.
Commenter (12617) provides examples of per- and polyfluoroalkylated substances (PFAS) research data sets that include extensive data sets analyzed with chemometrics, data reduction, and commercial or customized computer programs. The commenter asserts that hosting this data and code would be no small task.
Commenter (12720) states that the supplemental proposal does not engage with the idea of "big data" and large datasets that may not lend themselves to data sharing.
Response: The EPA agrees with the commenters and has limited the scope of this final rule to dose-response data underlying pivotal science. Rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). This limited scope increases the feasibility of this rule at this stage, and addresses the commenters' concerns.
<IP_ADDRESS>.2 Capability of EPA to Host Data
Comment: Commenters (11192, 11193, 11361) state that EPA is not prepared to host information in a manner sufficient for the proposal. Specific concerns include:
Commenters (11192, 11361) assert the EPA must identify and create a structure for publicly available data before requiring studies to make their data public or make decisions about what data can safely be made public through that architecture. The commenters state that, if the EPA seeks to make more studies' data public, then the Agency should, at the minimum: (1) create a public repository and user interface for such data, (2) provide funding explicitly for data de-identification in new federally funded research grants, and (3) admit de-identified data to the repository and into full consideration for influential scientific information.
Commenter (11193) contends that the Agency has provided no methods or structure by which they will inventory, store, process, and analyze the data, analyses, and models they would require to be made public. The commenter notes that currently, the Agency struggles to maintain its data systems consistent with current standards and to serve their own data efficiently to the public. According to the commenter, requiring the Agency to inventory, manage, distribute, analyze, etc. the quantity and density of data that would be involved in the proposed rule and supplemental proposal has not been thought out and no presentation of any such management system or standard operating procedure has been provided. The commenter states that no infrastructure exists for such a system, such that there is effectively no way to implement this program. Also, the commenter notes that the management of such a data system will involve a substantial cost for storage, management of flow, and security. The commenter asserts that no description of the Agency's ability to pay for this effort was presented.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenters (9372, 11900) suggest that, to adequately protect privacy, all data associated with human study participants should only be accessible through secure data enclaves. Commenter (9372) states that, even for data that is already anonymized, most data providers are unwilling to budge on the requirement that data are kept on secure servers.
Response: The EPA agrees and clarifies that in this final rule the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
Comment: Commenter (11176) provides reservations concerning the data sharing infrastructure for classes of data that could inform EPA actions:
 Data owned by the EPA or other federal entity: While there is a push to provide expanded access to data in the federal government, the commenter states that the effort is still in its infancy. The commenter provides that the National Center for Health Statistics (NCHS) Research Data Center (RDC) is one example for providing some level of tiered access, and the National Opinion Research Center (NORC) Data Enclave and the Inter-university Consortium for Political and Social Research (ICPSR) Virtual Data Enclave may provide additional flexibility of providing access. However, the commenter notes that EPA's involvement in such programs is minimal, limited to a single survey. Given this limited involvement, it is premature to enact this rule.
 Federally funded non-EPA data: For a snapshot of the status of data repositories for federally funded research, the commenter considers the NIH, the federal government's likely leader in federally funded data sharing, an effort started more than a decade ago. The commenter provides that, in its frequently asked questions (FAQ) page for data sharing, the NIH responds, "maybe," to question #20 on whether researchers should consider contributing their data to a data archive. The commenter contends that the reply equivocates, of course, because of the many factors involved but also indicates how far the NIH is from achieving full data sharing for the research it funds. For the next question of where to find guidance on preparing data for sharing and archiving, the commenter notes that the NIH FAQ does not provide its own guidance but, except for molecular biology information, refers readers of the guidance to an outside entity, the afore-mentioned ICPSR. According to the commenter, EPA is a relative newcomer to encouraging data sharing and, while it will benefit from the advances and experiences of other federal entities, is still likely to be years away from providing or encouraging the data access assumed in this rule. The commenter also notes that EPA does not seem to require a data management plan, unlike the National Science Foundation and NIH, which it planned to do by 2018 in the 2016 EPA Plan. The commenter states that such a requirement might be a first step towards the eventual goal of EPA funded research data being more widely accessible. And while data management plans and sharing strategies remain work in progress, the commenter contends that they are a mechanism to ensure that large swaths of data become accessible to as broad a public as possible, including through some of the same restricted-access mechanisms mentioned previously. According to the commenter, without ready access to this category of data, the commenter believes it is premature to enact the rule but encourages EPA's continued work to make such data available
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11361) states that there is currently no central place for academics to publish their data, let alone to publish data with tiered access. The commenter provides that when discussing tiered access, EPA offers the Centers for Disease Control and Prevention (CDC) RDC as an example repository and system. However, the commenter notes that EPA fails to discuss, however, the costs associated with building, maintaining, and providing access to that repository, nor the cost of reviewing and evaluating applications for access. According the commenter, EPA must identify and create a structure for publicly available data before requiring studies to make their data public or make decisions about what data can safely be made public through that architecture. The commenter suggests that the Agency should, at the minimum: (1) create a public repository and user interface for such data, (2) provide funding explicitly for data de-identification in new federally funded research grants, and (3) admit de-identified data to the repository and into full consideration for influential scientific information.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11479) states that even with the clarifications provided in the supplemental proposal, there is no information on where the raw data collected would reside, how the resources to manage the data would be obtained, or how EPA would deal with legally and ethically protecting confidential or sensitive data. The commenter notes that ensuring all this extra information is received, stored, and made publicly accessible would be no small undertaking. Additionally, the commenter provides that EPA's SNPRM references a system used by the NIH to collect health data, but the NIH system is not publicly accessible and requires significant resources to manage and share with researchers  -  a clear contradiction to the administration's apparent interest in transparency. The commenter contends that at a time when EPA's budget and capacity continues to decrease, it is unreasonable for the Agency to initiate such an undertaking.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11408) quotes a 2013 study that reviewed requests through the Information Quality Act between 2002 and 2012 that suggests the Agency should establish a database that "registers studies and obtains systematic sets of parameters and results" rather than making all raw data available, such as the NIH's clinical trials database.
Response: The EPA disagrees, and notes that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
Comment: Commenters (11894, 11900) express concern that information regarding the RDC pilot project has not been provided for public comment. Commenter (11900) requests that the details and results of that pilot study be made public.
Commenter (11894) states that EPA staff confirmed that the EPA was working with the NCHS at the CDC to conduct a pilot study using a secure data enclave to host EPA data sets in a restricted use environment to examine how PII data could be protected. The commenter states that they suggested the pilot would be completed in about six months (i.e. end of March) and that "the final rule would build upon the information obtained from the pilot." The commenter states that apparently, this work, which was to be a foundation for the final rule, was not completed in time to inform the SNPRM or the EPA chose to move forward without waiting to get its results. The commenter states that the preamble states that "EPA is currently conducting a pilot study using the RDC's secure data enclave" referring to the Research Data Center (RDC) of the NCHS/CDC. The commenter states that this key study that the EPA told SAB would be "used to inform the final rule" has not been made available for public comment. The commenter states that, apparently, in its place, the preamble notes that the RDC itself is "a model of tiered access for data involving PII." The commenter states that the EPA's inability to define tiered access is emblematic of the utter disregard of both the tradition of sound science/policy principles and the rulemaking process at EPA. The commenter states that this is a violation of adequate notice and opportunity for comment.
Response: The EPA disagrees with the commenter because this final rule does not depend on the RDC pilot study. This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
Comment: Commenter (11900) asks how the EPA intends to address issues of ownership and responsibility for data deposited in the restricted enclave. The commenter states that stewardship of data involves maintenance of the infrastructure for storing and accessing the data, accountability for the data quality and underlying methods that generated it, and moral responsibility to the study participants. The commenter states that researchers may be reluctant -- or unable -- to cede moral responsibility and accountability for the data to EPA, even if they can cede maintenance of the data. The commenter states that, in light of their ongoing responsibilities, the original study investigators should be granted some input over who is allowed access to their data and for what purposes. The commenter states that one way to do this would be to give the original study investigators an automatic seat on the Data Access Committee for any application seeking to use their data. The commenter states that, in this way study investigators will have the opportunity to review and comment on research proposals seeking to use their data.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Further, it does not direct parties outside the EPA on how to provide access to data deposited in a restricted enclave. This is a rule of internal agency procedure and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
Comment: Commenters (11184, 11407, 12412, 12474, 13549) state that, in August 2018, the National Tribal Air Association (NTAA) expressed concerns about EPA's belief that confidential or private information could be dealt with by the application of solutions in use across other parts of the federal government. The commenter states that the NTAA also expressed concerns about the cost of doing so and the reduction in studies this would cause the EPA to consider. The commenter states that eighteen months later the SNPRM did not address any of these concerns or analyze the costs or impacts. The commenter states that, before the EPA can finalize a rule that relies on certain approaches to provide, yet protect data, it must lay out and analyze those approaches for public review and comment.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenters (12546, 12720) express concern that, while the SNPRM describes development of standard data repositories as "ongoing," the EPA does not explain why the EPA should introduce burdensome new public data release requirements at a time when such repositories have not demonstrated the ability to handle such a massive influx of data and models in a way that actually improves public access to data in a meaningful way.
Commenter (12546) notes that in the SNPRM, the EPA indicates that it "...is currently conducting a pilot study using the RDC's secure data enclave to host EPA datasets in a restricted use environment. Development of standard data repositories is still ongoing." The commenter states that the EPA is proposing to change the scientific basis of their rulemaking based on an untested and unverified method for data security enclaves. The commenter states that the EPA needs to put out a rule that has a tested security method in place, not a `plan' for a system that has not been tested and evaluated. The commenter states that the EPA should not be basing regulations on pilots, but rather should focus on implementing the EPA's own 2016 Plan to increase access to results of EPA-funded scientific research.
Response: The EPA disagrees that the practices informing this final rule are untested. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act.[,]
<IP_ADDRESS> Addressing Data From Various Sources
<IP_ADDRESS>.1 Application to States and the Federal Government
Requirements of Federal Agency Research
Comment: Commenter (12402) recommends that studies and data developed under EPA testing programs, where EPA test guidelines or other validated and accepted test methods such as those developed by the Organisation for Economic Co-operation and Development (OECD), American Society for Testing and Materials (ASTM) International have been used, and EPA Good Laboratory Practices have been followed and certified should be candidates for exemption from the SNPRM. It can reasonably be assumed that such studies and data have already been subject to adequate validation review by the Agency and have been subject to appropriate public notice and comment in accordance with statutory requirements of the relevant EPA programs. For example, the commenter acknowledges an extensive chemical fate and environmental effects testing program under an enforceable testing Consent Order under the TSCA for which EPA was involved in all aspects of the data development. The commenter notes these same studies were reviewed again by the Office of Water as part of an extensive review of available data for the development of Ambient Aquatic Life Water Quality Criteria for Nonylphenol (NP) in 2005. According to the commenter, it is likely that these same studies will be reviewed again when EPA conducts its TSCA risk evaluation on this substance, which is listed on EPA's 2014 update to the TSCA Work Plan. Considering that these studies are now thirty years old, the commenter states that providing public access to raw data will likely be difficult and should be unnecessary considering that the studies were developed under EPA oversight and reviewed previously under EPA programs.
Response: The EPA believes that its efforts under this rule can be additive to existing validated and accepted test methods. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Further, the EPA directs the commenter to the factors in 40 CFR 30.5(d) that the EPA shall evaluate when determining the amount of consideration to grant pivotal science where the underlying dose-response data are not available for independent validation.
Comment: Commenter (12720) asserts that EPA has not explained or anticipated the impact of the supplemental proposal on other federal agencies that rely on similar types of scientific data and models to develop influential scientific information. The commenter provides the recent joint publication of the Safer Affordable Fuel-Efficient(SAFE) Vehicles Rule for Model Years 2021-2026 Passenger Cars and Light Trucks by EPA and the National Highway Traffic Safety Administration as an example of potentially impacted projects. Additionally, the commenter states that the supplemental proposal does not explain how requirements introduced at EPA would affect significant scientific work at other federal agencies and appears to undermine longstanding scientific review processes at other federal agencies, including the Department of the Interior.[,]
Response: This is a rule of agency procedure internal to the EPA. Where the EPA is collaborating on a significant regulatory action or influential scientific information with another federal agency, this rule would apply. However, the EPA notes that The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act.[.]
States
Comment: Commenter (12456) asserts that it may not be possible for certain information to be made public due to state requirements that prevent the release of specific information related to private lands under the Colorado Open Records Act. It is also noted that the data covered by this Act should not be at the discretion of the EPA Administrator, nor excluded from consideration due to the study taking place on private lands in Colorado.
Response: The EPA does not require external parties to make their dose-response data available. Where the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBICBICBICBI, and is sensitive to national security.
Comment: Commenter (10974) is concerned that the supplemental proposal could undermine state-level efforts at developing health-based benchmark values used in air toxics and permitting reviews.
Response: The EPA disagrees with the commenter that this final rule will undermine state-level air toxics activities or permitting reviews. The Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
<IP_ADDRESS>.2 Treatment of Studies From Subscription and Open Source Journals
Comment: Commenter (12455) raises a concern that depending on the nature of the journal the data is published in; the data will be treated differently.
Response: Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.[,] When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Only after a study has been designated as pivotal science, the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
<IP_ADDRESS> Reporting Funding Sources
Comment: Commenter (11892) recommends that the rule make it clear that the same stringent standards for transparency and quality of data to all information used in decision-making regardless of its source, whether provided by public agencies, the academic community, regulated entities or other sources. Moreover, the commenter states transparency would be increased by requiring information regarding financial support for the research. According to the commenter, funding by federal agencies and many foundations are typically identified in research reports, but corporate and other private funders may not be.
Response: The EPA notes that the final rule codifies in 40 CFR 30.3: "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The EPA anticipates that giving greater consideration to pivotal science where the underlying dose-response data is available for independent validation will offer increased transparency, but notes that this rule is an incremental part to increasing transparency in EPA significant regulatory actions and influential scientific information.

 Chapter 5: Influential Scientific Information; Expansion of Proposal; Determining and Weighting ISI (Supplemental Notice of Proposed Rulemaking (SNPRM))
In the supplemental notice of proposed rulemaking (SNPRM), the EPA proposes to expand the scope of the notice of proposed rulemaking (NPRM) to apply to "influential scientific information", defined as follows:
      Influential Scientific Information means scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions.
The SNPRM also provides revisions to the NPRM regulatory text at 40 Code of Federal Regulations (CFR) 30.5 from that in the Proposal. Under the proposed new approach at 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the EPA would only use pivotal regulatory science and/or pivotal science if the data and models are available in a manner sufficient for independent validation. The SNPRM provides that this includes studies with data and models that are publicly available as well as studies with restricted data and models (i.e., those that include confidential business information (CBI), proprietary data, or personally identifiable information (PII) that cannot be sufficiently de-identified to protect the data subjects) if there is tiered access to these data and models in a manner sufficient for independent validation.
As an alternative, the SNPRM preamble states that under proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing , other things being equal, the EPA would give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because they are publicly available or because they are available through tiered access when the data includes CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. The SNPRM preamble further states that EPA would identify those studies that are given greater consideration and provide a short description of why greater consideration was given.
Several comments were received regarding the SNPRM coverage and use of influential scientific information. These comments are organized under the following topics:
 Definition of Influential Scientific Information
 Expansion of Scope to Include Influential Scientific Information
 Determining What Constitutes Influential Scientific Information
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
5.1 Definition of Influential Scientific Information
 5.1.1 General Support/Dissent for Definition
Comment: Commenter (14397) states that the term "influential scientific information" remains vague and unworkable.
Commenter (11390) asserts that the new definitions provided in the supplemental proposal are so expansive and vague as to render the proposed rule applicable to any scientific study that EPA considers for nearly any meaningful purpose, outside of the supplemental proposal's exceptions for adjudications and enforcement activities.
Commenter (12546) states that this definition of "influential scientific information" essentially includes all decisions and documents that EPA generates which use science, in addition to the already expanded definition of "pivotal regulatory science." The commenter also notes that the definition does not include impact on human health/public health, which is EPA's mission.
Conversely, commenters (9587, 11401, 11499, 12406, 12471) support EPA's proposed definition for "influential scientific information". Specific comments include:
Commenter (12471) states that having such a definition seems appropriate as the term appears in the Memorandum and yet the Memorandum does not specify a definition, nor is one found in the 2002 Office of Management and Budget (OMB) Guideline. The commenter suggests a minor revision to the definition of influential scientific information as follows:
      Influential scientific information means pivotal science in an area of inquiry, when taken together, represents the synthesis of scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions," in order to link the definition to pivotal science.
Response: The definition of influential scientific information in 30.2 of the final rule remains, "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." The Agency will not be taking suggestions to modify the definition because this definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. In addition, the Agency does not believe this definition is expansive or vague. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. Given that the definition is both established and has been routinely applied by the EPA, the EPA disagrees that the term is inherently too narrow or too broad. Rather than modify the proposed 40 CFR 30.2 definition of influential scientific information, the EPA is modifying 40 CFR 30.3 to clarify the Agency's intent that the requirements in 40 CFR Part 30 will apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Final Information Quality Bulletin for Peer Review. Consistent with this approach, the EPA is finalizing the definition of "influential scientific information" as proposed in the 2020 SNPRM. The EPA's Peer Review Handbook provides greater guidance on how the Agency identifies influential scientific information.
5.1.2 What is Covered by the Definition of Influential Scientific Information
Comment: Commenter (11381) provides that EPA defines influential scientific information without defining the predicate noun. The commenter notes the supplemental proposal definition approximates a key Information Quality Guidelines (IQG) definition, but that the supplemental proposal definition is narrower  -  it excludes financial or statistical information  -  without provision of a rational basis for the departure from the IQG. As such, the commenter asserts this leaves the Agency's definition ambiguous at best. The commenter states that the only subsets of information that assuredly lie within the definition are data and models, and thus leave the public and Agency staff confused. Furthermore, the commenter questions if the supplemental proposal includes only data and models, or all information ("any communication or representation of knowledge such as facts or data, in any medium or form") that is influential ("information the Agency can reasonably determine that dissemination ... will have or does have a clear and substantial impact on important public policies or important private sector decisions"). The commenter notes that because elsewhere the supplemental proposal focuses on data and models, this is not an insignificant concern and that, unless the matter is clarified, it will be the cause of wholly unnecessary and pointless argument.
Commenter (11390) notes that the defined terms are so broad they ostensibly could apply even to science that never actually impacts any Agency policy or regulation: The commenter states that if EPA finds science reasonably "will have" an impact in the future, that is enough to bar the Agency from considering key data in the present, whether or not such an impact ultimately occurs. Moreover, the commenter states that a wide range of EPA activities might be found to impact "private sector decisions" outside the scope of the Agency's own regulatory authority, making the potential reach of this provision virtually limitless.
Commenter (11399) provides that the definition provided in the supplemental proposal does little to clarify the meaning of "influential scientific information," which is troubling in a rule concerned with transparency. Thus, the commenter does not support this expansion to the proposed regulatory language.
Commenter (12645) asserts that a quick look at the recent Health Effects Institute (HEI) Review Committee reports shows a median of over 20 references that are cited by the reviewers. According to the commenter, the reviewers do not look at the raw data in order to write their influential analysis, nor would all cited references have such data available. Therefore, the commenter questions if EPA plans to tell HEI not to allow its Review committee to consider any literature for which the committee members have not personally evaluated the completed data, or at least are certain that the completed data are available? Similarly, the commenter notes the EPA's own analyses, such as the Integrated Science Assessments (ISA) reviewed by the Clean Air Science Advisory Committee (CASAC), often simply cites previous EPA documents, or other reviews such as those performed by the National Academy of Science (NAS), as foundations on which they then overlay the more recent scientific studies. The commenter questions whether the EPA now plans to remove these foundations, or to subject the literally thousands of papers that went into the weight of evidence for previous EPA analyses to considerations as to which are pivotal or influential.
Commenter (12727) states that there is not an adequate notice of the types of influential scientific information to which the rule could apply. The commenter notes that although some influential scientific information could have been indirectly excluded from use by the Agency by the Proposal  -- e.g., if destined to support a significant regulatory decision -- the supplemental proposal imposes an independent and immediate restriction on the development of all influential scientific information.
Commenter (12715) provides that the supplemental proposal's statement that the requirement only apply to studies that "could drive [EPA's] subsequent decisions," 85 Fed. Reg. at 15,399, is inconsistent with information EPA provided to staff of the House Committee on Science, Space, and Technology that for scientific evaluations designated as influential scientific decisions "every study considered by the Agency in writing a rule, regulation, risk assessment, and more would be subject to these data transparency requirements."
Response: The definition of influential scientific information in 30.2 of the final rule remains, "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." This definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. Given that the definition is both established and has been routinely applied by the EPA, the EPA disagrees that the term is inherently too narrow or too broad. The EPA's Peer Review Handbook provides greater guidance on how the Agency identifies influential scientific information. All Agency influential scientific information, besides going through peer review, will also be subject to public notice and comment.

However, in the 2020 SNPRM, the EPA did not incorporate the OMB Bulletin for Peer Review Section IX exemptions to peer review, including routine statistical information and financial information. Rather than modify the proposed 40 CFR 30.2 definition of influential scientific information, the EPA is modifying 40 CFR 30.3 to clarify the Agency's intent. As a result, the requirements in 40 CFR Part 30 will apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Bulletin for Peer Review. Consistent with this approach, the EPA is finalizing the definition of "influential scientific information" as proposed in the 2020 SNPRM. It is the Agency's intent to be consistent with the OMB Bulletin for Peer Review regardless of how commenters may interpret that intent or potential inconsistency.
 The application of this final rule to influential scientific information applies only during the develop of the study and then is narrowed even further as it only addresses dose-response data underlying the pivotal science used to develop that influential scientific information. To address the commenter's concern directly, though there may be "thousands of papers that went into the weight of evidence for previous EPA analyses," only a small number of studies will be consider pivotal science when applying the provisions of this final rule to either significant regulatory actions or influential scientific information.

 In addition, the final rule focuses solely on dose-response data. The scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. In some instances, this group will consist of a handful of studies. In other instances, where there are multiple toxicity endpoints, there may be more studies that are crucial to characterizing dose-response relationships. In some other cases, there may be a large number of studies that are used to characterize a dose-response relationship (e.g., where the dose-response is based on a meta-regression of epidemiology studies). However, not all of these studies would be considered pivotal science (see Section III.C.6 for the definition of "pivotal science").

Past briefings to Congress were intended to informational as of that moment during the Agency's deliberative process. The final rule, including its applicability, definitions, etc., considers all comments and represents the Agency's final decisions.
The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
And finally, this final rule does not regulate any entity outside the EPA (e.g. the commenters reference to HEI). Rather, the requirements modify the EPA's internal procedures regarding the transparency of pivotal science underlying significant regulatory actions and influential scientific information.
5.2 Expansion of Scope to Include Influential Scientific Information
 5.2.1 General Support/Dissent for Expansion
Comment: Several commenters (10865, 11388, 11407, 11895, 12702, 13788) generally oppose the expansion of coverage of the rule to include influential scientific information. Specific comment examples include:
 Commenter (11388) does not support the expansion due to the complexity and impracticality of including influential scientific information and instead recommends the final rule only apply to underlying studies, data, and models when promulgating significant regulatory actions.
 Commenter (11895, 11407) states that with the expansion, the supplemental proposal takes a bad proposal and makes it worse and that the expansion does nothing to address the crux of the problem: there are many valid and important scientific studies that cannot legally, ethically, or efficiently publicly release all their data.
 Commenter (10865) states that it is troubling that the scope of the SNPRM now includes "influential scientific information" in addition to significant regulatory actions.
 Commenter (12702) provides that the expansion broadens uncertainty and possible regulatory conflict.
 Commenter (13788) states that the expansion of the scope to include ISI is unwarranted and inconsistent with the IQA.
Conversely, several commenters (11397, 11401, 11499, 11500, 11922, 12402, 12612, 12699) generally support the expanded scope as defined. Specific comment examples include:
 Commenter (11397) asserts that the expansion is clearly in keeping with the goal of the rulemaking, and of Executive Orders 13777 and 13783, to apply transparency provisions not just when EPA is promulgating regulatory decisions or engaging in notice and comment rulemaking but also when the Agency is finalizing key scientific information upon which EPA may rely in the future. The commenter believes that the currently proposed extension of transparency to finalization of key scientific information  --  regardless of whether it is attached to a particular regulatory decision  --  is the correct approach.
 Commenter (11401) states that all scientific information used by EPA should adhere to the highest scientific standards.
 Commenter (11500) notes that the final regulation should make it clear that such policies or decisions include potential use of the information in public or private litigation. The commenter asserts that the need to apply the transparency rules to all key scientific information is highlighted by the EPA's current Integrated Science Assessment (ISA) for particulate matter (PM). The commenter notes that the EPA's CASAC did not support the ISA's findings that PM that have a diameter of less than 2.5 micrometers (PM2.5) is "likely to be causal" for cancer and that the PM that have a diameter of 10 micrometers and smaller (PM10) data are "suggestive" of causality, yet they remain in the final ISA, which can be relied upon in toxic tort litigation or other proceedings as official EPA findings.
 Commenter (11922) believes these expansions along with the SNPRM's proposed definitions and clarifications would enhance the transparency proposal's objectives to ensure that the data and models underlying the science supporting significant regulatory actions are publicly available in a manner to allow for analysis and validation. The commenter states that the more transparent the underlying scientific and regulatory science utilized in formulating policy decisions effecting regulated entities, the more confidence both the public at large and regulated entities will have in EPA and its implementation of policies and regulatory programs.
 Commenter (12612) notes that the expansion is an improvement that will increase the functionality of the Proposal in increasing EPA transparency.
 Commenter (12699) states that the expansion is an important change that indicates EPA's recognition that the science the Agency disseminates even outside the regulatory context can have a major impact. The commenter notes the definition of influential scientific information used in the supplemental proposal is drawn from OMB's "Final Information Quality Bulletin for Peer Review".
 Commenter (11499) supports retroactive application in stating that existing data and models should be covered if they underlie "pivotal regulatory science" in a rulemaking or "pivotal science" in the development of influential scientific information.
Commenter (13788) states that the expansion of the scope to include ISI is unwarranted and inconsistent with the IQA.
Response: Influential scientific information remains in the scope of the final rule. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment. Where underlying dose-response data in pivotal science are available, subject matter experts can independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
The EPA is finalizing the applicability of this final rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
5.2.2 Justification/Necessity for Expanding Scope
Comment: Commenters (10048, 10866, 11392, 11404, 11894, 12456, 12727, 12715) note that the supplemental proposal does not provide adequate justification for the expansion in scope to include influential scientific information. Specific comment examples include:
 Commenter (10048) states that the vague and broad scope is problematic and the supplemental proposal does not provide justifications for these changes, just as there is no justification for the proposal itself.
 Commenter (10866) notes the supplemental proposal fails to adequately justify the proposed rulemaking as it applies to "influential scientific information as well as significant regulatory actions."
 Commenter (11392) states that the supplemental proposal does not provide justification for the sweeping expansion, which makes it difficult for the public to assess the extent of the impact that the rule would have on the Agency's ability to consider the best available science. As such, the commenter asserts that EPA should remove influential scientific information from the scope of the rule. The commenter notes that, if EPA insists on maintaining influential scientific information within the scope of the rule, EPA must address why it decided to apply the rule to influential scientific information as well as significant regulatory actions and provide a detailed explanation of the factual evidence that it used to justify its decision.
 Commenter (11404) provides that the expansion is unnecessary, vague, and dangerous.
 Commenter (11894) states that the supplemental proposal does not in any way address either the legal support for this proposal or the policy need for it. Nor does it delineate its scope or how it would work.
 Commenter (12456) notes such a vague and broad scope is problematic, and the supplemental proposal does not provide justification for these changes.
 Commenter (12727) asserts that the closest the Agency comes to an explanation is a passing reference to OMB implementation updates recommending that "[a]gencies should prioritize increased access to the data and analytic frameworks (e.g., models) used to generate influential information." The commenter contends that the provisions in the proposed rule that would allow the EPA to disregard data and analytic frameworks in developing influential information are inconsistent with the OMB implementation updates. As such, the commenter provides that absent any rationale for expanding the scope of its rule to influential scientific information, finalizing the supplemental proposal would be arbitrary.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. OMB defines both significant regulatory actions and influential scientific information and has provided guidance as to how to identify and manage these actions because of the potential impact of these actions on American lives and livelihoods. This narrow but impactful scope of Agency actions is an important first step towards creating greater transparency.
These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information. The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
5.2.3 Application Guidance for EPA Programs
Comment: Commenter (12406) supports the inclusion of "influential scientific information", but believes that each program office at EPA should establish criteria and guidance to determine the circumstances under which the rule will be applied to "influential scientific information" within the context of each statutory and regulatory scheme, subject to public notice and comment. The commenter states this would reduce the risk and uncertainty of an overly expansive definition that might needlessly slow implementation within the context of some statutes. For example, the commenter notes that applying the Strengthening Transparency rule to Toxics Substances Control Act (TSCA) Section 5 new chemicals program may render review of premanufacture notices impossible within the 90-day statutory timeframe. For these reasons, the commenter recommends that EPA clarify applicability with subsequent criteria and guidance.
Response: In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA intends to issue implementation guidelines and statute-specific rulemakings, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. In this action, the EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation.
5.2.4 Public Comment on Expanded Scope
Comment: Commenters (11361, 11377, 11396, 11492, 11920) state that the expansion to include "influential scientific information" will undercut EPA's ability to fulfill its mission to protect human health and the environment. Commenter (11377) asserts that systematically and arbitrarily favoring fully public data for even more of the Agency's work only further imperils citizens whose health depends on EPA's use of the best science, not just the most public.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.

Expanding this final rule to include influential scientific information will not undercut the Agency's missions but rather by having a better understanding of the underlying data, this final rule will enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.

The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
Comment: Commenters (11393, 11407) state that if EPA refuses to consider a valid scientific study when finalizing "influential scientific information," the public might never be informed, have the opportunity to comment, or bring a legal challenge.
Response: The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation.
As stated in section 30.5, "The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given." This will ensure that the public is properly informed and has the opportunity to comment.
Comment: Commenters (9493, 11894, 12727) contend that the expansion of the proposal requires additional comment to fully address the new reach of the rule. Specifically:
 Commenter (9493) provides that due to the sprawling set of documents that play critical roles in protecting human health and the environment listed in the Agency's most recent Peer Review Agenda, additional comment is necessary to fully address all of these materials.
 Commenter (11894) asserts that EPA must afford the public notice and opportunity to comment on findings on the rationale concerning the actual effects of its supplemental proposal as it is presently unavailable to commenters.
 Commenter (12727) provides that commenters cannot anticipate the reach of the supplemental proposal or account for any potential overlap with the original due to a lack of evaluation on the breadth of the rule in regards to influential scientific information.
Response: The EPA disagrees with these comments. The SNPRM clearly states the potential new scope of the rule, provides background documentation and definitions (e.g. what is influential scientific information) to allow the public to understand the scope and rational and asks the public to comment on various options presented.
5.3 Determining What Constitutes Influential Scientific Information
Comment: Commenters (9362, 11390, 11404, 11892, 12430, 12465, 12688, 12720) contend that the supplemental proposal states the applicability to "influential scientific information" but provides no clarification as to what information the Agency will consider to fall under this category. Specific comments include:
Commenter (9362) questions how "the Agency reasonably can determine" whether a scientific study "will have or does have a clear and substantial impact on important public policies and private sector decisions."
Commenter (11390) provides that the supplemental proposal does not explain how EPA will determine particular science has such a "clear and substantial impact"; there are no factors for the Agency to consider when making such a determination, nor examples of scientific information that would be encapsulated within the definition.
Commenter (11404) notes that determining a level of "influence: and deeming what knowledge qualifies as "pivotal" is too subjective. According to the commenter, science is science; it should be evaluated as such and sufficient methods of evaluation already exist.
Commenter (11892) states that it is unclear how "influential scientific information" will be used differently than has already been done by EPA in the past, while maintaining consistency with the OMB guidance on peer reviews and the Data Quality Act. Additionally, the commenter notes it is unclear how EPA will make the decision on whether something is "influential scientific information" and what the detailed criteria would be for that evaluation. The commenter suggests that the final rule make this clear.
Commenter (12430) asserts that the definition of influential scientific information offers no parameters for determining whether scientific information could have "a clear and substantial impact" or which hypothetical "public policies or private sector decisions" should be considered "important." Accordingly, the commenter states that the breadth of the definition would empower the administrator to make subjective, ad hoc -- and potentially inconsistent or politically motivated -- determinations about whether theoretical "policies" and "decisions" would be "important" and thus would justify the application of the supplemental proposal.
Commenter (12465) provides that the supplemental proposal is unclear what transparent criteria for the dissemination of "influential scientific information" would meet a "novel, controversial, or precedent-setting" determination. Additionally, the commenter states that scientific advancements are invariably "novel, controversial, or precedent-setting" which would tend to discourage advancement of science and is counter to basic scientific principles. The commenter contends that it is also not clear how the scope of the regulation interplays with scientific information that is outside of the scope initially but later becomes within the scope. For example, the commenter notes that a series of smaller studies outside of the scope of the rule would not be held to the transparency requirements, however, if they are appropriately scientifically rigorous, and are at some future point incorporated into a larger activity, they would be precluded from inclusion.
Commenter (12688) notes that the supplemental proposal sets out no test, or even parameters or factors to guide the Agency in determining what information might have a "clear and substantial impact," thus amplifying all the concerns the commenter had with the Proposal.
Commenter (12720) states that the definition of "influential scientific information" is overly vague, as it does not describe how, when, or why the Agency makes such a determination about any such information. Furthermore, the commenter notes the SNPRM does not define what a "clear and substantial" impact is and does not specify if and how such a determination is made or documented. Additionally, according to the commenter the definition is not specific to EPA's research review context and is vague and overly expansive in not describing the types of "public policies" and "private sector decisions" are affected by the nature of influence of the science itself, rather than EPA's determinations about that scientific content within the larger context of all available research. Notably, the commenter asserts EPA does not restrict "influential scientific information" to include only "publicly available" scientific information, because a large portion of important science may or may not be publicly available. As such, the commenter provides that the inconsistency between what EPA currently considers to be influential science on its own web site and how it is defined in the SNPRM signals EPA's lack of adequate advance consideration to define this term.
Response: The definition of influential scientific information in 30.2 of the final rule remains, "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." This definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review.

The interpretation of the term "influential" is consistent with OMB's government-wide information quality guidelines (IQG) and the IQG of the Agency. At EPA, scientific and technical work products that will have or do have a clear and substantial impact on important public policies or private-sector decisions would be considered influential. The following factors (See Chapter 3.2 EPA' Peer Review Handbook, 4[th] Edition) are considered when determining whether a product is likely to be influential:
 Establishes a significant precedent, model or methodology.
 Is likely to have an annual effect on the economy of $100 million or more, or adversely affect in a material way the economy; a sector of the economy; productivity; competition; jobs; the environment; public health or safety; or state, tribal or local governments or communities.
 Addresses significant controversial issues.
 Focuses on significant emerging issues.
 Has significant cross-agency and/or interagency implications.
 Involves a significant investment of agency resources.
 Considers an innovative approach for a previously defined problem, process, or methodology.
 Satisfies a statutory or other legal mandate for peer review.
The Agency disagrees with commenters that the definition, use or application of the term influential scientific information is vague or unclear.
Comment: Commenter (12715) provides that the current supplemental proposal states EPA would continue to use standard processes for identifying, evaluating, and reviewing available data, models, and studies," and that the requirement that studies have data and models available for independent validation would only apply to studies that "could drive [EPA's] subsequent decisions." The commenter asserts the distinction does not appear to be logical or defensible. Using human health risk assessment as an example, the commenter notes that the elements of the scientific evaluation other than the quantitative basis of the toxicity factor are just as important. Additionally, the commenter states that it is not logical that EPA would exclude studies from consideration in the quantitative part of its scientific evaluation but would consider these same studies in other equally important components of the scientific evaluation.
Response: The final rules defines pivotal science as "the specific dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information." The rule also states, "This action establishes how the Environmental Protection Agency (EPA) will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation."

Based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). The EPA considered commenters' assertions that the scope of the 2018 proposed rule would be so broad as to make implementation infeasible. The 2018 proposed definition of "dose-response data and models" would apply to dose-response data [and models] "used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact." This relationship of the dose-response data to the magnitude of a predicted health or environmental impact would require the consideration of an array of studies beyond those that characterize dose-response relationships, including, for example, studies that inform the dose-response modeling (e.g., benchmark response selection); studies that identify data for toxicokinetic adjustments that inform calculation of a human-equivalent point of departure (POD); and studies that inform the selection of uncertainty factors. The number of studies that are used to establish the relationship between dose-response data and models and the magnitude of a predicted health or environmental impact can potentially be very large. This may make implementing the rule as proposed infeasible for at least some significant regulatory actions and influential scientific information. While transparency in EPA decision making is the purpose of this action, transparency cannot be effectively accomplished if the means to that transparency are not feasible. Rather than having this final rule apply to all the studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact, the EPA is balancing transparency and feasibility by focusing on those studies that describe the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Specifically, the scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. In some instances, this group will consist of a handful of studies. In other instances, where there are multiple toxicity endpoints, there may be more studies that are crucial to characterizing dose-response relationships. In some other cases, there may be a large number of studies that are used to characterize a dose-response relationship (e.g., where the dose-response is based on a meta-regression of epidemiology studies). However, not all of these studies would be considered pivotal science (see Section III.C.6 of the final rule for the definition of "pivotal science").
Based on comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency on dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. This final rule provides an important step in furthering the progress toward maximizing transparency and will provide insight for future statute-specific requirements.
Comment: Commenter (9455) states that the only measurable criterion of monetary cost in the supplemental proposal is that "influential scientific information which sways public policies and private enterprise decisions and highly influential scientific information which leads to large policies and decisions that would cost $500 million in a given year," fails EPA's fundamental purposes and negates environmental laws. Instead, the commenter provides that bins of scientific information and science should be defined by numbers of lives saved or lost, numbers of cases of chronic conditions avoided or incurred, costs of medical care avoided or incurred, acreage of land contamination avoided or incurred, and other measures of success or failure of policies and decisions.
Response: There are many ways that scientific information can be defined and put into bins, but with respect to this final rule and with regard to influential scientific information, the interpretation of the term "influential" is consistent with OMB's government-wide information quality guidelines (IQG) and the IQG of the Agency. At EPA, scientific and technical work products that will have or do have a clear and substantial impact on important public policies or private-sector decisions would be considered influential. The following factors (See Chapter 3.2 EPA' Peer Review Handbook, 4[th] Edition) are considered when determining whether a product is likely to be influential:
 Establishes a significant precedent, model or methodology.
 Is likely to have an annual effect on the economy of $100 million or more, or adversely affect in a material way the economy; a sector of the economy; productivity; competition; jobs; the environment; public health or safety; or state, tribal or local governments or communities.
 Addresses significant controversial issues.
 Focuses on significant emerging issues.
 Has significant cross-agency and/or interagency implications.
 Involves a significant investment of agency resources.
 Considers an innovative approach for a previously defined problem, process, or methodology.
 Satisfies a statutory or other legal mandate for peer review.
Comment: Commenter (12645) states that studies can only be considered "scientifically influential" in retrospect, i.e., this term is more suitable to the history of science. The commenter posits that if the original Atomic Energy funded studies of radioactive particles had not been confirmed in many ways, we would not have had such an interest in fine particles. So, the commenter queries, aren't these confirmatory studies scientifically influential? To the commenter, the definition given in the supplemental proposal is little more than handwaving and can be used by a variety of parties. Finally, the commenter states that citation of research within an EPA document on which a regulatory decision is based could also be used to define pivotal or influential science, and while imperfect, it is still far more understandable and accountable than any of the definitions in the supplemental proposal.
Response: The definition of influential scientific information in 30.2 of the final rule remains, "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." This definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. The intent of prospectively designating studies as influential scientific information is to gain from the additional attention and required peer review. Retrospective designation may be more accurate but provides no benefits to the Agency in terms of ensure a sounder more scientifically robust product.

The interpretation of the term "influential" is consistent with OMB's government-wide information quality guidelines (IQG) and the IQG of the Agency. At EPA, scientific and technical work products that will have or do have a clear and substantial impact on important public policies or private-sector decisions would be considered influential. The following factors (See Chapter 3.2 EPA' Peer Review Handbook, 4[th] Edition) are considered when determining whether a product is likely to be influential:
 Establishes a significant precedent, model or methodology.
 Is likely to have an annual effect on the economy of $100 million or more, or adversely affect in a material way the economy; a sector of the economy; productivity; competition; jobs; the environment; public health or safety; or state, tribal or local governments or communities.
 Addresses significant controversial issues.
 Focuses on significant emerging issues.
 Has significant cross-agency and/or interagency implications.
 Involves a significant investment of agency resources.
 Considers an innovative approach for a previously defined problem, process, or methodology.
 Satisfies a statutory or other legal mandate for peer review.
With regard to pivotal science, the Agency has considered comments and Part 30.2 of the final rule provides an final definition.
5.3.1 Limits to EPA's Application of the Rule
Comment: Commenter (12435) asserts that the supplemental proposal's coverage of influential scientific information essentially removes any limit to what information is applicable per the Agency's determination, so the effect is to allow the rule to apply at the Agency's discretion.
Response: The EPA disagrees with this comment. The definition of influential scientific information in 30.2 is "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." This definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. The interpretation of the term "influential" is consistent with OMB's government-wide information quality guidelines (IQG) and the IQG of the Agency. At EPA, scientific and technical work products that will have or do have a clear and substantial impact on important public policies or private-sector decisions would be considered influential. The following factors (See Chapter 3.2 EPA' Peer Review Handbook, 4[th] Edition) are considered when determining whether a product is likely to be influential:
 Establishes a significant precedent, model or methodology.
 Is likely to have an annual effect on the economy of $100 million or more, or adversely affect in a material way the economy; a sector of the economy; productivity; competition; jobs; the environment; public health or safety; or state, tribal or local governments or communities.
 Addresses significant controversial issues.
 Focuses on significant emerging issues.
 Has significant cross-agency and/or interagency implications.
 Involves a significant investment of agency resources.
 Considers an innovative approach for a previously defined problem, process, or methodology.
 Satisfies a statutory or other legal mandate for peer review.

The requirements of this final rule will apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Peer Review Bulletin.
 Chapter 6: Pivotal Science
The supplemental notice of proposed rulemaking (SNPRM) includes a definition for "pivotal science" at 40 CFR 30.2 that would pertain to the science underlying influential scientific information, which is not regulatory, and made conforming changes to the notice of proposed rulemaking (NPRM) regulatory text at 40 CFR 30.3, 30.5, 30.6 and 30.7. These proposed revisions were made to provide clarity on key terminology used in the regulatory text in the 2018 proposed rulemaking as well as in this supplemental proposal.
The SNPRM proposed "pivotal science" definition follows:
      Pivotal science means the specific scientific studies or analyses that underly influential scientific information.
Several comments were received regarding the definition of "pivotal science" and scope of coverage. In this section these comments are organized under the following sections:
Section 6.1: NPRM Pivotal Regulatory Science
 How Should the Proposed Rule Apply to Dose Response Data and Models Underlying Pivotal Regulatory Science If Those Data and Models were Developed Prior to the Effective Date?
 Could Prospective or Retrospective Application Introduce Bias Regarding the Timeliness and Quality of Scientific Information Available?
 How Does What this Rule Would Require Compare to Requirements of Environmental Statutes?
 Recommended Changes to the Definition of Pivotal Regulatory Science at §30.2
Section 6.2: SNPRM Pivotal Science
 Need for Additional Clarity/Specificity
 Support for Expanded Scope
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
6.1 NPRM Pivotal Regulatory Science
6.1.1 How Should the Proposed Rule Apply to Dose Response Data and Models Underlying Pivotal Regulatory Science Developed Prior to the Effective Date?
<IP_ADDRESS> Support Some Application of the Rule to Data and Models Developed Prior to the Effective Date
<IP_ADDRESS>.1 Recommend that EPA Revisit Regulatory Decisions Based on Studies Conducted in the Past that have had Major Impacts to Regulated Entities
Comment: Commenter (3971) suggests that for regulations that have had major impact on the economic well-being of certain corporations or the national economy, the EPA should selectively revisit decisions of the past. The commenter states that many older studies that that the Agency relied upon as a basis for regulation were outdated, poorly conducted, and improperly interpreted. According to the commenter, the Agency often said that although the data were weak, it was all that was available, and suggested that action is warranted. The commenter notes that this bar may have been adequate in the past when we were dealing with major problems, such as a serious ambient air problem in cities, but that the trend toward invoking the Precautionary Principle in times of doubt has sometimes led to this Principle to be misapplied. The commenter contends that more studies and a proper interpretation of them will allow EPA to drift away from policy-driven regulation.
 Response: EPA disagrees that this rule should require EPA to require the dose-response data underlying pivotal science to be publicly available for past rules. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
<IP_ADDRESS>.2 Recommend that Studies Used as Pivotal Regulatory Science be Made Available Early in the Rulemaking Process
Comment: Commenter (6375) states that, in general, they did not believe that EPA needed to perform a retrospective review of all past final regulations to ensure compliance with the proposed rule when the rule is published. The commenter suggests that, at a minimum, however, the proposed rule should apply to any rule in the Advance Notice of Proposed Rulemaking (ANPRM) or proposed rule phase as of the effective date of this regulation and that enough time be available prior to any statutory or court-ordered deadlines to make public all data and models from dose-response studies as early as possible. The commenter states that this approach should apply to dose-response studies even if they were developed prior to the effective date of this rule. The commenter adds that, if there are instances when EPA cannot apply this rule, then the Agency should identify in the final rule which dose-response studies used as pivotal regulatory science have not been made publicly available and a schedule for making those studies available.
Response: EPA agrees with the commenter that this rule should not apply retrospectively to past final regulations, but should apply to studies developed prior to the effective date of this rule. This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. EPA further notes that it sees no need to include the proposed rule stage of significant regulatory actions in the regulatory text because as a practical matter proposed rules must comply with this final rule before being finalized. As a general matter, the EPA does not introduce the studies and analyses it relies on for a rulemaking at the final rule stage. The scientific basis for a rulemaking is provided for public review and comment in the public docket when the proposed rule is issued or, if subsequently added to the docket, through a separate opportunity for public comment. Advance notices of proposed rulemakings are not consistent with the purpose of this rule, given their preliminary nature and frequent focus on soliciting comments on a regulatory issue or approach.
<IP_ADDRESS>.3 Recommend that Rule Apply to All Pivotal Regulatory Science Regardless of When Data and Models Developed
Comment: Commenter (6921) recommends that apply the enhanced transparency requirements included in the Proposed Rule to all pivotal regulatory science regardless of when the data and models were developed. The commenter asserts that, given that EPA generally relies on previously published research, applying the rule prospectively to any new study would undermine any effect from the rulemaking for years. According to the commenter, this would allow EPA to continue to use research that has not been replicated or validated. The commenter notes that, given the importance of EPA regulatory decisions that can impose millions, if not billions, of dollars on the regulated community and the American public, the EPA should make every effort to assure public access to all models and data used to support significant regulatory actions, regardless of when the study was first published or relied upon for regulatory decisions. The commenter asserts that the relative importance of the study to significant regulatory actions, rather than its age, should be the main determinant of the applicability of the enhanced transparency requirements.
Response: EPA agrees with the commenter. The final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
<IP_ADDRESS> Oppose Application of the Rule to Data and Models Developed Prior to the Effective Date
Comment: Commenters (1012, 5170, 6373, 6448, 6916) take the position that this rule should not apply to data and models developed prior to the rule's effective date. Specific comments include:
Commenters (1012, 5170) assert that data and research that were conducted prior to a final date of this proposed rule should be exempted as the protocols were already approved. Commenter (5170) states that this approach would be similar to the manner in which pesticides that EPA approved for use in the past with less current research protocols have been and continue to be `grandfathered in', often for widespread use.
Commenter (6373) states that it would be difficult, if not impossible, to apply the Rule's new requirements related to dose response assumptions and models to studies where the methodology was developed prior to the effective date of the Rule.
Commenter (6916) states that application of the proposed rule to dose response data and models developed prior to the effective date would have the potential effect, on Clean Air Act (CAA) rulemakings, of prohibiting the use of the well-developed dose response information resulting from two landmark studies, the Harvard Six Cities Study and the American Cancer Society Study of Particulate Air Pollution and Mortality, that have been called "the poster children of good-science reforms." The commenter expresses opposition to such an outcome, as it would remove from EPA's rulemaking toolbox the "best available science" on the link between air quality and health and is both unnecessary and unjustified.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The requirements of this final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: Commenter (1012) states that data, studies and research that were initiated prior to the final date of this proposed rule should be excepted from this proposed rule as the protocols were already set before the rule was finalized. Similarly, commenter (6448) recommends that the provisions of this rule not apply to data and models currently underway.
Commenter (6373) requests that EPA state whether the Rule will apply to scientific studies that have already commenced, where human participants have already signed data use agreements or other agreements that place limits on the use of their private data.
Response: For future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The requirements of this final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
6.1.2 Could Prospective or Retrospective Application Introduce Bias Regarding the Timeliness and Quality of Scientific Information Available?
<IP_ADDRESS> Conditional Retrospective Application to Past Studies Used as Pivotal Regulatory Science
Comment: Commenter (6141) states that, in the event that EPA elects to apply the transparency rules retrospectively, the EPA should consider developing exemptions for those past studies for which it is not possible to release a dataset publicly for legitimate reasons pertaining to the use, for example, of human subject data.
Response: EPA notes that this final rule applies prospectively to future significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. That said, EPA agrees with the commenter that additional reasons for exemption from this rule are merited, and has amended 40 CFR 30.7 (proposed 40 CFR 30.9) to read,
   "The Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration is warranted because:
       technological or other barriers render sharing of the dose-response data infeasible;
       the development of the dose-response data was completed or updated before the effective date of the final rule;
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or