Comment: Commenters (11408, 11479) state that the demand for obtaining the raw data related to EPA rulemaking does not appear to be significant and would not justify the excessive burden it would place on the Agency. The commenter states that one study, which looked at the 79 requests made between 2002 and 2012 through the Information Quality Act to EPA that asked the Agency to correct or reconsider the data supporting its regulatory decisions, found only two requests asking for the raw data. The commenter states that one possible explanation for this low number is that the public already has access to the science the EPA relies on and can fully access methods, summary data, results, and interpretations, as can all peer reviewers. The commenter states that in developing the NAAQS, for example, the EPA collects all relevant data and studies used to develop the Integrated Science Assessment and the merits of those scientific studies are publicly debated by the Clean Air Scientific Advisory Committee and pollutant review panel. The commenter states that the need for access to raw data is entirely unnecessary and arbitrarily precludes the use of critical studies.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenters (11403, 11498, 11576, 12718) object to the proposal package and assert that access to raw data is not determinative of the quality of the research.
Commenter (11403) asserts that public availability of raw data is not a rational metric for deciding whether a study is of high enough quality to be used for setting thresholds or enacting regulatory protections. The commenter states that the EPA must ensure that it uses high quality data to analyze environmental justice impacts vulnerable populations, regardless of the public availability of the raw data.
Commenter (11498) contends that there are credible procedures for testing results and verifying outcomes with methodologies that do not require access to raw data. The commenter states that the original proposed rule and now this SNPRM are de facto rejecting credible practices used by the scientific community and replacing them with a non-scientific metric in the evaluation of a study beyond its immediate quality. The commenter states that, as the EPA's own Scientific Advisory Board has stated, this decision risks politicizing science by using an unscientific standard to assess the validity of science.
Commenter (11576) states that there is no need to limit consideration of reliable science simply because the underlying data is not publicly available. The commenter states that less than 30 percent of scholarly literature is open-access, and only a small fraction of these include access to underlying raw data. The commenter states that journal-specific transition plans to increase access involve long timeframes to accommodate these changes, which are often voluntary opt-ins for participating researchers. The commenter states that these open-access programs typically refer only to public access to the published studies themselves rather than to underlying raw data as required in the proposed rule, which is contrary to standard scientific practice.
Commenters (11576, 11920) refer to a joint statement by a group of scientists and editors-in-chief of leading scientific journals which states that "the merits of studies relying on data that cannot be made publicly available can still be judged." Commenters state that the group further noted that scientific studies typically contain all the information required to judge the accuracy and robustness of their findings without access to raw data as scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results.
Commenter (12718) states that the Multi-Society comments on the SNPRM espouse a principle that must be given careful consideration: "access to raw data is not determinative of the quality of the research." The commenter states that the EPA's proposed approach in the SNPRM presumes that access to data or information must be  -  as an absolute  - accessible to EPA officials, even when that level of access might be inconsistent with Institutional Review Board (IRB) requirements, consent statements, or other ethical procedures that should be paramount in government's commitment to its citizens for protecting data and confidentiality.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Justification of How Availability Impacts Validity
Comment: Commenters (10048, 11408, 11576, 12430, 12727, 12731, 13788) express concern that the EPA has not provided any justification as to how or why the availability of confidential information or PII may impact the validity of data, studies, and models.
Commenter (11576) states that, while the EPA argues the proposed rule will strengthen the quality of the Agency's science, it points to no evidence to support that statement. The commenter states that, instead, the evidence shows that the rule's emphasis on data availability rather than accuracy will actually weaken the body of evidence available to the Agency. The commenter states that, while full consideration of the range of observations reported in peer-reviewed scientific literature provides a method for comprehensively estimating trends, means, and variances in data, arbitrarily omitting studies can result in skewed interpretations.
Commenter (12430) states that the EPA does not articulate, let alone provide, any basis for alleging that the scientific community's established practices are faulty or inadequate. The commenter states that, nonetheless, it would replace these practices with a single threshold requirement: public availability of the underlying data. The commenter states that the fact that the proposed rule is at odds with EPA's and the scientific community's established practices -- as underscored by the many unique, substantive comments submitted in opposition to the rule by scientists, scientific organizations, and scientific journals -- suggests that the EPA intends for the SNPRM to advance political priorities rather than the quality of science the Agency uses.
Commenter (12727) indicates that EPA has not demonstrated -- and cannot demonstrate -- that a study cannot be scientifically valid unless the underlying data and models are publicly disclosed. The commenter states that mechanisms are already in place to validate studies for which disclosure of underlying data and models is either illegal or impracticable.
Commenter (12731) states that researchers do not make data public to improve the strength or quality of their findings, and the EPA has offered no evidence to suggest that studies with publicly available underlying data are more likely to represent strong science than studies without such data availability. The commenter asserts that, like the original proposal, the SNPRM lacks any demonstration that the public unavailability of a study's underlying data or models necessarily -- or even likely -- renders the study invalid or unreliable. The commenter states that the proposal failed to identify an actual problem that needed to be addressed, or to point to even a single example of an EPA action or rulemaking that was later found to be defective because it rested on a study for which data was not publicly available.
Commenter (13788) contends that the basic premise of this rulemaking is that public health data -- including human health data -- has to be made publicly available to ensure the validity of scientific studies. The commenter states that this premise is completely unsupported and totally incorrect. The commenter states that the Agency's long standing position is that "[w]hether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications." The commenter states that the EPA fails to explain why it now believes the contrary -- that study validity and public data availability are (or should be) connected.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
<IP_ADDRESS>.2 Approaches to Validity Other Than Raw Data
Comment: Commenters (10866, 11377, 11400, 11489, 12403, 12731) note that the proposal package calls for underlying data and models to be "publicly available in a manner sufficient for independent validation." These commenters object to the proposal package and generally assert that such independent validation is not the only means of determining credibility of studies.
Commenter (10866) states that the EPA fails in making its proposal to acknowledge that research studies can be both evaluated and replicated without the release of such individual data. Commenter (11377) asserts that transparency can be achieved in different ways that satisfy the demands of different kinds of science, while still maintaining respect for persons and intellectual property. Commenter (11489) states that, in cases where it is not appropriate for data to be made publicly available, there are other mechanisms intrinsic to the scientific process for substantiating the relevance and validity of research results. Commenter (12403) states that the proposed rule is not necessary to allow for independent validation of scientific studies and it would undermine existing successful science-based policy making. Commenter (12731) notes that comments on the original proposal demonstrate that mechanisms are already in place to validate studies for which disclosure of underlying data and models is either illegal or impracticable.
Response: The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (11378) contends that the integrity of much environmental and health research requires the tradeoff that certain business, proprietary, and PII remain confidential. The commenter states that making such information publicly available would potentially violate long-standing human subjects research protocols (i.e. the 1974 National Research Act), as well as legally binding proprietary data agreements.
Commenter (11926) asserts that the notion that the public release of raw data is a protection against fraud, or the generation of misleading results is also flawed. The commenter states that there are instances in which an unscrupulous researcher alters or even fabricates data to fit a preconceived hypothesis, however, there is no way to tell from the data itself that it is not accurate. The commenter states that this is why the weight of evidence, replication of findings from multiple sources, and peer review by scientific journals are so important -- these factors, not the public availability of raw data, are the main safeguards against flawed or fraudulent science.
Commenter (13788) states that the EPA provides no explanation as to why those policies are not sufficient to allow validation of scientific studies and information relied on by the Agency. The commenter states that simply stating that "EPA shall conduct independent peer review on all pivotal regulatory science used to justify significant regulatory decisions and on all pivotal science underlying influential scientific information, consistent with the requirements of the Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review," fails to explain why this does not already occur under the Agency's past and current practice. The commenter contends that, if validation is truly the aim of this rulemaking, the EPA must explain why existing practices for peer review are insufficient to validate scientific studies.
Response: As discussed in Section III.B of this preamble, based on a consideration of the public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA is also further clarifying how the Agency will determine the consideration to afford to pivotal science in either significant regulatory actions or influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect.
Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information:
 Soundness  -  The extent to which the scientific and technical procedures, measures, methods or models employed to generate the information are reasonable for, and consistent with, the intended application.
 Applicability and Utility  -  The extent to which the information is relevant for the Agency's intended use.
 Clarity and Completeness  -  The degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented.
 Uncertainty and Variability  -  The extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods or models are evaluated and characterized.
 Evaluation and Review  -  The extent of independent verification, validation and peer review of the information or of the procedures, measures, methods or models.
When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E).
Comment: Commenters (10040, 10748, 11480) suggest the Agency rely, in part, on well-established principles for good science such as disclosure of conflicts of interest.
Commenter (10040) urges the EPA to withdraw its proposal to restrict scientific research and follow the current, effective measures in place to ensure the use of robust, uncensored scientific research to protect the health of our citizens and our communities. The commenter states that existing protections are in place to protect and assure the quality of the science used in the process of EPA's regulatory decision making.
Response: Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information:
 Soundness  -  The extent to which the scientific and technical procedures, measures, methods or models employed to generate the information are reasonable for, and consistent with, the intended application.
 Applicability and Utility  -  The extent to which the information is relevant for the Agency's intended use.
 Clarity and Completeness  -  The degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented.
 Uncertainty and Variability  -  The extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods or models are evaluated and characterized.
 Evaluation and Review  -  The extent of independent verification, validation and peer review of the information or of the procedures, measures, methods or models.
When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E).
Comment: Commenter (11390) states that the SNPRM's approach would, in a wide range of circumstances, replace the EPA's existing, successful protocols -- already in step with established scientific procedures -- with a process driven by arbitrary, non-scientific gatekeeping.
Commenter (12702) states that the EPA has established existing protocols and statutory requirements for the rigorous scientific review for any rulemakings, and those requirements are not tied to whether the data are publicly available. The commenter states that, thus, robust peer review, can, and sometimes should, proceed without the publication of sensitive or confidential data. The commenter urges the EPA to ensure that any final rule is consistent with the scientific journal standards cited in the rule.
Response: As discussed in Section III.B of this preamble, based on a consideration of the public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA is also further clarifying how the Agency will determine the consideration to afford to pivotal science in either significant regulatory actions or influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect.
Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
Comment: Commenters (11387, 11395, 11890, 11926) suggest that there is no need for this rule as mechanisms exist for researchers to share data in order to confirm and validate studies.
Commenters (11387, 11890) suggest that researchers in one institution can transmit data in a confidential manner to researchers in one or multiple other institutions so the data can be evaluated and analyzed in a completely new and different analysis. The commenters assert that this allows the study to be confirmed and validated, while still maintaining privacy protections of the people in the study.
Commenter (11395) states that, as discussed in comments on the proposed rule submitted by a large group of scientists in August 2018, representatives of academia, medical scientists, industry and publishers have developed protocols and guidelines specifically designed to improve reporting and evaluation of studies to improve the quality and transparency of the interpretation of findings. The commenter states that these protocols and guidelines are designed to improve the scientific basis of data evaluations without requiring public access to all study data. The commenter states that the EPA should utilize these existing tools to evaluate study quality and should build on the work already done by the scientific community.
Commenter (11890) states that, in order to honor study commitments to participants to guard their privacy and laws governing CBI and intellectual property, this could best be accomplished by a data sharing agreement with a trusted third party that agrees to uphold confidentiality of the study data. An example of this model is the Health Effects Institute reanalysis of Harvard's "Six Cities" Study and the American Cancer Society's "Study of Particulate Air Pollution and Mortality."
Commenter (11926) states that one approach is the establishment of data enclaves, in which an independent expert can access the raw data for statistical analysis, generate results to replicate the original findings and perform additional analyses, without taking possession of the data itself. The commenter states that the technology to create a secure data enclave is well-established, and many examples exist. The commenter states that this approach allows rigorous independent assessment of the raw data without compromising the sensitive information provided by human subjects.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11489) expresses concern that the SNPRM does not address other mechanisms that could be used to give the EPA confidence that the findings should be considered in rulemaking. The commenter states that such mechanisms include comparing outcomes of several trials from different groups that reach the same conclusions and vetting the science through expert scientific panels convened for this purpose. The commenter states that these mechanisms are especially important for past studies where the underlying datasets may be unsuitable or unavailable for public review. The commenter also asserts that the public notice and comment period provide an opportunity for the Agency to communicate the scientific research that supports its proposal.
Response: When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E).
Comment: Commenters (10040, 10339, 10973, 11378, 11395, 11399, 11480, 11489, 11498, 11926, 12404, 12430, 12617, 13788, 14397) suggest that peer review is an appropriate mechanism to determine the validity of studies.
Commenter (10040) asserts the use of peer-reviewed research is the foundation for EPA's decision making. Commenter (11378) encourages the EPA to utilize the most appropriate, scientifically peer-reviewed research to guide the development of environmental regulations and scientific information. Commenter (11399) states that additional "independent validation" that goes above and beyond the peer review process and the EPA existing science advisory infrastructure is not necessary to assure scientific validity and accuracy.
Commenter (10339) expresses concern that the Agency provides no evidence that the EPA decisions have been misled by the use of data published in the peer-review literature. The commenter suggests that the stated reason for this rule lacks scientific justification and credibility.
Commenter (10973) states that the EPA often appropriately publishes research through peer-reviewed journals. The commenter states that, ensuring that the Agency makes this information publicly available, whether through open access journals (or open access options within traditional journals), pre/post-prints, publicly available data, or some other method or combination of methods would help assure that all of EPA's processes can align to this policy and be of the greatest utility to all who may benefit.
Commenter (11395) asserts that risking re-identification by increasing access to study data is not necessary, given the availability of data to other scientists through National Institute of Health (NIH) repositories and peer-review journals, and the alternative techniques available to verify conclusions of scientific studies. The commenter states that, under the proposed rule, a peer reviewed study that has been replicated many times by different investigators using different data may be excluded from consideration, while one that uses an inferior database that is publicly available would be included.
Commenter (11489) states that the incorporation of conclusions from peer-reviewed, quality science is critical to the integrity and effectiveness of the EPA's regulatory and rulemaking processes and its mission "to protect human health and the environment."
Commenter (11498) states that the model of down-weighting research maintains the major flaw in deeming data that cannot be made completely public  -  for legitimate and legal reasons to protect the privacy of health and CBI  -  less scientifically valid or valuable. The commenter states that it mischaracterizes the scientific process and the range of mechanisms for disclosing and protecting scientific research results for decision-making, implying that peer-reviewed scientific research data that are not available in its raw form is not rigorous enough for use in policy; this is simply not true. Commenter (11480) states that much of the proposal hinges on data availability, which the Agency appears to erroneously conflate with accuracy instead of relying on well-established principles for good science, such as peer review.
Commenter (11926) states that a key factor used in assessing research findings is publication in a peer-reviewed journal. Commenter (12617) states that peer review includes the evaluation of the authors' study design, research questions, methods, results, data presentation, interpretation, and conclusions. Commenter (12404) states that the proposed exclusion from consideration of any study for which underlying data are not made publicly available is not consistent with sound scientific practice because the scientific community uses other tools to validate studies without access to all underlying "raw data" including peer review. Commenter (13788) states that the EPA has long relied on peer review and already has extensive policies in place regarding influential scientific information to assess the quality of the scientific research.
Commenter (14397) asserts that the standard process of peer review already ensures that the underlying research data and models used in scientific decision making "are publicly available in a manner sufficient for independent validation" and ensures that these studies adhere to the appropriate standards of transparency, repeatability, accuracy, appropriateness of data, and explain all assumptions included in the research. The commenter states that the internationally accepted peer review system already includes mechanisms for excluding CBI and PII. The commenter states that, for these reasons, the EPA's proposal is unnecessary and purports to solve a problem that does not exist.
Commenters (11395, 12404, 12617) suggest there are several mechanisms available for validating studies, including reanalysis, replication, and reproduction of data, which do not require access to all underlying raw data. Commenters (11395, 12404) state that available tools include replication of a study using the same methodology but different data sources and/or reproduction of a study's conclusions using different methodologies and data. Commenter (11395) states that, under the proposed rule, a study that has been replicated many times by different investigators using different data may be excluded from consideration, while one that uses an inferior database that is publicly available would be included.
Response: The EPA finds that comments received from the 2018 proposed rule and the 2020 SNPRM have merit, in part. In some cases, prior peer review or publication may be adequate, in which case initiating a new peer review may unnecessarily delay the EPA's development of a significant regulatory action or influential scientific information. However, as detailed in OMB's Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states, that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary". Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule.The EPA also agrees with commenters that the scientific community and government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms. Many scientific publications, for example, require authors to make a data availability or data access statement, which discloses where and under what conditions the underlying study data are available. Yet the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited.
Comment: Commenters (10748, 11926) support a weight of evidence approach. Commenter (10748) states that it is the replication of the science and weight of evidence of numerous  - sometimes hundreds  -  of studies that actually influence scientific opinion.
Commenter (11926) states that, in addition to the weight of evidence from multiple domains, regulators have also examined consistency in findings from independently conducted studies. The commenter states that, in the case of humans, the finding that a chemical agent is harmful becomes more compelling as studies conducted by unrelated investigators, using different subject pools, all point to the same conclusion.
Commenter (11926) contends that the proposal ignores the history of science-based government regulation, which has always been based on the weight of evidence obtained from a variety of sources: laboratory-based chemical and biological (test-tube) analysis, laboratory-based testing on non-human animals, and observational research of human subjects. The commenter states that an example is the weight of compelling evidence from multiple sources that led to the regulation of tobacco marketing and distribution; as is well-known, research in all of the above domains established the fact that use of tobacco products caused cancer, a variety of other diseases, and early death.
Response: The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. As further described in Section II.B, the EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
Comment: Commenter (12617) states that scientific understanding progresses by researchers performing studies related to others in their field, by questioning one another's methods, by updating or expanding upon their own work or that of others. The commenter states that it also progresses by scientists performing systematic reviews and meta-analyses to understand large bodies of research. The commenter states that poorly designed studies and unsupported conclusions are called out by others without needing someone to recreate every step of data processing, analysis, and calculation for every study.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the public's access to the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
The EPA's attention to data transparency is also responsive to the broader interest in greater data transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions (Refs. 19, 20, 21, 22, 23). The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
Comment: Commenters (11915, 12454, 12596) see merit in both the Tiered Access Option and Consideration Option.
Commenters (12454, 12596) suggest the final rule should combine both approaches to establish a procedural transparency assessment framework robust enough to inform the Administrator of scientific transparency in the course of using policy-making authority. The commenters state that such a framework should be premised on the Consideration Option's reliance on policymaking consideration while also leveraging the Tiered Access Option's appreciation of different levels of transparency. The commenters state that general tiers of access  -  broadly speaking, full public access, restricted researcher access, and little outside access  -  can be used to establish a procedural transparency assessment framework that signals the extent to which a study's transparency supports its conclusions or, alternatively, introduces uncertainties into its methodology. The commenters state that this framework would serve to highlight those studies that deserve greater consideration based on the transparency of their underlying data and models.
Commenter (12596) states that the procedural transparency assessment framework outlined in these comments would inherently mitigate concerns that transparency requirements could lead to the disclosure of sensitive information. The commenter states that, under the process described herein, the EPA would generally not be in the position of disclosing data and models. The commenter states that researchers would instead be incentivized to disclose this information in order to improve their studies' transparency assessments.
Commenter (12596) suggests that the final transparency rule can be effective without, as suggested by the Tiered Access Option, dictating when the Agency will consider a study in CAA rulemaking. The commenter states that the Final Transparency Rule can enhance transparency without, as suggested by the Consideration Option, directing how the Agency should consider science in CAA rulemakings. Instead, the Final Transparency Rule can combine both approaches to establish a procedural transparency assessment framework robust enough to inform the Administrator of scientific transparency in the course of using policy-making authority under the CAA. The commenter states that such a procedural transparency assessment framework, authorized only by the Federal Housekeeping Statute, would exist separate from the specific statutory and regulatory processes already established for reviewing science under the CAA. The commenter states that, as such, the EPA should implement the framework in a manner that complements, and does not interfere with, these processes, with the goal of better informing final policy outcomes. The commenter states that, at the same time, assessments resulting from the framework should be formatted in a manner that is readily accessible not only to the Administrator, but also the general public. The commenter suggests that the EPA can accomplish these objectives through a "Transparency Scoring System."
Commenter (12596) explains that, under this System, each study covered by the Final Transparency Rule would be given a score based on the ability of outside parties to assess the study's underlying data and models. The commenter states that the scoring parameters would be based largely on categorization as outlined by the Tiered Data Option. The commenter states that, borrowing from the Consideration Option, the Administrator would then have the ability to use this score when considering studies in the rulemaking process. The commenter states that the EPA could make this system simple or more complex, depending on how the Agency feels it would be best implemented. The commenter states that a simpler approach would assign basic "whole number" transparency scores to studies, for example:
 Transparency Score 3: Study's authors make all data and models publicly available.
 Transparency Score 2: Study's authors make a majority of data and models publicly available.
 Transparency Score 1: Study's authors make a majority of data and models only available for review by select outside researchers.
 Transparency Score 0: Study's authors make less than a majority of data and models available for review by select outside researchers.
Commenter (12596) states that the EPA could also choose to implement a slightly more complex Transparency Scoring System, to address more specific scenarios, by incorporating "interval scores." The commenter states that, for example, if all of a study's data and models were made public, but sufficient contextual information to fully replicate the study (such as underlying assumptions) was not made available, the score could be 2.75 (or some other appropriate interval). The commenter states that the final rule could provide for guidance to address the details of various intervals. The commenter states that, in assessing these specifics, the EPA's guiding principle should be to implement a balanced and workable procedural transparency assessment framework. The commenter states that, on the one hand, the framework should reflect the variables implicating transparency, even if that tends towards a more complex system. The commenter states that, on the other hand, the framework should not be so burdensome that it inhibits rulemakings. The commenter states that, either way, the Final Transparency Rule needs to either address or provide for guidance on key underlying issues.
Response: As discussed in Section III.B of this preamble, based on a consideration of the public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA is also further clarifying how the Agency will determine the consideration to afford to pivotal science in either significant regulatory actions or influential scientific information.
Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
<IP_ADDRESS> International Concerns
Comment: Commenters (11395, 11496, 11890, 11894, 12404, 12405, 12727, 12731) express concern that the EPA has not described how science from other countries would be incorporated.
Commenter (11890) states that researchers in other countries could not hand over private information to unauthorized people. The commenter states that data-sharing agreements do not allow sharing personal health data even with trusted friends.
Commenter (11894) states that seven recent North American studies were done using Canadian cohorts, for which access is extremely limited. The commenter states that access is also highly limited for all ten of the European studies under the EU General Data Protection Regulation. The commenter states that the EU requires that all PII data must be controlled by a data controller, who must show that any use of the data has been consented to by the individuals involved.
Commenter (11894) states that 30 of the 40 plus cohort studies of PM and mortality listed in the Integrated Science Assessment (ISA) for Particulate Matter (December 2019) could not be considered at all in the first version of 30.5. The commenter states that, under the alternative, 25 of them could not be given full consideration. The commenter states that this illustrates how arbitrary the SNPRM is, and indicates the proposal violates CAA section 108 (a)(2), which defines air quality criteria, among other things, as "the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such air pollutant in the ambient air." The commenter states that it is possible to say with a straight face that the Medicare studies, which have a statistical power orders of magnitude greater than any previous PM epidemiologic study, are not "useful" in identifying the effects of PM2.5 in the ambient air; yet the SNPRM would either bar them from consideration or a priori downgrade their worth.
Commenter (12404) state the proposed rule and SNPRM place an undue burden on private companies that have invested in REACH (Registration, Evaluation and Authorization of Chemicals) registration. The commenter states that, in order to register a chemical under the European REACH program so that it can be used in commerce in Europe, private sector companies have to conduct studies of their chemicals (most of these are animal or in vitro studies) and submit them to the European Chemicals Agency (ECHA). The commenter states that these studies are expensive to conduct and are considered by the companies who do so to be an investment for the purpose of gaining access to the European market. The commenter states that robust summaries of these studies are available to the public, but data, associated protocols, computer codes and models, recorded factual materials and detailed descriptions of how to access and use such information are usually not made available to the public. The commenter states that one reason for this, is that companies spend a lot of money to develop these materials. The commenter states that requiring information developed at some cost to be made public in this way could give competitors access to the European market without having to make similar expenditures; thereby giving competitors an unfair advantage. The commenter states that the EPA relied on 20 ECHA Robust summaries for its risk evaluation of Pigment Violet 29 under TSCA and on five ECHA Robust summaries for its risk evaluation of 1-Bromopropane. The commenter states that the EPA would not be able to do so under this rule.
Commenter (12405) states that international studies raise additional challenges because participants are protected by European Union (EU) law. The commenter states that, going forward, an EU study's compliance with the proposed rule will need to be reconciled with the new General Data Protection Regulation, which is seen as more restrictive than the United States' Health Insurance Portability and Accountability Act (HIPAA). The commenter states that it is clear that many studies involving people located in the European Union (EU) will have a difficult time both complying with the new directive and providing enough information to EPA to be considered. The commenter states that studies coming from other countries are vital to health determinations in the United States because people in other countries are exposed to chemicals at different rates than in the U.S. The commenter states that the inability to review and use international research in determinations will guarantee the EPA is missing major findings and important data.
Commenter (12731) refers to comments submitted by Bernard Goldstein of the Environmental Protection Network for the SAB's August 2019 consultation. The commenter states that Goldstein's comments pointed to the example of a 2010 study on the effects of formaldehyde exposure that had 34 co-authors from seven different institutions and three different countries. The commenter states that Goldstein observed that sharing the underlying data from that study would potentially require the unanimous agreement of all 34 co-authors, many of which answer to different IRBs that must approve such sharing, and some of whom are subject to different privacy laws than those that apply in the United States. The commenter states that, as Goldstein also observed, "international studies with or without significant U.S. collaboration are becoming an increasing percentage of total environmental health research. The commenter suggests that one of the inevitable outcomes of EPA's proposal is a cumbersome barrier to being able to put together a team of international investigators for studies of humans that are potentially relevant to EPA's environmental regulations" The commenter states that neither the original proposal nor the SNPRM meaningfully considers this critical question and the implications that it poses for EPA's ability to consider regulatory science.
Commenter (12727) notes the possible limitation of use of international studies due to international barriers. The commenter references Bernard Goldstein's comments to the SAB, who pointed to the example of a 2010 study on the effects of formaldehyde exposure that had 34 co-authors from seven different institutions and three different countries. According to the commenter, Goldstein observed that sharing the underlying data from that study would potentially require the unanimous agreement of all 34 co-authors, many of which answer to different IRBs that must approve such sharing, and some of whom are subject to different privacy laws than those that apply in the United States. The commenter states, as Goldstein also observed, "international studies with or without significant U.S. collaboration are becoming an increasing percentage of total environmental health research. Isn't one of the inevitable outcomes of EPA's SNPRM a cumbersome barrier to being able to put together a team of international investigators for studies of humans that are potentially relevant to EPA's environmental regulations?" To the commenter, neither the Proposal nor the SNPRM meaningfully considers this critical question and the implications that it poses for EPA's ability to consider regulatory science.
Commenter (11395) reiterates the International Society for Environmental Epidemiology's comments on the Proposal that the Canadian Community Health Survey Cohort and the Evaluation Study of Congestive Heart Failure and Pulmonary Artery Catheterization Effectiveness (ESCAPE) are unlikely to meet the requirements of the rule due to stringent Canadian and European privacy laws and thus be excluded or devalued.,￼,￼￼ Failing to consider these and similar studies, according to the commenter, would limit the breadth and quality of information available to EPA for critical decisions.
Response: The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements.
Further, the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited.
<IP_ADDRESS> Implementation Issues
<IP_ADDRESS>.1 Breach of Contract
Comment: Commenter (11576) states that relationships with hospitals and other medical facilities rely on the confidentiality agreed upon by both researchers and the medical network. The commenter states that, in instances where the EPA is requiring private medical information to be disclosed in order for scientific studies to be used for decision-making purposes, the risk of breach of contract is not addressed in the SNPRM. The commenter states that such agreements cannot be easily renegotiated, especially for existing published studies and data. The commenter states that the alternative proposal provided in the SNPRM does not address situations where researchers would be contractually precluded from releasing confidential information and does not offer a solution for data that cannot be contractually released. The commenter states that, even under a tiered-access approach, the EPA could effectively ignore such data and the EPA fails to provide a solution for considering such studies and underlying data under the tiered-access scenario.
Response: When promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data, the Agency shall follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship.
The Agency shall also give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation. For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9. The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given.
<IP_ADDRESS>.2 Description of the Structure for Implementing the Rule
Comment: Commenters (11192, 11361, 11890, 11919, 12617, 12720) express concern that the proposal does not identify a structure within which to implement the rule.
Commenters (11890, 12720) contend the proposal should not move forward until it is updated with supporting analysis and reposted for public review.
Commenter (11890) states that it appears that the EPA is still gathering ideas for how the goals of the rule might be achieved. The commenter states that it is unclear whether the rule intends to create a passive system where the data owner only provides data if requested or an active system that will require submission and storage data being considered in a science assessment or rule. The commenter states that the proposal does not specify how the EPA proposes to screen studies that would be subject to the new requirements, determine who owns the data for a given study, and request access to data from authors or institutions. The commenter states that there is no structure given for how this new rule would make data models publicly available that are protected by intellectual property and possibly patent laws. The commenter states that the rule is also silent on how and where data would be stored and protected, how the data would be shared, with whom, how much this would cost, and who would pay for it.
Commenter (12720) states that because the proposal does not include consideration of the questions listed below, it should be withdrawn. The commenter asks:
 How will the EPA ensure that studies are considered in perpetuity if it does not constantly verify and re-verify that underlying data and models remain publicly available (e.g., if journal open-access policies were to change)?
 Who at the EPA will certify that a study's underlying data and models have been made "available" so as to satisfy the requirements of the rule?
 When will such a determination be made, since weeks and months can elapse between when a paper is accepted to when it is formally published?
 What contingency has the EPA planned in the case that there arises a need to verify that all (rather than a portion) of a study's underlying data has been made, and who at EPA would be qualified to make that determination?
 If requests for data access are granted, would study data be provided only to the requestor, or made available to a broader group?
 How would the EPA ensure that anyone with access to the data make appropriate use of it, and not share it with other individuals or groups?
Commenter (12720) refers to the 2020 SAB Letter to the Administrator which suggests a mechanism for interested researchers to access datasets for reanalysis. The commenter states that it is not clear who would staff such a committee and draft protocols. The commenter states that the protocols themselves are likely to conflict (or at the very least, intersect) with existing, signed agreements for many studies, including major cohort studies that involve large participation. The commenter states that, as a result, such a proposal for an added layer of federal scrutiny and approval over academic studies introduces unwieldy and impractical restrictions on already well-functioning systems. The commenter states that the proposal does not address these Commenter (12720) states that the EPA has not indicated whether or not the Agency would provide the underlying data or simply link to it. The commenter states that the EPA has not made it clear whether any Agency-initiated cleaning or transformation of the data would be publicly available.
Commenter (10042) urges the EPA to:
 Define, clarify and specify the limitations of transparency when used by the EPA.
 Determine who outside the EPA in the rulemaking process will have access to human health information and data and when in the process those non-EPA personnel will contribute to human health risk assessments.
 Codify its approach for how, when and under what circumstances it will use human health data and information in and for rulemaking and setting standards.
 Adopt and develop SOPs for managing human health data and information which will likely have a strong anonymization component that can or will be used to make human health risk assessments.
 Establish the criteria for revisiting previously conducted human health risk assessments for which human clinical health information and data were collected.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
<IP_ADDRESS> Consistent with the Law and Protects Privacy
Comment: Commenters (11387, 11480, 12457, 13788) express concern that the proposal's requirement to make data publicly available could be in violation of state and/or federal law.
Commenter (11387) contends that the proposed rule would require all studies used by EPA to have all their data publicly available on an internet website. The commenter states that putting study data on the internet could be in violation of the federal code that requires protection of confidentiality in human subjects research, as well as the HIPAA. Commenter states that, as highlighted in comments by the EPA Science Review Board, requiring public availability of data ignores legitimate confidentiality and privacy concerns.
Commenter (12457) states that the proposed revisions should include a detailed analysis of how this will impact patient privacy in health-related studies, what additional privacy safeguards will be implemented, and how the EPA will support transparency compliance for health data while also protecting privacy in accordance with HIPAA.
Commenter (13788) notes that the supporting private personal information that the EPA in the SNPRM proposes would have to be publicly released prior to EPA's reliance on public health studies is the kind of information that the EPA itself is generally prohibited from releasing, even when faced with a FOIA request. The commenter states that FOIA exempts "personnel and medical files and similar files the disclosure of which would constitute a clearly unwarranted invasion of personal privacy." The commenter states that, in the SNPRM, the EPA by contrast makes clear that the information subject to this rulemaking would include "[p]ersonnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy." The commenter states that, not only is this information exempt from FOIA, but Congress passed the Privacy Act of 1974 in order to prohibit agencies from releasing precisely this kind of information. The commenter states that the Privacy Act concerns the release of records, and the definition of "record" explicitly includes medical history. The commenter states that requiring public disclosure of information, where EPA itself is prohibited from disclosing it, as the threshold to allowing a study's use in regulatory decision-making, is not only unjustified, it is absurd.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11894) states that, in general under EPA practice, if a study is funded via any kind of assistance agreement, the recipient of the funding owns the data. The commenter states that the EPA lacks access to part or all of the data and models in many cases and does not have the authority to provide public access to part or all of the data and models. The commenter states that, if the EPA chooses to constrain its ability to the use of data and models only to those which are publicly available, the system will essentially shut down.
Response: This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make the underlying dose-response data available. Researchers may choose to make more data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
7.1.3 40 CFR 30.5: Option 1
<IP_ADDRESS> Support/Oppose Option 1
<IP_ADDRESS>.1 Support Option 1
Comment: Commenters (10973, 11397, 11401, 11499, 12397, 12410, 12612, 12692, 12712) express general support for the proposed Option 1.
Commenter (10973) states that the EPA's final rule should recognize evolving and historic open data access policies. The commenter states that open data access principles are increasingly, but not uniformly, an expected element of modern research. The commenter states that this is true especially when the research is sponsored by public sources or by prominent private foundations. The commenter states that there is likewise a growing expectation for open data access in the peer-review process and upon publication. The commenter states that, despite this trend, at present only a modest percentage scientific research articles are published in an open access format, and the publication of full data sets and models alongside articles is growing but small. The commenter states that different fields of study face different data constraints based upon privacy, logistical concerns, and other drivers. The commenter states that the "tiered access" model proposed in this version of the rule will help to alleviate some of these concerns.
Commenter (11397) agrees with the Option 1 approach because it provides clear direction to the Agency: do not use pivotal science unless the underlying data and models are available and allow for independent review and scrutiny. The commenter states that this is in keeping with Executive Orders 13777 and 13783, and it also eliminates gray areas and confusion that might exist if the EPA were merely directed by the rule to "give greater consideration" to publicly available data.
Commenter (11401) recommends that any study which has fully open public data can be considered the basis of a regulatory action, assuming of course, that the study supports the action that the EPA wishes to take. The commenter states that studies based on "CBI, proprietary data or PII with tiered access in a manner sufficient for independent validation" could likewise be considered foundational as a basis for regulatory action, but require more oversight and assurance that the system for allowing tiered access is robust.
Commenter (11499) supports the SNPRM's modified regulatory text that provides greater scope to consider studies in which underlying data can be reasonably available through tiered access so that CBI, proprietary data, or PII remains protected while still allowing sufficient opportunity for independent validation.
Commenter (12397) cites the Columbia River Inter-Tribal Fish Commission Consumption Survey (CRITFC 1994) that was conducted in 1991/1992. The commenter states that the calculated Feed Conversion Ratio (FCR) from this study has been the basis of FCR in the States of Washington and Oregon for a number of years. The commenter states that the data underlying the calculated FCR has never been made publicly available for the purposes trying to validate the "given" FCR. The commenter states that uncertainties associated with this study are significant. The commenter states that having this information publicly available would have enabled third parties (such as the regulated community) to understand how the FCR value was derived and to have done its own calculations as to an appropriate FCR, especially for specific tribes and states.
Commenter (12410) tiering access based on clearance levels seems like an appropriate measure to limit exposure of sensitive data. Commenter (12612) states that the tiered approach to data access will increase the functionality of the rule in increasing EPA transparency.
Commenter (12712) states that, with EPA regulations imposing such profound, significant harm to human health, their benefits must be verifiable by independent outside researchers. The commenter supports the modified transparency rule but only on the understanding that: no significant regulatory decision will be made on the basis of pivotal scientific studies that do not at the least ensure what the modified rule calls tiered access to the data, where such tiering is consistent with applicable privacy statutes; however, every attempt will be made to de-identify data to permit wider access to the data. The commenter states that, if tiered access to the data underlying a study cannot for whatever reason be granted to outside researchers, the EPA will not rely on the study for any significant regulatory decision.
Response: The EPA acknowledges and agrees with commenters that there may be pivotal science where the underlying data are not publicly available or available through restricted access. The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
<IP_ADDRESS>.2 Oppose Option 1
Comment: Commenter (9587) states that the EPA's proposed modification gives claims of privacy and/or confidentiality priority over transparency requirements. The commenter states that this modification practically will impose substantial limits on the effect of the rule since privacy, confidentiality, and tiered access are all concepts and practices that inhibit full transparency. The commenter endorses the previous, unmodified transparency requirements, and states that transparency should take priority over privacy and confidentiality in EPA policymaking.
Commenter (11182) asserts that Options 1 renders the SNPRM unfeasible, unethical, and possibly illegal.
Commenter (11500) urges the EPA to finalize the proposal under which unavailable data will not be used. The commenter states that public access to data and models is necessary not only for independent validation, but also for independent scrutiny of the data and methodologies. The commenter states that, if such materials are unavailable, the public is denied a meaningful opportunity for comment.
Commenters (10853, 11354, 11404, 12463) generally oppose proposed Option 1.
Commenter (10853) states that increasing transparency in science is an important goal that the EPA could more productively support by funding and creating data-sharing infrastructure that researchers from all sectors can use. The commenter states that the approaches in the proposed rule are too far downstream in the scientific process; unduly prescriptive; likely to increase costs and reduce supply of useful science; and likely to introduce bias and inefficiency in EPA's process of evaluating and developing regulatory science. Commenter (11404) states that the SNPRM's modified approach of tiered access is insufficient to address privacy concerns.
Response: The EPA disagrees with commenters that the requirements of this rule will result in any meaningful delay in promulgating regulations. While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review). Further, with this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes that the EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
<IP_ADDRESS>.3 Revision of Option 1
Comment: Commenter (12471) suggests that, if the EPA finalizes Option 1, the following modification is offered:
      When promulgating significant regulatory decisions or finalizing influential scientific information, the Agency will only use pivotal regulatory science and/or pivotal science that includes studies with restricted data and models (i.e., those that include confidential business information (CBI), proprietary data, or Personally Identifiable Information (PII) that cannot be sufficiently de-identified to protect the data subjects) if there is tiered access to these data and models in a manner sufficient for independent validation, and studies that do not include restricted data and models if the data and models are publicly available in a manner sufficient for independent validation.
Response: The EPA agrees with commenters that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (12692) states that it may be many years before a significant fraction of the studies on which EPA relies is independently validated. The commenter suggests that, until studies are independently validated, the EPA should follow Lutter and Zorn's recommendation to former House science chairman Lamar Smith, whose HONEST Act (H.R. 1430) is the legislative inspiration for the Transparency proposal:
      The bill should also allow agencies to regulate in instances where they do not possess data. Specifically, agencies may rely on research published in peer-reviewed journals, even if there is not public access to such data and the agency cannot acquire it, provided that the agency states it has unsuccessfully sought the data under an agreement providing for privacy of human subjects and protecting trade secrets. In this case the agency, however, would have to state that it is using research based on non-public data and explain why it believes the research is nonetheless sufficiently reliable to be used for regulation.
Response: Since the purpose of 40 CFR 30.5 is to determine the consideration to afford to studies based on, among other factors, the availability of the underlying dose-response data that would support independent validation via reanalysis of the data underlying pivotal science, the appropriate stage of data would not be the processed data (data that have been computed and analyzed to extract relevant information) or the final clean data set that is provided with a publication. At these two stages of data, the analysis has already been conducted, and the results have already been determined. In order to determine if these results are valid, data that had not already been computed and analyzed are needed.
In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
<IP_ADDRESS> Definitions
Comment: Commenter (11890) states that, in the SNPRM, new terms are introduced that require more explanation and transparency. The commenter states that the language "appropriate techniques for tiered access" is used, yet "appropriate techniques" are not defined.
Commenters (9587, 11576, 11890, 11894) express concern that "tiered access" is not defined.
Commenter (9587) asserts that the EPA should define tiered access precisely, to maximize transparency. The commenter states that the EPA should scrutinize its regulatory models carefully, and make sure that its own definition of tiered access defines all terms with absolute clarity, provides equally clear procedures designed to maximize transparency, and facilitates reproduction studies.
Response: Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
<IP_ADDRESS> Data That Cannot Be Sufficiently De-identified
Comment: Commenters (9455, 10866, 11361, 11395, 11473, 11576, 11890, 12403, 12546, 12720) contend that the tiered structure presented in the SNPRM does not adequately address concerns about re-identification of study participants, disclosure of private health data, and/or negative effects on future study participation, as described below.
Commenter (9455) asserts the proposed rule's system of tiered access would violate medical confidentiality laws and standards because it only reduces the risk of re-identification and it allows, illegally and unethically, sharing of confidential data. The commenter states that posting into the system imposes risk in ways that have not been properly evaluated in the proposed rule. The commenter states that, as the tiered program could only "reduce the risk of reidentification and mitigate disclosure privacy risks," it would break laws on privacy and standards of medical confidentiality. The commenter states that it would make all the data sitting ducks for hackers.
Commenter (10866) asserts that the proposed tiered access strategy for consideration of protected health information does not adequately protect the identity and health information of participants included in research studies. Commenter (11890) states that the rule must recognize that the re-identification of study participants is possible even when stripped of the variables typically considered PII (e.g., under HIPAA).
Commenter (11395) is concerned about the potential for re-identification of study participants in the tiered access structure. The commenter states that, because environmental studies by necessity need to use location data to establish exposures, a very significant percentage of study participants can be identified unless the data are masked to a degree that would not allow reanalysis. The commenter states that, recently, a peer reviewed study examined the identifiability of records from an environmental health study in northern California. The commenter states that, using data considered by the HIPAA to be sufficiently de-identified to be made public and which involved far fewer variables than would be required for the reanalysis proposed by EPA, the researchers were able to correctly identify over 25 percent of the study participants. The commenter also refers to the 2002 National Academy of Sciences reports which states that:
      In an experiment to discover whether confidentiality could be preserved while opening the data for public review, the study investigators attempted to disguise the identity of the study participants. They deleted as many features as possible from the questionnaires, such as the name, the state file number, the mother's maiden name, and the name of the person providing the information. However, they needed to retain a minimum set of features if other scientists were to be able to replicate the basic findings of the study... They found that even this minimum set of features could allow for identification of research participants.
Commenter (11473) asserts that the new "tiered" approach does not prevent loss of privacy. The commenter refers to an editorial published in the Lancet (Thorp et al., 2019) and signed by Editors of Science, Nature, PLOS, PNAS, Cell, and Lancet:
      As leaders of peer-reviewed journals, we support open sharing of research data, but we also recognise the validity of scientific studies that, for confidentiality reasons, cannot indiscriminately share absolutely all data. Datasets featuring personal identifiers -- including studies evaluating genomes of thousands of people to characterise medically relevant genetic variants -- are but one example. Such data may be critical to developing new drugs or diagnostic tools but cannot be shared openly; even anonymised personal data can be subject to reidentification (e.g., Sweeney, 2013; De Montjoye et al., 2015; Rocher et al., 2019) (attached), and it has been a longstanding practice for agencies and journals to acknowledge the value of data privacy adjustments. The principles of careful data management, as they inform medicine, are just as applicable to data regarding environmental influences on public health. Discounting evidence from the decision-making process on the basis that some data are confidential runs counter to the EPA stated mission `to reduce environmental risks...based on the best available scientific information'."(USEPA, 2016).
Commenter (11576) states that the supplemental notice admits that under a tiered approach, the EPA will not fully protect confidential information. The commenter states that, while the EPA claims that its tiered-access approach will allow for a reduction in the risk of re-identification, it cannot guarantee that it can fully prevent all such data from being reidentified. The commenter states that an analysis published in 2019 assessed the risks associated with the sharing of data within environmental-health studies, including genetic and medical records, and found an increased likelihood of re-identification. The commenter states that there is little discussion as to the criteria the EPA will rely upon to develop such an approach and even then, the EPA can only guarantee a reduction of risk -- which means there would remain individuals whose private medical information is at risk of being disclosed.
Commenter (11926) states that there are substantial risks involved in sharing of individual level research data, especially if data are available to anyone in a public release. The commenter states that enormous technological advances have been made in data linkage techniques, and substantial resources have been devoted to data linkage as businesses, government agencies and sophisticated rogue hacking groups seek to find out as much as possible about as many people as possible.
Commenter (12546) states that researchers in 2015 were able to reidentify 90 percent of their study population as unique individuals and to uncover their records, knowing just four random pieces of information, due to the uniqueness of human behavior. The commenter states that a more recent study in 2019 modeled the likelihood of individual reidentification in a "heavily incomplete" anonymized dataset, and found "that 99.98% of Americans would be correctly re-identified in any dataset using 15 demographic attributes." The commenter states that this research shows how, with only a few pieces of information, current technology can already deidentify data that scientists currently consider sufficiently anonymized.
Commenter (12720) expresses concern that the EPA does not define a level to which study data can be "sufficiently" de-identified in the proposal, which leaves this determination open to arbitrary influence. The commenter states that this description of a tiered access system is vague in not defining a "sufficient" level of de-identification. The commenter states that the EPA has not submitted any specific plan for such a determination to be made, nor has any independent group evaluated such an approach.
Commenters (9455, 11361) express concern that the tiering approach may not be consistent with IRB requirements. Commenter (9455) states that the possibility that medical data, even if ostensibly de-identified, will be posted for review would never get past any good IRB. The commenter states that the requirements of various privacy protection laws and of medical ethics preclude such posting. The commenter states that IRBs enforce these requirements in their reviews of grant proposals. The commenter states that most federal agencies that fund health research, including EPA and National Institute of Environmental Health Sciences (NIEHS), require strict medical confidentiality, and IRBs make sure that all grant proposals ensure confidentiality.
Commenter (11186) states that, while the rule discusses risk reduction techniques and creating multiple versions of a single dataset with varying levels of specificity and protection, in fact, creating more datasets can involve more risk of reidentification and breaches of confidentiality.
Commenters (11387, 12404, 12423) assert that health studies of environmental exposures require sufficient location information to determine exposure, and that this location information makes it impossible to adequately de-identify health information. Commenter (12404) states that this is particularly true in some air pollution studies, in which participants' addresses are used to estimate exposure.
Commenter (11387) contends that the requirement that all study data be available on an internet website, means it would always be possible to use diagnosis and location information to figure out the identities of one or more people in the study. The commenter states that this violates the person's confidentiality rights, and means the study cannot be used (if it was completed in the past) or conducted (if it is a proposed future study).
Commenter (12423) asserts that the small populations of most tribes and tribal communities and their restricted geographic locations (i.e. specific reservations and villages) make it impossible to sufficiently de-identify raw data. The commenter states that tiered access does not alleviate this issue. The commenter states that, under the proposed rule, these studies would either be excluded or heavily discounted. The commenter states that this scenario would be played out hundreds to thousands of times throughout Indian Country. The commenter states that it should be noted that rural America in general shares some features with tribes, such as substandard housing, exposure to dust from unpaved roads, poor healthcare infrastructure, greater number of businesses that are below a range of federal statute threshold reporting levels and are more likely to rely on water systems and private wells unregulated by the Clean Water Act (CWA) and Safe Drinking Water Act (SDWA). The commenter states that, for example, there are 229 Alaska tribes living in remote and isolated villages averaging just 330 people. The commenter states that they are unconnected by road to any other populations, so that any community's geographic, hydrologic, and geological features relevant to participant selection criteria could unmask the tribe.
Commenters (9455, 11576, 12404, 12423, 12464) contend that certain studies show that it is difficult to make data public without also making participants identifiable.
Commenter (9455) contends that there is no way to completely ensure that re-identification will not happen. The commenter states that the references in Boronow demonstrate that the environmental health research community has futilely grappled with this problem for many years.
Commenter (11576) refers to a study which discusses how even limited information made available from multiple studies involving the same dataset allowed researchers to successfully re-identify study participants. The commenter states that researchers in the study were able to re-identify study participants despite the exclusion and redaction of information that could not be shared under the safe-harbor provision of HIPAA. The commenter concludes that the EPA's proposal poses significant threats to personal privacy -- threats the EPA has yet to acknowledge and address. The commenter states that, while the EPA claims that its tiered-access approach will allow for a reduction in the risk of re-identification, it cannot guarantee that it can fully prevent all such data from being reidentified. The commenter states that there is little discussion as to the criteria the EPA will rely upon to develop such an approach and even then, the EPA can only guarantee a reduction of risk -- which means there would remain individuals whose private medical information is at risk of being disclosed.
Commenter (12404) states that it is difficult to make data public without also making participants identifiable. The commenter states that this is particularly true in some air pollution studies, in which participants' addresses are used to estimate exposure. The commenter states that this means, that in some cases, information about participants' exposures could be used to figure out their addresses. The commenter states that a 2002 study by the National Research Council of the National Academy of Sciences found that, even after the deletion of some personal information from participants, if all the information necessary to reproduce a study's findings remained, it was possible to identify the participants.
Commenter (12423) asserts that the small populations of most tribes and tribal communities and their restricted geographic locations make it impossible to sufficiently de-identify raw data. The commenter states that, for example, a retrospective longitudinal cohort study was carried out in 2001 to examine whether there was a significant association between a range of short-term health risks and the surrogate inhalation exposures to solid waste in four Alaska Native villages. The commenter states that information was collected from 95 percent of residents, including education level, tobacco use, honey bucket use (plumbing), self-reported asthma, self-reported short-term health symptoms, and approximate total subsistence food consumption. The commenter states that, in another study conducted in 2006, and published in the American Journal of Epidemiology, birth records were used to examine the association between hazardous waste content in Alaska Village landfills and a number of birth outcomes, including congenital anomalies. The commenter states that these two studies are seminal in that they are the first and only systematic studies carried out on impacts in rural Alaska Native villages. The commenter states that they have since informed EPA policy, as well as delegated state Resource Conservation and Recovery Act (RCRA) implementation and policy. The commenter states that a substantial portion of results pertain as well to the rest of rural Alaska communities, representing in total about half of the state's population. The commenter states that, in the case of the retrospective cohort study, obtaining participant consent 19 years later is not possible. The commenter states that, in the case of the second study, the use of the Alaska Birth Record Registry was conditioned upon strict confidentiality and scrubbing procedures because of the medical records and sensitive information contained. The commenter states that, with communities as small as 50 people and just a single annual birth, it would be straightforward to identify individuals and individual medical records. The commenter states that permission from the State to modify the research agreement and release the data would not be granted.
Commenter (12464) states that recent research suggests that the risk of re-identification for environmental health data is greater than traditionally thought. The commenter states that a recent paper demonstrates that it is possible to successfully re-identify environmental health study participants using information that is publicly or commercially available. The commenter states that, if data with geographic information about the study participants becomes public, that data can be combined with information that is readily available online such as voter lists, tax and real estate information, or information that is available from data brokers. The commenter states that, in light of the risk of re-identification with environmental health data, researchers would have a strong interest in ensuring the data is restricted and not publicly available. The commenter asks if researchers are able to make this determination or will the EPA demand that certain studies be made public? The commenter asks if the EPA will decide, what process will be available if the researchers and the EPA disagree about the risk of re-identification?
Commenter (11576) states that, in the original notice, the EPA mentioned "[o]ther federal agencies [that] have developed tools and methods to de-identify private information for a variety of disciplines" as examples of approaches to de-identify and protect confidential medical information. The commenter states that, however, the Health and Human Services guidance referenced by the EPA acknowledges that de-identification does not fully protect patient information, stating that "[b]oth methods, even when properly applied, yield de identified data that retains some risk of identification. The commenter states that, although the risk is very small, it is not zero, and there is a possibility that de-identified data could be linked back to the identity of the patient to which it corresponds." The commenter states that, as commenters noted in 2018, de-identifying personal information has thus far proven to be ineffective. The commenter states that the SNPRM does not offer a resolution that would fully prevent de-identified information from being re-identified.
Response: Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA follows an objective, unbiased process for identifying and evaluating scientific studies and already identifies key or pivotal studies in some of its actions (e.g., Integrated Risk Information System (IRIS) assessments). The EPA intends to issue implementation guidelines and statute-specific rulemakings that will further describe these criteria and how the EPA will identify pivotal science in its assessments and rulemakings.
The EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
<IP_ADDRESS> Exclusion or Discounting of Data and Studies in Tiered System
Comment: Commenters (11391, 11479, 12404, 12688, 12731) express concern that, under Option 1, the EPA would not prevent important and relevant scientific information from being disregarded.
Commenter (11391) states that, as scientists discussing the feasibility and usefulness of a tiered or controlled-access approach have pointed out, "controlled access does not permit researchers to curtail national legal and ethical policies on privacy and data protection." The commenter states that, while the EPA seems to suggest that in many instances privacy issues can be sufficiently addressed through de-identification, the EPA's own SAB has concluded that "even de-identified datasets present risks of re-identification." The commenter states that, as such, there are some circumstances where scientists simply cannot share data, even if they promise only to share it with a select group of people through a tiered access regime. The commenter states that, even under Option 1, then, the EPA would still be forced to disregard large amounts of valuable scientific information where a tiered or controlled-access approach is not legally or ethically permitted.
Commenter (11479) states that the tiered-access approach fails to address a central concern -- that it would require the release of private data for EPA to be able to use studies that rely on such data. The commenter states that, under a tiered-access approach, which would allow select researchers to gain access to sensitive data, the vast majority of studies that rely on nonpublic data  -  data such as personally identifying information, CBI, and intellectual property  -  would still be prohibited from use by EPA. The commenter states that this is because restrictions on disclosure of such data (as detailed in the next section) would still apply, even when such data are released to a select few researchers in a tiered-access system. The commenter states that, for example, the EPA relies on many studies that use personally identifying health data. The commenter states that, to use such data, researchers obtain ethics approval from IRBs and sign legal agreements with hospitals, health maintenance organizations, and other bodies that prevent them from sharing health data publicly. The commenter states that these legal and ethical obligations still apply, even if a tiered-access system exists. The commenter states that, thus, sharing health data in a tiered-access system would require researchers to breech legal and ethical agreements, as described in detail in the next section. The commenter states that researchers would be unlikely to participate in such a system if their research relies on sensitive data, consequently, the EPA will not be able to use much relevant research relying on nonpublic data, even if a tiered-access system is offered.
Commenter (11390) states that, while the EPA's revised proposal contemplates that "tiered access," such as restricted access to confidential data for independent validation by "authorized researchers," would allow research institutions and researchers to authorize restricted access. The commenter states, however, this requirement is not consistent with many contractual agreements with research subjects, and will be infeasible or impossible to implement both retroactively and prospectively in a way that allows the Agency to rely on high-quality science.
Commenter (11498) states that this restricted access approach presupposes permission and participation from the scientists doing the research. The commenter states that, since the EPA does not have the authority to grant access to confidential and private research data, this approach would depend on the consent and approval of the scientists and institutions who conducted the research. The commenter states that it is unclear if this is at all possible, as researchers and institutions, in the example of many groundbreaking epidemiological studies, enter into contracts with participants to keep their health and other sensitive information private. The commenter states that it is improbable the researchers would or could alter these legal contracts after studies are concluded to allow individuals selected by EPA to gain access to such protected information, even in a tiered scheme. The commenter states that, thus, this approach would likely exclude many credible and valuable studies, including ones containing private information that EPA has benefited from in the past. The commenter states that this reinforces our ongoing serious concern that this proposal threatens the use of the best available science in its decision-making.
Commenter (12404) states that the tiered approach would permit data that cannot be made broadly public to be shared with a limited number of scientists. The commenter states that, if this proposal were to become a final rule, it is possible to imagine that researchers might seek and obtain informed consent from study participants to do this kind of sharing of identifiable information. The commenter states that, if they did, it might be more difficult to find willing study participants, especially in communities of color, which have good historical reasons to distrust medical researchers and which are often the most heavily exposed to pollution. The commenter states that, even if researchers were able to seek and obtain consent for this kind of limited sharing in future studies, and even if doing so did not reduce study participation in communities of color or elsewhere, this rule would still exclude a tremendous amount of scientific research conducted before there was any reason to obtain such consent.
Commenter (12688) states that Option 1, which would require consent and approval from the researchers and institutions that collect data, ignores the realities of public health research involving human subjects, which often necessarily involve contractual arrangements with subjects to protect privacy and confidentiality. The commenter states that it is certain that many past studies would not be amenable to "tiered access" to data, and this requirement would inevitably impede the use of many future studies to inform Agency actions as well.
Commenter (12731) states that, in altering its proposed approach, the EPA purports to address objections to the Agency's exclusion of valid data and models containing confidential, proprietary, or personal data that cannot be sufficiently de-identified to protect data subjects. The commenter states that the EPA's tiered access approach is an inadequate response to these objections. The commenter states that, if tiered access does not exist, for whatever reason, the Agency will ignore all studies for which data and models are not publicly available. The commenter states that, thus, the tiered access modification to the original proposal fails to cure its essential defect: the Agency would continue to disregard valid studies without any rational grounding in its statutory authorities, and in plain violation of numerous statutory requirements.
Commenter (11500) states that, when excluding materials unavailable in a manner sufficient for independent validation, the Agency should clearly identify the materials that have been excluded and explain the precise bases for exclusion.
Response: Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA follows an objective, unbiased process for identifying and evaluating scientific studies and already identifies key or pivotal studies in some of its actions (e.g., IRIS assessments). The EPA intends to issue implementation guidelines and statute-specific rulemakings that will further describe these criteria and how the EPA will identify pivotal science in its assessments and rulemakings.
The EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
<IP_ADDRESS> Research Data Centers
Comment: Commenter (11392) notes that the SNPRM references a pilot project with the Centers for Disease Control and Prevention (CDC) involving a secure data enclave for sensitive EPA datasets. The commenter asks if the CDC was involved in writing this rule, do they have the resources to host the EPA's data and models and moderate tiered access, will the CDC be making the decision as to who can access the data and at what level, and will CDC be reviewing applications for access? The commenter states that the SNPRM does not address any of these questions. The commenter states that, if the EPA intends to make its arrangement with the CDC permanent, and establish the CDC as the host of a secure data enclave for studies considered in Agency rulemaking, the EPA must clearly articulate its rationale for partnering with a separate federal agency rather than performing the work internally. The commenter states that the EPA must also describe how such an unusual arrangement would be structured; it must be clear that the CDC is involved in this rulemaking.
Response: Proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of "research data." In the 2020 SNPRM, the EPA deleted the proposed definition of "research data." The EPA is not including a definition of "research data" in this final rule given that it is finalizing the definition of "data."
Comment: Commenters (11479, 12435, 12710, 12720, 12731) assert that the proposed model of tiered access for data involving and the precedents the SNPRM cites for it, are inappropriate to the work of the EPA.
Commenter (11479) states that the EPA's SNPRM points to the CDC's Research Data Center (RDC) as a model, and the EPA has referenced other practices by CDC and the U.S. Food and Drug Administration (FDA) as relevant models for the Agency to follow in implementing a tiered-access approach. The commenter states that there are significant reasons that these CDC and FDA systems will not work for EPA. The commenter states that, unlike CDC and FDA, the EPA is primarily focused on environmental threats, and deidentification of personal health data is challenging and sometimes impossible, as noted above. The commenter states that, by contrast, data from clinical drug trials used by FDA and medical data aggregated at the hospital or county level used by CDC can often more readily be disclosed publicly, because doing so will not reveal identities of study subjects.
Commenter (12435) states that the types of studies that inform the FDA are fundamentally different from those that inform the work of the EPA. The commenter states that the RDC of the CDC is not easily or affordably accessible and does not provide the raw data access that this rule demands.
Commenter (12710) asserts that the model for tiered access cited in the preamble (from the RDC within CDC), is relevant only to data owned by the federal government. The commenter states that no model exists, nor does the EPA propose one, for tiered access to studies generated outside the federal government (e.g., states, universities, non-governmental organizations (NGOs)).
Commenter (12720) states that the EPA does not explain why the RDC approach would work in the EPA context and the wide scope of scientific data and models that would be subject to new public data release requirements under the proposed rule. Commenter (12731) asserts that the federal tiered-access models the EPA has pointed to are relevant only to "a small fraction of published epidemiology research" and that these platforms do not address access to datasets held by universities or by private organizations like the American Cancer Society.
Commenter (12731) states that the closest the EPA comes to describing its envisioned tiered access approach is a brief reference to the RDC. The commenter states that the EPA's reference to the RDC is wholly inappropriate, as the data features are entirely different from the scope of scientific information the EPA is capturing in the SNPRM. The commenter states that the data housed within the RDC are either developed directly by the federal government or collected through federal government- designed and managed programs. The commenter states that these data are highly standardized, and the platforms housing the data are developed and managed by the government. The commenter states that the RDC is a highly controlled, integrated, and coordinated data infrastructure. The commenter states that, in contrast, the EPA's proposal targets all potential "data and models" to be considered by the Agency that are developed by a wide breadth of entities inside and outside government and that are highly variable including in terms of how they are generated, collected, curated, and stored. The commenter states that any reference to the RDC as an analogous potential model for EPA's tiered access approach is highly inappropriate and disingenuous.
Commenter (12720) states that the EPA does not grapple with the fact that the RDC restricted-use data is not "publicly available" in that there are multiple barriers to data access.
Response: This rule is designed to build upon prior EPA actions in response to government-wide data access and sharing policies. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator (see Section III.G). See Section III.E for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
Further, proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of "research data." In the 2020 SNPRM, the EPA deleted the proposed definition of "research data." The EPA is not including a definition of "research data" in this final rule given that it is finalizing the definition of "data."
Comment: Commenters (10038, 11176) support the concept of tiered access to data in the current draft of the rule and for piloting the use of the RDC of the CDC National Center for Health Statistics (NCHS).
Commenter (11176) states that the NCHS RDC is one example for providing some level of tiered access. The commenter states that the NCHS also, along with more than a dozen other federal entities, provides data access through Federal Statistical Research Data Centers (FSRDCs). The commenter states that the EPA actually provides access to its Pollution Abatement Costs and Expenditures (PACE) survey through the FSRDCs. The commenter states that one potential limitation of the FSRDC network is that work conducted therein must be for a statistical purpose and must be for research as well as the following stipulation: "An agency with regulatory or enforcement roles that also conducts or supports research may be able to house their data in RDCs if there is a clear delineation between enforcement and research in the agency. Further, the mission of RDCs is to facilitate research." The commenter states that other existing access methods may provide additional flexibility and are actively being used by federal agencies. The commenter states that two such access examples are the NORC Data Enclave and the Inter-university Consortium for Political and Social Research (ICPSR) Virtual Data Enclave. The commenter states that the former is used by the two United States Department of Agriculture (USDA) statistical agencies, for example, and the latter by the Bureau of Justice Statistics. The commenter states that, if more restricted access is needed, programs such as the Bureau of Economic Analysis' Special Sworn Researcher Program provides access to highly confidential firm data, under tight security. The commenter states that, while the EPA is actively working to provide more access to its data, its involvement in the sharing of its sensitive data through research data centers and data enclaves is minimal, limited to a single survey to our knowledge.