Response: The EPA disagrees with the commenter that its analyses are often political or grounded in false information. The EPA performs objective, high-quality analyses using the best available science. Through this rule, and consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.[,] When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E of the final rule preamble).
EPA agrees that relying on measures beyond peer review can increase transparency. However, EPA notes that peer review is still a valuable process. As noted in EPA's Peer Review Handbook (4[th] Edition): "Peer review is conducted to ensure that activities are technically defensible, competently performed, properly documented and consistent with established quality criteria. Peer review is an in-depth assessment of the assumptions, calculations, extrapolations, alternate interpretations, methodology, acceptance criteria and conclusions pertaining to the scientific or technical work product, and of the documentation that supports them." Peer review does serve to increase the credibility of a work product. This is why the final rule at 40 CFR 30.6 requires the EPA to conduct independent peer review consistent with the requirements of the Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review and the exemptions described therein, and authorizes the EPA to evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review.
Finally, EPA disagrees with the commenter that this rule is needed to convince funders of scientific studies to change policies restricting data disclosure that are consistent with laws governing privacy and intellectual property. Furthermore, this is an internal rule of agency procedure and not intended to impose requirements or burdens upon the public.
<IP_ADDRESS> Are Studies More Valid If Data Publicly Available?
Comment: Commenters (6133, 9227) assert that EPA provides no basis for its assumption that science or studies for which data are publicly available yield more valid or reliable results than the best available, peer-reviewed, independent, credible science, for which the underlying data are not publicly available. Commenter (6133) states that neither the Proposal nor docket contains any factual, scientific, technical, logical, or legal support for the suggestion that science and data that are "publicly available in a manner sufficient for independent validation" are necessary elements for the "validity," "reliability," or "transparency" of scientific information.
Commenter (9227) states that the EPA irrationally conflates scientific "validity" and "transparency" with data availability, incorrectly assuming that eliminating the use of studies without publicly available data will improve scientific validity and transparency. The commenter states that, in the preamble to the proposed rule, EPA states that the intent of the regulation is "to strengthen the transparency of EPA regulatory science." The commenter states that, later in the preamble, EPA states: "[e]nhancing the transparency and validity of the scientific information relied upon by EPA strengthens the integrity of EPA's regulatory actions and its obligation to ensure the Agency is not arbitrary in its conclusions." The commenter states that EPA then leaps to the unexplained conclusion that barring the use of studies without publicly available data will enhance transparency and validity. The commenter states that EPA's assumption that data availability (or "transparency" in the form of data availability) ensures the use of valid science or its equivalent to using the best available science is manifestly incorrect, and hence provides an irrational basis for the proposed rule. The commenter states that neither data availability in particular, nor transparency in general, is equivalent to or a guarantee of "validity" in scientific studies.
Commenter (6133) states that the Proposal arbitrarily fails to address, much less explain, why prior EPA regulatory actions that relied upon studies, data, or other information did not reflect the "best available science" or why they were otherwise unreliable, despite failing to meet the Proposal's standards.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to national ambient air quality standards (NAAQS), risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines (IQG) or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
<IP_ADDRESS> The Proposal is not an Exclusionary Rule
Comment: Commenter (6179) states that some have mischaracterized the Proposal as an exclusionary rule that would bar EPA from considering studies where underlying information cannot be made public. The commenter states that a plain reading of the Proposal's specific regulatory language shows no such requirement. The commenter asserts that the Proposal states that: (1) EPA should generally release regulatory science where practicable; (2) EPA shall ensure certain information underlying studies cited in significant rulemaking is made publicly available; and, (3) the Administrator has authority to exempt studies cited in significant rulemakings containing certain information, including that which is private, sensitive, or confidential, from these requirements. The commenter states that these are reasonable steps to improve the evidentiary basis for EPA's regulatory policies and improve regulatory outcomes by targeting resources to where they can achieve the largest benefits. The commenter states that it would be unsound policy for EPA to summarily exclude science from regulatory consideration  -  and the Proposal does not do so. The commenter states that the proposal presumes proactive management of private, sensitive, or confidential information and only requires disclosure of dose response data and models in appropriate circumstances. The commenter states that, rather than rigidly excluding important studies, the Proposal provides a flexible, case-by-case exemption process.
Commenter (6179) states that the Proposal is not, and does not need to be, an exclusionary rule in order to accomplish its underlying goal of informing public discourse on significant regulations. The commenter asserts that the rule can be implemented to protect private, sensitive, or confidential information while still seeking maximum possible transparency. According to the commenter, the presence of private, sensitive, or confidential information does not necessarily make disclosure of dose response data and models impossible.
Commenter (1302) states that many have objected that this transparency ruling would undermine the ability to collect relevant data. The commenter notes that the ruling specifically allows the Agency to make data or models available in a manner that is consistent with law, protects privacy, confidentiality, CBICBI, and is sensitive to national and homeland security. The commenter adds that the data made publicly available are only those required to enable "independent validation," that is that the data provided is sufficient "for the public to understand, assess, and replicate findings." According to the commenter, since such information does not (except perhaps in extraordinary circumstances) include the identity of subjects, it is difficult to imagine how confidentiality agreements would be violated by disclosing information regarding anonymous subjects.
Commenter (0563) suggests that §30.5 assures that the Agency, in making data or models publicly available, would do so in a fashion that is consistent with law, protects privacy, confidentiality, CBICBI, and is sensitive to national and homeland security. The commenter adds that §30.9 allows the Administrator to grant exemptions on a case-by-case basis if compliance is impracticable. The commenter concludes that the rule would not prevent the EPA from using studies that involved confidential information, such as personal health data or corporate proprietary information.
Response: The EPA agrees that this final rule is not an exclusionary rule. The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
Comment: Commenter (6376) suggests that EPA should establish guidelines to ensure transparency does not come at the cost of widespread exclusion due to inadequate protections for the variety of sensitive data included in different pertinent studies. The commenter states that scientific studies involving or including sensitive health data should not be at risk of exclusion over concerns related to personal information being exposed.
Response: The EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Therefore, the EPA is not finalizing the primary proposal in the 2020 SNPRM that would have categorically required that for studies to be considered pivotal science the underlying data would need to be available for independent validation. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science.
Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
<IP_ADDRESS> EPA Would Exclude Studies and Other Valid Data from Regulatory Decision-Making
<IP_ADDRESS>.1 EPA Should Identify Studies that Would Be Barred
Comment: Commenter (6125) expresses concern that the proposal says nothing about what studies it will exclude, aside from those identified but not mentioned by name in footnote 3 of the preamble. The commenter requests that EPA should identify the key studies that its proposal would bar that have been used to support regulations in the past and evaluate the impact of their loss on those regulations and thereby on human health and the environment.
Response: Under this final rule, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
<IP_ADDRESS>.2 Best Available Science
Comment: Commenters (1012, 1433, 1796, 1945, 1973, 1994, 2170, 2180, 2182, 2190, 2249, 2335, 2425, 2746, 2911, 3135, 4595, 4827, 4828, 4845, 4877, 5115, 5131, 5176, 5178, 5180, 5981, 6116, 6122, 6125, 6128, 6133, 6134, 6137, 6153, 6157, 6181, 6194, 6195, 6355, 6368, 6373, 6873, 6882, 6895, 6906, 6909, 6932, 6942, 8272, 8282-PH10, 8281-PH38, 9225, 9227) oppose this proposed rule since it could undermine EPA's ability to use studies that represent high-quality studies/best-available science and thereby potentially leading to weakening protections for public health and the environment. Comments include:
 Commenter (6373) asserts that EPA must consider the "best available science" to guide policies and practices and should avoid eliminating existing data and standards until such time as new and better information is available, regardless of the source of the existing data.
 Commenter (6932) states that risk assessment processes can be improved and recommendations have been issued in reports like the 2014 NAS report, but limiting the types of studies that can be used is not the way to improve them. The commenter notes that the best available science means that EPA should use all scientific studies  -  especially independent science from peer-reviewed journals-- available in order to inform its decisions.
 Commenter (6125) What justification is there to impose such an absolute ban on EPA studies? Would the ban extend to using research developed and relied on by other agencies?
 Commenter (6116) states that EPA does not explain how its proposed approach will ensure that it relies on the "best available science" which is an oft-repeated standard in various environmental laws. The commenter states that it is possible that EPA's proposed regulation will actually result in a failure to rely on the "best available science" as EPA will arbitrarily disqualify important scientific studies if the underlying data cannot be disclosed to the public and/or cannot be independently validated.
 Commenter (6137) states that EPA has provided no scientific justification for ignoring an entire class of health science simply because the underlying data has not been disclosed. The commenter adds that EPA has not demonstrated that it is consistent with scientific principles to categorically exclude peer-reviewed scientific information from all consideration in a rulemaking, as EPA proposes to do. The commenter states that, if EPA has any doubts or concerns regarding the merits of a particular study, it must address those doubts or concerns for that particular study in the context of a given rulemaking, where agency staff, internal scientific experts, scientific advisory committees, or commenters contend that study is relevant. The commenter notes that, whether underlying data on which a study relies is made public or not simply has no bearing on whether a scientific study is good science, is accurate, is reliable, and is relevant to a scientific question (such as the health effects of air pollution).
 Commenter (2911) states that the only criterion that should be used in deciding whether to use the information from a scientific study is whether the study is valid or not. Commenters (6873, 6906) expand on this idea by noting that the proposal, as written, could provide the EPA Administrator and offices with an incentive to exclude studies with useful information either inadvertently or intentionally because underlying dose response information and information about models are not available. Commenter (6906) suggests that EPA close this potential loophole by amending proposed 40 CFR §30.4 to include instructions that this rule shall not be used to exclude a study from EPA's review and consideration of conclusions drawn therein, where EPA determines the study is compatible with the "best available science." Commenter (6873) recommends that EPA close this loophole by any number of minor adjustments -- for instance, holding public hearings for each study to be questioned by the EPA might allow scientific findings to be vetted without ruling them out entirely.
 Commenter (1012) asserts that EPA must use the best available science as the foundation of its regulatory actions. Commenter (1945) states that regulatory decisions should be based on the best science, and that there is often much to be gained by allowing data to be scrutinized.
 Commenter (1012) asserts that oftentimes the best available science (or the only available science) may include data of a sensitive nature like personal health information or CBI. Commenter (5131) states that peer-reviewed studies assessing the effects of pollution exposure on humans, based on actual personal health data, which by law must be kept confidential, are among the best science available to the Agency.
 Commenter (3135) states that, for years, both Congress and successive administrations have required the EPA to use the best science for its decisions. The commenter states that it is a major departure for this proposal to direct EPA scientists to exclude key studies merely because they cannot meet the proposed disclosure requirement.
 Commenter (2170) states that the best available science can include research vetted through means other than the publishing of all data used. The commenter states that the protection of human health and the environment, as well as the legal, ethical, and moral prohibitions on publishing certain kinds of data, require that the best available science be inclusive of such research. The commenter states that it is unclear from a reading of the Proposed Rule whether scientific research not relying on published data sources would be (1) required to publish the data before the research could be used, even where publishing would be in clear violation of legal, ethical and/or moral standards, (2) simply ignored by EPA as if it were nonexistent, or (3) some combination of 1 and 2. The commenter states that the practical effect would be the same: a whole segment of research critical to the protection of human health, representing the best available science, would be eliminated from regulatory use.
 Commenter (4595) asserts that it has been recognized since 2002 by the D.C. Circuit Court that it would be "impractical and unnecessary" for the EPA to require that all underlying data be made public. The commenter notes that the court recognized the importance of EPA to have access to as much scientific information as possible to protect public health and the environment. The commenter adds that there remain cases when the transparent and proper use of the best available science requires that some of the underlying data cannot be made publicly available for legitimate reasons.
Response: The EPA disagrees with the commenters because this final rule does not require the EPA to categorically exclude any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Further, this rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider "pivotal science in accordance with the provisions of this rule," unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations.
Some commenters interpreted 40 CFR 30.4 as requiring the EPA to make publicly available all science relied upon in final agency actions. The EPA clarifies that in the final rule, the requirements of 40 CFR 30.4 are consistent with the EPA's existing practice of making science that serves as the basis for informing a significant regulatory action available in the public docket as part of the rulemaking.
<IP_ADDRESS>.3 Epidemiologic Studies
Comment: Commenters (1012, 2180, 4844, 5981, 6103, 6114, 6116, 6119, 6130, 6131, 6170, 6450, 6884, 6915, 6932, 8272) express concern that this proposal would likely prevent the EPA from using any studies that could not be independently reproduced and would therefore disallow many extremely beneficial types of studies, such as those examining the results of natural disasters or industrial/occupational accidents/exposures. The commenters assert that such incidents could not be replicated from practical, safety, or ethical points of view, yet the information they provide on real world exposure and dose-response interactions is invaluable. These studies include the following:
 Exposure to high levels of smoke from wildfires or to high levels of chemicals resulting from industrial accidents or exposures. (6114, 6116, 6131)
 Unique events such as the 2010 Deepwater Horizon oil spill. (1012, 4844, 6103, 6450)
 The Flint water crisis and the 1979 Three Mile Island accident. (6450)
 Health impacts resulting from asbestos exposure, exposure to Dichlorodiphenyltrichloroethane (DDT), and birth defects caused by thalidomide. (6450)
 The effects of radiation from atomic bomb survivors. (6915)
 Data on wildlife toxicity from the Exxon Valdez oil spill. (6915)
 Data on the human health impacts of the September 11, 2001 World Trade Center disaster. (6915, 8913)
 Impacts of neurotoxicants from mercury contaminated Minamata Bay. (8272)
 The BP Transworld oil spill, the Katrina pollutant disaster, the Hurricane Harvey-related toxic chemical discharges, and the Church Rock uranium mill spill. (8913)
Commenter (6915) states that, for older epidemiology data, such as data from studies on occupational exposures to workers in factories before the advent of strict Institutional Review Board (IRB) requirements, raw data are seldom if ever still available. The commenter states that such data, including high quality data generated by major corporations in conjunction with academic institutions, would not be available to EPA under the proposed rule. The commenter states that, effectively, the proposed rule would restrict the epidemiology data available for use by EPA, even where the weight of the evidence clearly supports a finding of causality and risk.
Response: The EPA disagrees with the commenters that this rule would likely prevent the EPA from using any studies that could not be independently reproduced and would therefore disallow many extremely beneficial types of studies, such as those examining the results of natural disasters or industrial/occupational accidents/exposures. In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Further, the rule does not require EPA or any other entity to reproduce or conduct independent validation of pivotal science used in significant regulatory actions and influential scientific information.
Comment: Commenters (2171, 2336, 4591, 4595, 5019, 5115, 5165, 6133, 6916, 6919, 8275, 8281-PH16, 8281-PH26, 8281-PH38) express concern that EPA's Proposal would exclude, including published supporting studies for the NAAQS, that the Agency uses to make informed policy decisions regarding major public health and environmental laws.
Commenters (1311, 1883, 1973, 1995, 2002, 2064, 2107, 2151, 2171, 2180, 2190, 2241, 2336, 4410, 4591, 4595, 4828, 5019, 5115, 5165, 5981, 5186, 6117, 6119, 6120, 6125, 6130, 6132, 6133, 6137, 6170, 6181, 6188, 6373, 6696, 6868, 6872, 6882, 6884, 6895, 6915, 6916, 6919, 6920, 6922, 6943, 6925, 6932, 6941, 8272, 8275, 8281-PH16, 8281-PH26, 8281-PH38) also express concern that EPA is proposing to establish a policy that could preclude it from using many of the best available peer-reviewed and scrutinized longitudinal medical and epidemiological and other studies from consideration in regulatory actions because these studies' underlying data includes confidential information. Commenter (6920) states that, by disqualifying high quality longitudinal epidemiological and clinical studies - often the most direct and relevant evidence of chemical impacts on humans - the proposed rule would diminish not strengthen the science underlying regulations. Commenter (4848) contends that longitudinal medical and epidemiological studies are often conducted over years, if not decades and is concerned that, under the Proposed Rule, they would be eliminated from consideration unless all underlying data is made public. Commenter (4595) asserts that requiring all raw data to be made publicly available before a study can be utilized in EPA decision-making will cut off EPA from foundational research that has informed EPA's work since the inception of the agency and may violate Federal laws and directives already in place. The commenters generally indicate that epidemiological studies are essential to effective environmental regulation. Comments include:
 Commenters (2171, 4410) state that laboratory work, toxicological research, and epidemiological studies are complementary, and each facet is necessary when it comes to understanding and quantifying the effects of a pollutant on human health. The commenters state that eliminating evidence from one of these three essential disciplines threatens the scientific basis for regulatory decisions and actions.
 Commenter (1973) thinks that the proposed rule would deprive policy makers of the real-world epidemiologic evidence, based on real exposures of real people, that has been and will continue to be vital for future revisions of the NAAQS, drinking water standard, pesticide standards, and other health based standards based on a false understanding of transparency and replication in science.
 Commenter (4828) notes that many of EPA's drinking water standards rely on epidemiological studies. The commenter states that often these studies last decades and follow hundreds, if not thousands, of patients, collecting confidential health data as well as other personal data, like the people's addresses, ages, and genders. Commenters (4828, 6872) add that, for most of these studies, the underlying data cannot be made public  -  even in redacted form  -  without sacrificing the participants' privacy or jeopardizing data confidentiality. The commenter states that studies like these established the original limits on lead, and this research continues to be essential today. Commenter (6872) states that, thus, the proposed rule could prevent EPA from considering some of the best, most up-to-date research, including some of the largest collections of epidemiological data developed over long periods of time, in assessing toxic pollutants in drinking water and the air and/or compromise data confidentiality.
Commenter (2064) states that, through peer-reviewed and validated research, paraquat is very strongly associated with an increased risk of Parkinson's disease. The commenter states that much of the research relies on large, population-based studies, which is the type of research most impacted by this rule. The commenter states that, without the ability to consider this research, the EPA will miss vital information needed to protect public health, thus placing Americans at risk. Commenter expresses concern that many of the most important studies involving environmental exposures and Parkinson's disease would be excluded from future determinations for one or more of the following reasons:
 The authors did not previously obtain the needed consents to release the data and would thus be subject to criminal penalty if they did so.
 The data cannot be deidentified appropriately while complying with federal privacy protection laws.
 The cost to deidentify would be very high and many research institutions do not have people on staff who can perform the work needed to do so.
Commenter (6884) asserts that, in proposing this rule, EPA is essentially discounting the scientific expertise, knowledge, experience, and integrity of tens of thousands of researchers around the country that engage in epidemiological research. The commenter states that the "transparency" that is the basis for this rule would virtually undermine any studies that look at real world environmental health impacts to the people we work with every day.
Commenter (2151) states that, while environmental regulation will never be perfect and the goal should not be over-regulation, the potential costs to public health of any rule which would disallow long-term epidemiological studies is not an acceptable risk. The commenter states that there is no product or corporate bottom line so valuable that could legitimize excluding medical studies relating to a child's developing brain or the sickness of any human's body. The commenter states that no particular scientific method or study should be sacrosanct, but whatever methodology the EPA adopts it should be one in which all of the available science is considered in decision-making.
Commenter (5118) believes epidemiological reports are valuable but more critical initial review is needed. The commenter states that there is no way to verify the procedures or results presented and the EPA reviewers are not identified.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
<IP_ADDRESS>.4 Reliance on Animal Toxicity Studies Over Epidemiological Studies
Comment: Commenters (1973, 2171, 4410, 6915, 6941, 8281-PH63) express concern that the proposed rule would exclude many human studies and, as a result, force EPA to increase its reliance on information from less relevant animal toxicity studies. Comments include:
 Commenter (1973) states that the major scientific issue challenging EPA's regulations is not the use of human studies that have been replicated multiple times, it is that EPA has to base many of its regulations on extrapolation from animal studies because human studies, particularly at relevant doses, have not been available. The commenter states that this proposal, by announcing that human studies, when done, will likely be ignored, will only exacerbate this problem.
 Commenters (2171, 4410, 8281-PH63) assert the proposed rule would put regulators tasked with protecting human health in an impossible situation of relying primarily on animal models or in-vivo models that cannot be directly extrapolated to human dose-response estimates.
 Commenter (6915) notes that, in general, and specifically in EPA's 2005 Guidelines for Carcinogen Risk Assessment, human (i.e., epidemiology) data are preferred to animal data as the basis for risk assessment toxicity factors when they are of sufficient quality and are amenable to dose response modeling. The commenter states that this is because animal data always carry inherent uncertainties in regard to their relevance to humans. The commenter states that epidemiology data collected over the last 40 years, however, have been generated under the auspices of IRBs working to protect the patient or participant information obtained by academic institutions, government entities, hospitals, and other organizations, and thus disclosure of that data would be difficult, if not impossible.
 Commenter (6941) asserts that the proposed rule could force EPA to exclude from its consideration epidemiologic studies that use confidential health data. The commenter notes that such studies are frequently the most relevant to protecting public health as they assess outcomes in humans following real world exposures. The commenter states that, without access to the information provided by such studies, EPA may instead increase its reliance on information from less relevant animal toxicity studies.
Response: The EPA disagrees with the commenters that this rule will necessarily lead the EPA to disregard human subject research in favor of animal studies. In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (6446) states that the proposed regulation would preclude EPA from considering a large number of important, peer-reviewed animal studies  --  studies that should serve as a foundation for regulatory action -- merely because of the unavailability of the raw data underlying the studies or the time and expense that would be required to obtain the raw data and provide it to the public. The commenter adds that a large amount of toxicological literature is based on animal studies that has accrued over the past 60 years that underlies the establishment of advisory levels and standards for hundreds of chemicals. The commenter notes that, in the scientific studies relied on for establishing these levels, data are generally provided in aggregated form due to journal page count limitations. According to the commenter, certain raw study data are available at an individual animal level (e.g., pathology data) online and from animal study reports performed by the National Toxicology Program (NTP), but such data are not included in study reports published by academic and industry researchers in peer-reviewed journals.
Commenter (6446) contends that a requirement to obtain an ad hoc exception to the proposed rule's requirements for large numbers of animal studies would be wholly unworkable.
Response: The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
The EPA disagrees that the process of the Administrator granting exemptions to the rule per 40 CFR 30.7 will be unworkable. The Agency may incur administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. The Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information.
<IP_ADDRESS>.5 Limitation of Use of Studies in Future Rulemakings
Comment: Commenters (0559, 6103, 6111, 6125, 6188, 6903, 6934, 8276) express concern that the proposed rule would shrink the universe of scientific information EPA can rely upon in support of rules in the future. Commenters provide the following examples:
 Commenter (6103) states that, in the upcoming review of soot standards, the proposed Transparency rule makes it unlikely that the EPA would be able to consider much of the scientific research that links soot pollution to adverse health effects, as it relies on medical records. The commenter adds that EPA will be forced to make decisions based on less information, making it more difficult for the agency to make science-based decisions that protect public health, as its mission requires.
 Commenter (6111) states that late in 2017 EPA issued a draft report identifying potential approaches to deriving a maximum contaminant level goal for perchlorate. The commenter states that, to develop these approaches, EPA focused on five epidemiological studies. The commenter provides that all five studies relied on confidential patient data and all five studies were carried out in Europe, where scientists may be subject to different data confidentiality requirements than in the United States. The commenter states that the proposed rule would also prohibit EPA from continuing to rely on Needleman's critically important study in future reconsiderations of the lead NAAQS and revisions to the Lead and Copper Rule. The commenter states that, as a result, the proposed rule risks undermining the scientific basis for these EPA actions.
 Commenter (6125) states that EPA must propose an updated rule on lead in soil, dust and paint this year and EPA will propose an updated lead and copper rule for drinking water by 2020. The commenter states that, because many of the foundational lead studies analyzed children with higher exposure and blood lead levels than are commonly seen today, it would be unethical to run the same tests today, exposing children to levels of lead that we know are unhealthy. The commenter notes that, due to numerous factors related to their age, it may well not be possible to recover the original data. The commenter states that, if use of those older studies were successfully challenged because they do not meet the data requirement policy, they could not be used. The commenter asserts that their exclusion would weaken the scientific support for EPA regulations that have clearly proved their worth in the dramatic declines in blood lead levels in American children.
 Commenter (6125) suggests that the standards for arsenic and nitrate standards will most likely be affected by the proposed rule. The commenter states that both drinking water standards rely on epidemiological data including confidential patient information, as well as on older studies. The commenter adds that data from such studies may have been discarded or lost or be in an unreadable form. The commenter contends that none of these studies could realistically or ethically be reproduced since they derive from a unique cohort, and it would be unethical to expose people to ingested pollutants.
 Commenter (6903) states that one study published in 2017 indicates that particulate matter (PM2.5) and ozone exposure at levels below current NAAQS is associated with increased mortality. The commenter notes that the Proposed Rule would make it near impossible for the Agency to consider the study.
 Commenter (6934) expresses that the rule would eliminate the use of studies like the one they hope to one day attain that rely on confidential health guidelines, discredit already peer-reviewed science, and make it impossible for citizens within their group and the countless other groups out there like theirs and individuals to gather preliminary health information for fear that this confidential information might become public once it is accepted into your review system and most of all create a double standard that allows polluting companies and industries to keep their data confidential.
Response: The EPA disagrees with the commenters that this rule will shrink the universe science the EPA can rely upon in significant regulatory actions or influential scientific information. In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
<IP_ADDRESS>.6 Important Studies that Could Be Excluded from Regulatory Decision-Making
Cancer Classifications and Reference Values
Comment: Commenter (6125) states that, if the proposed rule were to apply retroactively with few or no exceptions/exemptions, most of the EPA cancer classifications and reference values would be vacated, aside from those values that are based upon accessible National Institute for Occupational Safety and Health (NIOSH) epidemiology data, National Toxicology Program (NTP) bioassay data or pesticide-related CBI data, if an exemption were made for the latter. The commenter provides a detailed analysis which focuses on two chemical categories for which a cancer classification has been assigned and/or a non-cancer Reference Dose (RfD) and/or Reference Concentration (RfC) has/have been calculated. The commenter provides the following observations/conclusions:
      1. Office of Pesticide Program (OPP) "Known" carcinogens all were assessed in collaboration with Office of Research and Development (ORD)-National Center for Environmental Assessment (NCEA) as those substances also have non-pesticidal uses and/or are an issue for other program offices. So access to their raw data would be a problem for OPP as well as everyone else, since they were not generated by registrants in response to FIFRA regulatory requirements.
      2. OPP has the raw data for chemicals on the 2013 and the RfD/RfC lists, but, of course, they cannot share them with the public because of FIFRA section 3(c)(1)(F) which provides for both "exclusive-use" and compensation rights for data submitted to EPA to support registration actions. In other words, they are CBI and one can be fined and/or jailed for releasing them without approval by their rightful owner.
      3. OPP's regulatory decisions rarely rise to the $100 million cost threshold and most are not considered "rules" in the traditional sense. And, of those that might meet the dollar threshold, most of them are not chemical-specific. Rules such as the Worker Protection Standard or the Container rule may have met that threshold. Actions on individual requests for approval or revocation of uses do not qualify. On rare occasions, the cancellation/revocation of the entire spectrum of uses could meet the threshold if the number of uses was large enough. The 2015 proposed revocation of all of the chlorpyrifos tolerances (~80) might qualify by virtue of numbers of uses although EPA concluded that "revoking all tolerances for chlorpyrifos will not have a significant economic impact on a substantial number of small entities," so maybe it didn't.
      4. Under the conditions of a final rule in which CBI data were exempted, OPP might be able to go forward with decision-making for chemicals based upon those data alone but would be hard-pressed to conduct scientifically-sound risk assessments for those chemicals for which the registration package is mixed. Even assessments for many of the conventional chemicals often are informed by studies from the open literature that were not funded by registrants and declared CBI. Important details in their toxicity or exposure profiles might have to be discarded leading to an underestimation of the actual risk.
      5. The Integrated Risk Information System (IRIS) assessments, for the most part, have significant age on them (late 1980's into the 1990's). This indicates that the studies used to generate those assessments are even older than that. Access to their raw data would be very much a challenge, unless the studies were conducted by NIOSH (epidemiology), the National Toxicology Program (animal studies) or the EPA, itself (human volunteer laboratory studies.
      6. If the proposed rule were to apply to both human and animal studies, then everything on IRIS could become compromised and many of the pesticides may be at risk, too, depending upon the nature of any exemption for them and the degree to which they depended upon non-CBI data.
      7. If a chemical exhibits suggestive to known human carcinogenic potential, and the cancer risk is estimated quantitatively, then this endpoint usually drives the risk assessment and the human data and/or the long-term animal bioassays become the key studies. Data from non-government-conducted epidemiology studies are likely difficult or impossible to acquire and share. Unless the animal data are CBI data from manufacturers or from NTP bioassays, gaining access to them also is likely to be very difficult. Most all of the cancer bioassay data on pesticides are from the manufacturers. Some number of the key studies for the non-pesticide IRIS chemicals are NTP bioassays. In these two cases, EPA will have access to the raw data but will be able to share only the NTP information.
      8. For chemicals for which cancer is not the driving endpoint, the RfDs and RfCs become the important factors in quantitative assessments. Again, for conventional pesticides and biopesticides, most of the key data are from the manufacturers. In the case of the antimicrobials, some submitted studies may come from alternative non-CBI sources, presenting a challenge to access the raw data for them. For the IRIS chemicals, a small number are calculated from NTP data. Most key studies are from the peer-reviewed literature, primarily academic research. Given the age of most of the IRIS assessments, these data would be difficult, if not impossible, to acquire.
      9. If the rule means to define reproducibility/replication as applying to the data/results of the key studies used rather than EPA's assessment of them, then the pesticide and commodity chemical manufacturers might have to submit TWO of every key study used in developing a regulatory decision, each being conducted in accordance with FIFRA Part 158 data requirements and data call-ins or Toxic Substances Control Act (TSCA) Section 4 or 5 test rules.
      10.An inability to acquire and share the raw data for the substances in both of these case examples would seriously comprise EPA's risk assessments and thwart its mission to protect public health.
Commenter (6920) notes that IRIS uses a rational weight-of-evidence method to assess available research and to provide access to a comprehensive source of toxicity data. The commenter adds that it increases the capacity to evaluate chemical hazards, and to quantify risk magnitudes and uncertainties, but it does not tell us how to manage risks. The commenter contends that the proposed exclusion of research lacking all underlying data from IRIS would be an unnecessary waste of information. Commenter (6133) notes that IRIS the Proposal would undermine decades of expert work to advance successful data evaluation methods described in the systematic review approach now underway in the EPA IRIS program.
Response: There are several important clarifications in response to the commenters' concerns. First, the final rule applies prospectively to significant regulatory actions and influential scientific information, not retrospectively. Second, this rule only applies to significant regulatory actions and influential scientific information. Many of the actions performed by the EPA Office of Pesticide Programs (OPP) are not significant regulatory actions, and therefore would not be subject to this rule. Finally, in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
Six Cities Study and American Cancer Society Cancer Prevention Study Would be Excluded in Regulatory Decision-Making
Comment: Commenters (0567, 2429, 2746, 2787, 4591, 4894, 5012, 5165, 5176, 5180, 6103, 6111, 6125, 6133, 6351, 6915, 6916, 6925, 8281-PH38, 9227) express concern that the proposed rule could exclude studies that utilize confidential data, such as the Harvard Six Cities ("Six Cities") (1993) and American Cancer Society (ACS) Cancer Prevention Study II (1995) studies, each of which tracked the medical histories of thousands of people and which are the two major epidemiological studies referenced in the particulate matter (PM) NAAQS.
Commenters (2787, 6925) state that the "Six Cities" study, which followed more than 8,000 participants for nearly twenty years was key in establishing a link between chronic air pollution exposure and increased mortality. Commenter (6925) provides that this epidemiological study relied on health data that is protected under confidentiality agreements, so these data cannot be publicly released without violating privacy.
Commenter (9227) states that the original findings of the Harvard Six Cities Study have been validated many times over, and they have been used to inform countless EPA rule makings that address particulate matter pollution. The commenter adds, however, that the Proposal would appear to preclude EPA from using them because -- while the Study has been reanalyzed and reproduced -- the underlying data is not publicly available because of patient confidentiality protections bound by individual contractual agreements between the scientists and the research participants and by the Health Insurance Portability and Accountability Act (HIPPA). The commenter states that these reasons are unrelated to the validity, integrity or quality of the Harvard Six Cities Study. The commenter asserts that the provisions of the proposed rule that would preclude the use of the study are inconsistent with OMB's data quality guidelines, which specifically point to the Harvard Six Cities Study as an example of how data may be validated or corroborated without public release of the underlying raw data.
Commenters (5165, 6925) suggest that EPA publicly confirm that it would continue to use existing literature such as the Six Cities Study in future rulemakings, should the proposed rule be enacted.
Commenters (2429, 2787, 6125, 6915) express concern that the proposed rule could exclude studies that utilize confidential data such as the ACS Cancer Prevention Study. Commenter (2429) states that, while results from the study have been subjected to independent reanalysis and sensitivity analyses, public access of these data will be difficult given that they were collected decades ago with assurances of confidentiality for research participants.
Commenter (4894) states that there are valid, peer reviewed studies that should be included in EPA's regulatory work, even though their underlying data sets cannot be released to the public, including the ACS's Cancer Prevention Study II. Commenter (2907) states that the ACS Cancer Prevention Study II tracked air pollution exposure and personal medical histories of nearly 670,000 people for more than two decades to understand the exact risk of air pollution on death and the results of this study underpin critical components of decision making with respect to national air quality standards.
Commenters (2787, 4827, 4894, 6925, 8272) state that the results of the Harvard Six Cities Study and the ACS Cancer Prevention Study II studies have stood up to extensive subsequent analysis, including the "Reanalysis of the Harvard Six Cities Study and the ACS Study of Particulate Air Pollution and Mortality,"  which highlights the strength of such research. Commenter (8272) states that since publication of these two landmark studies, dozens of additional peer-reviewed studies, as well as follow-up research involving the Six Cities and ACS cohorts, have replicated and expanded upon their findings and provided insights into specific effects of particulate pollution on certain groups.,
Commenter (6915) states that under the proposed rule, EPA could ignore these two foundational studies and other peer-reviewed studies built upon them in setting health-based air quality standards for particulate matter and other pollutants. The commenter states that the effect could be devastating and deadly, as EPA estimates that reductions in ambient particulate matter under the 1990 Clean Air Act (CAA) Amendments will prevent 230,000 adult deaths by 2020.
Commenter (6925) states that the fact that patient personal information is held confidential in no way undermines or casts doubt on its findings. The commenter states that this study is one of the most critical studies that has been used to establish air pollution standards. Commenter (4894) states that the studies remain valuable today.,
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes the Harvard Six Cities ("Six Cities") (1993) and ACS Cancer Prevention Study II (1995) studies. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of personally identifiable information (PII), CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Other Important Studies that Would be Excluded from Regulatory Decision-Making
Comment: Commenters (2336, 4827, 6920) state that, if adopted, the proposed rule will deprive EPA policymakers from real-world evidence and studies that are vital to EPA's review of the NAAQS into the future, such as "Air Pollution and Mortality in the Medicare Population." Commenter (6920) states that the NAAQS review process is a model of scientific transparency. The commenter notes that its reviews are renowned for their high quality. In regard to the studies on which they are based, the commenter states that the costs of publication as well as requirements for the privacy of study subjects have prohibited making all the data obtained publicly available. The commenter strongly disagrees that this diminishes the value of such epidemiology and toxicology studies to regulatory science. The commenter contends that the exclusion of such studies would significantly reduce the amount of scientific data available to establish appropriate standards to protect public health and the environment.
Similarly, commenter (6125) states that one of the most recent fine particle mortality studies created a cohort of 60 million subjects from the Medicare database. The commenter states that this study found even larger effects of fine particles at levels well below EPA's current standards. The commenter states that the Medicare database is available to any research group that pays a fee and can guarantee confidentiality of the personal data. The commenter states that, because these data are not fully available to the public, under the proposed rule, the Administrator could choose to bar consideration of even this powerful study.
Commenter (2336) provides that the history of the NAAQS standards for PM2.5, and the dose-response models which underpin them, demonstrates that a blanket public disclosure rule such as the Proposal would have been counterproductive if it had been in effect during the 1990s when the new NAAQS standard for PM2.5 was being evaluated and the underlying dose response models were being vetted. The commenter states that the rule proposed would have required that the health-outcomes dataset be made public, or the results would have been thrown out. The commenter adds that, had the rule proposed been in effect in the 1990s, it would have created a Hobson's choice for the EPA. The commenter states that it would have had to discard a pair of robust groundbreaking studies demonstrating that there is an intense dose-response function between PM2.5 and mortality rates, or it would have had to violate the privacy interests of the studies' participants and jeopardize the prospect of performing similar studies in the future.
Commenter (8272) references two recent studies and states that they add to knowledge on susceptible populations., The commenter states that both of these studies involve data that can be obtained by investigators who apply to use it and are approved by the relevant IRB or Privacy Board, but their data sets are not publicly available.
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes the "Air Pollution and Mortality in the Medicare Population" study.  Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (2429) states that a recent example of an important study that could be excluded under the Proposal is the Multi-Ethnic Study of Atherosclerosis and Air Pollution (MESA-Air), a 10-year prospective cohort study of more than 6,000 participants. Using these data, the commenter notes that the MESA-Air researchers have produced over 100 scientific publications examining the cardiovascular impacts of long-term exposure to air pollution, all of which could be excluded from consideration in the regulatory process.
Commenter (2977) expresses concern that the proposed rule would restrict the scientific evidence that can inform essential environmental regulations; regulations that have reduced emissions, protected our drinking water, reduced lead exposure, and saved lives. Commenter states that the studies listed below, which EPA has used to help establish acceptable levels of exposure to pollutants, would not be considered if rigid standards like the ones proposed in this transparency rule were in effect.
Commenter (2977) provided a sample list of population-based studies (included in an Appendix to their comment letter) from the National Institute of Environmental Health Sciences/Environmental Protection Agency's Children's Environmental Health and Disease Prevention Research Centers that were considered or included in EPA regulations:
 Berhane K, Zhang Y, Linn WS, et al. The Effect of Ambient Air Pollution on Exhaled Nitric Oxide in the Children's Health Study. The European respiratory journal. 2011;37(5):1029-1036. doi:10.1183/09031936.00081410.
 Bradman A, Salvatore AL, Boeniger M, et al. Community-Based Intervention to Reduce Pesticide Exposure to Farmworkers and Potential Take-Home Exposure to their Families. Journal of exposure science & environmental epidemiology. 2009;19(1):79-89. doi:10.1038/jes.2008.18.
 Breton CV, Salam MT, Vora H, Gauderman WJ, Gilliland FD. Genetic Variation in the Glutathione Synthesis Pathway, Air Pollution, and Children's Lung Function Growth. American Journal of Respiratory and Critical Care Medicine. 2011;183(2):243-248. doi:10.1164/rccm.201006-0849OC.
 Coronado GD, Holte SE, Vigoren EM, et al. Do workplace and home protective practices protect farm workers? Findings from the For Healthy Kids Study. Journal of occupational and environmental medicine / American College of Occupational and Environmental Medicine. 2012;54(9):10.1097/JOM.0b013e31825902f5. doi:10.1097/JOM.0b013e31825902f5.
 Islam T, Berhane K, McConnell R, et al. Glutathione-S-Transferase (GST) P1, GSTM1, Exercise, Ozone and Asthma Incidence in School Children. Thorax. 2009;64(3):197-202. doi:10.1136/thx.2008.099366.
 Keet CA, McCormack MC, Pollack CE, Peng RD, McGowan E, Matsui EC. Neighborhood Poverty, Urban Residence, Race/ethnicity and Asthma: Rethinking the Inner-city Asthma Epidemic. The Journal of allergy and clinical immunology. 2015;135(3):655-662. doi:10.1016/j.jaci.2014.11.022.
 Li Y-F, Gauderman WJ, Avol E, Dubeau L, Gilliland FD. Associations of Tumor Necrosis Factor G-308A with Childhood Asthma and Wheezing. American Journal of Respiratory and Critical Care Medicine. 2006;173(9):970-976. doi:10.1164/rccm.200508-1256OC.
 Linn WS, Rappaport EB, Berhane KT, Bastain TM, Avol EL, Gilliland FD. Exhaled nitric oxide in a population-based study of Southern California school children. Respiratory Research. 2009;10(1):28. doi:10.1186/1465-9921-10-28.
 McCormack MC, Breysse PN, Hansel NN, et al. Common Household Activities are Associated with Elevated Particulate Matter Concentrations in Bedrooms of Inner-City Baltimore Pre-School Children. Environmental research. 2008;106(2):148-155. doi:10.1016/j.envres.2007.08.012.
 McConnell R, Islam T, Shankardass K, et al. Childhood Incident Asthma and Traffic-Related Air Pollution at Home and School. Environmental Health Perspectives. 2010;118(7):1021-1026. doi:10.1289/ehp.0901232.
 Salam MT, Bastain TM, Rappaport EB, et al. Genetic Variations in Nitric Oxide Synthase and Arginase Influence Exhaled Nitric Oxide Levels in Children. Allergy. 2011;66(3):412-419. doi:10.1111/j.13989995.2010.02492.x.
 Salam MT, Islam T, Gauderman WJ, Gilliland FD. Roles of Arginase variants, Atopy and Ozone in Childhood Asthma. The Journal of allergy and clinical immunology. 2009;123(3):596-602.e8. doi:10.1016/j.jaci.2008.12.020.
 Salvatore AL, Bradman A, Castorina R, et al. Occupational Behaviors and Farmworkers' Pesticide Exposure: Findings From a Study in Monterey County, California. American journal of industrial medicine. 2008;51(10):782-794. doi:10.1002/ajim.20622.
 Salvatore, A. L., Chevrier, J., Bradman, A., et al. A community-based participatory worksite intervention to reduce pesticide exposures to farmworkers and their families. American journal of public health, 2009; 99(S3), S578-S581. doi:10.2105/AJPH.2008.149146
 Thompson B, Griffith WC, Barr DB, Coronado GD, Vigoren EM, Faustman EM. Variability in the TakeHome Pathway: Farmworkers and Non-farmworkers and their Children. Journal of exposure science & environmental epidemiology. 2014;24(5):522-531. doi:10.1038/jes.2014.12.
 Wenten M, Gauderman WJ, Berhane K, Lin P-C, Peters J, Gilliland FD. Functional Variants in the Catalase and Myeloperoxidase Genes, Ambient Air Pollution, and Respiratory-related School Absences: An Example of Epistasis in Gene-Environment Interactions. American Journal of Epidemiology. 2009;170(12):1494-1501. doi:10.1093/aje/kwp310.
 Wright, John Paul, et al. "Association of prenatal and childhood blood lead concentrations with criminal arrests in early adulthood." PLoS Medicine 5.5. 2008. doi:10.1371/journal.pmed.0050101
Commenter (4844) states that, by requiring reproducibility, this rule may also exclude many landmark public health studies that were so scientifically rigorous and resource intensive that they could not be reproduced, such as the Framingham Heart Study, a 70-year long cardiovascular epidemiologic study, the Diabetes Control and Complications Trial, a 10-year study of the effects of intensive diabetes control on long-term complications, or the Nurses' Health Study, a decades-long study into the risk factors for major chronic diseases in women.
Commenter (5012) expresses concern that the Children's Health Study conducted in southern California would also be eliminated.
Commenter (5177) submitted four attachments and a summary of each as examples of important studies that may be excluded in regulatory decision-making:
      1. As part of a large cohort study of cancer and hematotoxicity among approximately 75,000 workers exposed to benzene sponsored by The Chinese Academy of Preventive Medicine and the U.S. National Cancer Institute, a cross-sectional study was performed of currently exposed workers in order to gain insight into early biologic effects associated with benzene exposure. The report presents the findings of the association between benzene exposure, benzene metabolite formation, and hematotoxicity.
      2. A study was conducted to determine the relation between the consumption of contaminated local fish and concentrations of total polychlorinated biphenyls (PCBs) and 68 PCB congeners in the milk of nursing Mohawk women residing near three hazardous waste sites. Estimated cumulative lifetime exposure from local fish consumption was significantly related to milk total PCB and to three congeners only among those Mohawks who gave birth from 1986 to 1989.
      3. This report summarizes the health implications associated with exposure to PCBs, primarily through fish consumption. The weight of evidence clearly indicates that populations continue to eat fish containing PCBs and that significant health consequences are associated with consumption of large amounts of some fish.
      4. The EPA has been performing its second Six-Year Review of the drinking water contaminants regulated under the Safe Drinking Water Act (SDWA). The primary purpose of this document is to provide a screening-level review of the health effects component of the Six-Year Review 2 effort.
Commenter (5419) states that complete exclusion of an entire study could limit access to scientifically defensible information to regulators and other stakeholders to achieve desired regulatory aims. The commenter provides three examples of research that the Electric Power Research Institute (EPRI) believes are important for consideration in regulatory activities with an explanation of the potential issues associated with making data required to understand human health effects and dose‐response relationships publicly available. These are summarized below:
      1. Occupational Epidemiology Study of Hexavalent Chromium Cancer Risk: EPRI funded a study to update the mortality, exposure reconstruction and dose‐response modeling for a cohort of workers from the Painesville, Ohio Chromate Production Plant who were exposed to hexavalent chromium, resulting in elevated rates of lung cancer (Proctor et al., 2017). This cohort is associated with significant regulatory precedent as it updated a study used by the Occupational Safety and Health Administration for risk assessment when the hexavalent chromium Permissible Exposure Limit was lowered in 2006. The current EPA inhalation cancer slope factor, originally set in 1984, was also based on an earlier study of workers from the Painesville plant, but the EPRI study uses more robust data and included short‐term workers, thus allowing examination of the dose‐response relationship in the low exposure range. Clearly this study is of interest for risk assessment (in particular, the EPA's IRIS program); however, the original data are not publicly available as they are owned by the company that employed the workers, and EPRI's research was subject to a strict confidentiality agreement. Complete study validation, as expressed in the proposed rule (i.e., data to be publicly available in a manner sufficient for independent validation), would not be possible.
      2. Children's Air Pollution Asthma Study (CAPAS): EPRI funded a panel study of asthmatic children involving residential exposure assessment and collection of pulmonary function and symptom data. Several papers resulted from the research related to the associations between indoor and outdoor PM2.5 and its individual components and asthma exacerbation (Habre et al., 2014a, 2014b; Rohr et al., 2014; Schachter et al., 2015; additional paper forthcoming). Additionally, CAPAS provided important information on the role of PM components in the health effects, which is a topic of interest to EPA and others, given that it is unlikely that all PM components are equally toxic. The consent form utilized in the study stated that indoor exposure data as well all questionnaire data would be kept strictly confidential.
      3. Low Dose Radiation Exposure Health Effects: EPRI has for nearly 10 years worked to increase the understanding of effects from low doses of ionizing radiation. Observational studies have been well‐utilized and form the basis for radiation protection standards around the world (Ozasa et al., 2012; Richardson et al., 2015). Since many of the data from human radiation studies are managed by other research organizations that have highly restrictive data access policies, it is unclear that special arrangements can be made with these entities to release the data for public access. EPRI is concerned that these seminal studies may be eliminated from future consideration as a result of this proposed rule.
Commenter (6111) states that studies that would be excluded from EPA consideration under the proposal form the basis for multiple regulatory actions that EPA and other agencies have taken over the course of many years. The commenter provides the following examples:
 Lead: Early studies on the neurological effects of low-dose exposure to lead such as Herbert Needleman's 1979 paper finding a negative relationship between the level of lead in children's teeth and IQ scores. EPA relied on this study in its 1986 Air Quality Criteria document for lead. EPA's Lead and Copper Rule, which established the federal regulations for lead under the SDWA, in turn relied on that Air Quality Criteria document to identify blood lead levels of concern. EPA relied on both the 1986 Air Quality Criteria and on Needleman's research directly in establishing standards for lead-based paint hazards under the TSCA. Needleman's work, and subsequent studies building upon it, also supported EPA's decision to revise the NAAQS for lead in 2008. After 40 years, and with the principal investigator no longer alive, it is not clear that the raw data from the Needleman study is available. Even if the data were, they could not be made publicly available without invading the privacy of the study participants. Importantly, it would not be possible to conduct that same study at this time, because children no longer have blood or dental lead levels as high as they did in the 1970s as a result of EPA's implementation of the statutes.
 Arsenic: EPA's drinking water standard for arsenic under the SDWA is dependent on studies that the agency would now be compelled to ignore under the proposed rule. EPA established a drinking water standard of 10 ppb for arsenic in 2001. The Food and Drug Administration ("FDA") then relied on EPA's determination. In setting this standard, EPA relied on a National Research Council review of the scientific evidence, which "concluded that [certain epidemiological] studies from Taiwan provided the current best available data for the risk assessment of inorganic arsenic-induced cancer." The Taiwanese papers looked at rates of skin cancer and blackfoot disease in villagers from southwestern Taiwan who were exposed to high levels of arsenic in their drinking water. These studies were based on data from clinical examinations of the research subjects and therefore included confidential patient data that likely cannot be released to the public. In addition, given that the first data were collected more than 50 years ago, the studies are based on data that may no longer be available.
 PCBs: EPA's regulations establishing water quality standards for PCBs under the CWA were based in part on long-term epidemiological studies of cancer rates in workers exposed to PCBs.
 Radionuclides: EPA's S regulation for radionuclides relied on epidemiological studies of survivors from the Hiroshima and Nagasaki atomic bomb attacks.
 Methylmercury: EPA's reference dose for methylmercury in fish that will be consumed by humans relied on data from human exposures in the Faroe Islands. Precluding reliance on these and other studies for the sole reason that the underlying raw data has not been or cannot be released to the public is arbitrary, capricious, contrary to professional best practices, and antithetical to protection of public health and safety as required by the Statutes. The proposed rule will prevent EPA from relying on the "best available science."
Commenter (6373) provides four examples of research and studies the City (NYC) has conducted to evaluate the impact of environmental exposures on populations within its borders. The commenter states that all these studies have informed or currently inform EPA decision-making and are considered regulatory science to which the Rule would apply. The commenter states that, if the Rule is finalized, the valuable findings that these studies offer may be discounted by EPA, to the detriment of New Yorkers and the country.
 The Department of Health and Mental Hygiene (DOHMH) used study designs to identify individual-level risk factors for the elevated risks of dying on extreme heat event days (e.g., ethnicity, age, and location at time of death). DOHMH used the analysis to develop the Heat Vulnerability Index. If DOHMH had to make the raw data available to public, the agency wouldn't be able to use these study designs while protecting individual privacy.
 In collaboration with Brown University, DOHMH conducted a series of analyses to examine the health impacts of air pollution on several birth outcomes, including birth weights, pre-term births, and hypertensive disorders of pregnancy.
 The World Trade Center Health Registry (WTCHR) is the largest registry in the United States that tracks the health effects of a disaster. The WTCHR collects personal identifiable information as well as protected health information, with permission from each enrollee.
 New York City Health & Nutrition Examination Survey (NYC HANES) is a community-based health survey first conducted in 2004 by DOHMH and then most recently in 2013-14 by City University of New York School of Public Health and DOHMH. The data collected enables DOHMH to learn how many New Yorkers suffer from basic conditions such as diabetes, high blood pressure, high cholesterol, and depression.
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available in a manner sufficient for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
<IP_ADDRESS>.7 Multi-Study Issues
Comment: Commenter (6125) states that the proposal has failed to consider the effect of the rule on consolidated models. The commenter states that the Integrated Exposure Uptake Biokinetic (IEUBK) model, a crucial tool to assess lead body burden in regulatory and other settings, is based on many individual studies to support various individual relationships. The commenter asks: given the number of studies needed to support what is clearly a highly influential tool in regulation, how would EPA determine which supporting studies needed to provide data and which would not? The commenter states that EPA has not fully considered these kinds of multi-study issues, and the potential effect on the model as a whole. The commenter notes that the contributing researchers may have little motivation to prepare and make available data to satisfy this new and unique policy.
Response: The EPA is finalizing the scope of the rule to apply to dose-response data, and not models. Specifically, where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenter (6125) states that EPA has not considered the potentially significant implications of this policy for researchers doing meta-analyses, which have been crucially important in understanding lead, and which consider many studies by many researchers, who may have varying views on releasing their individual data for public review. The commenter provides that one example of a possible regulatory issue would be the 2006-2008 lead air standards, which were based in part on a pooled analysis, for which two of the seven primary investigators declined to provide the raw data to the public.
Commenter (6895) asserts that the rule would exclude high quality, peer-reviewed science. The commenter states that loss of information would be particularly egregious in an area called out in the notice, the "point-of-departure from which a reference value is calculated." The commenter states that reference value determinations all too often suffer from a paucity of data and will be further hindered by excluding sound science that does not meet the proposed availability requirement.
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The EPA is finalizing the scope of the rule to apply to dose-response data. Specifically, where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Furthermore, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
<IP_ADDRESS>.8 Non-US Studies and International Studies
Comment: Commenters (1973, 6188, 6906, 8913) express concern that the proposed rule could limit recognition of analysis conducted and information available in foreign jurisdictions. Comments include:
 Commenter (1973) states that essentially all non-U.S. studies would be legally unable to meet EPA's requirements, including the Canadian Community Health Study data, the ESCAPE study, the UK national cohort study, the Netherlands National Cohort study, the Danish National Birth Cohort and the Norwegian Birth Cohort (both funded by National Institutes of Health (NIH)). The commenter states that EPA will achieve no greater "transparency", they will only deny themselves the ability to set standards based on the studies that the rest of the world relies on.
 Commenter (6188) states that the ESCAPE study relied upon data protected by European privacy laws to combine multiple cohorts across Europe and examine the association of air pollution with mortality in study participants. The commenter notes that the findings of that study are consistent with multiple other studies that examine the effects of particulate matter exposure using different methodologies.
 Commenter (6906) states that consideration of hazard and risk assessments from other jurisdictions for Risk Evaluations under TSCA promotes a uniform international market by coming to a common understanding of risk. The commenter notes that recognition of internationally available data also reduces EPA's burden in performing evaluations, assuming EPA can verify the quality of information meets the standard of "best available science." The commenter states that it also reduces industry's burden by allowing industry to rely on and/or provide data across countries.
 Commenter (8913) states that many studies span decades, involve multiple collaborations, and use pooled international data. The commenter adds that a prime example is a recent Lancet study by an international team which included scientists from the Radiation Epidemiology Branch of the National Cancer Institute (NCI).
Response: The EPA clarifies that in this final rule the Agency is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes studies from foreign jurisdictions. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
<IP_ADDRESS>.9 Older Studies
Comment: Commenters express concern that many older studies could not be used under this rule for the following reasons:
 Older studies where the authors or data sources may not be accessible. (1006, 9225)
 Open data requirements would be a new qualification for the field not in line with prior publication standards; as a result, many quality studies would be disregarded due to the old societal standards under which they were published. (1481)
 Most existing environmental health studies were completed before the movement toward public access in science, and such existing studies cannot reasonably be expected to retroactively release their data to be eligible for consideration in future regulatory decisions. (1994)
 Public access to data from older studies might not be feasible. (2472)
 Older studies that have data stored on older technology that can't be recovered. (8281-PH38)
Commenter (6125) states that, even for controlled human and animal studies, where subject data might ethically be released, the underlying data may no longer be accessible years after publication.
Commenter (6907) is concerned that, under the proposed rule, older studies on pesticides may not be considered. The commenter states that this is of particular concern for older pesticides which have now been unregistered but for which we have older studies that indicate that these pesticides pose harm to humans or the environment. The commenter asserts, for example, that there is no sound reason to redo the studies related to DDT.
Commenter (6915) expresses concern that many of the standards that are developed or updated by EPA are for chemicals that have an extensive, older body of scientific literature on their effects, but that are not currently being actively researched. The commenter states that the vast majority of studies considered for standard-setting are not new and were not conducted, designed, or published with the goal of ensuring data availability. The commenter states that, accordingly, their data are likely unavailable and, even if data were kept, the formats in which older data are stored may not be accessible from currently available computers, potentially invalidating the use of those studies as the basis for future regulatory standards.
Response: The EPA clarifies that in this final rule the Agency is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes older studies. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
Comment: Commenter (6125) expresses concern that EPA has not considered or examined the number of potentially significant studies for which EPA cannot obtain raw data or methodology due to age. The commenter includes a review of EPA's IRIS and pesticide data bases to show the age of many of the studies that underlie risk assessments for carcinogens and reference doses and concentrations (Rfd, Rfc) for a number of substances that are regulated under legislation that addresses hazardous wastes, toxic chemicals, clean water, drinking water, clean air and pesticides. The commenter notes that perhaps the most revealing and stark finding of this review comes from the examination of the IRIS data base used to determine cancer classifications, RfDs and RfCs for IRIS chemicals. The commenter provides that most of the entries are over 20 years old, and some over 30 years old. The commenter states that this guarantees that the studies used to develop the findings are at least that old, and, perhaps, decades older still. The commenter asserts that, without examining the support document for each entry, one cannot rule out that the age of the key studies may far exceed that of the entry. The commenter notes that, if a regulatory action that relied on one or more of these assessments was challenged due to the lack of the original data, under the requirements of the proposed rule, most of the cancer classifications and reference values could be vacated, aside from those values that are based upon certain NIOSH epidemiology data, NTP bioassay data or pesticide-related CBI data, and then, only if an exemption were made for the latter.
Response: The EPA is finalizing the scope of the rule to apply to dose-response data. Specifically, where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis for various reasons, including the age of the pivotal science or its underlying dose-response data. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
Furthermore, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
<IP_ADDRESS>.10 Studies Used to Support Other Agency Standards
Comment: Commenter (0671) states that no other federal agency has such a broad standard as the one envisioned here. The commenter states that EPA knows that IRBs will not allow such studies' data to be made public, so to require public data from these studies will make them invisible to the rulemaking process.
Commenter (1973) thinks that the proposed rule would put EPA in direct opposition to other agencies' standards for considering evidence:
 Health effects of smoking, high cholesterol, etc. by the Centers for Disease Control and Prevention (CDC).
 The studies considered for issuing guidelines generated by the NIH.
 The FDA reliance on non-publicly available studies to approve drugs for use by people.
Commenter (5022) provides that some of EPA's greatest public health accomplishments, such as eliminating lead in gasoline and classifying secondhand smoke as a cause of cancer (which led to banning smoking from indoor public places), were based on the kinds of data that would be discarded under the proposal. The commenter states that such data are widely used in rulemaking proceedings by other U.S. government agencies and around the world.
Commenter (6117) references the following policies of other agencies and asserts that none of these policies require that all individual-level underlying data and models be made available to the public:
 NIH requires that researchers conducting agency-funded research pre-register clinical trials, pre-specify study endpoints and statistical plans, and report research results.
 All NIH applications for $500,000 or more in direct funding in any given year must contain a data sharing plan, and some institutes of the NIH share data with researchers via a secure website.
 Many federal agencies, including the EPA, require that any published articles based on federally funded research be made available via the PubMed Central website.
 The International Committee of Medical Journal Editors has adopted a policy requiring pre-registration of clinical trials, and publication of a data sharing statement.
 The Institute of Medicine published a report recommending that trials be registered prospectively, that summary-level results be shared with the public after trial completion, and that the metadata and patient-level data be responsibly shared with researchers 6 months after scientific publication, or 30 days after regulatory approval.
Commenter (6125) states that the CDC and other Federal Agencies involved in reducing lead risk, would not be subject to the restrictions on use of science imposed by the proposal, and thus would continue to use the older studies as evidence, creating an unwarranted science practice and policy schism in the federal government. The commenter provides the following examples:
 The CDC has acknowledged that no safe level of lead in blood has been identified and has identified a "reference level" to define especially high-risk populations and geographic areas most in need of primary prevention. Even some of the studies that the Federal government's most respected lead advisory group (CDC's Advisory Committee on Childhood Lead Poisoning Prevention, or ACCLPP) used to develop background justification for the reference level could be precluded from EPA use under this policy. If some of those 92 plus references and studies fail to meet this policy, EPA could not rely on them in developing regulations, and thus could issue regulations that fail adequately to address CDC's high-risk exposures. Moreover, EPA cost-benefit analyses could not use the CDC health effects work in calculating regulatory benefits. Further, the additional requirements in the proposal could require that EPA re-evaluate the appropriateness of the CDC dose-response relationship and give "explicit consideration" to other dose-response models even if, as in this case, they have been vetted and rejected by CDC and its advisory committee.
 Both EPA and the U.S. Department of Housing and Urban Development (HUD) regulate similar residential sources of lead (paint, soil, and dust) - EPA in private housing, and HUD in Federally supported housing, including Federally supported units in private buildings. This policy shift increases the potential for disparate treatment and public confusion stemming from those situations. The identical risks addressed by the two programs should be analyzed consistently. HUD has already acted to incorporate CDC's statement into its programs by making its policy on dust-lead clearance levels more stringent; is still considering its response. But under this proposal, EPA might not be able to rely on as wide a range of studies as would be available to the rest of the federal family, which could result in different risk assessments and regulatory outcomes for similar, and possibly even adjacent, dwellings.
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act[.].
<IP_ADDRESS>.11 Privacy Data
Confidential Business Information (CBI)
Comment: Commenters (1006, 2787, 4877, 6114, 6116, 6128, 6131, 6707, 8281-PH38, 9225) express concern that this proposal would likely prevent the EPA from using many studies performed using data that contains CBI. Comments include:
 Commenter (2787) states that confidential industry data are vital to understanding the nature of chemical pollutants, the impacts of pollution, and the most effective ways to protect the environment and public health.
 Commenter (4877) states that many studies contain CBI and this data should be exempt from blanket data sharing requirements. The commenter states, however, that studies based on this data should not be automatically excluded from the EPA's policy-making progress, which is what proposed rule is designed to do.
 Commenters (6114, 6116, 6131) express concern that this proposal would likely prevent the EPA from using many studies performed using data from industrial sources because they contain CBI, such as product formulations. The commenters state that federal law imposes criminal penalties for disclosing CBI, which could result in liability for the EPA, if they follow down the path of making research data publicly available.
 Commenter (6707) states that some data that are not ready for public release (such as proprietary or temporarily embargoed data) could not be used since they simply cannot be made public. The commenter notes, for example, that there is a famous case involving the study of the health effects of living near a major hog farming operation. The commenter provides that, because this work included the study of the effects on children, this data could not be fully released under this law, and thus this major, relevant study would not be considered when the EPA makes a decision on the environmental impacts of that hog farm.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes studies that use data containing CBI. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information, and will continue to handle studies and underlying data in accordance with existing CBI requirements and procedures. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Confidential Health Data
Comment: Commenters (2425, 4838, 4843, 4844, 6130, 6868) express concern that this rule would create procedural barriers to the use of studies that rely on confidential health information, thereby restricting the EPA's ability to consider significant amounts of relevant scientific evidence bearing directly on its core mission. Commenter (4838) states that EPA should not provide blanket limits on the use of studies that cannot be made public because they contain confidential health or business information.
Commenters (1170, 1536, 1973, 4933, 5981, 6788) assert that since most medical research uses patient data that is subject to stringent privacy requirements, the proposed rule automatically eliminates from consideration a majority of medical research. Commenter (1536) states that the proposed rule dismisses virtually all medical research because the research protects the privacy of the patients that volunteered even though these legal obligations are a routine part of HIPAA.
Commenter (6896) states that it is vital to have accurate information about the most pressing health and safety issues to prevent work-related injuries, illnesses, and fatalities and to protect the confidentiality of personal identifying data. The commenter asserts that quality science protects the confidentiality of construction workers personal identifying data.
Commenter (8281-PH63) states that important studies by Minnesota's Department of Health help Minnesota derive health protected values under state law and help the state of Minnesota in litigation. These studies are only possible because we provided absolute guarantees to the participants that their data would be protected and that we would assure its confidentiality. The proposed rule will make it unlikely that public health data such as these would be available for states to use, but even more so for the EPA to use in its decision-making.
Commenters (2429, 2907, 2908, 4884, 5013, 6188, 6696, 6882, 6915, 6943, 8281-PH26) express concern that the proposed rule would limit access to many important and valid peer-reviewed studies that have relied on confidential health data, undermining EPA's mission to protect public health. Comments include:
 Commenter (4884) states that peer-reviewed studies have ensured the sound basis of EPA rules. The commenter states that valid health effect studies are often non-public because they rely upon confidential medical records. The commenter states that, as long as the analysis of the raw data is available and correct, such studies must be used to achieve the legislative intent of the environmental laws.
 Commenter (6943) asserts the EPA should continue its current system in basing its rulemaking decisions on peer-‐reviewed scientific studies while protecting the privacy of citizens in the human database.
 Commenter (6882) states that peer-reviewed studies assessing the effects of pollution exposure on humans, based on actual personal health data -- which by law must be kept confidential -- are among the best science available to the Agency. The commenter states that the proposal would allow EPA to ignore this data unless that confidential personal health data were made publicly available. The commenter states that we must take these studies into account when creating our policies in order to protect the public's health.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes studies that use confidential health data. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenters (1006, 1518, 1981, 4933, 5013, 5178, 6450, 6195, 6877, 6882, 6896, 8272, 8281-PH63) express concern that many important studies that rely on confidential health data could not be used. Comments include:
 Commenter (1006) states that Google's AI study release would not be able to function with this proposal due to confidential data. The commenter states that they both have hidden the public data and the means of which the AI is coded. The commenter states that, if they wanted to do anything with the EPA in relation to a hospital it would be impossible for them.
 Commenters (1518, 4933, 5013, 6882) express concern that, if the transparency regulation were in place, all health studies on fracking would be simply not considered because the research could not be conducted due to non-disclosure agreements. Commenter (1518) states that, if the EPA were unable to consider such studies due to a requirement to release patient information, the Agency would be ill-equipped to weigh benefits vs. costs. Commenter (5013) references three studies:
 One study found that drilling and fracking activity was associated with increased rates of hospitalization in Pennsylvania. In communities with the most wells, the rate of cardiology hospitalizations was 27 percent higher than in control communities with no fracking.
 Another study found that living near fracking operations significantly increases asthma attacks.
 Another study found that pregnant women who lived near active fracking operations in Pennsylvania were at a 40 percent increased risk of giving birth prematurely.
 Commenter (1981) does not agree with the purposed rule's claims that it "will help ensure that EPA is pursuing its mission of protecting public health and the environment in a manner that the public can trust and understand." The commenter states that, where the rule would cause data affecting tribal members to be excluded as a matter of cause, we refute this claim of trust and understanding. The commenter states that fish surveys demonstrate that the proposed rule results in inequitable data representation.
 Commenter (1981) states that, under the proposed rule, fish surveys would be excluded. The commenter states that almost no activity on the reservation has more potential for significantly affecting the economic and political integrity and the health and welfare of all reservation citizens than water use, quality, and regulation. The commenter states that the proposed rule threatens to silence tribal interests where it potentially prevents the use of the fish surveys. The commenter states that, in our view, this seems contrary to "openness, clarity, [and] unobstructed access."
 Commenter (1981) states that the fish surveys are scientifically rigorous while also protecting the fiduciary interest between the researcher and the research subject. The commenter states that one fish survey provided by the Columbia River Inter-Tribal Fish Commission included a confidentiality agreement for instance -- barring the release private information about tribal member participants -- but also included limitations, peer review, procedural information, and quality control. The commenter states that the fish surveys demonstrate the distinctive relationship between Tribes and the health of our Nation's waterways, and in so doing highlights the need for diverse representation in EPA data collection. While the general population eats 19-56 grams of fish and shellfish on average per day, tribal members in the Pacific Northwest can eat up to 797 grams (1.75 pounds) every day. Tribal populations are exposed to toxins that bioaccumulate in aquatic life due to factory effluent, urban wastewater, and runoff from agriculture and cities.