Response: Access to restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation.
<IP_ADDRESS> May Be Limited to Authorized Officials and Researchers
Comment: Commenters (11487, 11890) express concern that the rule does not define "authorized researchers." Commenter (11890) states that, in the SNPRM, new terms are introduced that require more explanation and transparency.
Commenter (11487) states that the SNPRM language regarding tiered access is far too vague and limited to engender confidence among researchers or those who might consider participating in studies. The commenter states that, for instance, it does not define "authorized researchers" or mention the IRBs that govern human-subjects research in academia.
Response: Proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of "research data." In the 2020 SNPRM, the EPA deleted the proposed definition of "research data." The EPA is not including a definition of "research data" in this final rule given that it is finalizing the definition of "data."
This rule does not require any researcher or other outside entity to provide data to the EPA. Nor does the rule categorically exclude studies -- even studies where the underlying dose-response data are not available for independent validation -- and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability.
Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11391) expresses concern that there is no discussion regarding who at the EPA would make decisions about when access to data should be limited or how, or how the EPA would manage the potentially significant additional capacity necessary to make and execute such decisions. The commenter states that, even an unambiguous tiered access system would not address the fundamental underlying problem: there are situations in which researchers simply cannot make data available, even if they or the EPA agree only to share that data with a select group of people.
Commenter (11403) contends that, since the data would be PII, it would be available to the public only through tiered access, and the vast majority of the members of the public would not have more access to the data than it would under the current system. The commenter states that the proposed rule attempts to allow for the use of some CBI and PII by permitting use only if there is tiered access to the data or if the data can be sufficiently de-identified. The commenter states that, in a tiered access system, public transparency is restricted, not just access to the data. The commenter states that, while the EPA identifies the RDC as the model for its approach, why the EPA believes it is appropriate to create another database for confidential health information within the EPA is mystifying. The commenter states that this data, according to EPA, "may be limited to authorized officials and researchers and not provided to the general public." The commenter states that it would be at the EPA's discretion to determine if a member of the public was qualified to access scientific information. The commenter states that the EPA's proposed system is vastly more "secret" than the method it thinks needs fixing.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
<IP_ADDRESS> Implementation of Tiered System
<IP_ADDRESS>.1 Costs and Burdens
Implementation Costs Not Fully Considered
Comment: Commenters (11176, 11391, 11890, 11894, 12430, 12731, 12464) express concern that administrative burdens associated with implementation of the rule are not fully considered in the proposal and SNPRM.
Commenter (11391) states that there is considerable resistance to tiered access in the scientific community, in no small part because of the additional administrative burdens imposed by the tiered-access agreements. The commenter states that there is no indication in the 2020 SNPRM that the EPA has considered those burdens or possible approaches for managing them. Commenter (12430) states that a requirement to provide multiple tiers of access would disproportionately burden academic, public, and even smaller corporate researchers. Commenter (12731) states that the tiered access approach would likely be difficult and costly to administer, but the EPA has not explained, even in general terms, how this approach would be implemented, the degree of burden it would impose, or who would bear the costs.
Commenter (12464) states that the SNPRM says nothing about the costs of the tiered access alternative nor who will bear those costs. The commenter states that, without any details, or cost analysis, it is not unreasonable to assume that the cost of the tiered access approach would fall on researchers, other federal agencies such as the CDC and other third parties.
Commenters (12430, 12617, 12731, 12464) express concern that the tiered access alternative will introduce additional costs for both the government and individual researchers. Commenter (12430) states that a requirement to provide multiple tiers of access would disproportionately burden academic, public, and even smaller corporate researchers.
Commenter (12617) states that researchers would have to dedicate time and resources to figuring out the logistics of sharing, especially with a tiered-access approach. The commenter states that researchers would have to determine how to safely protect medical and proprietary information while designing different levels of access. The commenter states that, while the SNPRM says researchers could create "multiple versions of a single dataset with varying levels of specificity and protection," the extra time and expense seems unreasonable and without any clear benefit to EPA's mission.
Commenter (12731) states that, in a recent briefing with staff on the House Committee on Science, Space, and Technology, EPA staff revealed that "key implementation responsibilities [for tiered access] will fall on the research community." The commenter states that the EPA's apparent expectation that it will impose massive new costs on the research community without acknowledging as much in the SNPRM is arbitrary and a violation of notice requirements. The commenter states that it would further privilege the EPA's reliance on studies from wealthy organizations or those with a financial incentive to expend the necessary resources for EPA to consider their work.
Commenter (12464) states that the tiered access alternative will be even more costly to implement than a requirement that all data and models be made available to the public. The commenter states that, for example, some researchers will have to submit data to a secure data repository, the operator of the repository will have to maintain and operate the facility, and researchers who want to reanalyze the data will typically need to pay fees to access the data as well as pay to travel to the repository.
Commenters (12645, 12731) express concern the proposal and SNPRM do not address the burden of obtaining the consent of researchers. Commenter (12645) notes that there is an increasing number of co-authors in each published study and asks if each author would have to agree to share the data so that any one author would have veto power over sharing? The commenter states that the EPA could find out by sampling the scientific community represented in cited publications underlying its regulatory decisions, but like so many other aspects of the initial transparency rule and this SNPRM, it has not bothered to do any preliminary analyses or to have such analyses done by the National Academy of Sciences (NAS) or the SAB.
Commenter (12731) states that the EPA's proposed "tiered access" approach fails to engage with or even acknowledge concerns that have been presented multiple times to the Agency regarding the cost and feasibility of establishing such arrangements on a routine basis. The commenter states that the Health Effects Institute (HEI) comments to SAB noted that tiered-access arrangements, such as proposed in the SNPRM, require the consent and collaboration of researchers and can be "challenging" to establish.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. As stated in Section III.A.2, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information.
Comment: Commenters (10038, 11392, 12464, 12715) express concern that the costs of data repositories such as the NCHS's RDC would exceed millions of dollars annually.
Commenter (10038) states that the annual operation of such data repositories costs several million dollars above and beyond the initial costs of establishing such centers. Commenter (12715) states that, given the expanded scope of the SNPRM, which now covers not just pivotal regulatory science but also pivotal science supporting influential scientific information, the costs of the proposal may well exceed the $100 million annual threshold for Executive Order 12866.
Commenter (11392) notes that the Secret Science Reform Act of 2014, a legislative precursor to this SNPRM, was estimated by the Congressional Budget Office to cost "$250 million annually over the next few years" should it have passed into law and asks how much will this SNPRM cost, and what money will the EPA use to undertake a project of this magnitude that has not been authorized by Congress? Commenter (12464) states that the tiered access alternative will introduce additional costs for both the government and individual researchers. The commenter states that, as explained in our previous comment letter, the Congressional Budget Office ("CBO") estimated that a bill similar to the proposal would cost the EPA as much as $250 million per year. The commenter states that a tiered access approach will be even more costly to implement than a requirement that all data and models be made available to the public.
Commenters (11176, 11192, 11479, 11487, 11576, 12435, 11900) express concern that the SNPRM does not detail the significant costs of establishing and maintaining an RDC-type system. Commenter (11900) states that, before requiring such a system, the EPA must clearly explain how it will fund the staff and infrastructure required to collect, process, store, and mediate access to this immense trove of data.
Commenter (11192) states that the EPA fails to discuss the costs associated with building, maintaining, and providing access to that repository, nor the cost of reviewing and evaluating applications for access. Commenter (11487) states that the fact that RDC charges researchers $3,000 to access a single year's data hints at the substantial costs associated with the enterprise. The commenter states that the EPA also has not explained who would be in charge of making those determinations -- a crucial issue given that communities disproportionately harmed by pollution might not trust the EPA to make them. Commenter (12435) states that the RDC is not affordably accessible and, from a financial standpoint, the construction of such a system could be very involved and should require an assessment of cost for review along with the proposed rule.
Commenter (11479) states that, even for the subset of EPA-relevant studies for which disclosure of data in a tiered-access system is feasible, such a system would be costly for the Agency to implement and maintain and impractical, if not impossible, for individual researchers to do so. The commenter states that, in turn, a system would likely be costly for those seeking to access the data. The commenter states that the CDC's RDC, for example, costs researchers thousands of dollars to access. The commenter states that it is unlikely that stakeholders such as the public and community groups would pay such high costs to access data that has already been vetted through peer-reviewed studies.
Commenter (11900) states that, in the case of the RDC, the data stored belongs to NCHS (or in some cases to other operating divisions of the Department of Health and Human Services [DHHS]). The commenter states that, however, the system employed by EPA will house extramural data produced by academic researchers in addition to data from their own scientists. The commenter states that the scope of data to be stored in the RDC -- essentially all epidemiologic research on environmental exposure -- is potentially massive and the EPA must explain how it will fund this system. The commenter states that the EPA must also define the burden to researchers of preparing data for entry into the repository, if such preparation is required.
Commenter (11900) states that, to ensure equitable use of the RDC, the EPA should not charge researchers to deposit or store their data in the RDC. The commenter is concerned that the costs of either depositing extramural data into an RDC or accessing data stored in an RDC will disproportionately impact certain types of research interests. The commenter states that, an alternative model, in which researchers are given unsupervised access to the raw restricted data, depends on the integrity of researchers and a system of penalties and enforcement for misuse; the commenter does not recommend this model.
Commenter (11900) states that a key feature of the RDC process is that researchers do not have access to the restricted data except while on site at the RDC, nor are researchers allowed to bring any external technology or datasets into the RDC. The commenter states that all computer code for running analyses at the RDC is reviewed in advance, activity inside the RDC is monitored, and the analysis output is reviewed again before being released back to the researcher. The commenter states that this process ensures that researchers never have access to the restricted data except under close supervision, which precludes any possibility of a linkage re-identification attack. The commenter states that this close oversight over the data comes at the cost of operating the RDC.
Commenter (12715) states that making the data underlying the studies used to generate all "influential scientific information" and "significant regulatory decisions" publicly available or ensuring that such data is publicly available in the future will require significant resources. The commenter states that the additional tiered access system the EPA now proposes is complicated and unclear and merely adds to the burden of evaluation and decision-making that the EPA must conduct in order to determine what data and studies to use when developing standards and health-based criteria. The commenter states that establishing tiered access to protect PII and other confidentiality protections involves creating multiple versions of a single dataset with varying levels of specificity and protection. The commenter states that, as agencies add more tiers, they must build in sufficient controls to monitor who is accessing the data and allow only for authorized access. The commenter states that the SNPRM does not specify how confidential health information covered by the HIPAA Privacy Rule can be protected outside of HIPAA-covered entities but, again, designing and implementing such protections would be expensive and time consuming. The commenter states that, although setting up and maintaining tiered access levels involves significant additional time and cost, the SNPRM is silent on the "logistics, processes, and funding for such data sharing."
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. The analysis by CBO noted by the commenter is not applicable to this rule because the analyses were not for this rule and the requirements analyzed are not the same as those in this rule.
Delay
Comment: Commenters (10038, 10853, 11890, 12731) contend that a tiered access program would delay Agency actions.
Commenters (10038, 10853) state that the rule will curtail the Agency's ability to respond to crises such as the current corona virus disease (COVID)-19 pandemic.
Commenters (11890, 12731) contend that the use of a tiered access program would further delay Agency assessments in critical programs that already struggle to meet their statutory or programmatic work plans. Commenter (12731) expresses concern that the EPA's SNPRM makes no mention of and gives no consideration to practical impediments to tiered access. Commenter (12731) refers to comments of Dr. Smith who highlighted that federal research centers allowing tiered access require researchers to visit in person and require a federal employee to approve all outputs taken from the research center; Center for Medicare and Medicaid Services (CMS) allows for remote use of data, but only after the researcher signs a contract with CMS that provides for the data to be maintained on a secure server -- a process that in Dr. Smith's experience "took some months to set up and (as I understand) a considerable sum of money."
Response: Some commenters on the 2018 proposed rule and 2020 SNPRM specifically asked why the EPA would need to peer review health and scientific studies and scientific literature that had already undergone independent peer review. The EPA disagrees with commenters that the requirements of this rule will result in any meaningful delay in promulgating regulations. While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
General
Comment: Commenter (11890) states that a tiered access program would be costly in critical programs that already struggle to meet their statutory or programmatic work plans.
Response: The EPA has identified some incremental costs that the Agency may incur as a result of this final rule. As stated in Section III.A.2, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information.
<IP_ADDRESS>.2 Timing and Planning for Implementation
Comment: Commenters (10038, 11176) assert that it is premature to enact this rule given the EPA's limited involvement with RDCs and data enclaves and no public plans for further involvement.
Commenter (10038) states that, based on the current status of available data repositories and these anticipated costs, the explicit inclusion of tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects in the proposed rule is premature.
Commenter (11176) asserts that the EPA should have an operations plan and an operational infrastructure for data sharing before implementing the requirements in the proposed rule as updated by the SNPRM. The commenter states that the operations plan should address financial support and logistics for placing research data into an enclave or RDC and carrying out the reanalysis and independent validation.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenters (11176, 12718) state that the EPA does not have an existing infrastructure in place to implement such tiering, therefore, it is premature to enact this proposed rule.
Commenter (11176) states that the data sharing infrastructure not in place and the mechanisms for data sharing are in the early stages of development and lacking proof of concept. The commenter states that making federally-funded research available for which the EPA does not own the data is a work in progress as the SNPRM's wording states: "Development of standard data repositories is still ongoing." The commenter states that the SNPRM also notes the White House Office of Science and Technology Policy recent call for comments on a "draft set of characteristics of data repositories used to locate, manage, share, and use data resulting from federally-funded research." The commenter states that, for a snapshot of the status of data repositories for federally funded research, consider the NIH, the federal government's likely leader in federally funded data sharing, an effort started more than a decade ago. The commenter states that, in its frequently asked questions (FAQ) page for data sharing, the NIH responds, "maybe," to question #20 on whether researchers should consider contributing their data to a data archive. The commenter states that the reply equivocates because of the many factors involved but also indicates how far the NIH is from achieving full data sharing for the research it funds. The commenter states that, for the next question of where to find guidance on preparing data for sharing and archiving, the NIH FAQ does not provide its own guidance but, except for molecular biology information, refers readers of the guidance to an outside entity, the ICPSR. The commenter states that the EPA is a relative newcomer to encouraging data sharing and, while it will benefit from the advances and experiences of other federal entities, is still likely to be years away from providing or encouraging the data access assumed in this rule. The commenter states that, unlike the NSF and NIH, the EPA does not seem to require a data management plan of its grantees, which it planned to do by 2018 in the 2016 EPA document, Plan to Increase Access to Results of EPA-Funded Scientific Research. The commenter states that NIH is seeking to expand its policy on data management and sharing (84 FR 60398), with the goal of making results more transparent to the public. The commenter states that such a requirement might be a first step towards the eventual goal of EPA funded research data being more widely accessible. The commenter states that data management plans and sharing strategies remain work in progress.
Commenter (12718) states that the EPA does not have an existing infrastructure in place to implement such tiering and has made not such efforts to satisfy capabilities that would be needed for appropriate tiered access structures in practice. The commenter states that the proposed regulation offers no meaningful articulation of how the EPA would build this capacity or whether it would instead rely on existing infrastructure. The commenter states that, if the latter, the EPA does not address how such an approach would be operationalized given legal restrictions under title 13, title 42, and law for use of Federal Statistical Research Data Centers, including for managing the risks of re-identification that could violate confidentiality protections under federal law, even if EPA employees are the source of such violations. The commenter states that application and use of tiered access models within the federal government's data architecture and infrastructure cannot be haphazardly applied. The commenter states that, if the EPA were to adopt such an approach, the Agency must clearly articulate appropriate safeguards beyond simply stating the Agency is "conducting a pilot study." The commenter states that, even then, the peculiar requirements for nongovernment funded research to be included raise concern about the breadth, applicability, and utility of such an approach as proposed.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11176) notes that the OMB has yet to release its expected guidance for tiered access. The commenter suggests the EPA should also consider the September 2019 workshop on tiered access hosted by the Council of Professional Associations on Federal Statistics, where the workshop's recommendations for OMB include encouragement of experimentation as well as further research and development.
Response: Thank you for the recommendation.
Comment: Commenter (12431) shares the concern, expressed by the Society of Toxicology in its comments on the SNPRM, that due to the high costs of establishing and operating data repositories with the capacity for anonymizing protected data sets, the inclusion of tiered access to restricted data and models is premature.
Commenters (10038, 12431) suggest the rule should exempt such data and models until the described data repositories have been established and are fully capable of providing tiered access to data.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
<IP_ADDRESS>.3 Tiering Plan
General
Comment: Commenters (10339, 10974, 11362, 11392, 11393, 11184, 11407, 11499, 12412, 12474, 12710, 13549, 13788, 14397) contend the proposal and SNPRM are unclear on several aspects of how the rule would be implemented, as described below.
Commenters (11393, 11184, 11407, 12412, 12474, 13549) state that saying a tiered approach to accessing data is possible, without providing details, is insufficient and unacceptable.
Commenter (10339) states that the practical details about the implementation of the rule remain unresolved, including many issues raised by the EPA's SAB draft report on the scientific and technical basis for the rule. The commenter states that it is not clear precisely if/how the EPA will pursue tiered access to data or make a final determination on whether the level of access is sufficient to be compliant with the rule.
Commenter (10974) states that the EPA does not sufficiently describe how the proposed rule would actively and effectively preserve individuals' privacy or CBI. The commenter states that this proposed rule weakens longstanding privacy protections afforded to U.S. citizens.
Commenter (11362) questions how the EPA will implement this system and determine how, when, and with whom it will share private information. Commenter (12710) states that criteria for tiered access have yet to be developed, and the EPA makes no attempt to define tiered access in the proposed regulatory language.
Commenters (11392, 11499, 13788, 14397) assert that the rule is vague in several respects. Commenter (11392) states that the SNPRM's description of tiered access as "creating multiple versions of a single dataset with varying levels of specificity and protection" is vague and impracticable. Commenter (11499) is concerned that the overall language on tiered access is vague and that clarification on who would have access to CBI and PII would be helpful, accompanied by clarification on how the tiered access would work. Commenter (13788) states that the EPA's discussion of tiered access is vague and neither the rule nor the preamble provides sufficient detail on how such a system would be actually be implemented. Commenter (14397) states that the EPA's proposed tiered approach remains vague and unworkable.
Commenter (11391) states that even where a tiered or controlled-access approach is theoretically possible, there may be valid practical reasons why such an approach cannot be adopted. The commenter states that there is an ongoing debate among scientists about the utility of tiered access models for human health data and how they can and should work. The commenter states that some researchers and organizations have indeed begun to use this kind of approach. The commenter states that there is still considerable resistance to tiered access in the scientific community and it has been far from universally adopted.
Commenter (9455) expresses concern that this proposed rule threatens the trust of the public in environmental health research. The commenter states that the ebbing of that trust has roots in reality and is not just a perception problem. The commenter states that medical data posted on a public website are open to sophisticated data and AI programs that could identify the individuals. The commenter states that medical literature includes large numbers of research papers on patients' need for medical confidentiality. The commenter states that medical confidentiality in research also has been found necessary, especially if the research involves parents giving consent for inclusion of children. The commenter states that even if no laws and ethical standards were in place, environmental health research would be throttled if even a rumor of confidentiality breach should arise.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Administrative Capacity
Comment: Commenters (9455, 10974, 12731) contend that the EPA has not demonstrated that the Agency has the capacity to administer a system of tiered access. Commenter (10974) states that the EPA has not demonstrated sufficient programming resources to comply with the legal requirements afforded to individuals' privacy or CBI in the tiered approach outlined in the SNPRM.
Commenter (9455) asserts that the effort to de-identify or access-tier records of volunteers and of certificates is beyond EPA's budget and staffing. The commenter states that it cannot be left to an automated program with only spot checks for QA/QC. The commenter states that the number of individuals whose data underpin environmental standards may run into the millions. The commenter states that the number of volunteers whose medical data underpin existing standards may be as large as one half to one million. The commenter states that they are not the sole source of confidential medical data. The commenter states that many studies involve birth certificates and death certificates in the thousands for each study. The commenter states that these are privacy-protected documents. The commenter states that, even if the re-identification problem were solvable, the logistics of thorough de-identification or detailed tiering of the data would stymie the intent to post medical data.
Commenter (12731) asserts that the EPA has not demonstrated that the Agency has the capacity to administer a system of tiered access to the myriad studies that support all of its regulatory decisions and influential scientific information. The commenter states that, as for CBI, the EPA points to its existing regulations on disclosure of CBI as a potential framework that would govern tiered access to such information. The commenter states that those regulations, however, provide a detailed process for applying the rules and adjudicating disputes as to confidentiality.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Arbitrary System
Comment: Commenters (11396, 11576, 12430, 12731, 13788) contend the rule's proposed tiering system is arbitrary in certain respects.
Commenter (11396) states that the Options in the SNPRM are vague and arbitrary. Commenter (12430) states that, rather than making the EPA's proposals consistent with data privacy law, the SNPRM's proposed tiering provision underscores the arbitrariness, unworkability, and unlawfulness of the proposal.
Commenter (11576) states that the proposed tiered-access approach is arbitrary and unexplained, as the EPA has failed to provide any analysis to justify the approach. The commenter adds that the EPA has not adequately responded to concerns previously raised in comments to substantiate how its tiered approach would remedy existing concerns.
Commenter (12731) asserts that the EPA's preferred approach, which "would allow Agency consideration of studies where there is tiered access to data and models," is arbitrarily vague and notional. The commenter states that nowhere does the Agency specify the types of data and models it envisions would be encompassed by each tier, or even how many tiers might exist. The commenter states that the EPA fails to provide sufficient description or necessary analysis of the Agency's envisioned tiered access approach.
Commenter (12731) asserts that the EPA has arbitrarily offered no explanation of how and to what extent legal restrictions on the sharing of confidential information would impede its preferred "tiered access" approach.
Commenter (12731) asserts that the EPA failed to consider the likelihood that tiered access would not be practical or ethical if study participants did not consent to their personal information being examined by outside parties, even with restrictions. The commenter states that it would be arbitrary for EPA to ignore this important aspect of the problem, especially in disregarding the comments of its own science advisers on these issues. The commenter states that several members of the SAB expressed concerns along these lines. The commenter states that the SAB expressed a similar concern in its majority report on the proposed rule:
      It may not be feasible to identify and make available data in epidemiological studies that arise from small datasets or targeted geographic areas, especially if the Informed Consent Form indicated that only the particular researchers who conducted the study would have access to the information and data. If the participants agreed to grant only a select group of researchers access to their personal information, then that consideration should be respected, and such information should not be supplied to additional people for validation. It would probably be impractical, to go back to the participants and request their approval to provide additional people access to personal information.
Commenter (12731) states that the vagueness of the tiered access approach, if finalized, would render it arbitrary for another reason as well: researchers whose work is relevant to the regulatory process would no longer be able to pursue studies to ensure that the EPA could consider their findings. The commenter states that, by replacing its prior policy of considering peer reviewed studies with a prerequisite of indeterminate "tiered access" to data and models, the Agency has disregarded these stakeholders' reliance interests in a predictable system for accepting and considering scientific information. The commenter states that the EPA has failed to provide the more detailed justification necessary for altering its policy in a way that would inject uncertainty into policy-oriented scientific inquiry.
Commenter (13788) asserts that the EPA's failure to offer a reasoned explanation about how its proposed tiered access system would be implemented under this rule renders it unsupported and arbitrary. The commenter states that it is not enough for EPA in the SNPRM to introduce and vaguely discuss "tiered access."
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Establishment of the Data Repository
Comment: Commenters (10974, 11487, 11496, 11576, 11894, 12710, 12720, 12731, 13788, 11900, 12464) express concern that the EPA has not yet established a data repository nor established who or what entity will create and manage the tiered access approach.
Commenter (10974) states that, while the EPA described a model of private data treatment by the RDC, NCHS, CDC, there is no suggestion or reference in the SNPRM for how the EPA would establish this level of data stewardship, administration, and curation. Commenter (11900) asks how will researchers access data stored in a restricted-access enclave?
Commenter (11487) states that the EPA's proposal forces an untenable choice on researchers: know that their study will be inappropriately down weighted, or accept the problems that can accompany submission of their research into a repository that has not been satisfactorily described. The commenter states that, when environmental health researchers inform potential study participants that their data will be placed into such a repository, they will likely find it harder to recruit. The commenter states that the problem will be particularly acute in historically marginalized and disproportionately polluted communities whose involvement is essential.
Commenter (11487) states that the EPA's invocation of the NCHS's RDC acknowledges the necessity of establishing a way to provide carefully controlled access to identifiable data, but it fails to demonstrate that it has fully considered how such a model might be used for the many studies the Agency ought to be considering. The commenter states that the SNPRM language merely states that the RDC is a model, and that the EPA is conducting a pilot study on how RDC might host EPA datasets. The commenter states that this falls far short of the kind of detail necessary before finalizing a rule. The commenter states that, because the RDC contains data collected by CDC and other federal agencies, not by independent academic researchers, a great deal of additional work and infrastructure would be necessary before it could host the full range of data that should be informing EPA work.
Commenter (11576) notes that the EPA suggests that the RDC is a viable model for its tiered-access protocol. The commenter states that, as the protocol has yet to be fully developed, however, members of the public have no way of providing sufficient review or comment on the possible model. The commenter states that, at the time of publication of the SNPRM, there was no indication as to how long this evaluation will take and if it will be a successful model for implementation. The commenter states the EPA still has not provided a sufficient explanation as to how the Agency will consider all information available within a published study.
Commenter (11894) contends that the new proposal is opaque on the role the EPA intends to play in implementing a rule requiring studies and models to have public or restricted access to all data or code. The commenter states that the SAB conducted a consultation on mechanisms for applying a tiered access approach for securing PII and confidential information in August 2019. The commenter states that no consensus was attempted but several SAB members were skeptical because the EPA had not provided any specifics of how to adapt such an approach for the rule.
Commenter (12720) states that the EPA highlights the RDC program as a model but does not explain how such an approach could be workable in the EPA context, nor does it explain the criteria that must be spelled out in a research proposal as a prerequisite for obtaining access to the data. The commenter states that the EPA apparently is planning to rely on CDC to fulfill major data management responsibilities made necessary in the SNPRM, but has not made that clear within the text of the SNPRM.
Commenters (12731, 12464) express concern that the EPA is entirely silent on who or what entity will create and manage the tiered access approach. Commenter (12464) states that, if the EPA intends to operate its own data enclave to administer the tiered access alternative, it should state that intention unambiguously, identify the source of its legal authority to do so, and address how it will pay for it. The commenter states that the SNPRM suggests that the EPA intends to address such issues in "implementation guidance," (85 FR 15403), but they are far too fundamental to be left to guidance documents.
Commenter (13788) notes that the SNPRM references models in the RDC and CDC, mentioning that the EPA is currently conducting a pilot study using the RDC's secure data enclave, but then the Agency also asserts that "[d]evelopment of standard data repositories is still ongoing." The commenter states that, thus, even the EPA acknowledges that it is far from clear how this would actually work.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
Comment: Commenter (12710) asks if confidentiality agreements concerning PII in those studies can be overruled by whatever tiered access plan the EPA devises for such studies and, if so, at what risk to individuals and researchers?
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12719) asserts that the EPA has not tested the proposed methods to ensure that private data would indeed remain secure. The commenter states that, without testing their tiered approach, the EPA cannot be sure that it has protected individuals' privacy, which raises serious ethical and legal concerns about data privacy.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Detailed Descriptions of Tiering System
Comment: Several commenters (11391, 11392, 11396, 11399, 11894, 12464) contend the proposed revisions should include a more detailed description for implementation of the tiered system as described below. Commenters suggest the following implementation questions need to be addressed:
 How "tiered access" will be achieved? (12464)
 Who will determine which data and models count as "restricted"? (12464).
 To whom would the EPA release the otherwise restricted data and under what circumstances? (11396)
 Where data will be stored? (12464)
 Who will determine who has access to the stored data? (11396, 12464)
 Who will pay for the operation of such a system? (12464)
 How would the tiered access process be administered and funded? (11396)
 Where will such a system be housed within the Agency? (11392)
 The EPA is not a data management agency  --  does it have the capability to manage a large amount of sensitive data? (11392)
 Does the EPA have a robust cybersecurity strategy? (11392)
 Does the EPA have staff who are capable of running this system? (11392)
 How many people will the EPA need to hire? (11392)
 How will members of the public apply to access the sensitive data? (11392)
 Would the recipient(s) of the restricted information be a subset of the public with demonstrated scientific expertise? (11396)
 How will it evaluate the adequacy and integrity of any tiered access review that follows? (11396)
 Who at the EPA will determine external access to the data, and at what tier? (11392)
 Who at the EPA will determine what parts of the datasets belong in each tier of the system? (11392)
 How will this address restrictions put on PII protected by the Health Insurance Portability and Accountability Act, the NIH, and IRBs, which contain no applicable "tiered access" exemptions from privacy requirements? (11392)
 What framework or level of "tiered access" would be deemed acceptable to EPA? (11399)
 What is the difference between tiered and restricted access? Option 2 does not use the term "tiered access," but nonetheless discusses (85 FR 15405) giving intermediate consideration to studies based upon data with "restricted access"? (11894, 12715)
 What standards will be used to determine what counts as "restricted data and models" subject to tiered access? (12464)
 Does the EPA mean to exclude restricted access in the first version and that tiered access is another term for publicly available? (11894)
 Is the EPA is proposing to consider studies where participants and researchers have agreed from the outset to a tiered access approach, or whether the Agency is proposing to itself provide tiered access to certain confidential data, even where researchers and study participants have not agreed to such sharing at the outset? (11391)
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenters (9455, 12457, 12731) contend that the EPA fails to provide sufficient description or necessary analysis of the Agency's envisioned tiered access approach. The commenters suggest that the following information is necessary:
 Discussion of the process of allowing access to its tiered program other than noting that the highest tiers would be accessed only by certified" researchers (9455).
 Description of "certified researchers" (9455).
 Description of how a demander of access would be judged to merit access (9455).
 Identification of the access fees, a potential barrier to the public but not to regulated industry (9455).
 Protocols for this tiered sharing approach (12457).
 Descriptions of who will be given access to the different data tiers (12457).
 Operational details associated with sharing data (12457).
 What data platforms will be used to publicly share data that will enable publicly shared tiered and non-tiered data (12457).
 What funding sources will be made available to cover the additional time and cost of data sharing (12457).
 Details relating to the determination of tiers (12731).
 Procedures for granting access to various tiers of data; operation, curation, management and oversight of data repositories; data repository ownership (12731).
 Whether, and if so, how repositories involving PII and CBI could be legal (12731).
 Safeguards to prevent hacking of web-based repositories (12731).
 Liability implications for researchers handling PII or CBI (12731).
 Considerations relating to the extent to which study authors would participate generally (12731).
 Costs (12731).
 How the Agency would ensure compliance with such statutory requirements or evaluate the extent of the restrictions they might place on its tiered access approach (12731).
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11576) notes that the SNPRM states that "[u]nder a tiered approach to accessing data and models that include CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects, access is more restricted for more sensitive data and models. Thus, the amount of information available for analysis is dictated by the tier. The greatest amount of information is made available at the most restricted access tier." The commenter states that this fails to describe what criteria must be met or what grounds any member of the public -- including researchers or advocates -- must have in order to access potentially sensitive information.
Commenter (11890) states that the alleged transparency sought by this SNPRM cannot be obtained for many public health studies if the EPA honors existing legal and ethical requirements for protecting participants' privacy. The commenter states that these study results would need to be restricted (tiered access) to researchers with legal data-sharing agreements and safeguards regarding protection of privacy.
Commenter (12720) contends that, even if access is restricted, the EPA needs to ensure that parties accessing the data (whether EPA staff, EPA consultants, or the public) are legally bound to protect the information, such as through a non-disclosure agreement, data use agreement or the like.
Commenter (12731) express concern that the proposed regulation should clearly address the issue of obtaining public access to datasets while maintaining the privacy of the participants and confidentiality of the data, because without such access, sensitive data and CBI could be excluded entirely from consideration as pivotal regulatory science."
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11900) notes that the SNPRM described the restricted access tier as "limited to authorized researchers and not possible for the general public." The commenter states that, however, the EPA does not define a process for vetting applicants or criteria for granting access. The commenter states that restricted access must be limited to researchers with a legitimate scientific interest in the data, otherwise restricted access is simply `accessible to the public by application.' The commenter states that the RDC, identified as a possible model by the SNPRM, requires researchers "submit a research proposal outlining the need for restricted-use data." The commenter states that a stated primary goal of the SNPRM in making data available is to support reanalysis of study data. The commenter states that it is unclear on what basis applications seeking to perform reanalysis would be judged, because reanalysis is not typically considered research. The commenter states that the EPA must clearly define how applications to access restricted-access data will be evaluated.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11900) states that an established practice for evaluating applications for controlled data is to have a Data Access Committee, rather than leave decision-making in the hands of a single individual. The commenter states that, for example, large intramural research studies conducted by the National Institute of Environmental Health Sciences (NIEHS), including the Sister Study and Agricultural Health Study, have committees to review data requests. The commenter states that advises the EPA to establish a Data Access Committee that includes equal representation by academic scientists, industry scientists, public-interest researchers, and community members who are nonscientists. The commenter states that, similar to requirements for IRB membership (21 CFR section 56.107), the committee should have diversity of membership (including race, gender, and cultural background), and members must not have any conflict of interest with the applications they are evaluating.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of the final rule will consider existing best practices for data governance.
Comment: Commenter (12699) suggests the any limiting of the release of underlying information, if required, should be narrowly tailored to cover only the specific information that is unable to be disclosed. The commenter states that through a tiered approach, researchers signing non-disclosure requirements should be able to gain access to the restricted information. The commenter states that one exception would be if there was a genuine concern for access to be provided even to researchers signing non-disclosure agreements or who have been subject to other privacy and related requirements.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Opportunity to Comment on Tiered System
Comment: Commenters (11393, 11184, 11407, 11576, 12403, 12412, 12474, 12731, 13549, 12464) assert that, in the SNPRM. the EPA has failed to provide the requisite public notice.
Commenters (11393, 11184, 11407, 11576, 12412, 12474, 13549) state that this vague aspiration -- saying a tiered approach to accessing data is possible, without details -- is insufficient and before the EPA can finalize the rule, it must lay out and analyze those approaches for public review and comment. Commenter (12731) asserts that, without any specific proposals as to how it would implement its preferred tiered access approach, the EPA has failed to provide the requisite notice.
Commenter (11576) asserts that the tiered-access approach introduced in the SNPRM problematically and arbitrarily relies on an approach to be developed in the future without providing commenters with the opportunity to comment on the exact tiered-access methodology it will use.
Commenter (12464) suggests that the EPA has not fully thought through the implications and legal challenges to a tiered access system. The commenter states that the public does not have enough information on such a program and therefore the EPA has not given the public a meaningful opportunity to comment on the proposal. The commenter states that it is not clear how the tiered access alternative would be implemented or who would be responsible for implementing it. The commenter states that the tiered-access alternative in the SNPRM contains multiple gaps and ambiguities.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance.
Comment: Commenters (11392, 12720, 13788, 14397) assert that it is inappropriate for EPA to use the public comment period to illuminate issues that the Agency itself should have clarified within the original proposal and SNPRM.
Commenter (11392) asserts that the Agency's solicitation for public comments on this matter is insufficient given the paucity of details on this proposed solution to the publication limitations of sensitive data.
Commenter (12720) states that it is clear that the EPA has not adequately planned for the major complications to science activities imposed by the SNPRM, because the EPA is seeking the public to resolve fundamental implementation issues through the public comment period. The commenter states that, given these clear deficiencies, the SNPRM should be withdrawn.
Commenter (13788) states that the EPA's assertion that "[i]nformation received during this public comment period will, among other things, help inform improved guidance and best practices related to preserving and providing access to data," does not allow commenters access to sufficient information about the course the Agency's final rule will take to permit meaningful comment. The commenter states that the whole point of an SNPRM is to clear things up and to allow enough release of information to permit the public to evaluate fully what the Agency is proposing. The commenter states that the EPA has failed in that regard concerning its proposed tiered access approach.
Commenter (14397) states that, not only is the EPA is proposing a major change to the way science is evaluated with regard to public health, but now the EPA doesn't even have a clear idea of how this policy should be implemented. The commenter states that this proposed approach would incorporate suggestions from commenters on policy implementation without giving the public a chance to review and comment on them. The commenter states that this approach ignores the nation's system of public input and amounts to no more than wishful thinking that an acceptable approach will present itself. The commenter states that this tiered approach proposal would rank studies in a way that is, ironically, lacking in transparency.
Response: Implementation of the final rule will consider existing best practices for data governance.
Privacy Concerns
Comment: Commenters (11192, 11361) state the proposal does not describe how tiered access would effectively mitigate privacy concerns, especially as it provides no description regarding those who could gain access to different tiers of data.
Commenter (12731) states that the EPA fails to assess the legal and practical barriers to implementing tiered access.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance.
Comment: Commenters (9455, 12715) express concern that the tiered process to gain access to CBI, proprietary information, or PII may lead to cybersecurity concerns.
Commenter (9455) states that, even if privacy protection could be accomplished satisfactorily, the EPA cannot guarantee that hackers could not enter the website and alter the data. The commenter states that protecting databases from intrusion is known to be an often-lost arms race. The commenter states that frequent news items about hacking of databases kept by large corporations and governmental agencies demonstrate this cybernetic problem. The commenter states that many parties have motivation to alter the data being proposed for EPA's public posting. The commenter states that the EPA proposes to post immense databases, analyses and models for all new standards and regulations spanning every aspect of environment and environmental health. The commenter states that the potential for massive disruption looms large. The commenter states that, while the EPA is testing tiered access by using the RDC program, most data in the RDC have nothing to do directly with industrially or commercially important standards and do not invite hacking except possibly by pranksters and ransomware blackmailers. The commenter states that they contrast with the data to be posted by EPA which would underlie standard-setting. The commenter states that the EPA data would be a prime target for funded hackers to impose re-identification of medical data and to falsify the database by making changes designed to disprove effects of pollution.
Commenter (12715) states that this presents yet another reason that data are not made publicly available, which the SNPR does not address. The commenter states that, accordingly, cybersecurity issues need to be addressed to protect the identity of all participants in a study. The commenter notes that the National Academy of Sciences states that the identification of personal information is a cybersecurity concern:
      In addition to cybersecurity concerns, computer scientists and cryptographers have demonstrated that statistical analyses of data sets that generate highly precise results -- such as geographic specificity or other characteristics that identify respondents -- may result in privacy breaches. This presents a new challenge that federal statistical agencies are just beginning to address.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
Comment: Commenter (11492) asserts the tiered-access approach, by which data that cannot be made public can be shared with a few for independent validation, is unworkable because it is illegal to share personal health data with anyone, so sharing it with one or a handful of people is just as illegal as sharing it with the general public.
Commenter (11399) states that it is questionable whether a tiered approach is feasible or could be managed to the extent that confidentiality could be maintained.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
Comment: Commenters (11192, 11361) contend that there are significant barriers to producing multiple datasets for different tiers of access, but perhaps the most obvious vulnerability in the proposal of a tiered access is that the EPA does not provide any criteria or suggested process regarding who would or could receive access to which tiers of data. Commenter (11361) states that, while the EPA mentions the potential creation of multiple versions of datasets that include privacy-related data transformations, the Agency fails to acknowledge that any data that needed to be transformed for any tier would need to be transformed for all tiers.
Commenter (11576) notes that the SNPRM states that "risk reduction techniques include creating multiple versions of a single dataset with varying levels of specificity and protection[,]" and that "[t]he benefit of tiered access is that data users who wish to conduct activities with a statistical purpose without first obtaining special authorization have access to the versions of the data in the least restricted tiers, allowing them to conduct research while protecting confidentiality." The commenter states that, while the notice is unreasonably short on specifics, this suggests how the tiered-access approach might operate, with tiers that are less or more restricted than others. The commenter states that the EPA does not describe which users would be granted access to the vague "higher restricted tiers" and, importantly, what type of information would be made available at each tier. The commenter states that there is not any indication of who would be responsible for reviewing access requests and otherwise managing the large databases the revised proposal would require. The commenter states that, based on the Agency's recent statements to the House Committee on Science, Space, and Technology, it appears the EPA intends for this burden to be shouldered by other agencies and private institutions.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
Recommendations
Comment: Commenter (9587) suggests the EPA be required to provide clear and explicit reasons why a proposed replication study should not be permitted.
Response: In this final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
Comment: Commenter (11401) recommends that the EPA, in conjunction with this rulemaking, review its program for ensuring rigorous oversight of CBI, proprietary data or PII program. The commenter states that, for those studies that are not either fully transparent or transparent with tiered access, the EPA should determine that they cannot be used solely as the basis for a regulatory action or when finalizing influential scientific information. The commenter states that such studies may be considered, but alone cannot form the basis of a regulatory action without the inclusion of a fully transparent or transparent with tiered access study.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
Comment: Commenter (12720) suggests the EPA consider developing tiers of public access that may provide a base level of information (e.g., a robust summary or final study report which is devoid of confidential protected information) for the general user or member of the public and then restrict the availability of additional information to a smaller group when access to confidential or copyrighted information is warranted. The commenter states that, in matters of public health, it is common to require disclosure to a trusted third party. The commenter states that, in the case of environmental epidemiological studies, micro-aggregation of data can be used to protect personal identity.
Commenter (12720) notes that the SAB letter states that it may not be practical for EPA to make all studies or other regulatory science used in a final regulatory action available to the public. The commenter suggests the EPA consider producing a list of study data considered in an evaluation and then strive to provide data for critical studies driving regulatory limits. The commenter states that the SAB notes that:
      the requirement to make data available for public inspection will be more easily implemented for some datasets than others (e.g., GLP [Good Laboratory Practices] studies include individual sample data as part of standard reporting). The expectation that data and methods will be available for all endpoints may be unrealistic. EPA could consider designating scores for information availability with the highest tier assigned to studies that define methods (e.g., protocol) as well as raw (individual sample level) data. Additional scores could be driven by sharing data on other related endpoints in the pivotal study to facilitate data interpretation. Scores could subsequently be used to evaluate data utility, uncertainty, etc. Ultimately, there are some datasets where CBI and/or PII data will require more onerous steps to protect data confidentiality. Based on EPA responses to SAB questions, it appears that the Agency is seeking approaches to manage these data issues.
Commenter (12720) notes that any such scoring method has not been detailed by EPA in the SNPRM and could be biased given the unclear definitions and lack of evaluation criteria. The commenter states that, while the EPA may be "seeking approaches" to manage data issues, those issues should have been resolved by EPA before issuing the SNPRM. The commenter states that any such approaches determined by EPA itself in preparation of a proposed rule must be set forth for robust public inspection and comment because they are central to the ultimate impact of the SNPRM within the Agency and on stakeholders outside of the federal government. The commenter states that the SAB comment identifies "data utility, uncertainty, etc." as potential components of a scoring tool, but this is a poor representation of the complexity and depth of information available in scientific studies that would require consideration.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
<IP_ADDRESS> Researchers
<IP_ADDRESS>.1 Burden
Comment: Commenter (11377) states that the expansion of scope represented by the SNPRM increases the already substantial burden that this rule would place on researchers to ensure the privacy and confidentiality of research subjects. The commenter states that the rule fails to acknowledge the enormously burdensome and costly complexities involved in redacting individuals' information in a way that both protects individuals' privacy and ensures that the data is still useful for future analysis. The commenter states that the SNPRM would make this problem worse, as burdensome redaction would be required for influential information, rather than only regulatory decision-making information.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11391) states that many of the studies the EPA has traditionally relied on in regulating environmental pollutants necessarily involve confidential human health data. The commenter states that researchers, and the institutions that employ them, have a legal and ethical obligation to protect the privacy of study participants. The commenter states that, if enacted, the rule -- particularly as expanded in the 2020 SNPRM -- would make it much more difficult for researchers to fulfill that obligation and credibly promise study participants that their patient information will remain confidential. The commenter states that this could have a serious chilling effect on the conduct of important public health research in the U.S. The commenter states that privacy concerns would almost certainly hamper such research. The commenter states that critical lines of scientific inquiry that would have been pursued may not be and the quality of data that is obtained may be poorer than it otherwise would have been.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11392) states that, if the EPA expects researchers to implement this system, then the Agency is creating enormous new requirements for the research community. The commenter states that describing such requirements simply as "multiple versions of a single dataset with varying levels of specificity and protection" is excessively vague for such a monumental overhaul of how the Agency considers science in its rulemaking. The commenter states that, if the EPA intends for the scientific community to adopt  --  and potentially implement  --  these dataset and access requirements, it is not offering researchers a meaningful description of the parameters on which they can comment. The commenter asks the following questions:
 If the intent is for the researcher to determine the appropriate level of access, who would determine whether a researcher's determined access levels appropriately met the requirements of the rule?
 Would researchers be expected to review the applications for reanalysis in perpetuity?
 Has a cost analysis been done to assess the burden this rule would place on researchers and on the institutions hosting the data?
 What happens if a research team has since disbanded?
 Does the EPA expect researchers in other countries to work with the Agency to manage such a data access system so their studies can be used by the United States government?
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11479) states that, even for the subset of research studies that could be shared within a tiered-accessed system, implementation of such a system would be onerous, costly, and infeasible for several reasons. The commenter states that de-identification of data would be an immense undertaking and is not feasible in many cases and for many study designs, particularly because the EPA is concerned with environmental threats. The commenter states that, to understand environmental threats, researchers often must know the precise locations of parameters such as environmental exposure and health effect occurrences in a subject population. The commenter states that, in many studies, these locations are personally identifying. The commenter states that it is difficult and often impossible to release this data in a way that protects study subjects' identities, and this exercise would be required under the tiered-access approach proposed in the EPA's SNPRM. The commenter states that, while the rule does not make this clear, researchers would bear the burden of deciding and implementing the tiered-access system for their data, according to EPA's conversations with Congress. The commenter states that the rule may make researchers wary of participating in policy-relevant research. The commenter states that this potential effect is antithetical to the stated goals of the proposed rule.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11495) states that, even before this new expansion of scope, it was already apparent that the Agency would be unable to handle the immense logistical and technical challenges this new rule would create. The commenter states that the EPA has previously not provided web-based or clearinghouse depots for the posting of data, assuring quality control, nor have they provided a system to ensure database integrity and maintenance. The commenter states that this has been the case even in more innocuous situations that don't involve personal privacy and proprietary issues. The commenter states that, even with this dramatic expansion in the scope of data sharing requirements, there has been no indication that the EPA is ready or willing to take the responsibility to develop such a clearinghouse website to house data and the associated information or metadata, including defining a structure with all protections and securities. The commenter states that such a data system is beyond the scope and complexity of any data system the EPA has previously developed for its internal use nor does it easily fit the existing models used by the Agency, such as the one used for its Air Quality System for monitoring data. The commenter states that this system is fully contracted, maintained and assessed outside the direct control of the Agency. The commenter states that, given that the EPA will be unable to handle all of these new responsibilities, it is clear that this SNPRM will instead impose tremendous burdens on researchers and scientists outside the Agency. The commenter states that scientists and researchers will have to take time-consuming steps to make the data underlying their studies publicly available in order to have their study considered by EPA.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
<IP_ADDRESS>.2 Funding
Comment: Commenters (10748, 12418, 12731) express concern that there is no financial support for researchers to redact data, host data, and/or provide tiered access infrastructure to the databases.
Commenter (10748) states that there is no financial or technical support for researchers, which would reduce the capacity of researchers to produce unbiased and credible research. The commenter states that the Agency would also need financial support to host data and provide tiered access infrastructure to the databases. The commenter states that this too would be expensive and burdensome to the Agency. The commenter states that, with the current effective review process and with leading scientific publications already encouraging data transparency, it does not make fiscal sense for the Agency to implement this rule. The commenter states that the proposed rule would, without doubt, impede the development and utilization of new science. The commenter states that the cost of doing research both in investigator time and required funding to develop regulated processes for broad access to data would be substantial, yet the SNPRM does not define any financial and technical support for researchers in this regard. The commenter states that, without defined financial and technical support, the result would be a crippling, if not an elimination of the strong, crucial science that has been the backbone of existent, validated regulatory reviews per the CAA mandate and found in other environmental statutes. Instead researchers may have to turn to other funding sources making their work biased and less credible.
Commenter (12418) states that the EPA has not specified the process for de-identifying data, ensuring study participant privacy, and covering associated costs, if the proposed rule is indeed adopted. The commenter states that the EPA has not indicated whether the researchers or the Agency will bear the cost of providing de-identified data and models upon request, and/or creating the additional versions of a dataset to support tiered access. The commenter states that, if the responsibility falls to researchers, this hurdle will likely further limit which studies will become available for consideration given that costs to de-identify data and additional resources to comply with data and model sharing requirements are not typically requested in grant budgets. The commenter states that, if instead the EPA will cover the cost of deidentifying data and models to sufficiently comply with the proposed rule, the anticipated costs should be assessed along with other proposed merits and drawbacks. The commenter states that the costs of de-identifying data, appropriately managing those data, and ensuring that no PII is released for the foreseeable future may far outweigh any perceived benefits of this rule.
Commenter (12731) states that recent information has also underscored the EPA's failure to consider the costs to researchers of demanding that older data be made available for reanalysis (either to the public or through a tiered access mechanism), and the implications that cost would have on EPA's access to regulatory science. The commenter states that, for example, SAB member Robert W. Merritt commented in the SAB's September 2019 consultation that data from older studies may be located in obsolete storage media or data formats that are either impossible or very costly to convert to a shareable form. The commenter states that, because the grants funding older studies have already lapsed, there would likely be little to no funding available for datasets owned by academic and non-profit institutions to be shared -- meaning that much of this research would not be eligible for EPA's consideration, and that the EPA might be drawing from a biased pool of for-profit industry studies. The commenter states that, previously, the EPA had accepted some responsibility for making information publicly available in a "cost-effective" way. The commenter states that now, it has disclaimed this role and abandoned any pretense of contributing to the transparency and validity it supposedly seeks, opting simply to discard studies out of hand (85 FR 15402).
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12464) states that an industry representative who wants access to such data faces no costs or burdens. The commenter states that, instead, academic researchers and universities would assume the responsibility and the financial burden of managing the logistics of making the data and models publicly available. The commenter states that, even then, the EPA is under no obligation to consider or use the study to inform its regulatory or informational processes. The commenter states that, even research, such as the Six Cities Study, that has been subject to a comprehensive reanalysis by the Health Effects Institute, would apparently not be eligible for consideration under the SNPRM, because its data are not available for further reanalysis by other researchers.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
<IP_ADDRESS>.3 Other
Comment: Commenter (11495) states that the SNPRM imposes implicit obligations on parties outside the federal government, including on researchers and potential study participants. The commenter states that researchers are often obligated to protect the confidentiality of study participants, and the SNPRM would require researchers to either breach those confidentiality agreements or risk the loss of willing study participants.
Response: The underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11497) asserts that the requirements for the public availability of data will not only undermine the inclusion and use of previously conducted epidemiological studies in the pool of "influential scientific information" and the science underlying "significant regulatory decisions"; they will also damage the ability of researchers to conduct future studies. The commenter states that researchers' ability to recruit potential study participants will be severely undermined if participants fear that their personal information may be made available for public scrutiny. The commenter states that this is particularly true for communities of color that are both disproportionately exposed to hazardous chemicals and have historical reasons to distrust researchers; yet, it is the exposures to these communities, and the resulting health effects, that most need to be understood and addressed.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. The underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12435) contends the SNPRM could increase the burdens facing science and health researchers. The commenter states that the proposal disadvantages scientists and clinical researchers whom this proposed rule would burden with reporting requirements, and study subjects whose participation and data would be prevented from informing public policy. The commenter states that the exclusion of many studies from informing policy has a direct impact on the scientific community whose work will be devalued under this proposal. The commenter states that pressure for underlying data to be public will induce participation bias in studies; people will be less likely to participate if their privacy is not assured. The commenter states that there is immeasurable harm done by politicization of the scientific basis of Agency rulemaking that Congress unquestionably intended to prevent when it mandated the role of scientific advisory boards under the CAA.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. The underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12457) states that the proposed revisions should include a detailed analysis that examines how these proposed revisions would impact laboratories and study investigators.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
<IP_ADDRESS> Potential Bias with Tiered System
Comment: Commenter (9455) states that the proposed rule fails to consider the probability of distorting studies because of minority distrust, privacy concerns, and allowing industrial access. The commenter states that the small amount of social science that analyzed the broad informed consent form indicates that members of traditionally discriminated against minority groups and people concerned about privacy shy away from giving broad consent. The commenter states that knowledge that for-profit companies will have access to the data also results in lower broad consent. The commenter states that participants in studies to be used for EPA regulation under the proposed rule would have to be informed that the data would be posted and that regulated industries would have access to them before they are asked to sign the broad informed consent forms.
Commenter (12435) asserts that pressure for underlying data to be public will induce participation bias in studies; people will be less likely to participate if their privacy is not assured. The commenter states that there is immeasurable harm done by politicization of the scientific basis of Agency rulemaking that Congress unquestionably intended to prevent when it mandated the role of scientific advisory boards under the CAA.
Commenters (11192, 11361) contend that there are significant barriers to producing multiple datasets for different tiers of access, but perhaps the most obvious vulnerability in the proposal of a tiered access is that the EPA does not provide any criteria or suggested process regarding who would or could receive access to which tiers of data. The commenters state that, as written, the EPA could grant access to industry personnel eager to poke holes in public health studies and decline access to academic researchers interested in evaluating industry studies.
Commenter (11192) states that the proposal does not describe how tiered access would effectively mitigate privacy concerns, especially as it provides no description or metrics regarding those who could gain access to different tiers of data. The commenter states that this omission introduces an extraordinary potential vulnerability to political bias.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11479) states that it is unlikely that stakeholders such as the public and community groups would pay the necessary high costs to access data that has already been vetted through peer-reviewed studies. The commenter states that the exception is regulated industry, which is likely to be the only EPA stakeholder group interested and able to access data housed in a tiered-access system. The commenter states that the SNPRM makes clear that the rule's intent is to allow stakeholders the opportunity to "reanalyze the data and models and explore the sensitivity of the conclusions to alternative assumptions..."
Response: This rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12430) states that discouraging individuals or particular groups of people from participating in future studies would introduce bias into the studies, to the detriment of the public interest in protecting health and the environment. The commenter states that, to the extent that researchers require the public availability of data for future studies, this could have a chilling effect on people's willingness to participate. The commenter states that the possibility of identification, with the specter of denial of future employment or benefits based on disclosure of medical information, may make some population groups less likely to participate in future studies. The commenter states that this chilling effect has important implications for environmental justice, as it may become more difficult to conduct studies in communities of concern.
 Commenter (12702) asserts that the SNPRM allows arbitrary decisions to be made about what science can and can't be used. The commenter states that environmental laws direct the EPA to use the "best available science" in countless situations. The commenter states that the SNPRM creates multiple confusing pathways for considering long standing public health studies, including the Option of "tiered access" or giving preference to some studies over others. The commenter states that the EPA's purported goal of "transparency" means studies will be evaluated not on their quality but on non-scientific ill-defined metrics.
Commenter (12720) expresses concern that the proposal does not sufficiently explain how a tiered access system would avoid biased handling of data and models at EPA.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12731) states that, because the grants funding older studies have already lapsed, there would likely be little to no funding available for datasets owned by academic and non-profit institutions to be shared -- meaning that much of this research would not be eligible for EPA's consideration, and that the EPA might be drawing from a biased pool of for-profit industry studies.
Response: This rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12731) asserts that the EPA has completely disregarded the potential for pro-industry bias by considering studies for which interested parties have arranged for tiered access. The commenter states that the EPA's recent briefing to House Science, Space, and Technology Committee staff underscores how the tiered access approach would preference industry interests: researchers would be responsible for managing the logistics of making data and models publicly available and establishing levels of tiered access; a time- consuming and labor-intensive process for investigators who have already concluded their work and published their findings, possibly having expended the full amount of funding available, unless specially interested third parties bankroll the process. The commenter states that, once established, tiered access would also allow financially interested, repeat players such as regulated entities to test the findings of whatever studies they dislike. The commenter states that the EPA has thus proposed to establish a requirement that would favor industry interests by compelling disclosure of data and methods that regulated entities can more readily furnish and that only regulated entities will likely have the resources and motivation to scrutinize with any frequency. The commenter states that such favoritism in the Agency's rule would be arbitrary unless somehow grounded in its authorizing statutes. The commenter states that the EPA would need to assess, consider, and balance the substantial unfair advantage industry would receive against any theoretical, marginal increase in the reliability of scientific information that could result from a tiered access approach.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. This rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12720) states that the claim that publicly available dose response data and models would allow for independent validation stands in direct contradiction to the legal privacy protections that apply to key data necessary for re-analysis. The commenter states that the proposed partial redaction of sensitive information poses a cascading set of problems, because the statistical models characteristic of epidemiologic investigations rely on the inclusion of potentially confounding variables (e.g., age, sex, home address, health status, diet and alcohol consumption, smoking history) in order to properly isolate the pollution-health relationship with precision. The commenter states that, to understand the dose-response connection, these studies analyze detailed health, demographic, spatial, and behavioral information from thousands of people. The commenter states that this information is extremely sensitive and collected at the individual level. The commenter states that, as such, our nation's health privacy laws and IRB protocols require researchers to keep the data secure and confidential to prevent misuse. The commenter states that, collectively, these data points help researchers understand and isolate the cause-effect relationship between exposure to air pollution and risks for various health problems. The commenter states that it would be extremely difficult if not impossible for anyone using partially-redacted data sets derived from epidemiologic cohort studies to "validate" the results of the original studies, because such investigators would not be working with complete data sets.
Commenter (12720) states that, as further demonstration, the 2009 Integrated Science Assessment for PM2.5 notes that "[a]ppropriate statistical adjustment for confounders requires identifying and measuring all reasonably expected confounders." The commenter states that, therefore, exclusion of some potentially sensitive confounding variables from an underlying dataset likely would lead a different team of investigators to a different result. The commenter states that causing this wrongheaded and indefensible outcome results from the core approach and conceit in the proposal, revealing it to be yet again, arbitrary and capricious and an abuse of EPA discretion. The commenter states that the quantitative findings of dose-response relationships would almost certainly differ, not as a result of any true difference in the quantitative exposure-effect relationship, but because the original work relied on complete data sets and the new analyses would not -- due to the proposal. The commenter states that the resulting discrepancies in quantitative findings could serve as motivation to call the original study results into question due to faulty and incomplete re-analyses.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.4 40 CFR 30.5: Option 2
<IP_ADDRESS> Support/Oppose Option 2
<IP_ADDRESS>.1 Support Option 2
Comment: Commenters (9362, 10973, 11358, 11388, 11395, 12395, 12397 12406, 12453, 12471) support Option 2.
Commenter (9362) asserts their preference for Option 2 of section 30.5 but notes the phrase "other things equal" should be deleted. The commenter questions who makes the determination that "other things are equal" and states that each study should stand on its own merits and those studies in the public domain should be given greater consideration. Additionally, the commenter approves of the statement "Furthermore, the Agency will identify those studies that are given greater consideration and provide a short description of why greater consideration was given," as long as all relevant scientific information has been considered and is documented in any EPA decision-making.
Commenter (10973) states that the EPA's alternative approach giving priority for publicly available studies is commendable. The commenter states that this is more of an "opt out" scenario where all appropriate scientific information is considered unless there is a specific reason to exclude it. The commenter states that, at the same time, this approach will continue to encourage open access to data and models. The commenter states that this approach is reasonable in that it furthers the goals of transparent rulemaking and open access to scientific data while recognizing that some relevant scientific information may not be fully publicly available. The commenter states that the EPA's attention to detail in this rulemaking is essential to assure the appropriate balance of transparency with inclusion of all relevant scientific information, while addressing necessary privacy and confidentiality concerns.