The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (6942) requests that EPA provide clarification and guidance for implementing the proposed regulation when human data are utilized for dose-response modeling and development of a chemical-specific cancer slope factor, reference dose, or reference concentration (e.g., risk assessments by the Integrated Risk Information System (IRIS)).
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Institutional Review Board (IRB) Approval of Disclosure of De-identified Data
Comment: Commenter (6942) recommends that, for human data, information available to researchers be limited to individual-level, de-identified data, to ensure that the privacy and confidentiality of the study participants is maintained. Commenters (1006, 2171) state that detailed data are similarly available for research uses, under the approval and guidance of IRBs that help regulate and evaluate studies that use important data so identify can be preserved. The commenter recommends that any disclosure of de-identified data should require approval by an IRB.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
<IP_ADDRESS>.3 Confidential Business Information
Comment: Commenter (6362) recommends that, in circumstances where company CBI and other intellectual property may be implicated, EPA should confer with the CBI data owner to determine how to make that data available to the greatest extent possible without disclosing the CBI within that data, study, or model. The commenter states that how this is handled will likely be impacted by the type of regulatory decision and statute involved. Commenter provides the following example:
      For example, under TSCA, while the summarized study results, analysis, and final report may be publicly available, the underlying data in a health and safety study may qualify as CBI when the underlying data are not in the public domain and that data provides a commercial value to its owner. In such circumstance, it is the availability of the underlying data that determines whether or not an unpublished study can be used by a competitor to support its notification or registration of a substance overseas without obtaining ownership or citation rights to use such data, depriving the data owner of the value of its investment in the underlying data. Current EPA regulations require chemical manufacturers to submit health and safety studies under some circumstances. However, it is noteworthy that none of these regulations routinely require study submitters to submit underlying data along with a final report. This indicates that the final report likely communicates sufficient information about the potential health and environmental effects to the public when a company has submitted health and safety studies in which it has a commercial interest in protecting.
Commenter (6362) believes that making a final study report publicly available where the underlying data are CBI would, in most circumstances, be an effective way to make relevant information publicly available about studies and data EPA may rely on, but which must be protected as CBI in circumstances triggering this policy. The commenter provides that, in these situations, EPA can access the underlying data to confirm the methods, models, and approaches are based on validated procedures, accessible data, etc. According to the commenter, if necessary, when specialized expertise is needed, EPA could contract with an independent third-party science reviewer to confirm those findings, although we believe this would likely only be necessary in unusual circumstances.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA will continue to follow existing requirements/guidance for handling CBI.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed.
Comment: Commenter (6130) states that the proposed rule establishes increased protections for CBI, diminishing the amount of information available to the public to inform policy, whether from the scientific community or from the business community.
Commenters (0559, 6120) assert that CBI may be necessary in some cases for scientific experts to evaluate the potential for hazards due to chemical exposures e.g., in consumer products. The commenters suggest that responsible authorities, including state agencies, the EPA, and independent scientific reviewers should be able to request and have access to CBI when necessary to evaluate these hazards.
Commenter (5170) states that because CBI may be necessary to evaluate health risks from exposures to some products, EPA and independent scientific reviewers should be able to access CBI in assessing these risks. The commenter states that, given the recent apparent scientific fraud committed by the founder of Theranos, it would seem that CBI and industry-based science are due substantial scrutiny and transparency.
Commenter (6120) asserts that one significant way to improve transparency is to increase access to CBI; this would give the public confidence that this information is independently reviewed by experts in relevant fields. The commenter recommends that EPA establish a group of independent scientists who could serve as scientific reviewers of CBI data. The commenter provides that this review could be done under strict confidentiality agreements and would ensure more confidence in the scientific vigor of the data submitted. The commenter urges the EPA to follow the OMB guidelines for peer review to ensure peer reviewers are independent of industry and include academic scientists and allow scientists who receive government research grants to offer "independent scientific advice to the agency on other projects."
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenters (6126, 6906, 6935) support the need for CBI provisions/standards to protect data from disclosure.
Commenter (6906) commends the Agency for acknowledging the importance of CBI and simultaneously establishing that while transparency is key, data transparency must be conducted in a way where CBI and intellectual property are retained. The commenter states that industry heavily relies upon the use of CBI provisions in various EPA programs to help retain competitive advantage between companies. According to the commenter, this could be especially pertinent when addressing industry-sponsored testing data, modeling and risk assessments. The commenter provides that, additionally, many studies (especially dose response data and models, which are the focus of this rule) have a significant amount of personal information that would need to remain confidential as well. The commenter states that, for both instances, EPA needs to determine a standard to either sanitizing certain aspects of the data, so that they can be made publicly available, or a procedure regarding selective publication so that certain aspects of these studies can also be made publicly available.
Commenter (6935) suggests that any final rule should include a detailed, objective and consistent process for safeguarding CBI and that fits seamlessly with the Administrator's potential exemption process after it is expanded to provide additional information on criteria to be considered and procedures for exemptions.
Commenter (6126) urges EPA to incorporate Recommendation 3-4 of the U.S. Commission on Evidence-Based Policymaking which encourages the adoption of activities that protect public trust for confidential data.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed.
Comment: Commenters (6112, 6115, 6855) support the Agency's proposal and state that its policy and guidance documents sufficiently address concerns about unauthorized access to confidential personal and business information. Commenter (6162) states that, to allay concerns regarding the disclosure of CBI, the Alliance supports the proposed regulatory text of §30.5:
      Where the Agency is making data or models publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national and homeland security.
Commenters (6448, 6918) supports EPA's efforts to balance appropriate protection for copyrighted or CBI, including where protected by law, with requirements for increased transparency of pivotal regulatory science. Commenter (6448) notes that there will be situations in which CBI cannot be shared publicly but this should not preclude its inclusion in regulatory decisions. Commenter (6918) states that EPA must, and should, continue to recognize limits on data access and modify, as appropriate, efforts to enhance transparency to reflect these lawful, reasonable limits.
Response: Section 30.5 in the final rules states, "The Agency shall also give greater consideration to pivotal science based on dose-response data that include confidential business information, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation." It also states, "For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9."
Comment: Commenter (6379) urges EPA to establish a process to maintain confidentiality through a peer review process to ensure EPA can continue to rely on CBI even though it is not publicly disclosed while also ensuring that scientific rigor is applied to consideration and use of this information.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed.
<IP_ADDRESS>.4 Proprietary Data
Comment: Commenters (6150, 6191, 6375) provide recommendations on how to balance protection and transparency with regards to proprietary data.
Commenter (6150) asserts that the protection of proprietary data, models and methodologies should not apply to research that is funded by EPA or was developed under EPA contract. The commenter states that utilization of private models or failure to disclose methodologies should be discouraged, if not deemed a disqualifier, as such actions fail to meet minimum transparency requirements.
Commenter (6191) requests that intellectual property considerations be noted and that data with intellectual property or confidentiality concerns should not be shared. The commenter states that the sharing of data containing patentable information or trade secrets should not be required due to the potential negative economic implications of such a requirement.
Commenter (6375) suggests EPA solicit input from EPA Offices (such as the Office of Pesticide Programs) or other federal agencies (such as FDA) that rely heavily on copyrighted information, proprietary studies, and CBI for strategies to balance these concerns with transparency. The commenter states that, for example, EPA Offices (e.g. Pesticides) that rely heavily on proprietary studies/CBI apparently have already devised mechanisms that balance protection with transparency of pivotal regulatory science. The commenter provides that one mechanism is Data Evaluation Records (DERs) that summarize proprietary studies. The commenter provides that another is human health risk assessments that integrate findings in DERs such that stakeholders can assess the impact of a particular endpoint (e.g. health effect) in a particular study (pivotal regulatory science) on the overall risk assessment. The commenter states that both DERs and risk assessments have been made available in the docket or by other means. According to the commenter, while not as transparent as making the underlying data and models fully available, sufficiently detailed DERs and risk assessments are arguably as detailed and transparent as many health studies and risk assessments published in the peer-reviewed scientific literature. The commenter states that, regarding copyright protection, a potential alternative for copyrighted works that use data that are federally funded may be to solicit from the authors the raw data (which often isn't copyrighted) that were used to produce the copyrighted work. The commenter notes that these raw data, along with EPA's analysis of these raw data, could then be made publicly available.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed.
Section 30.5 in the final rules states, "The Agency shall also give greater consideration to pivotal science based on dose-response data that include confidential business information, proprietary information or personally identifiable information if these data are available through restricted access in a manner sufficient for independent validation." It also states, "For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9."
In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
<IP_ADDRESS>.5 Non-CBI Animal Studies
Comment: Commenter (6942) recommends that the Hexavalent Chromium Mode of Action Research study serve as a model for other research programs that desire to foster collaboration in the research community and promote transparency and validity of the scientific data used in regulatory risk assessment. The commenter provides that similar approaches, using the internet and web-based repositories, could be implemented for both government-funded and stakeholder-funded research. The commenter states that additional resources may be needed to accommodate the proposed rule, but with regard to animal research data, they do not anticipate that significant hurdles would arise, unless the pivotal study is highly dated, or the best available science. The commenter provides the following information regarding their experience:
      For recent research and risk assessment projects, we have used confidential, personal identifying information, with Institutional Review Board (IRB) approval, including data collected at the federal level (e.g., Social Security and National Death Index records), and we have also used de-identified personal information generated by the National Institute for Occupational Safety and Health (NIOSH) and the Institute of Medicine (IOM) for research projects related to the development of pivotal data for risk assessment. Therefore, we believe we offer highly informed comments to the proposed regulation based on specifically-relevant experience.
      In that regard, we offer the example of our Hexavalent Chromium Mode of Action (MOA) Research study. During this nine-year project, ToxStrategies created a vast repository of in vitro and experimental animal data, including original toxicology data, modeling code, and toxicogenomics, and that body of data has been made freely available for use by other scientists and regulators seeking to analyze the raw data (https://cr6study.info). Both U.S. EPA and California EPA have accessed and downloaded the raw data sets. The Hexavalent Chromium Mode of Action Research study was developed to better understand how hexavalent chromium caused cancer in rodents from drinking-water exposures. Key objectives of the research study were to (i) evaluate the MOA, (ii) develop and make assessible physiologically based pharmacokinetic (PBPK) data and models allowing for inter- and intra-species extrapolations in risk assessment, and (iii) ultimately to provide data that will improve upon the science used for risk assessment and setting drinking-water standards for Cr(VI). This research study has been conducted as a collaborative effort among multiple organizations, universities, and government researchers, and full disclosure of the original data has always been our objective.
Response: The final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
<IP_ADDRESS> Definition, Guidance, Standards and Training are Needed
Comment: Commenters (1008, 1011, 6126, 6362) recommend that EPA provide training/guidance to researchers to support making research data publicly-accessible. Commenter (1008) states that, should this proposed rule place transparency requirements on EPA-funded researchers, these researchers must be provided appropriate training and support to make research data created in future research studies publicly accessible. Commenter (6126) states that, if the intent of EPA is to rely solely on de-identified data, some researchers may lack the appropriate expertise to sufficiently de-identify data without additional training, support, and guidance on techniques and approaches for doing so. Commenter (6362) suggests EPA develop guidance on protecting privacy, de-identifying data, and settling disputes should a breach occur. Commenter (1011) suggests that EPA provide researchers guidance regarding proper channels for data dissemination.
Response: The final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
14.1.6 Other Implementation Issues that May Require Special Consideration
<IP_ADDRESS> Consideration of Differing Regulations and Requirements
Comment: Commenter (6191) states that, as research is increasingly the product of international collaboration, they recommend that differing regulations and requirements worldwide be considered.
Response: In section 30.9 of the rule it states, "The Administrator may grant an exemption to this part for a study on a case-by -case basis if he or she determines that greater consideration is warranted because: (1)technological or other barriers render sharing of the dose-response data infeasible; . . ."
<IP_ADDRESS> Consideration of Secondary Data Analysis Data Use Agreements
Comment: Commenter (6191) states that where secondary data analysis is being conducted, data use agreements prohibit broader dissemination of data, which would also prohibit data from being released publicly.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
<IP_ADDRESS> Regulatory Determinations Should be Based on More than One Study
Comment: Commenter (6875) encourages EPA to not base any regulatory determination on a single study, regardless of whether or not the data is publicly available. The commenter suggests that relying on a single study, no matter how robust it may be, can bring bias to a model or regulatory decision. If EPA must base a decision off of a single study, the commenter recommends that the data be publicly available as well as the reasoning and methodology behind why the study was chosen, how it is being used and why no other studies were deemed sufficient to be included.
Response: The rule will apply to the pivotal science underlying significant regulatory actions and influential scientific information. Section 30.2 states, "Pivotal science means the dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information." Pivotal science is not necessarily a single study or analysis.
14.1.7 Timing of Implementation
<IP_ADDRESS> Do Not Delay Implementation of Proposal
Comment: Commenter (6150) encourages EPA to not delay implementing core components of this proposal until the Final Rule is issued. The commenter asserts that it is appropriate and consistent with federal statutes for EPA to immediately begin to identify key studies that are likely to be pivotal in significant upcoming rulemakings immediately. The commenter provides that EPA could seek public comment on a list of these potential studies/models, allowing the public to nominate other studies or models. The commenter states that identification now would allow time for EPA to contact researchers and develop appropriate data sharing approaches. The commenter adds that such early action could also assist in identifying research used by EPA in the past that may need additional disclosures.
Commenter (6867) suggests that the EPA should not delay implementation of this rule unless required to do so by statute. The commenter states that certain commenters claim the EPA has not complied with specific statutory obligations. The commenter notes that they do not have a position on the EPA's statutory obligations, but they encourage the EPA to carefully comply with applicable procedural requirements so as to minimize the risk of dilatory litigation.
Response: The final rule is effective on the date of publication in the Federal Register. The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Also, the rule does not require any researcher or other outside entity to provide data to the EPA. Researchers may choose to make more data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
30.7 of the final rule states, "To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption." Forthcoming guidance/substantive rulemakings subject to notice and comment could be further opportunity for input from the public.
<IP_ADDRESS> Request for Implementation to be Delayed Until After Proper Procedures and Infrastructure are Established
Comment: Commenter (6908) suggests that EPA not implement all or portions of the regulations until the proper procedure and infrastructure, such as IT systems and databases, are in place and functional.
Response: The final rule is effective on the date of publication in the Federal Register. However, the Agency plans to have implementation guidance available by the date of publication in the Federal Register. However, as outlined in 40 CFR 30.3, the provisions of the rule apply to significant regulatory actions for which a proposed rule was published in the Federal Register after the effective date of the rule and influential scientific information submitted for peer review after the effective date of the rule. Further, EPA intends to issue implementation guidelines and statute-specific rulemakings that will provide further clarity.
<IP_ADDRESS> Ensure Data Access Early in the Rulemaking Process
Comment: Commenter (6375) suggests EPA should ensure that data access occurs early enough in the rulemaking process so that the public has sufficient time to analyze the data.
Response: The rule requires the EPA to identify and make publicly available the science that serves as the basis for informing a significant regulatory action at the proposed or draft stage to the extent practicable. The definition of publicly available in the final rule includes, "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment.
14.1.8 Impacts on other Federal Agencies, States and Tribes
<IP_ADDRESS> State and Tribal Implementation
Comment: Commenter (6125) expresses concern that the proposal does not describe how it will be implemented in regard to states and tribes. The commenter states that federal standards may govern the level of environmental protection within states and on tribal lands. According to the commenter, those levels of protection would be reduced under a new regulation that ignores evidence because of the new rule. The commenter asks the following related questions:
 Would states or tribes have any recourse when this happened, or could they keep the old standards?
 Under what circumstances would or could EPA disapprove a program for failing to adhere to the new federal law requirement?
 Does EPA believe that a state with an authorized program based on actual science would still meet federal standards after the program becomes regulated under censored science providing for reduced federal standards?
 Would EPA disapprove a state program if the state failed to relax its standards to reflect new federal requirements?
 What kind of oversight would there be when states make permitting or enforcement decisions, actions that are not covered by the proposal but possibly implementing regulations that are covered.
 Has EPA considered whether it would be consistent with the federal trust responsibility to tribes to relax requirements or standards, and thus reduce protection for tribal lands solely because of a new regulation based on ignoring reliable and relevant science?
 Would such a reduction in protection on tribal lands be consistent with principles of environmental justice?
Response: EPA plans to issue implementation guidelines and statute-specific transparency regulations or programmatic regulations. The final rule states, "The action imposes no enforceable duty on any state, local or tribal governments or the private sector." The EPA does not agree that this rulemaking would weaken human health protections. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
<IP_ADDRESS> Relationship Between EPA-Funded Research and Research Funded by Other Agencies
Comment: Commenter (6126) suggests that EPA has ill-considered the relationship between EPA funded research relevant for regulatory actions, and the vast amount of research funded by other agencies (e.g., NSF and NIH) or through non-federal funding sources (e.g., many universities, states, foundations, private firms). The commenter asks:
 Would EPA request data when needed or would researchers be expected to provide it unsolicited or on-demand?
 What obligations would researchers have to provide such information?
 What would be the consequences for not doing so?
 If EPA cannot obtain data from studies and thus excludes those studies from consideration, how can EPA ensure use of the "best available science?"
Response: This final rule does not regulate any entity outside the EPA. Rather, the requirements modify the EPA's internal procedures regarding the transparency of pivotal science underlying significant regulatory actions and influential scientific information.
14.1.9 Communications
Comment: Commenter (6126) suggests that government agencies develop a "learning agenda," which would be operationalized as a strategic plan for evidence-building activities. The commenter states that, to the extent EPA is planning regulatory activities well in advance, the learning agenda is intended to signal to researchers inside and outside government what the priorities are and could also be used to indicate data priorities.
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies -- even studies where the underlying dose-response data are not available for independent validation -- and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
If researchers want to increase the likelihood that their studies receive greater consideration by the EPA, they may take steps to ensure that the underlying dose-response data are available to the greatest extent possible. But any such response to this final rule would be purely voluntary. It is not required by this rule.
Comment: Commenter (5185) suggests that EPA ensure that relevant transparency information is incorporated into public communications and marketing materials associated with regulatory initiatives. For example, if the claimed benefits of a proposed action are based on non-transparent information, the commenter recommends that the Agency adopt a practice of disclosing this in press releases and fact sheets. The commenter states that, paired with an enhanced focus on risk communication, this practice will lead to greater public understanding of the scientific and other inputs that drive important EPA policies.
Response: Section 30.4 of the final rule states, "The EPA shall clearly identify all science that serves as the basis for informing a final significant regulatory action. The EPA shall make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law"
14.1.10 Implementation Costs
<IP_ADDRESS> Inclusion of Cost Analyses in the Proposal
<IP_ADDRESS>.1 More Robust Analyses of Implementation Costs Are Need
Proposal Fails to Adequately Consider the Costs of Implementation
Comment: Commenters (1012, 1281, 1346, 1388, 2024, 2787, 3135, 5021, 5022, 5165, 5579, 6109, 6111, 6122, 6126, 6133, 6185, 6233, 6168, 6373, 6375, 6925, 8272, 9227) generally assert that EPA has failed to adequately consider the costs of implementation.
Commenter (1388) states that in its proposal, the EPA has not fully accounted for the cost and consequences of excluding valuable information. Commenter (5579) provides that data transparency comes at costs which the EPA does not elaborate on in this rule and which leads the current rule as it stands to be incomplete. Commenter (6109) states that EPA does not explain how it will pay for compliance with the proposed rule or where the personnel hours will come from. Commenter (6375) suggests EPA should assign designated personnel and allocate sufficient funds to facilitate data access.
Commenter (6126) states that, in the absence of incentives to provide information as requested by EPA for regulatory actions, researchers that are not funded or mandated by EPA appear to have no direct incentive to comply with EPA's regulation. Commenter (6133) states that EPA does not confront the financial dimensions or the need for financial incentives to support the unprecedented data release requirements in the rule.
Commenter (9227) contends that the proposed rule fails to acknowledge or provide for costs of the rule related to data masking, de-identification, deposition in public data repositories, and controlled access in federal RDCs. The commenter states that there are various costs to the agency of administering the regulation, which include contacting researchers, gathering data, and ensuring that patient confidentiality and CBI are not disclosed. The commenter states that additional costs could also be incurred because of the need to conduct additional peer reviews required by proposed §30.7 and any additional analyses imposed by proposed §30.6's requirement that "EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions."
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits to the Agency. The final rule acknowledges that the EPA will incur some administrative costs associated with the rule, but the EPA finds that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. See III.J of the final rule preamble for further discussion of benefits and costs of the rule.
The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. The final rule does not require additional peer review, though the EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review.
Comment: Commenter (6915) expresses concern that, despite the existing peer review process, EPA apparently proposes to require that EPA itself conduct an additional "independent" review. See 83 Fed. Reg. at 18774. Yet, the commenter notes that the proposal nowhere discusses how EPA would vet reviewers to identify persons who are purportedly more competent than those already used in past or current peer review processes, or the level of EPA staffing and associated costs that would be needed for additional review -- only stating that EPA will implement the proposed rule in a manner "that minimizes costs." Id. at 18774. The commenter adds that the EPA fails to acknowledge the rigor of existing processes in statutes, policies and federal procedures, or to explain how its proposal would provide any added value and minimize costs.
Response: The EPA is clarifying that 40 CFR 30.6 does not require de novo peer review on studies that have already undergone adequate peer review. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. The EPA's determination of what constitutes adequate peer review is made consistent with the EPA's 2015 Peer Review Handbook.
Paperwork Reduction Act, Executive Orders, and OMB Guidance
Comment: Commenter (2024) states that EPA should do a burden estimate under the Paperwork Reduction Act (PRA).
Commenter (6125) states that the proposal mentions cost-effective strategies for providing public or restricted access to shared data and methods in repositories that might involve collaboration with other federal agencies, but never commits to any path, nor indicates that the agency would bear the cost in time and effort for researchers who would be willing to use them. The commenter states that, instead, EPA claims that the proposal "does not contain any information collection activities and therefore does not impose an information collection burden under the PRA." According to the commenter, EPA cannot have it both ways: either researchers are compelled by a new requirement to expend significant resources on arrangements to ensure the implications of their research for policy are considered or they have no burden because EPA will pay for it.
Response: The rule does not require any action by any party other than the EPA and imposes no paperwork collection burdens or other costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Response: In the final rule the EPA has identified and described some incremental costs that the Agency may incur as a result of the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (6137) states that EPA's proposal will have serious implications on public health, privacy, the environment, and the judicial review process, and thus the costs will be significant. The commenter asserts that there is no evidence that EPA evaluated any of these adverse impacts, quantitatively or qualitatively. According to the commenter, the EPA cannot ignore these harms in its cost-benefit analysis, even if the costs are not readily quantifiable. See, e.g., Michigan v. EPA, 135 S. Ct. 2699, 2707 (2015) (costs include "harms that regulation might do to human health or the environment"); Executive Order 12866, 58 Fed. Reg. 51,735, 51,735 (Oct. 4, 1993) (it is "essential to consider" the "qualitative measures of costs and benefits that are difficult to quantify"); see also Food Labeling: Nutrition Labeling of Standard Menu Items in restaurants and Similar Retail Food Establishments; Extension of Compliance Date; Request for Comments, 82 Fed. Reg. 20,825, 20,828 (May 4, 2017) (acknowledging that delaying a nutrition labeling requirement would lead to millions of dollars in lost health benefits).
Response: The EPA disagrees that the proposal will have the serious implications described by the commenter. As described in the rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations.
Comment: Commenters (6125, 9227) assert that Agencies are encouraged to weigh the costs and benefits of developing higher information quality in OMB's Information Quality Guidelines. The commenters note that costs that the Guidelines encourage agencies to consider include costs attributable to agency processing effort, respondent burden, maintenance of needed privacy, and assurances of suitable confidentiality. Commenter (9227) states that EPA's existing information quality guidelines track the OMB Guidelines closely. Commenter (6125) states that the proposal contains no evidence that the EPA has conducted such an assessment; instead the proposal simply states the belief that the benefits of the rule will be greater than the monetary costs (and potential disbenefits), without providing any quantitative evidence to support that belief. Commenter (9227) argues that EPA's disregard of the Guidelines' recommended weighing of costs and benefits further contributes to the arbitrariness of EPA's failure to consider the costs of the proposal.
Response: The EPA agrees that the benefits and costs of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. However, in the final rule the EPA has identified and described the benefits of the rule and some of the incremental costs that the Agency may incur as a result of the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Mercatus Paper
Comment: Commenters (6111, 6916, 9227) express concern that the proposal inappropriately relies on the Mercatus Paper.
Commenter (6111) argues that EPA cannot abdicate its responsibility to conduct its own analysis of the costs and benefits of this regulation by relying on the paper by Randall Lutter and David Zorn for the Mercatus Center cited in the proposal, which arrives at a lower cost estimate than the Congressional Budget Office (CBO), to support its conclusion that "the benefits of this proposed rule justify the costs."
Commenters (6916, 9227) assert that the only thing EPA relied upon in discussing the costs and benefits of the proposal was the industry-funded Mercatus working paper which rejects a CBO projection that proposed legislation, with terms like the Proposed Rule here, would cost the country $250 million a year. Commenter (6916) states that the Mercatus Paper does not find the new policy to be cost-free; instead it reports that compliance with the proposed legislation would add $2,558 to the costs of each study.
Commenter (9227) states that the working paper of the Mercatus Center asserts that improvements in reproducibility "can be thought of as increasing the net benefits of regulation because they would avoid situations in which costs or benefits are wrongly estimated to occur or in which regulatory costs are imposed without corresponding benefits." The commenter asserts that, setting aside the lack of substantiation for this assertion, it entirely omits situations in which costs and benefits are wrongly estimated because the relevant science is not used -- and the costs that would be imposed on society if EPA inadequately protects communities from harmful pollution or toxic exposures.
Commenter (9227) states that even the Mercatus working paper notes that "[t]he cost of providing access to data has been one of the primary concerns about requiring access to data used by the federal government." The commenter states that, far from concluding, as the proposal suggests, an increase in net benefits from greater reproducibility, the Mercatus working paper simply explained a figure the authors were suggesting could be calculated (the point where net benefits would be positive); the authors do not themselves calculate the benefits, and admit that their "estimates of the benefits of public access to data supporting federal regulatory decisions fall short of proving that the benefits outweigh the associated costs." The commenter asserts that, while the Mercatus working paper disagrees with CBO's cost estimates, it does not argue that requiring access to data is cost-less; as it discusses the "costly activities and services that need to be performed," including activities related to "data collection and data accessibility." The commenter states that, according to that working paper, data collection requires "correspond[ing] with researchers and publishers to obtain the data, review[ing] the data for confidentiality concerns, format[ting] the data for public access, publicly post[ing] the computer code and models used in each study's analysis, and provid[ing] descriptions and documentation on how to obtain the date." The commenter states that data accessibility requires "computer processing services to construct and maintain data bases to store study-related information." According to the commenter, while the actual calculations put forward by the Mercatus working paper appear faulty (for example, it entirely omits the cost to researchers to compile and make their data public, does not include the costs of ensuring patient privacy is protected, and makes assumptions about the similarity of a chemical manufacturer collecting its own studies and EPA collecting and disseminating information of other researchers), the working paper acknowledges that there are costs, which the commenter states the EPA's proposal ignores.
Response: The final rule does not draw upon or characterize the findings of the Mercatus working paper. The EPA acknowledges that the benefits and costs of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. However, in the final rule the EPA has identified and described the benefits of the rule and some of the incremental costs that the Agency may incur as a result of the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Costs Related to Deidentifying/Redacting Data
Comment: Commenters (1008, 2477, 2479, 4591, 5176, 6103, 6114, 6116, 6158, 6361, 6772, 6915, 9227) assert that proposal fails to identify the substantial costs related to the time, effort and resources required to make research data involving PII publicly accessible.
Commenters (1395, 2171, 6133, 6915) state that the CBO estimated the cost of redacting data at $100 million per year. Commenter (6915) states that these costs would encompass obtaining the underlying data, review of the data to address confidentiality concerns, formatting the data for public access, providing computer codes and models used, and providing directions for accessibility of the data, but would not include the additional costs related to the potential need for contractors due to EPA staffing issues to assist with this work.
Commenters (1433, 2249, 6464) state that the process for removing confidential personal and business information before making it public would be virtually impossible, especially considering smaller Agency budgets. Commenter (4878) states that, even if there was an acceptable way to mask personal data while maintaining enough information to comply with the rule, costs of such anonymizing are prohibitively expensive.
Commenter (1008) states that the deidentification process is even more time consuming for data gathered in the past, where research subjects did not consent to public access of information gathered about them. Commenter (2477) states that one cannot ignore that the cost to protect privacy by anonymizing all data is substantial. Commenter (4591) states that de-identification to maintain privacy is not simply a matter of redacting names from documents, but a complex and resource-intensive process. Commenters (6114, 6116, 6361) state that it is hard to imagine that these extra costs could be borne in an era where EPA funding has been cut, or at best has remained stagnant. Commenter (6915) states that redacting confidential data from large studies is a huge job that can take thousands of hours, at commensurately high cost.
Commenters (5176, 9227) assert that, in proposing to redact private information, the proposed rule fails to acknowledge the enormously burdensome and costly complexities involved in redacting such information in a way that both protects individuals' privacy and ensures that the data is still useful for future analysis. Commenter (9227) states that while EPA mentions a range of options for facilitating secure access to confidential data, EPA fails to recognize the significant costs associated with implementing most of these options or the risks to privacy that remain even if these methods are employed.
Response: The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. The analysis by CBO noted by the commenter is not applicable to this rule because the analyses were not for this rule and the requirements analyzed are not the same as those in this rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (2006) expresses concern that, while the proposal suggests that access to private data may be addressed through a RDC, RDCs are not for casual users or the general public and have serious limitations and costs, even for researchers. The commenter states that the proposal in footnote 16 refers, in part, to the U.S. DHHS, which is responsible for the National Health and Nutrition Examination Survey (NHANES). The commenter states that confidential data are available only through an RDC. According to the commenter, public access to confidential NHANES data at RDCs requires payment of "management fees" (i.e. fees to have the data made available at the RDC, ranging from $1,800 to $15,000 per project, skills with specific statistical software (usually SAS), and statistical expertise. The commenter states that access is contingent upon approval of a research proposal, which takes time for the approval as well as to schedule a date to visit the RDC. The commenter states that additional costs would also include transportation and lodging costs for RDCs which are not in one's home town. According to the commenter, no hard copy materials are to be brought into or removed from the RDC which means that analyses must be thoroughly designed before the visit and one must be able to interactively analyze the data as findings are made during the analysis.
Commenter (9227) states that deidentification is complicated and costly. The commenter asserts that, while EPA states that "[o]ther federal agencies have developed tools and methods to deidentify private information," the EPA cites only one source, the U.S. DHHS' Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the HIPAA Privacy Rule. According to the commenter, this guidance provides two methods for de-identifying data: (1) expert determination method, where an expert determines that, after application of statistical and scientific principals and methods, the risk is very small that the information alone or with other available information could be used to identify the subject; and (2) the safe harbor method, requiring that a number of identifiers are removed. The commenter states that the first method requires case-by-case work, and EPA has provided no information regarding how EPA or others could potentially implement it or how much it might cost. The commenter states that the second method is also costly (which EPA disregards) and requires removal of much information that may be necessary to be able to reanalyze or reproduce the research results, so it is unclear whether it would satisfy EPA's requirements in the proposal.
Response: The EPA recognizes that there may be costs associated with accessing data through RDCs and conducting research. However, the rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. See III.J of the preamble for further discussion of benefits and costs of the rule.
Data Management and Acquisition Costs Should Be Addressed
Comment: Commenters (1008, 1012, 2006, 2472,3135, 5419, 6046, 6126, 6127, 6144, 6449, 9227) express concern that the proposed rule offers no information on where the raw data collected would reside, how the resources to manage the data would be obtained, or how the agency would deal with legally and ethically protecting data that is confidential or sensitive. Commenter (1008) states that EPA should allocate resources to develop the digital infrastructure necessary to publicly share research data, especially sensitive data.
Commenters (6046, 6449, 9227) express concern that the rule needs to take into consideration the cost of establishing and maintaining the electronic infrastructure necessary for obtaining and storing data and models. Commenter (6046) recommends that, while the proposed rule does not estimate EPA's cost for an RDC (including space, costs for storage, ongoing maintenance costs to assure security, and staff to manage the data), nor assign responsibility for such a center to any EPA division, the EPA should investigate all options, including the options' feasibility, costs, etc, for keeping data confidential if the data are to be made public. Commenter (6449) states that the cost of safeguarding integrity of the data also needs to be considered.
Commenters (2006, 6046, 6127, 9227) state that the expense of data storage in a public repository can be significant. Commenter (9227) provides that even smaller costs could be prohibitive for many researchers.
Commenters (1011, 2006, 6126, 9227) suggest EPA needs to provide funding for data infrastructure. Commenter (1011) states that EPA should provide sufficient funding to cover the indirect costs associated with preparing and disseminating data generated by researchers funded through grants or cooperative agreements. Commenter (2006) suggests EPA should set aside money to grant to scientists for public data storage. Commenter (6126) states that EPA must ensure resources are available to enable a data infrastructure and responsible use of evidence, consistent with the principles for evidence-based policymaking.
Commenters (0039, 4537) provide the following recommendations regarding EPA funding:
 EPA prioritize its funding toward upgrading existing research data so that it may meet the standard of best available science; e.g., by anonymizing research data so as to preserve privacy, confidentiality, etc.
 EPA should provide substantial funding for a "reproducibility architecture" of hardware and software to facilitate the production of reproducible science by all American scientists whose research informs the EPA's decision-making.
 EPA should provide substantial funding so as to make it possible and then required for all EPA notices, proposed rules, regulations, etc. to include easily accessible links to all scientific materials used to justify these EPA actions. These links should include all relevant scientific research that meets the BARS standard. but that does not support the proposed EPA action.
 EPA's granting program should establish a funding category, with funding priority over all other categories, for meta-analysis and research into publication bias.
Commenter (6144) states that, while EPA's proposed rule references a system used by the NIH to collect health data, there are barriers and questions to how such a system could apply to EPA. The commenter states that, while the administration asserts that the proposal is about transparency, the NIH system is not publicly accessible and requires significant resources to manage and share with researchers. The commenter expresses concern that, at a time when EPA's budget and capacity continues to decrease, it seems unreasonable for the Agency to initiate such an undertaking.
Response: The EPA recognizes that there are costs associated with hosting data for public access. However, this rule does not require EPA or any party to disseminate scientific data and incur associated costs. The rule does not require any action by any party other than the EPA and imposes no costs on other parties. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenters (1941, 5419) recommend that EPA take the steps necessary to ensure that future efforts to gain access to the data underlying pivotal study data are streamlined and cost-effective.
Commenter (1941) describes the Diesel Exhaust in Miners Study (DEMS) study conducted by National Institute for Occupational Safety and Health (NIOSH) and National Cancer Institute (NCI) published in 2012 and the burdensome processes and expenses incurred by the Truck & Engine Manufacturers Association (EMA) to obtain access to full DEMS data sets. The commenter states that EMA's efforts to help facilitate independent investigators' access to the full DEMS data sets in a manner allowing for replication and reanalysis of the DEMS results took well over three years and well over $200,000.
Commenter (5419) encourages EPA to work with the Centers for Medicare and Medicaid Services (CMS) to streamline and simplify the data acquisition process and make the data available to the public free of charge. The commenter states that, otherwise, the cost and security requirements of obtaining these data may place an undue burden on stakeholders seeking to replicate analyses. The commenter provides that data accessed for an Electric Power Research Institute (EPRI) project were acquired at a significant cost (> $100,000) and a comprehensive data management plan that outlined rigorous security protocols was required. The commenter contends that these requirements would make replication of the work untenable for many researchers. The commenter expresses concern that the challenges of making these data available in an equitable fashion may result in the data and insights from this study being removed from consideration in standard‐setting activities for PM2.5, ozone, and NO2.
Response: The EPA first notes this rule does not require EPA or any party to disseminate scientific data and incur associated costs. However, the EPA may opt to undertake activities to gain access to data or to disseminate data when it is in the public interest to do so and will take the points from the commenters into consideration when applicable.
Costs to Researchers Need to Be Addressed - Overview
Comment: Commenters (0671, 1011, 2024, 4542, 4840, 4933, 6127, 6133, 6181, 6373, 6877, 6894, 6924, 6937, 6942, 9227) express concern that the rule will increase costs for researchers and thus have a burdensome effect on them. Many of these commenters contend that this will decrease the resources available for new research.
Commenters (0671, 2024, 1011) assert that EPA's proposed regulation will require a lot of work by researchers, especially the labor necessary to post all their data in a publicly accessible manner no matter what form the data take and to certify to EPA that their work is properly available, such that EPA can use it. One commenter (6133) notes that scientists do not typically receive funding to make the data underlying peer-reviewed studies available for public inspection (6133). Other commenters (6181, 6894, 9227) contend that the cost of sanitizing such data would be immense and would be borne by researchers who do not necessarily have the resources or institutional support to complete the process. Thus, as commenter (6937) notes highly‐trained researchers and public health officials would need to spend time complying with the regulation rather than doing research. Commenter (6373) asserts that the level of effort required to prepare a public version of confidential data, including in the manner described in section 30.5 of the proposed rule, can be enormous. The commenter further states that the onerous requirements and expensive processes that the proposed rule would require most likely would put smaller laboratories conducting academic research at a disadvantage. Another commenter (4542) contends that even for purposes of reanalysis, simply making data available is insufficient; the participation of the original researchers is often required because of their unique expertise in the design, collection and analysis, especially for large or complex datasets. This is burdensome.
Several of these commenters contend that unless the EPA pays the compliance costs of researchers, this action will have a significant economic impact on small entities such as researchers. Several of these commenters also suggest that EPA should increase Federal grant funding to off-set this increased burden.
Response: The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (0844) states that the rule "does not directly regulate any entity outside the federal government" and would be of interest "to entities that conduct research and other science activity relevant to EPA's regulatory activity." The commenter notes that researchers should be prepared to release their underlying data if they want their research to be considered by the EPA to promulgate regulations.
Response: The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. As described in the rule EPA will continue to consider all relevant and appropriate science regardless of data availability but will give greater consideration to studies where dose-response data are available in a manner sufficient for independent validation.
Costs to EPA
Comment: Commenters (0061, 1012, 4845, 5011, 6125, 6915, 8281-PH84, 9227) express concern that this rule would place a substantial new burden on the Agency and further hamstring the regulatory decision-making process to protect public health and the environment.
Commenter (6125) states that EPA officials told the CBO that the Agency would likely greatly reduce the number of studies it relied on and would not take on the cost of disseminating the underlying data.
The commenter (6915) suggests that any requirement for EPA to conduct additional review would entail additional significant costs, contrary to the proposal's assertion. 83 Fed. Reg at 18772. The commenter states that the practical outcome of the proposal is that EPA may end up relying on a much smaller number of studies and/or on a less robust subset of relevant available studies, thus undermining the regulatory decision-making process.
Commenter (9227) expresses concern that the proposal includes no discussion of whether funds would be made available to implement the rule, whether other EPA activities would be sacrificed, how these tradeoffs would be justified, and what the overall impacts might be on public health and the environment. The commenter states that with respect to the potentially very large costs that would accrue to EPA, EPA's proposal provides no indication that any funding to support such activities would be available. The commenter asserts that EPA funding is at its lowest level since the 1980s and, absent a significant change in Congressional priorities, any EPA expenditures for the purposes of supporting making data publicly available would necessarily require cutbacks in other critical areas of environmental protection, which might include supporting additional research, conducting inspections, issuing permits, setting standards, or many other activities.
Commenter (6125) believes that EPA is not staffed to reanalyze the number of pivotal studies that could reasonably arise for all of its significant rules under all statutes, and EPA provides no reason to believe that staffing levels would be adequate. Therefore, the commenter suggests that the agency might need to enlist external analysts. The commenter contends that this would take substantial time and money. As a point of comparison, the commenter provides that the reanalysis of the American Cancer Society (ACS) and Six-City Studies, which included analysis of alternative model specification, cost the Health Effects Institute (HEI) approximately one million dollar. If just half of recent cohort studies were to submit their data through some limited use agreement, it could cost EPA on the order of $10 million to hire outside analysts to do the work. The commenter notes that if this level of analysis were required for even one or two studies in every major regulation, it would likely impair significantly the agency's ability to meet regulatory deadlines for the NAAQS or other programs, among other things precluding the agency from meeting its avowed goal of completing NAAQS reviews for particulate matter (PM) and Ozone by 2020.
Commenter (6144) suggests that it is clear that the current proposal would create a tremendous time and resource burden on EPA and on scientists outside of the agency, as they would scramble to track down, collect, compile, and post (including any statistical methods needed to conceal private information) all the data, models, code, etc. for each of the hundreds of studies EPA cites on any given decision.
Response: The rule does not require EPA nor any other party to reanalyze data, only that EPA determine whether dose-response data for pivotal science are publicly available in a manner sufficient for independent validation. The final rule acknowledges that the EPA will incur some administrative costs associated with the rule, but the EPA anticipates that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule. The analysis by CBO noted by the commenter is not applicable to this rule because the analyses were not for this rule and the requirements analyzed are not the same as those in this rule.
Comment: Commenter (2561) expresses concern that this rule will open the EPA to numerous state and locality lawsuits, at the expense of the American taxpayer.
Commenters (4839, 4845, 6196) express concern that the proposed regulation includes a case-by-case exemption process for data in significant regulatory decisions. The commenters contend that the result will be more cost, more bureaucracy, and may result in litigation over whether exemptions are justified.
Response: The EPA does not agree that this rule will lead to more and more costly litigation. The EPA does acknowledge the Agency will incur some administrative costs associated with the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Who Is Responsible for Covering Implementation Costs?
Comment: Commenters express concern that EPA has not identified who will pay for rule implementation. Commenters (5419, 6142, 6125, 6446) state that EPA should develop a plan and either provide the resources to compensate researchers for the burdens imposed by this rule or identify who will pay specifically for which aspect of the rule, including, (1) who is responsible for the collection, de-identification, storage, and dissemination of sensitive data (2429, 2746, 2906, 6158, 6447), (2) who is going to compensate researchers for the increased compliance costs (0671, 6909, 6915, 6924), (3) how will EPA fund its activities (6125, 6126, 8274, 9227), and (4) who will be responsible for implementation of the rule and associated costs within the agency (6868).
Commenter (6909) states that many improvements to existing technologies and infrastructure are necessary before uniform data sharing can become a universal reality. Commenter states that it is not at all clear who will pay for these improvements. Commenter states that this proposed rule implicitly requires all these associated costly investments in improved technology and infrastructure without providing any funding for them. Commenter states that this is a huge unfunded mandate that researchers will not be able to cope with quickly, and the inevitable result will be sound science being ignored by EPA regulators who should be paying close attention to it.
Commenter (6125) states that the proposal mentions "cost-effective" strategies for providing public or restricted access to shared data and methods in repositories that might involve collaboration with other federal agencies, but never commits to any particular path, nor indicates that the agency would bear the cost in time and effort for researchers who would be willing to use them. The commenter states that, by not offering any support for the process, the proposal betrays its real motive namely to reduce the availability of scientific information that the agency now sees as inconvenient in view of its policy goal to retreat from environmental rules that have provided significant public health benefits.
Response: The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. The EPA acknowledges that the Agency will incur some administrative costs associated with the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Costs will Result in Delays in Necessary Environmental/Public Health Standards
Comment: Commenter (5120) expresses concern that the agency will not be able to support the proposal's requirements within a timely manner, given that the Agency has been targeted for a twenty-three percent decrease in its budget.
Commenter (9227) states that, if EPA complies with the regulation not by spending the money to make data publicly available, and if the research community does not bear those costs itself, then it appears that EPA would simply ignore studies that do not comply with the regulation (EPA is proposing to exercise its discretionary authority to establish a policy that would preclude it from using such data in future regulatory actions). The commenter states that course of action has its own significant costs, and EPA provides no analysis in the proposal of the magnitude of studies that it has previously relied upon that it could no longer rely upon in regulating.
Commenter (6872) states that it is critical that the impacts of the proposed rule on EPA resources and capacities to complete toxicity assessments, and their downstream impacts on the states, be evaluated and addressed. The commenter asserts that EPA completing assessments of chemical toxicity and health effects are critically needed by the states to protect public health and the environment. The commenter states that a delay in EPA action on completing toxicity assessments and standards as a result of the proposed rule will necessitate state resources to complete these toxicity assessments and standards in order to protect public health and the environment.
Response: The final rule EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. EPA acknowledges that the Agency will incur some administrative costs associated with the rule, but finds that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. The Agency believes the incremental burden of assessing the availability of data and models for pivotal science is unlikely to result in significant delays in promulgating regulations or publishing influential scientific information. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenters (6163, 6358, 8272) suggest that EPA should assess the cost of eliminating science from consideration.
Commenters (6358, 8272) state that the proposal should include an examination of the studies that have previously informed regulation and the extent to which they will meet the proposed criteria; an analysis of regulations that would not have been promulgated, would have included less-protective limits, or that could be weakened in the future under the proposed criteria; an analysis of the time and costs required to bring all relevant studies into compliance with the criteria; and a calculation of the foregone public-health benefits and associated costs to the U.S. economy when regulations are delayed or are less protective due to this rule.
Response: The final rule states that the EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. The Agency does not believe it is feasible or necessary to attempt an analysis of how past regulatory decisions may have been made differently. The rule does not require EPA nor any other party to change relevant studies or bring them into compliance with any standard. The EPA disagrees with commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS> Proposal Fails to Adequately Consider Public Health-Related Costs
Several comments express concerns that there will be an increase in public health-related costs as a result of the proposed rule due to: (1) the delay of necessary regulations to protect public health, (2) the inability to use the best available science when making regulatory decisions, and (3) because persons will choose not to participate in studies because of confidentiality concerns. The commenters state that these public health costs were not adequately considered in the development of the proposed rule. An overview of these comments is provided below.
<IP_ADDRESS>.1 Public Health-Related Costs as a Result of the Delay of Regulations Needed to Protect Public Health
Comment: Several commenters (3135, 5579, 6358, 6373, 6872, 6915, 6920, 8274) express concern that the added costs and efforts to comply with this rule may significantly delay the development of EPA regulations that improve public health. The commenters assert that the result of delaying the development of EPA regulations that improve public health will be the imposition of significant healthcare costs on American citizens.
Commenters (0671, 6122, 6358, 6373, 6872, 6920, 8272, 8274) assert that this proposed rule does not include the foregone public health benefits that would result when regulations are delayed. Commenters (6358, 8272) state that the proposal should include a calculation of the foregone public-health benefits and associated costs to the U.S. economy when regulations are delayed due to this rule.
Response: The EPA disagrees with commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. Further, with this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
<IP_ADDRESS>.2 Health Costs Because of the Inability to Use Studies that Represent the Best Available Science
Comment: Commenters (2336, 6125, 6137, 8281-PH63) express concern that the costs to public health from implementation of this rule could be significant due to the inability of the EPA to use the "best available science" under the proposed rule which could result in a weakening of regulations. Commenter (2336) explains, for example, if the "six cities" and "150 cities" studies had been rejected, and, consequently, the NAAQS for PM2.5 had not been tightened in 1997, the harm to public health would have amounted to several hundred billion dollars.
Commenter (6125) states that the CBO analysis does not tell the whole story as it did not assess the disbenefits to the regulatory process and to public health of being unable to base regulations on numerous influential studies for which data could not be made available. The commenter asserts that considering only the science policy value of the information in multiple original studies that are likely to be lost (e.g. cohort epidemiology studies, which must limit access to protect privacy), it is reasonable to conclude that their loss would almost certainly outweigh the value of any information gained by subsequent re-analyses of a more limited set of studies, which rely on publicly available databases that are often inferior to those that contain more relevant information.
Commenter (6137) states that, in a recent report on rulemakings related to health, EPA and OMB quantified some of the benefits of various regulations that would be dramatically weakened under the new rule. The commenter states that these recent studies show health rulemakings, particularly air rulemakings, create significant benefits (including health and protection of life, reductions in the need for health care and health care costs, as well as job creation, and other economic values in avoiding days lost at work and school) that are quantitatively larger and more qualitatively valuable than the costs of pollution controls or other economic costs of the regulations.
Commenters (5021, 6163, 6185, 6188, 6916, 8281-PH63, 9227) express concern that the EPA offers no accounting of the public health costs that would inevitably result if EPA is constrained in its ability to use the best research to support its future decision-making.
Response: The EPA disagrees with commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. The final rule states that the EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. The EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
<IP_ADDRESS>.3 Public Health-Related Costs Associated with Reduced Participation of Persons in Studies Due to Confidentiality Concerns
Comment: Commenter (9227) states that the proposed rule does not disclose the cost that would be associated with the resulting reduced study participation due to perceived risks to privacy and confidentiality. The commenter states that NAS explains that this "threatens the research enterprise itself, because concerns about privacy and confidentiality are among the reasons often given by potential respondents for refusing to participate in surveys, and those concerns have been shown to affect behavior as well. , The commenter provides that the NAS panel emphasized: "Any confidentiality breach that became known would be likely to heighten such concerns and, correspondingly, reduce survey response rates." According to the commenter, efforts to increase researchers' access to data must, therefore, take into account the need to avoid increasing the actual and perceived risks of confidentiality breaches. The commenter states that the proposal does not take into account this potential cost and that this confidentiality risk has a further cost: it affects the quality of the data collected, as the NAS explained:
      The reason for confidentiality pledges and for stringent procedures to prevent disclosure is that they improve the quality of data collected from individuals, households, and firms. It is essential that respondents believe they can provide accurate, complete information without any fear that the information will be disclosed inappropriately. Indeed, if the information was disclosed, harm might come to an individual respondent.
Commenter (9227) asserts that the proposal's only acknowledgment of this complex problem and cost is its statement that "EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government." The commenter states that EPA does not cite a single example of these common solutions, nor does it include an analysis of these alleged solutions and their costs and benefits, it does not even explain what the solutions are that EPA believes address this concern.
Response: The rule does not require EPA or any party to disseminate scientific data, nor does the rule does not require any action by any party other than the EPA. EPA anticipates that researchers will abide by any agreements regarding confidentiality that they have made with participants or other parties. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data and model availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make dose-response data available, weighing their own benefits and costs and make choices that they deem appropriate. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS> The Rule May Be Prohibitively Expensive
Comment: Commenters (1433, 2249, 6111, 6126, 6464) express concern that the proposal would be unrealistically expensive.
Commenter (6126) states that EPA will not likely have the resources or capability to assess the entire suite of regulations and the evidence-base retrospectively. The commenter states that, for example, EPA is technically required to evaluate every regulation promulgated under the Resource Conservation and Recovery Act (RCRA) on a triennial basis; recent research suggests that statutory requirement has been largely unfulfilled due to a lack of resources to comply with the decades-old mandate. The commenter states that, in the absence of a major influx of new resources, EPA would be unlikely to engage in such meaningful and transparent reviews while also complying with statutory obligations. The commenter recommends that, if implemented as stated, the regulation should be applied prospectively.
Several commenters (0061, 1102, 1395, 1433, 2171, 2249, 2336, 2787, 2906, 2907, 4840, 4845, 4878, 5022, 6103, 6114, 6116, 6122, 6125, 6132, 6133, 6137, 6144, 6158, 6185, 6188, 6196, 6450, 6464, 6937, 6915, 8274, 9227) express concern over the potential cost to implement the proposal. The commenters point out the HONEST Act CBO ,  and EPA estimates of similar measures introduced in Congress indicate that the rule could cost millions of dollars a year to implement.
Commenters (0061, 1012, 1433, 2336, 2787, 2906, 2907, 4840, 4845, 5022, 6114, 6116, 6122, 6132, 6133, 6137, 6158, 6196, 6464, 6937, 8274) state the HONEST Act CBO indicates that the proposed rule could cost $250 million initially to implement, and $1 million to $100 million (depending on the approach taken by EPA in assessing studies) per year thereafter; at an estimated cost of $10,000-$30,000 per study, but some studies may cost closer to $1 million. Commenters (2907, 6133) state that when EPA staffers in 2017 considered the potential effects of the HONEST Act that mirrors the approach of the proposal, they calculated  that efforts to anonymize health data and CBI could top $1 million per study.
Commenter (9227) states that the CBO concluded, just with respect to the costs to EPA, that "based on information from the EPA and other federal agencies, as well as organizations and researchers in the scientific community that publish in peer-reviewed journals," EPA "could spend between a few million dollars per year to more than one hundred million dollars per year ... to ensure that data and other information underlying studies are publicly available in a format sufficient to allow others to substantially reproduce the results of studies." The commenter states that, in the 2017 estimate, the CBO concluded that "[i]f the EPA continued to rely on as many scientific studies as it has used in recent years ... then the CBO estimates that the agency would need to spend at least $100 million dollars per year to upgrade the format and availability of those studies' data," "on average, $10,000 per scientific study." According to the commenter, such costs would cover the costs of "obtaining all the underlying data used in a study, reviewing the data to address any confidentiality concerns, formatting the data for public access, providing access to the computer codes and models used in the study's analysis, and providing descriptions and documentation on how to access the data." The commenter states that, as Deputy Assistant Administrator Nancy Beck noted, during the development of the proposal, requiring "a huge amount of data to be submitted to the agency" would "be incredibly burdensome" and "not practical."
Response: The cost analyses referred to by the commenter are not analyses of this rule and assume costs of disseminating information. The rule does not require EPA or any party to disseminate scientific data and incur associated costs. The rule does require EPA to determine if dose-response data for pivotal science are publicly available in a manner sufficient for independent validation and the EPA acknowledges that the Agency will incur some administrative costs associated with the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS> Costs of Replicating, Reproducing, or Reanalyzing a Study (§30.5)
Comment: Commenters (1012,4848, 4933, 5015, 6196, 6373, 6925, 9227) express concern that replicating, reproducing, or reanalyzing a study (especially epidemiological studies) is a highly cost-intensive and time-consuming endeavor and resources spent trying to recreate a study identically expends time and resources that might otherwise be used to conduct new research.
Commenter (4848) states that timing and the cuts in EPA funding makes replicating studies (as a condition of promulgating regulations) an impossibility. Commenter (4933) states that the very nature of longitudinal public health studies where health and toxins intersect are by design, large, expensive and require years or decades before results are found. Commenter (4933) states that because these studies are so big, they are often too expensive to repeat.
Commenter (9227) asserts that it is critically important to note that reanalysis projects are not simple or inexpensive. The commenter states that the reanalysis of just the Harvard Six Cities Study and the ACS CPSII took three years to complete and cost $899,046 in direct expenditures, without accounting for costs incurred by Health Effects Institute for oversight and review as well as staff compensation.
Response: The rule does not require EPA nor any other party to perform or conduct any reanalysis or replication of a study. The rule does require EPA to determine if dose-response data for pivotal science are publicly available in a manner sufficient for independent validation and the EPA acknowledges that the Agency will incur some administrative costs associated with the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (6941) expresses concern that, under the proposed rule, chemical manufacturers or importers could be forced to incur the cost of unnecessarily repeating the use of vertebrate animals in the testing of chemical substances. The commenter states that the Toxic Substances Control Act (TSCA) directs EPA to reduce and replace the use of vertebrate animals in the testing of chemical substances to the extent practicable, scientifically justified, and consistent with its policies. According to the commenter, TSCA identifies the formation of industry consortia to develop information as one means to advance this goal by reducing duplicative animal testing. The commenter states that, for some chemical substances, testing may already have been conducted under other regulatory initiatives, such as the European Union's Registration, Evaluation, Authorization and Restriction of Chemicals program. The commenter asserts that, while full study data may not be publicly available, robust summaries of the objectives, methods, results and conclusions of such testing can provide sufficient information to independently assess its value.
Response: The rule does not require EPA nor any other party to perform or conduct any reanalysis or replication of a study. Further, with this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations.
<IP_ADDRESS> Costs to Describe Assumptions, Methods, Variability and Uncertainty (§30.6)
Comment: Commenter (6125) states that the most puzzling, and perhaps costly requirement in the rule (§30.6) appears only in the wording of a specific requirement in the actual proposed rule regarding use of dose-response data and models from pivotal studies: "EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions." The commenter expresses that this appears to require the agency not only to explain the basis of each model-related assumption in the original study but also to provide an analysis of the original data using alternative assumptions. The commenter states that EPA is not staffed to reanalyze the number of pivotal studies that could reasonably arise for all its significant rules under all statutes, and EPA provides no reason to believe that staffing levels would be adequate. Therefore, the commenter suggests that the Agency might need to enlist external analysts, which would take substantial time and money. The commenter states that, as a point of comparison, the reanalysis of the ACS and The Harvard Six Cities Study, which included analysis of alternative model specification, cost HEI approximately one million dollars. According to the commenter, if just half of recent cohort studies were to submit their data through some limited use agreement, it could cost EPA on the order of $10 million to hire outside analysts to do the work. The commenter states that, if this level of analysis were required for even one or two studies in every major regulation, it would likely impair significantly the Agency's ability to meet regulatory deadlines for the NAAQS or other programs, among other things precluding the Agency from meeting its avowed goal of completing NAAQS reviews for PM and Ozone by 2020.
Response: The EPA has removed proposed 40 CFR 30.6 and transferred some of the requirements to 40 CFR 30.5(e). The EPA has revised requirements and limited their application to dose-response assessments. The final rule requires EPA to describe and document only critical assumptions and methods and to characterize variability and uncertainty of the assessment. It requires an explanation of the scientific basis for critical assumptions used in the dose-response assessment. These changes from the proposed requirements result in only small incremental costs for the EPA. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS> Costs of Independent Peer Review (§ 30.7)
Comment: Commenters (1012, 4839, 4931, 6125, 6142, 6866, 6868, 6915) express concern that having the Agency conduct an "independent peer review on all pivotal regulatory science used to justify regulatory decisions" would be time-consuming, duplicative of current practice and costly to implement, without adding commensurate environmental and public health value. Commenter (6125) states that independent peer review is a cost and impact which the Agency has failed to acknowledge or account for.
Commenter (1012) states that having the Agency conduct an "independent peer review on all pivotal regulatory science used to justify regulatory decisions" would likely hamstring the Agency if additional resources and staff are not provided for this express purpose. Commenter (6866) asserts that implementing independent Agency peer review of all such studies would significantly slow down rulemaking and may invite lawsuits that further stymie regulatory processes.
Commenter (6142) states that there is no value in this second peer review without guidelines ensuring the review is rigorous and unbiased in nature and conducted by subject matter experts. The commenter states that this rule proposes adding this costly step to the process for developing EPA policy without setting aside resources for its execution. According to the commenter, the rule is likely to hinder the application of potentially important scientific findings to public and environmental health without improving the quality of data used to guide EPA regulatory policy.
Commenter (6915) states the proposal nowhere discusses how EPA would vet reviewers to identify persons who are purportedly more competent than those already used in past or current peer review processes, or the level of EPA staffing and associated costs that would be needed for additional review -- only stating that EPA will implement the proposed rule in a manner "that minimizes costs." The commenter states that the practical outcome of the proposal is that EPA may end up relying on a much smaller number of studies and/or on a less robust subset of relevant available studies, thus undermining the regulatory decision-making process. According to the commenter, the EPA fails to acknowledge the rigor of existing processes in statutes, policies and federal procedures, or to explain how its proposal would provide any added value and minimize costs.
Response: This rule requires that the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. The EPA anticipates that the incremental peer review costs to the Agency will be small. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS> Do Benefits Justify the Costs of the Rule?
<IP_ADDRESS>.1 EPA Needs to Quantify the Costs and Benefits of the Rule
Comment: Commenters (1012, 2171, 2787, 5022, 6046, 6125, 6137, 6143, 6168, 6196, 6358, 6874, 6925, 8272, 9227) assert that, while EPA claims benefits outweigh costs, there is no published record or analysis to demonstrate that this claim is valid and that the EPA should commit to quantify these costs and benefits (including foregone health benefits) and implement the rule in a manner that optimizes the costs and benefits.
Commenter (2787) states that EPA only stated that the Agency will implement the provisions "in a manner that minimizes cost." Commenter (6046) states that it is unclear who bears the burden of the cost and who benefits, by how much and how. Commenter (6137) states that, despite the lack of evidence in the regulatory docket, the costs of this Proposed Rule are far-reaching and substantial, while EPA has failed to identify any alleged benefits at all, other than vague and unsupported references to improved transparency. Commenter (6874) states that the theoretical  -  but undemonstrated  -  benefits of EPA's proposal must be weighed against the extensive and unequally distributed costs of such an approach.
Commenters (6916, 9227) argue that EPA makes unsupported statements about its "belie[f that] the benefits of the proposed rule justify the costs." Commenters (6916, 9227) assert that EPA's finding is not consistent with the conclusions of the National Academies, as the proposal suggests. Commenter (9227) states that the NAS report highlighted both the risks and benefits of making data publicly available and nowhere concluded that there were benefits to excluding data from the agency's regulatory decisions simply because the underlying data was not publicly available. Commenter (6916) states that the NRC explicitly states that its work does not "weigh the potential harm posed by disclosure against the benefits potentially foregone."
Commenter (9227) states that EPA has included only a cursory mention of the expected qualitative benefits of the proposal, with no discussion of the anticipated likelihood, scope, or impact of the suggested benefits, let alone any effort to quantify them, much less monetize them. Commenter states that EPA simply assumes that the proposal will "improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing an exploration of key data sets" without any analysis or evidence.
Commenter (6137) notes that, in performing certain CAA and other rulemakings, EPA may not consider the economic implications of considering or excluding certain science. The commenter asserts that EPA's failure to provide any lawful or rational justification for its costs/benefits statement for the proposed rule suggests that economic costs may have impermissibly played a role in its analysis. According to the commenter, had EPA focused on health rather than cost, EPA could not possibly find that the benefits of ignoring health science outweigh the costs for public health rulemakings.
Commenter (9227) states that there is no reason to think that excluding relevant science merely because the underlying data is not publicly available would increase the net benefits of a regulation. According to the commenter, for example, it appears that under the proposed rule EPA would exclude a peer-reviewed, published study whose conclusion had been reproduced based upon numerous different datasets (and whose underlying data, though not publicly available, had been reevaluated by outside experts), while including a study that had had no peer review, was not published, had no corroborating studies, and had not actually been replicated or reproduced, merely because the underlying data was made publicly available. The commenter asserts that is simply not a recipe for more accurate decision making.
Response: The EPA agrees that the benefits and costs of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM and has described these briefly in the final rule. The EPA acknowledges that the Agency will incur some administrative costs associated with the rule, but the EPA finds that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (0671) states that the proposed rule will impose compliance costs without providing any meaningful benefit to the EPA, to the regulatory process, or to the American people. Commenters (6450, 8272) argue that, considering there is no evidence that current data transparency policies are deficient, this added cost to the American taxpayer is neither necessary nor appropriate.
Response: The EPA acknowledges that the Agency will incur some administrative costs associated with the rule, but the EPA finds that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. The EPA believes that greater transparency is inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science these analyses rely upon. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (1346) states that the rule talks about the costs of compliance but it does not talk about the benefits of compliance appropriately. The commenter asserts that, for example, the carbon rules have a cost to power companies but a clearly defined benefit to the health of human beings living near power plants. According to the commenter, it is more ethical to weigh the benefits to the lives and well-being of people higher than the benefits to a company.
Response: The Agency agrees that there are important and valuable health benefits for many regulations. The EPA follows its own peer-reviewed guidance, as well as government-wide guidance from OMB, when estimating both benefits and costs of regulations.
<IP_ADDRESS>.2 Benefits of the Strengthening Transparency Proposal
Comment: Commenter (3971) states that, if rigorous and transparent analyses underlie future proposals, there should be fewer legal challenges, and passing this regulation should permit the Agency to implement rules more quickly, not less expeditiously.
Commenter (4534) states that increasing transparency by using peer reviewed data and publishing data for public review may increase the time and cost to bring a new or modified rule package into practice. The commenter states that the additional time and cost to develop a new regulation is small compared to the time and cost for regulated parties to comply with the regulation. The commenter asserts that many of the rule packages cost industry more than $100 million to comply and require several years to develop engineering solutions. According to the commenter, the EPA and industry both have incentive for the rule package to be right the first time.
Response: The EPA agrees that rigorous and transparent analysis has significant value and improves the regulatory process. EPA also agrees that the incremental time and costs associated with implementing the requirements of this rule are small. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS>.3 EPA Must Re-Propose After Analyzing its Costs
Comment: Commenter (9227) states that the proposal's failure to analyze and disclose costs and benefits cannot be cured in a final regulation. The commenter states that, should EPA not abandon this misguided proposal, it must re-propose it after first analyzing its costs (both to public health, to researchers, and to the agency itself) and benefits, and providing the requisite opportunity for public comment on its analysis. According to the commenter, the public cannot meaningfully comment on the proposed rule without understanding the actual costs and benefits of the proposal, the alternatives EPA considered, and the analyses underlying EPA's assessments.
Response: The EPA agrees that the benefits and costs of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. However, in the final rule the EPA has identified and described the benefits of the rule and some of the incremental costs that the Agency may incur as a result of the rule. The EPA disagrees that it must re-propose this rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS> Minimization of Subpart Implementation Costs (§30.8)
<IP_ADDRESS>.1 What Costs Will Be Minimized and How?
Comment: In order to achieve the goals of the proposal in a streamlined and cost-effective manner, commenter (1941) recommends the following:
 EPA should develop standardized data-sharing and data-security agreements (including non-disclosure agreements) to utilize with the authors/investigators of pivotal studies to facilitate and streamline the receipt and transfer of study data to the RDC or to EPA's other approved datahousing site(s);
 EPA should develop standardized data-access and confidentiality agreements that third-party investigators can use to gain authorized access to the pivotal-study data that will be housed at the RDC or at other EPA-sanctioned data-housing site(s); and
 EPA should work to ensure that any fees for gaining authorized access to pivotal study data at EPA's authorized data-housing site(s) are kept to a minimum to enhance the prospects that qualified third-party investigators will be able to conduct their reviews and reanalyses in a timely and cost-effective manner.
Commenters (6125, 6373) express concern that, while the proposal states that costs will be low because EPA will "implement" the rule "in a manner that minimizes costs", there is no explanation of what specific types of costs may be incurred, much less how they will be minimized. Commenter (6125) states that this type of vague and unsupported pronouncement does not provide an adequate basis for public comment.
Commenter (6158) states that, in suggesting that the potential costs of the rule would be minimal, the EPA ignores the costs to researchers (who would have to pay to set up and maintain data sharing for their previously published studies to be considered), to the EPA (for presumably conducting the multiple re-analyses required in section 30.6 of the rule), and to the public's health (for the very real costs in dollars and human lives of undermining existing regulations by limiting access to science).
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits. In the final rule the EPA has identified and described the benefits of the rule and some of the incremental costs that the Agency may incur as a result of the rule. The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
<IP_ADDRESS>.2 Will EPA Consider Fewer Studies?
Comment: Commenters (2336, 5021, 6125, 6137, 6158, 6163, 6185, 6358, 6874, 6916, 8272, 9227) generally express concern that the cost of this rule will be less science  --  not better science  --  and weaker public and environmental health decision making.
Commenters (2336, 4840, 4845, 6109, 6114, 6116, 6122, 6137, 6181, 6188, 6196, 6915) express concern that the proposal does not address the possibility that EPA would reduce the number of studies used in developing policies and regulations because of the cost of complying with this rule's requirements. Commenter (6181) states that, if the EPA restricts the science it relies upon to only those studies that publish open datasets, the outcomes and data from key areas of scientific inquiry related to EPA's decision-making will be put beyond the reach of Agency staff.
Commenters (2336, 4840, 6109, 6133) assert that CBO analysis suggests that EPA may rely on significantly fewer studies each year in support of its mission. Commenters (4840, 9227) state that the cost estimate made by the current administration for a substantially similar law dropped to $1 million annually because EPA officials explained to the CBO that the Agency would implement the legislation with minimal funding and generally would not disseminate information for the scientific studies that it uses to support covered actions.
Commenters (6114, 6116) state that an EPA staff study predicted that this policy would reduce by half the number of studies the EPA would rely upon in developing policies and regulations because of the cost of complying with its requirements.
Commenter (6137) states that the exclusionary intent of the Proposed Rule is demonstrated by proposed 40 CFR §30.8, which requires that the rule be implemented to "minimize costs." The commenter suggests that, if EPA must minimize costs, the exclusion of science, rather than taking complicated and expensive steps to hide confidential medical data, is likely to be the approach followed.
Response: The final rule EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. The EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
Comment: Commenter (2478) requests that the EPA quantify how much this proposed rulemaking would result in a reduction in sponsored research in climate science. The commenter inquires how the EPA defines climate science, how much funding is allocated from the federal government to climate science, and how much will this proposed rule impact funding allocated from the federal government to climate science?
Response: Funding for climate science varies and EPA does not have at its disposal total funding across the federal government. The EPA does not foresee any meaningful connection between this rule and funding allocated from the federal government to climate science.
<IP_ADDRESS>.3 Regulatory Decisions on Cost Inconsistent with Statutory Mandates
Comment: Commenter (6137) states that proposed Section 30.8, which would require EPA to implement the provisions of the Rule "in a manner that minimizes cost," is unlawful and arbitrary, for the following reasons.
      First, because EPA has only the authority conferred to it by statute, the Agency must identify the statutory authority upon which it bases its regulatory decisions about what data and models to consider on the minimization of cost. Clean Air Council v. Pruitt, 862 F.3d 1, 9 (D.C. Cir. 2017) (quoting Verizon v. FCC, 740 F.3d 623, 632 (D.C. Cir. 2014) ("It is `axiomatic' that `administrative agencies may act only pursuant to authority delegated to them by Congress.'") (alteration and citations omitted)). The Agency has failed to do so here.
       Second, EPA's proposal to base these decisions on cost is inconsistent with statutory mandates requiring the Agency to base regulatory decisions on the best available science. See, e.g., 42 U.S.C. § 300g-1(b)(1)(B)(ii)(II) (directing EPA to base its determination about whether to regulate any contaminant "on the best available public health information"); 15 U.S.C. § 2625(h) (requiring EPA to act "consistent with the best available science"). Plain meaning and common usage confirm that "best available science" does not refer to whatever science is least costly. See also supra, Section IV.A.
      Third, EPA's proposal to base its data quality decisions on cost is inconsistent with applicable statutory mandates requiring EPA to make regulatory decisions based on considerations of public health. Whitman v. Am. Trucking Ass'ns, 531 U.S. 457, 469 (2001) (cost "is both so indirectly related to public health and so full of potential for canceling the conclusions drawn from direct health effects that it would surely have been expressly mentioned . . . had Congress meant it to be considered."). If anything, EPA's proposal to base decisions on consideration of cost here is even more irrational than it was in Whitman, because in this case, decision-making about the science used to inform EPA's regulatory decisions will result in a proliferation of costs and benefits that resist quantification, monetization, and comparison. These include substantial and important non-market values, including privacy, transparency, health and environmental considerations, and even the value of scientific knowledge itself.
      Fourth, section 30.8 is also unlawfully vague. As the Supreme Court has explained, "`cost' includes more than the expense of complying with regulations; any disadvantage could be termed a cost." Michigan v. EPA, 135 S. Ct. at 2707. Yet the provision does not specify to what types of costs it refers. And even though various types of costs would necessarily need to be weighed against one another when applying this provision, the Proposed Rule is notably silent as to how that will be done. As proposed, it appears to confer nearly limitless discretion on EPA to decline to collect and utilize relevant science on grounds of cost, in contravention of applicable statutory mandates and reasoned decision-making. See U.S. Sugar v. EPA, 830 F.3d 579, 644 (D.C. Cir. 2016) ("in light of the unambiguous statutory command [to regulate toxic pollution sources] . . . [t]he Agency was obligated to collect the data it needed").
      Finally, even if EPA had authority to base its data quality decisions on cost, it would be irrational and arbitrary for EPA to ignore all the costs and benefits to the public  -  including public health and environmental costs and benefits and privacy-related costs and benefits  - that may flow from its decisions. EPA must treat costs and benefits alike and may not "put a thumb on the scale by undervaluing the benefits and overvaluing the costs of more stringent standards." Ctr. for Biological Diversity v. Nat'l Highway Traffic Safety Admin., 538 F.3d 1172, 1198 (9th Cir. 2008). And it may not ignore the public health, environmental, and privacy-related costs of its action or inaction of its decision just because they are not quantified. See, e.g., Exec. Order No. 13,563 § 1, 76 Fed. Reg. at 3821; Exec. Order No. 12,866 § 1, 58 Fed. Reg. at 51,735 (it is "essential to consider" the "qualitative measures of costs and benefits that are difficult to quantify"). OMB Circular A-4 cautions agencies against ignoring the potential magnitude of unquantified benefits, because the approach with the "largest quantified and monetized . . . estimate" is not necessarily the most cost-justified. Under all these authorities, a "full accounting" of the costs and benefits of a rule requires that indirect benefits be counted "equivalently" with other costs and benefits.
Response: The EPA acknowledges that the Agency will incur some administrative costs associated with the rule and will implement the rule in a manner that minimizes costs while fully complying with the rule EPA's other statutory obligations. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule. The EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The final rule EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. The EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule.
<IP_ADDRESS> Costs May Lead to Study Selection Biases
Comment: Commenters (6168, 6446, 6924, 9227) are concerned that the implementation costs of this rule may lead to selection biases regarding which studies EPA will include in its analyses.
Commenter (9227) states that, at a time when federal funding for research in environmental and public health-related fields has largely flat-lined, academic researchers are likely to have few additional funds available to undertake these activities. Commenters (6168, 6446, 6924) assert that, the lack of any commitment by EPA to fund implementation of procedures to protect privacy, confidentiality, security and other interests would privilege industry science, where funds would be available to comply with the proposed rule's requirements and could result in the exclusion of robust academic and other science from EPA's consideration only because of insufficient funds to comply with the proposed rule's onerous and unnecessary requirements. Commenter (6924) states that this would create a situation in which data EPA considers in decisions would be heavily skewed towards industry studies  -  and it is well documented that financial sponsorship (i.e. source of funding) introduces a risk of bias in the results and conclusions in favor of the regulated industry's interests., , ,  The commenter asserts that an influx of industry-funded studies with financial conflicts of interest would likely bias the final results of EPA's analysis, leading to less stringent regulations and policies than with consideration of unbiased studies -- with the ultimate result of reducing protections for the health of families. The commenter expresses concern that EPA's proposed approach could institutionalize a dangerous bias in the source of studies that EPA can use for regulatory activity.
Response: Under the final rule EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. All such studies will be subject to extensive review for scientific quality. Though greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation, the Agency does not agree that this necessarily institutionalizes biases in the source of studies or would reduce protections for the health of families. The EPA notes that it is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by
14.2 SNPRM Implementation
 14.2.1 Inter-organizational Efforts
Comment: Commenter (11479) points out that other federal and state agencies are involved in EPA's decision-making, yet there is little detail about how this rule would impact that coordination. Consequently, the commenter suggests that there is the possibility that this rule would hamper EPA's ability to quickly and effectively coordinate with other agencies during a disease outbreak, such as COVID-19. For example, the commenter notes that key studies which rely on confidential medical information may be excluded, "derailing any interagency effort to incorporate the study's findings into a federal pandemic response. In other words, the commenter contends that the EPA's proposed rule could, in times of crisis, slow or stifle life-saving discoveries in public health."
Commenter (12727) states that the SNPRM expansion of the 2018 proposal, has inadvertently and irrationally hampered its ability to cooperate with other agencies in conducting joint or coordinated rulemakings, beyond the examples noted previously. The commenter references the finalized emission standards for greenhouse gases from light-duty vehicles, which EPA intended to harmonize with the National Highway Traffic Safety Administration's (NHTSA) corporate average fuel economy standards (CAFE). The commenter notes that both sets of standards are based on scientific analysis for which underlying data or models are not available, and that EPA could not have relied on this analysis in setting greenhouse gas standards had the proposal's requirements been in place, even if the analysis did not alter the level of the standards. The commenter conceives that there would be instances in which discrepancies in the rulemaking records considered by EPA and NHTSA, respectively, would result in different standards and that it would be bizarre if, going forward, EPA and NHTSA were to attempt another harmonized rulemaking and be prevented from doing so by the fact that NHTSA relied on studies without publicly available underlying data or models while EPA would not.