When EPA develops significant regulations using public resources, including regulations for which the public is likely to bear the cost of compliance, EPA should ensure that the data and models underlying scientific studies that are pivotal to the regulatory action are available to the public. This proposed rule is designed to increase transparency in the preparation, identification, and use of science in policymaking.
Id. at 18,769/3.
      Regulatory determinations based on science should describe and document any assumptions and methods used and should address variability and uncertainty.
Id. at 18,770/2.
      Pivotal regulatory science' is the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of-departure from which a reference value is calculated. In other words, they are critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, risks and other impacts on which a final regulation is based.
      Id.
      This [P]roposal will help ensure that EPA is pursuing its mission of protecting public health and the environment in a manner that the public can trust and understand.
Id. at 18,769/1.
In this section of our comments, we make the following points opposing the Proposal and supporting its withdrawal:
 First, the Proposal as written sweeps broadly to capture -- and thereby to prohibit EPA from considering -- studies, models, and analyses that are integral to the functioning of EPA regulatory programs, implementation of statutes like the CAA, and protection of public health and the environmental. It is both destructive and unlawful for EPA to refuse or fail to consider these additional studies, models, and analyses. We discuss numerous examples below.
 Second, to the extent that the Proposal does capture one or more of the studies, models, or analyses below, the Proposal would require EPA to conduct independent peer review of these materials before considering or using them, or before continuing to make them available for public use and awareness. See 83 Fed. Reg. at 18,774 (proposed § 30.7). This is objectionable and absurd. It is also unlawful for the same reasons that the Proposal is unlawful, as detailed in these comments and others.
 Third, to the extent that EPA disagrees that one or more of these studies, models, and analyses are captured by the Proposal, continuing to consider these materials while prohibiting EPA from considering other materials would be arbitrary and capricious. This is because these studies, models and analyses have the same hallmarks as "pivotal regulatory science" that the Proposal would exclude, as discussed in greater detail below. We emphasize that we do not believe EPA should or that EPA may fail to consider these other studies, models, or data, for the reasons set forth in these comments. Rather, our point is that continuing to consider these materials demonstrates additionally that the Proposal is arbitrary, capricious, and an abuse of discretion.
The Proposal states that "[t]he provisions of this subpart apply to dose response data and models underlying pivotal regulatory science that are used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the regulatory science." 83 Fed. Reg. at 18,773/3 (proposed § 30.3). Next, the Proposal defines "dose response data and models" to mean:
      the data and models used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact. Such functions typically underlie pivotal regulatory science that drives the size of benefit-cost calculations, the level of a standard, and/or the points of departure from which reference values (reference doses or reference concentrations) are calculated.
Id. at 18,773/2 (proposed § 30.2). Then, the Proposals defines "pivotal regulatory science" to mean "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." Id. Finally, the Proposal defines "regulatory science" to mean "scientific information, including assessments, models, criteria documents, and regulatory impact analyses, that provide the basis for EPA final significant regulatory decisions." Id.
The Proposal either covers on its face, or appears to cover, examples of studies, models, and analyses that are integral to the functioning of EPA regulatory programs, implementation of statutes like the CAA, and protection of public health and the environment. The commenter states that it would be harmful, unlawful, arbitrary, and capricious, and an abuse of EPA's discretion to include these materials within the sweep of the Proposal's prohibitions.
Alternatively, according to the commenter, if EPA disagrees that the following examples are covered by the Proposal, then continuing to consider these materials that have the same hallmarks as the prohibited materials, and that raise the same issues and concerns that cause EPA to prohibit their consideration, demonstrates that the Proposal is arbitrary and capricious, biased, and internally inconsistent and contradictory. Moreover, the commenter states that in this case, the Proposal would suffer from fatal failures to explain why EPA may consider these materials, while the Proposal would prohibit EPA from considering other materials.
Response: See response to Comment 4829-2022. EPA notes that the rule requires any peer review to be conducted in accordance with applicable OMB guidance. EPA notes that the final rule does not apply to models and applies to only the subset of data that consists of dose-response data.
Comment [6163-2935]: Commenter (6133) states that dose response data and models is defined as "the data and models used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact. Such functions typically underlie pivotal regulatory science that drives the size of benefit-cost calculations, the level of a standard and/or the points of departure from which reference values (reference doses or reference calculations) are calculated." (§ 30.2). Despite being an important phrase repeated through the Proposal and the proposed text, this compound definition is vague and arbitrary. It also is circular -- the very terms being defined are used in the definition. It's unclear what data EPA is referring to in this phrase and definition. Moreover, it's unclear what EPA means by "[s]uch functions typically under pivotal regulatory science . . ." And the problems with "pivotal regulatory science" have already been discussed [in their comment letter]. As explained in section XII [of their comment letter], the definition does not adequately describe what the proposal covers. This definition, according to the commenter, along with the rest of the Proposal, is arbitrary and capricious and must be withdrawn.
Response: EPA has clarified the distinction between dose-response data and data. See Sections III.B, III. C and III.E of the final rule preamble.
Comment [6163-3025]: Commenter (6133) states that in proposed § 30.6, EPA proposes to "evaluate the appropriateness of using default assumptions, including assumptions of a linear, no-threshold dose response, on a case-by-case basis." 83 Fed. Reg. at 18,774. In proposed § 30.9, the Proposal grants the Administrator the ability to "grant an exemption to this subpart on a case-by-case basis if he or she determines that compliance is impracticable" for a number of enumerated reasons. The commenter contends that soth of these provisions inject additional arbitrariness into the rule, in that they ensure that the Proposal may be applied unevenly -- for certain rulemakings, the "rules" of the Proposal can be discarded or ignored where desired. This, in addition to and with other sections of the Proposal, underscores that it is arbitrary and capricious and must be withdrawn.
Response: See response to Comment 6163-2871
Comment [6163-3346]: Commenter (6133) states that the Proposal at section 30.3 states that:
      Except where explicitly stated otherwise, the provisions of this subpart do not apply to any other type of agency action, including individual party adjudications, enforcement activities, or permit proceedings.
83 Fed. Reg. at 18,773.
This provision most clearly highlights one of the arbitrary and capricious advantages that industry stakeholders enjoy under the Proposal: it exempts from its censoring coverage EPA activities where industry is the primary party likely to submit confidential information that EPA may consider and rely upon. This, notwithstanding that the submitted information is not "publicly available in a manner sufficient for independent validation," while still being highly relevant and even integral to EPA's legal responsibilities. Id. at 18,768.
Permitting activities are one key example. For permitting actions taken under the CAA, RCRA, CWA, etc., the Proposal arbitrarily and capriciously allows EPA to continue to rely on highly relevant regulatory science and other information supplied by industry that is not "publicly available in a manner sufficient for independent validation." Id. at 18,771 - 73. A company seeking a permit or permit revision may submit regulatory science, confidential business information or other non-confidential information that is not "publicly available in a manner sufficient for independent validation." Id. EPA could consider non-peer reviewed, nontransparent industry science or information to conclude that a non-transparent industry model demonstrates no adverse air quality impact on a neighboring national park or wilderness area. This, despite the inputs and assumptions behind the model being unavailable to the public. An applicant could assert that there are safe exposure levels for PM2.5 or lead, and therefore EPA need not require any mitigation measures at concentrations below NAAQS levels in attainment areas. Industry applicants could rely upon hidden CBI to project no emissions increases for purposes of NSR permitting under the so-called "demand growth" exclusion, notwithstanding the unavailability of information critical to industry's claim and EPA's acceptance of that claim. Considering this and other non-transparent information, EPA could conclude that permits or permit revisions may be granted in situations where they should not lawfully be granted, notwithstanding that the non-transparent, unavailable information is scientifically erroneous and even absurd.
A second example is public information submitted during enforcement proceedings. The Proposal arbitrarily and capriciously allows EPA to continue to rely on highly relevant regulatory science and other information supplied to the agency by industry during enforcement proceedings, even when that information is not "publicly available in a manner sufficient for independent validation." Id. at 18,771 - 73. Consider, for example, a company that receives a notice of violation from EPA and meets with the agency to make the case that EPA and the DOJ should not file a complaint. The company may submit regulatory science, CBI, or other non-confidential information that is not "publicly available in a manner sufficient for independent validation." Id. at 18,768. EPA could consider non-peer reviewed, non-transparent, erroneous industry science to conclude that formaldehyde or asbestos are not carcinogens, or that PM2.5 or lead have safe exposure levels, or that CO2 does not endanger public health or welfare. Considering this and other non-transparent information, EPA could conclude that prosecution is not warranted, or that the information represents mitigating factors for penalties or injunctive relief, notwithstanding that the non-transparent, unavailable information is scientifically erroneous and even absurd.
The third case is public information submitted during individual party adjudications. Id. at 18,771 - 73. The Proposal arbitrarily and capriciously allows EPA to continue to rely on highly relevant regulatory science and other information supplied to the agency by industry during individual party adjudications, even when that information is not "publicly available in a manner sufficient for independent validation." Id. at 18,768. Consider, for example, a company facing an EPA order or applicability determination that qualifies as an adjudication under the APA or one of the federal statutes that the agency administers.
The company may submit regulatory science, CBI or other non-confidential information that is not "publicly available in a manner sufficient for independent validation." EPA could consider non-peer reviewed, non-transparent industry science to conclude that formaldehyde or asbestos are not carcinogens, or that PM2.5 or lead have safe exposure levels, or that CO2 does not endanger public health or welfare. Considering this and other non-transparent information during the individual party adjudication, EPA could conclude that adoption of the order is not warranted, or that agency regulations should be interpreted in a way that does not apply to that company's actions. Indeed, EPA could conclude, after considering the non-transparent, unavailable information, that the regulations should not apply in ways that would affect an entire industrial sector favorably, while harming the public meant to be protected by those regulations. Under proposed section 30.3, EPA could consider the non-transparent, unavailable information to reach these objectionable outcomes, notwithstanding that the information is scientifically erroneous and even absurd. The Proposal nowhere explains why it is valid and consistent with EPA's statutory authorities and responsibilities to consider information that is not "publicly available in a manner sufficient for independent validation" under the situations allowed in proposed section 30.3 (individual party adjudications, enforcement activities, or permit proceedings), while prohibiting EPA consideration of that information in situations covered by the Proposal's prohibitions. Indeed, it is striking that the Proposal does not even attempt any such explanation or justification. Id. at 18,771 - 73. This is undoubtedly because there is no coherent, lawful justification or explanation that the agency could muster; it is unsurprising that the Proposal cannot overcome this.
Indeed, it is a hallmark of the Proposal's inherent arbitrariness and capriciousness that the Proposal prohibits EPA from considering the identical regulatory science, studies, and information in some regulatory situations, while allowing EPA to consider the identical regulatory science, studies, and information in other regulatory situations -- based merely upon the type of situation, rather than any differences in availability, replicability, verifiability, or validation concerning the information. Proposed section 30.3 prohibits EPA from considering information that is not "publicly available in a manner sufficient for independent validation"
102 during so-called "significant regulatory decisions," while prohibiting EPA from considering that identical regulatory science, studies, or information during "any other type of agency action, including individual party adjudications, enforcement activities, or permit proceedings." 83 Fed. Reg. at 18,768, 18,771. The Proposal does not and cannot explain or justify this differential treatment, so the Proposal does not even try.
Finally, proposed section 30.3 is unlawfully vague, open-ended, and arbitrary due to the capacious and unlimited way that EPA has drafted the exclusion from the Proposal's prohibitions. Section 30.3 indicates that "the provisions of this subpart do not apply to any other type of agency action." This grants EPA capacious and effectively unlimited discretion and authority to decide what "any other type of agency action" is and is not, without providing the public or regulated entities any criteria, understanding or advance notice as to how EPA will exercise that discretion and authority. That is the essence of arbitrary and capricious agency action. Indeed, the Proposal is structured in such a way that EPA will be exercising that discretion and authority -- to decide what "any other type of agency action" does and does not cover -- in secret, with no public input and no public awareness, concerning the situations in which EPA will and will not consider non-transparent, unavailable information. The commenter contends that, in addition to this being perversely ironic, considering the "transparency" title of the Proposal, this fact renders the Proposal even more arbitrary and capricious and unlawful.
Response: By focusing on significant regulatory actions and influential scientific information, EPA is applying this rule to the actions that have the potential to have the greatest impact on the American public.
Comment [6163-3479]: Commenter (6133) states that the term "pivotal regulatory science" is perhaps the most vague, unexplained, and internally inconsistent term used in the Proposal. The term has no statutory basis in any statute cited by EPA, or otherwise. Beyond having no statutory underpinning, the meaning of the phrase is neither self-evident nor adequately defined in the Proposal.
EPA's choice to modify "regulatory science" with the adjective "pivotal" does nothing to clarify the scope of scientific studies and information encompassed by the Proposal. "Pivotal regulatory science" is defined within the regulation as "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." 83 Fed. Reg. at 18,773/3 (proposed § 30.2). This definition is as unclear and unsupported as the term itself.
The use of the phrase "drive the requirements" within the C.F.R. definition is particularly incoherent. What does "drive the requirements" mean? The Proposal nowhere says. Does the definition apply only to scientific studies that were outcome determinative? Does it encompass any scientific study that was considered in making the requirements? What about studies that were useful but not determinative? Something else entirely? Can more than one study be "pivotal" to the regulatory decision, or does the term "drive the requirements" imply that only one study could be "pivotal" to a given decision? Furthermore, are most of the studies used by EPA considered to "drive the requirements" or is this term limited in some fashion, unrevealed to the public? Will EPA "know it when it sees it," making it up as the agency goes along?
According to the commenter, it is arbitrarily, vague, and unexplained under the Proposal which science would be considered "pivotal," and under what conditions. Because the term was created out of thin air to serve EPA's purposes and has no statutory grounding or intuitive meaning, this ambiguity-ridden definition is woefully inadequate. It is also arbitrary and capricious and an abuse of EPA's discretion. EPA is well aware of the insufficiency of the definition, as is evident in the agency's solicitation of comments on the definitions of "pivotal regulatory science" and "dose response data and models" within the Proposal. See 83 Fed. Reg. at 18,771.
Notably, the proposed CFR definition also differs substantially from a definition of "pivotal regulatory science" appearing earlier in the Proposal, which defines the term as "the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of departure from which a reference value is calculated. In other words, they are critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, or risks and other impacts on which a final regulation is based." 83 Fed. Reg. at 18,770.
Next, it bears repeating that EPA does not and cannot identify any statutory basis -- in federal environmental statutes, the APA or otherwise -- to apply the Proposal's approach "to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, or risks and other impacts on which a final regulation is based." 83 Fed. Reg. at 18,770. EPA simply makes this up.
EPA's separate explanation here suffers from additional defects, namely an internal inconsistency, incoherency and unbounded reach that do not accord with the proposed C.F.R. definition. EPA's preambular explanation says that "pivotal regulatory science" is "critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, or risks and other impacts on which a final regulation is based." 83 Fed. Reg. at 18,770 (emphasis added) The Proposal nowhere explains what these "other impacts" are. Nor does the Proposal limit or bound these "other impacts," nor link them to the sentence's incoherent notion of what is "critical" and what is not. Moreover, the preambular gloss is inconsistent with the proposed C.F.R. definition. The former says "pivotal regulatory science" is critical to hopelessly vague "other impacts" on which a final rule is based. Id. The proposed C.F.R. definition, by contrast, says "pivotal regulatory science" "drive[s] the requirements and/or quantitative analysis of EPA final significant regulatory decisions." 83 Fed. Reg. at 18,773/3 (proposed § 30.2). The Proposal does not square the contradictions between science that drives a final rule's requirements and science that is "critical" to "other impacts" in a final rule.
Furthermore, the commenter states that the EPA not only fails to provide a passable definition for its invented term, "pivotal regulatory science," the agency fails to provide its rationale for limiting the scope of the rule to so-called "pivotal regulatory science." Within the unlawful, arbitrary, and capricious worldview reflected in the Proposal, why is the "public availability of science and data in a manner sufficient for independent validation" any less important or necessary or justified when the science is not "pivotal" or "critical" to a regulatory decision? Why should not all science, studies, data, and information considered by EPA meet the standards for transparency, verifiability, independent validation, and trustworthiness that are the abiding concerns of the Proposal? Why is it not arbitrary and capricious for EPA to continue to consider science and data that are unavailable and insufficient for independent validation in areas outside the reach of the Proposal? EPA offers no explanation for this disparate treatment; the agency's reasoning, such as it is, is entirely conclusory.
By way of explanation for the limitation, EPA only suggests that the imposed standards "are of paramount importance when the government relies on science to inform its significant regulatory decisions." 83 Fed. Reg. at 18,769. This explanation is hopelessly circular and ultimately incoherent. For starters, EPA does not explain why it believes this explanation to be true. Next, the Proposal just substitutes the word, `paramount,' for the word, `critical,' that it substitutes for the word, `pivotal.' (The Proposal's drafters evidently were just flipping through a thesaurus.) This failure to thoroughly explain both the term "pivotal regulatory science" in a way that meaningfully defines the scope of the regulation, and the rationale behind limiting the application only to pivotal (critical, paramount) science, makes it impossible for interested parties to comment fully and meaningfully on the Proposal. The commenter suggests that, should EPA intend to finalize this unlawful proposal, EPA first must withdraw the Proposal, then issue a supplemental proposal with the necessary definitions and explanations. Better yet, the commenter suggests that EPA abandon the illegal and harmful proposal altogether.
Response: Regarding compliance with the APA, see response to Comment 2157-149. EPA notes that the APA does not require a proposal to be withdrawn before a supplemental proposal is issued. Response to comments on pivotal science are addressed in the final rule preamble (See Sections III.C and III.E) and in Chapter 6 of this document.
Comment [6168-4103]: Commenter (6178) states that the Agencies' proposed action in the NPRM  -  adopting a new standard for all scientific research to be relied on by the EPA going forward  -  is itself a substantive agency action, triggering certain requirements of the APA. Specifically, the requirement of the APA that agency action not be `arbitrary and capricious' as interpreted by the Supreme Court of the United States in Motor Vehicle Manufacturers' Association v. State Farm Mutual Automobile Insurance Co.(`State Farm').
Under Section 706 of the APA, a reviewing court can set aside agency action if the action is found to be "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law." The Supreme Court provided clear guidance as to what constitutes an arbitrary and capricious agency action in State Farm where the Court stated: "Normally, an agency rule would be arbitrary and capricious if the agency has... relied on factors which Congress has not intended it to consider...."
As stated above, a plain reading of the Congressional Acts cited by the EPA as support for its proposed Transparency Rule demonstrates that transparency, as contemplated by the NPRM, is not a factor on which Congress intended the EPA to rely on when evaluating what scientific research it would consider when adopting regulations pursuant to the Acts. In fact, to do so would very likely contradict the stated Congressional intent that such research be conducted with a focus on cooperation and coordination with other like-minded institutions. For these reasons, the commenter asserts that the Transparency Rule must be considered arbitrary and capricious under the APA as it would require the EPA to `rely on factors which Congress has not intended it to consider'.
Response: See response to Comment 2157-149. This final rule builds upon and is consistent with existing federal government open data initiatives and supporting federal agency guidance.
EPA notes that the final rule does not cite any of the statutes that it administers as authority for this rule. This procedural rule does not interpret or apply provisions of the statutes that it administers. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield. As discussed in the final rule preamble at III.A, EPA plans to issue statute-specific transparency regulations or programmatic regulations. EPA will address any statutory conflicts as appropriate when EPA develops these subsequent rulemakings.
Comment [6336-2687]: Commenter (6358) states that the EPA's process for promulgating this rule represents a gross deviation from the agency's typical action development process. EPA failed to perform an appropriate regulatory impact analysis, despite indications that implementation costs could be substantial. EPA has not received sufficient input from the OIRA, which conducted a highly truncated review and does not appear to have met with outside groups during that process, or from its SAB, which states that the rule "deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board." Taken together, these deviations from typical process leave EPA unable to consider its proposal accurately, and constitute an arbitrary and capricious approach to rulemaking.
Response: See response to Comment 2157-149. EPA's Action Development Process is not legally binding and is a purely internal, collaborative process. In any event, the supplemental proposal underwent OMB and interagency review and was thoroughly reviewed by the SAB. The final rule discusses the SAB's findings and the SAB's report is available in the rulemaking docket.
Comment [6339-3188]: Commenter (6361) suggests that while EPA states that the new rule would be applied prospectively, it notes that the rule could impact prior records for older rules that are subject to recurring updates and reviews. The new rule does not point to any basis in law or fact to question the validity and rationale of prior EPA rulemakings, and thus it would be improper and contrary to the APA for EPA to reopen prior regulations for review due to alleged lack of transparency in past scientific studies. Without specific evidence, particular findings, and legal analysis regarding individualized federal agency decisions, the commenter states that it is unorthodox and legally suspect for EPA to issue a new regulation based on blanket assertions of a generalized, non-specific nature.
Response: This rule does not apply retrospectively to EPA regulations.
Comment [6340-1690]: Commenter (4879) states that they strongly support EPA's demonstrated commitment in this proposal to build upon the principles underlying the APA, Executive Orders 12866, 13777, and 13783, and guidance of OMB.
Response: Thank you for the comment.
Comment [6425-2643]: Commenter (6447) states that the current proposal would require EPA to ignore some of the best available science, and to ignore certain information submitted during rulemaking. In particular, the proposal would make it difficult or impossible to use most epidemiological studies, where the source data are almost always comprised of sensitive personal information. The Agency does not the option of suppressing this information, and courts have held that EPA "cannot ignore existing data." In short, the commenter contends that the proposed action would be illegal, and would not survive a legal challenge.
Response: See response to Comment 4829-2022.
Comment [6428-1352]: Commenter (6450) states that the proposed rule is arbitrary and capricious for myriad reasons. EPA has failed to demonstrate the benefits of the proposal or how any such benefits would outweigh the associated costs.
Response: Regarding cost-benefit analysis, see response to Comment 6112-4452.
Comment [6845-1604]: Commenter (6867) provides that the APA, 5 U.S.C. § 500 et seq., prohibits agency action, findings, and conclusions that are arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law. See 5 U.S.C. § 706(1). In view of the overwhelming evidence that (i) a very high percentage of published studies are not reproducible, (ii) pre-publication peer review is not sufficient to ensure that a study lacks major errors, and (iii) pre-publication peer review is not sufficient to ensure that the computational code underlying a study lacks major errors, we respectfully submit that the Proposed Rule is necessary and arguably required by the APA. Indeed, according to the commenter, an agency action based on a study without fully disclosed data, methodology, and computational code is arguably so lacking in reasoned foundation as to be arbitrary and capricious.
Response: The commenter has not supported any of these assertions, which they contend are supported with "overwhelming evidence" with any evidence.
Comment [6888-4166]: Commenter (6909) opposes the proposed rule because it will do harm to important public health safeguards and will also negatively impact researchers and the scientific endeavor generally. The commenter states that the proposed rule's attempt to exclude perfectly reliable scientific studies from EPA's consideration is a thinly veiled attempt to allow the EPA to ignore sound science at its discretion if that science does not conform with the current Administration's political agenda. The commenter adds that the proposed rule would be very damaging and would fail to fulfill EPA's obligations under the APA for numerous reasons.
The commenter states that a ban on EPA considering relevant studies that have been vetted through peer review but whose underlying data cannot been made publicly available or which cannot be replicated will not strengthen EPA's decision-making process and its ultimate policy making, it will weaken them.
Response: See response to Comment 4829-2022.
Comment [6894-3595]: Commenter (6915) states that the EPA's proposal would violate the APA, 5 U.S.C. §501 et seq., both because it is arbitrary and capricious, and because it flouts the Act's important procedural requirements. EPA claims that the entire basis for the proposed rule is to ensure that the "pivotal regulatory science" underlying EPA regulations is transparent. But EPA ignores existing laws and policies that already do exactly that and which also take into account the need to protect medical data and other confidential information. This proposed rule would promote transparency in name only; in truth, it would mean that EPA's important decisions would no longer be informed by the latest, best available, and generally accepted science. Disturbingly, the proposed rule's only failsafe is the EPA Administrator's sole discretion to determine on a case-by-case basis that compliance is "impracticable" when making data publicly available is "not feasible." But the proposal provides no standards to govern the Administrator's exercise of discretion in determining "impracticability or "feasibility" -- a recipe for the very arbitrariness that the APA prohibits.
Response: See response to Comments 2157-149, 4829-2022, 6112-3062, and 6163-5030.
Comment [6895-3615]: Commenter (4879) states that the EPA's failure to solicit input from the SAB and other scientific groups is exacerbated by its failure to meet the fundamental legal requirements for a valid rulemaking proposal under the APA. The APA requires that "general notice of proposed rulemaking shall be published in the Federal Register," including the "terms or substance of the proposed rule." 5 U.S.C. §553(b). The straightforward purpose of this requirement is to give the affected public an opportunity to provide meaningfully informed comment on an agency's proposal. See Home Box Office, Inc. v. Fed. Commc'n Comm'n, 567 F.2d 9, 35-36 (D.C. Cir. 1977). But here, EPA's NPRM is vague as to the actual parameters of the proposed rule, is open-ended in terms of the alternatives under consideration, and fails to provide key information such as projected costs. Courts will not hesitate to strike down final rules based on proposals so lacking in specificity. See, e.g., Horsehead Res. Dev. Co. v. Browner, 16 F.3d 1246, 1268 (D.C. Cir. 1994) (noting that "general notice that a new standard will be adopted affords the parties scant opportunity for comment").
Far from meeting the requirement to "disclose in detail the thinking that has animated the form of a proposed rule and the data upon which that rule is based," Home Box Office, 567 F.2d at 35 - 36, the proposal at issue here creates far more questions than it answers. Most fundamentally, the proposal fails to provide a rationale for EPA to act contrary to accepted scientific practice, i.e., to preclude consideration of probative scientific information that has been subject to rigorous peer review for the sole reason that underlying data are confidential and therefore not publicly available. The proposal states that "EPA believes the benefits of this proposed rule justify the costs," 83 Fed. Reg. at 18,772, but fails to provide any specific information, quantification, or analysis as to what EPA believes are the proposed rule's purported benefits or expected costs, including the significant costs from the loss of probative information that the proposed rule would work to exclude. For example, Section 30.7 of the proposed rule could be read to require EPA to undertake very costly independent review of "pivotal" science on which it relies, but Section 30.8, entitled "How is EPA to account for cost under this subpart?" states only that EPA will "minimize costs." Id. at 18,774. The absence of data and analysis in support of EPA's cost-benefit conclusion deprives the public of a meaningful opportunity to evaluate the proposal and thus violates EPA's duty under the APA.
Response: See response to Comments 2157-149 and 6112-4452.
Comment [6894-3630]: Commenter (6915) asserts that the proposed rule disregards the APA's bedrock requirement that an agency's decision-making be based on a consideration of the relevant factors and data. See Motor Vehicle Mfrs., 463 U.S. at 42-43 (articulating standard and citing numerous cases). An agency's action is arbitrary and capricious not only if the agency "entirely fail[s] to consider an important aspect of the problem," but also if it "relie[s] on factors which Congress has not intended it to consider." Id. at 43. The proposed rule would call for EPA to do both. First, in excluding studies and models from its consideration based only on whether the underlying data are publicly available or have been subject to additional independent review by EPA, EPA would be excluding studies and models that Congress has instructed it to consider by requiring it to use, for example, the "best available science" or "latest scientific knowledge." Second, because none of the statutes EPA administers specify that, in setting standards, it shall consider whether the studies and models it uses have publicly available data or have been independently reviewed by EPA, EPA would be using factors that Congress did not intend it to rely on in deciding to exclude studies and models based on the proposed rule. See Am. Trucking, 283 F.3d at 372 (finding that the CAA does not require EPA to "obtain and publicize the data underlying the studies on which the Agency relies"). According to the commenter, EPA's failure to consider otherwise relevant studies and models that do not meet the proposed rule's requirements would therefore be arbitrary and capricious. See Motor Vehicle Mfrs., 463 U.S. at 42-43.
Response: See response to Comment 4829-2022.
Comment [6895-3232]: Commenter (6895) states that the Proposal claims to be "consistent with the principles underlying the Administrative Procedure Act". But, the Proposal does not clearly identify the authority under which it is promulgated, is so vague as to undermine the ability to meaningfully comment, and in its substance contradicts the APA requirement that agencies must consider all the relevant matter presented in a rulemaking record.
i. The Proposal does not clearly identify the authority under which EPA has promulgated it.
As discussed above, the CAA does not authorize the Proposal, which is clearly aimed at limiting the studies EPA can rely on and consider in CAA rulemaking. Additionally, APA section 553(b)(2) requires a proposed rule to provide a "reference to the legal authority under which the rule is proposed." While the Proposal's preamble lists various statutes the Agency claims support its action, a closer look reveals that none of those sections actually authorizes EPA to disregard scientific studies. Certainly that is the case for the CAA sections listed by the Agency in the Proposal, including the general rulemaking authority under the Act which requires the evaluation of all material in the record. EPA's suggestion that it has "general authority" to make rules to comply with its obligations is similarly unavailing here, as the general authority does not exist to make rules that are outside EPA's statutory obligations, or in violation of them  -  and a rule that would enable EPA to disregard relevant studies in a CAA rulemaking record is clearly outside the Agency's authority. Nowhere does the Act permit EPA to limit its own access to relevant science.
ii. The Proposal's terms are so vague as to fail to afford an opportunity for meaningful comment.
Significant issues are presented in the Proposal only in the broadest terms, despite an APA requirement that a proposed rule includes "the terms and substance of the proposed rule or a description of the subjects and issues involved," 5 U.S.C. § 553(b)(3), and with enough specificity "to permit interested parties to comment meaningfully." As noted supra, we assume (as we must, given no other information from the Agency), that the Proposal is aimed at precluding EPA's reliance on studies for which the data are not made public. That is the natural reading of the proposed regulatory language, which is the most choate information offered in the Proposal. But even that reading is not certain, given the very broad discretionary nature of the exemptions provided for in the rule text (see, e.g., 83 Fed. Reg. 18,774, proposed 40 C.F.R. § 30.9, giving the Administrator the ability to grant an exemption to the data publication requirement if that is "impracticable," which is undefined in the Proposal). As a result, it is unclear how this rule would apply to EPA decision making, and how it would be different than the Agency's current protocol for considering studies relevant to specific rulemaking proposals.
As one example, EPA alludes to "transparency" multiple times, but without defining what that concept means, or explaining why it believes existing statutes, regulations, guidelines and protocols fail to meet it. While asserting an interest in "ensur[ing] that the data and models underlying science is [sic] publicly available in a manner sufficient for validation and analysis," there is no more specific statement explaining how that would be implemented or why releasing confidential health and business information is necessary. Nor is the Agency's rationale completely clear in the Proposal. When commenters are left to guess, that is not lawfully sufficient notice of the subjects and issues involved.
iii. The Proposal's substance conflicts with the APA requirement that Agencies consider relevant matter presented in the rulemaking record.
Section 553(c) of the APA requires that when finalizing a proposed rule, and after taking comment, an agency must "consider[] the relevant matter presented" in the record. To the extent that the Proposal envisions authorizing EPA to ignore or disregard or not to consider some studies submitted during a rulemaking comment period, such Agency action would be taken in direct violation of this provision of the APA. Nor, according to the commenter, is there any basis in the APA for EPA to decide to exclude or prohibit the consideration of otherwise relevant matter because it is based on confidential information not available to the general public.
Response: See response to Comment 2157-149.
Comment [6895-4397]: Commenter (6916) states that an agency's decision is arbitrary and capricious where it fails to consider important aspects of the problem, is counter to the evidence before it, or relies on factors Congress has not intended it to consider. When the action rescinds, or represents a significant about face from, previous agency rules or longstanding practice, it must provide a reasoned basis for doing so. If finalized as proposed, EPA's attempt to cut off access to significant scientific results would fall short on all of these factors and would also lead to further arbitrary and capricious agency decision making.
Response: EPA did not finalize the proposal as is. See responses to Comments 2157-149 and 4829-2022.
Comment [6895-4399]: Commenter (6916) asserts that the failure to adequately justify a changed agency position is one of the hallmarks of arbitrary and capricious decision making. The Agency furthermore must "provide a more detailed justification than would suffice for a new policy...when, for example, its [revised] policy rests upon factual findings that contradict those which underlay its prior policy; or when its prior policy has engendered serious reliance interests that must be taken into account. It would be arbitrary and capricious to ignore such matters."
EPA, in the Proposal, fails to engage with the reasoning or facts underlying the Agency's longstanding policy position, which respected the confidentiality of underlying data and accounted for all relevant science. Further, this position has engendered significant reliance interests. Many of the "dose response" studies the Agency singles out for attention have been relied on in decision making for many decades, and have been the basis for many other subsequent work, on which the Agency also has relied. The Agency seems not to understand this or have a plan for how to manage the fallout from a final rule that denies access to all of that body of work  -  whether in an exclusively forward-looking way, or retroactively. But, "[a]n agency cannot simply disregard contrary or inconvenient factual determinations that it made in the past."
iii. If finalized, the Proposal's requirement to disregard relevant studies would ensure further arbitrary decision making.
Were this rule to be finalized as proposed, it would permit or even require the Agency to disregard certain studies placed in the record in future substantive rulemakings under the CAA, among other statutes. Not only would that render the future substantive rules unlawful under the Act's requirements, but it would also make them arbitrary and capricious, as "failure to consider the evidence proffered renders [a decision] arbitrary and capricious" and the Agency may not "hastily discount[]" relevant studies. The Agency seems completely unaware that its new policy, if finalized, could lead to future rulemaking actions that are unlawful, arbitrary, and capricious.
Response: See response to Comment 4829-2022.
Comment [6912-3753]: Commenter (6933) suggests that the NPRM be withdrawn because to the extent it can be understood what it requires -- through the haze of vague definitions and exceptions -- it both fails to address the issue which it purports to be concerned with, and is inconsistent with EPA's obligations to use best available science under both statutory requirements and caselaw. In essence, the rule identifies the goal of improving the quality and transparency of EPA's science and uses it as an excuse to adopt policies that actually reduce the quality, and likely the transparency, of EPA's scientific decision-making. According to the commenter, there is a fundamental mismatch between the problem identified and the solution adopted. Such an approach is plainly arbitrary and capricious and inconsistent both with the APA and with the environmental statutes under which EPA purports to act.
Response: See final rule preamble at Section III.A for a discussion of the match between the problem and the solution provided by this rule. See response to Comment 4829-2022
Comment [8750-1475]: Commenter (9227) states that in addition to violating the requirements of the various statutes that EPA administers or is subject to, the Proposal suffers from a total failure to consider important dimensions of the profound shift in policy that it implements. In the Proposal, EPA neglects to consider the many legitimate reasons why a study's underlying data may not be publicly available -- reasons that have nothing to do with the quality of the study -- and fails to offer solutions consistent with these legitimate limitations. EPA makes vague gestures to various guidelines and practices issued by other agencies and scientific organizations, none of which actually support the Proposal's radical position that EPA should exclude consideration of studies that rely upon confidential data. EPA does not even establish that there is a real problem that the Proposal would actually address: nowhere in the Proposal does EPA identify any prior agency action that has been called into serious question due to a failure to release study data. According to the commenter, the EPA's utter failure "to consider an important aspect of the problem" and to provide an explanation for the Proposal that is consistent with the evidence before the agency renders the Proposal wholly arbitrary and capricious. See Motor Vehicle Mfrs. Ass'n. v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 43 (1983). Likewise, the commenter suggests that the EPA's failure to explain its 180-degree change in position from its former belief that the lack of publicly available data does not render a study inappropriate for consideration in regulating is a hallmark of arbitrary and capricious decision-making. FCC v. Fox Telev. Stations, Inc., 556 U.S. 502, 515-16 (2009).
Response: See the final rule preamble at Section III.A for an explanation of why a change in policy is needed. See response to Comment 4829-2022.
Comment [8750-1480]: Commenter (9227) states that there are multiple reasons why underlying data are not publicly available for all studies.
There are legal and ethical requirements that restrict making public the data underlying studies, including rules to shield private personal information, requirements to maintain confidential business information, situations where obtaining the necessary permissions to release data are logistically difficult or impossible, and situations in which researchers have made significant investments in developing datasets that they intend to continue to work with for future studies. Not all of these barriers can be overcome, nor can they be overcome in every case. While there are ways potentially to address some of them, they can be extremely costly and burdensome, and/or may harm the prospects for further research. Accordingly, while the scientific community has made efforts to make more data publicly available, to the best of our knowledge all of the policies adopted by government and academic journals recognize that data is not, and need not be, publicly available to evaluate their quality.
a) Strong legal and ethical requirements limit the release of data in human subjects studies.
Particularly with respect to human subjects, there are strong legal and ethical privacy and confidentiality protections, which researchers are bound to respect. In some cases, researchers would be subject to civil or criminal penalties for violations.
The environmental health dose response studies targeted by EPA's proposal are likely to include human population studies (or epidemiological studies). Often the best available epidemiological studies contain extensive and sensitive data on individuals, such as environmental exposures, medical history (such as infant reproductive developmental abnormalities, children's behavioral and development problems, heart attacks or dementia among the elderly), dates of birth, residential address, drug use, race, socio-economic status (income, education), status of subjects' marriages, employment history, etc. For example, air pollution studies commonly use residential address information to assign air pollution exposures and link them to health effects. Other studies focused on genetically susceptible populations may also be linked to genetic databases or contain information on key genetic mutations that are strongly predictive of serious health risks, such as risk of Alzheimer's disease, and are thus very sensitive.
To conduct these studies, investigators must obtain informed consent from the study participants to collect protected health information, and investigators must sign documents promising to protect the privacy of this individually identifiable health information. Absent complex, difficult, and costly de-identification and redaction techniques, these data simply cannot be released publicly. As discussed below in section II.A.2.b), in some cases such techniques are simply not applicable or still leave significant risk of breach of privacy.
Additional protections apply to specific types of human subject information. For example, medical records are subject to strict requirements governing the use and disclosure of such information under the HIPAA of 1996. HIPAA requires researchers to protect identifiable information, and it provides that such information may only be disclosed for research purposes with the written consent of the person providing the information.
Another limitation on public availability of data is the requirement under the Federal Policy for the Protection of Human Subjects (also known as the Common Rule) that for all federally funded studies involving human research subjects, researchers must first obtain IRB approval and informed consent from study participants.
An IRB reviews each human subjects research project to ensure that the specific research protocol protects individual rights. Participants must be notified about the degree to which the confidentiality of their records will be maintained, and must receive appropriate notification and give consent if study data is to be shared outside the research team. The IRB also considers risks to the participants and how use of the information obtained may adversely impact the rights and welfare of the subjects. Most institutions have committed to comply with the Common Rule for all of their research, even when it is not federally-funded.
For studies that had received IRB approval prior to finalization of this proposed rule, there may be no practical opportunity to make the data publicly available. Even for new studies going forward, it may be extremely difficult, require additional (often unavailable) funding for elaborate protective measures, or simply impossible to obtain IRB approval for protocols that would allow the data to be made publicly available.
EPA's own SAB voiced these concerns that EPA was discounting the challenges to making even limited releases of data, saying:
The proposed rule oversimplifies the argument that "concerns about access to confidential or private information can, in many case, be addressed through the application of solutions commonly in use across some parts of the Federal government." For studies already completed or underway, the participation of human subjects is undertaken according to terms approved by the cognizant IRB. These terms can vary from study to study. In some cases, the data cannot be released simply by redacting portions of it. For example, data may have been collected with an assurance to the participating individuals that their data would be kept confidential.
Some researchers might respond by choosing to work only on public administrative datasets, but this would harm rather than strengthen science quality by curtailing scientific inquiry. Thus, the effects of EPA's proposed approach would cause some researchers to choose not to pursue research with human subjects, stifling scientific discovery, while others would forgo compliance with EPA's regulatory requirements and have their research ignored by EPA. As a result, EPA's proposal would both discourage the development of best available science as well as EPA's use of it.
b) There are especially significant barriers to public release of underlying data and models from studies that have already been completed.
With respect to studies that have already been completed, there are additional formidable barriers to public release of underlying data and models. Particularly, with older studies, simply finding the data sets and determining ownership may be expensive or impossible. For older studies with human subjects, obtaining consent to release of data may be practically impossible, and the data may have been collected in ways that would make protecting privacy with release difficult or impossible.
For some studies, administrative issues related to the data could be the most difficult barrier to overcome in providing for public release. Larger and more costly studies are often performed by groups of researchers within a university, across multiple institutions, or across multiple individual companies. Over time, the data itself may become lost or misplaced, or it may become unclear who actually owns and controls access to the data. Academics move among institutions, companies merge and spin off, and the initial agreements were not always clear in the first instance. Obtaining consent from multiple institutional players takes extensive time and resources, at minimum, and simply may no longer be possible in some instances.
These problems are exacerbated with respect to human subject studies. Researchers are legally and ethically obliged either to protect the privacy of the individual study subjects or attain each subject's consent to share data. This can be impractical for older studies and virtually impossible for larger studies, and extremely burdensome. For example, the Harvard Six cities study was started in 1975 and had 8,111 participants. The ACS CPSII extended analysis by Krewski in 2009, which is central to PM2.5 NAAQS standards, was initiated in 1979 and encompassed data from 500,000 study participants who lived in 116 metropolitan areas. For these types of situations, tracking down participants (or where the participants have passed away, their family members) to get consent is simply not realistically possible.
Even in situations where investigators might theoretically be able to attain consent, it would require extensive financial and human resources, which are usually simply not available, especially to academic researchers or to EPA. EPA ignores this prohibitive constraint and makes no attempt to address it.
c) There are additional significant barriers to public release of data in some situations, even for prospective studies.
Even with respect to prospective application of EPA's proposal, providing for public release of underlying data and models is costly and resource intensive, creating a serious disincentive for researchers to meet EPA's proposed requirements. Investigators willing to make their study underlying data publicly available would still face the logistical hurdle of making the data and models available in a manner sufficient for independent validation by the public. In addition to the cost of thoughtful and effective deidentification or redaction of sensitive information, the proposed text would likely require researchers to prepare annotated manuals including precise detail as to what variables were collected, how information was collected, and the rationale for each step taken. Some manuals alone run into hundreds of pages. One press account noted the example of publicly available datasets from the National Center for Health Statistics (NCHS), which can come with 100-page manuals; researchers would need to hire additional staff to meet such requirements. Yet EPA fails even to recognize (much less propose any means to address) the cost to researchers in time and money, on top of the constraints on academic research already imposed by the very limited funding available for this type of work.
In addition, there are other barriers to public release of underlying data. Studies conducted on behalf of industry or with industry cooperation may contain confidential business information, the release of which could jeopardize a company's competitiveness.
Also, in some instances, researchers cannot make their data sets public without losing much of the value to the researcher of these laboriously and meticulously collected sets of information. Research, especially those studies that include large numbers of human subjects, are incredibly human and capital-intensive endeavors. Moreover researchers may base years of work and multiple papers on unique datasets they developed and hold, and many scientists build their careers on carefully harvesting information from single large studies for years to come. It is not only unreasonable, but also unfair, to expect academic scientists to turn over their intellectual property and research investments, forgoing potential earnings and career advancements. Moreover, EPA's myopic and inflexible approach to data access gives no consideration to data sharing arrangements between researchers and the agency that could be developed to support EPA's consideration and integration of research.
If scientists are forced to choose between giving away their hard-earned data or forgoing any regulatory impact, it will discourage scientists from engaging in critical science that is targeted to help prevent disease and disability in our population. It appears that in many cases, scientists will choose to retain their datasets, with a worst-of-both-worlds result -- EPA will be deprived of valid scientific information and the scientific community will be discouraged from contributing their critical expertise to policymaking. EPA's Proposal does not consider the real-world implications of forcing such choices on researchers.
The commenter argues that the agency's failure to consider or examine any of these legitimate reasons for not making data publicly available is arbitrary and capricious.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency. See response to Comment 4829-2022. EPA notes that the final rule does not preclude EPA from considering studies based on data that is not publicly available. See § 30.5 of the final rule.
Comment [8750-1497]: Commenter (9227) states that in the proposal, the EPA blithely and irrationally ignores or assumes away real and significant issues, suggesting that existing mechanisms and techniques can be used to protect privacy and confidentiality while making underlying research data publicly available. In fact, according to the commenter, the evidence (including several of the sources that EPA cites) indicates that the potential mechanisms alluded to by EPA would only have the potential to address some of the barriers cited above, have serious limitations even for those, and are actually becoming less effective as it becomes easier to combine and manipulate public data sets.
Response: This rule does not require that dose-response data be made publicly available, nor does this rule categorically exclude any dose-response data based on the availability of the dose-response data. As discussed in Section III.E of the final rule preamble, EPA will give greater consideration to pivotal science if the underlying dose-response data is available, including through tiered or restricted access, for independent validation.
Comment [8750-1637]: Commenter (9227) states that the EPA begins by stating that the "proposed rule is consistent with the principles underlying the Administrative Procedure Act and programmatic statutes that EPA administers to disclose to the public the bases for agency rules and to rationally execute and adequately explain agency actions." While EPA is correct that it must disclose the basis and provide an adequate explanation for rulemaking (principles EPA manifestly fails to follow in this Proposal), it does not follow that these principles either require or support the quite specific notion that dose response data and models must be publicly available. Nor does EPA attempt to explain how these broadest of rulemaking principles support EPA's specific proposed approach here.
Response: This rule does not require that any data be made publicly available.
Comment [8750-1678]: Commenter (9227) states that the Proposal turns away from EPA's Scientific Integrity Policy, which stresses "a firm commitment to evidence," endorses use of "the best available science" and "[r]equire[s] reviews. . . regarding the content of a scientific product to be based only on scientific quality considerations." The Proposal, on the other hand, inhibits use of sound scientific information and evidence by arbitrarily excluding science for reasons unrelated to its quality. While the policy "[r]ecognizes the value of independent validation of scientific methods" and facilitating "the free flow of scientific information" by making information available "including access to data and non-proprietary models underlying Agency policy decisions," this is proposed as a flexible standard and an ideal to aspire to, not an absolute rule that takes priority over other competing interests -- such as use of the best scientific information. As discussed more in Section VII.C [of the commenter's letter], this Administration has blatantly violated key aspects of the policy by silencing scientists and the dissemination of scientific information, which this Proposal seems aimed at continuing, directly undoing "EPA's longstanding commitment to the timely and unfiltered dissemination of its scientific information  -  uncompromised by political or other interference" and goal to communicate scientific findings openly and actively to the public. By now placing "transparency" ahead of use of the best available science, aside from violating statutory requirements, EPA is changing its own policies and priorities and must justify this new position.
In footnote 2, EPA dubiously claims the Proposal is consistent with the Memorandum for the Heads of Executive Department and Agencies on Scientific Integrity (Mar. 9, 2009). Notably, the Memorandum specifies, "Except for information that is properly restricted from disclosure under procedures established in accordance with statute, regulation, Executive Order, or Presidential Memorandum, each agency should make available to the public the scientific or technological findings or conclusions considered or relied on in policy decisions." Not only does the Memorandum provide no support for the notion that agencies should be barred from relying on studies where the underlying data is properly restricted from disclosure it additionally discusses disclosure only of findings and conclusions, not underlying data.
Thus, despite EPA's claims to the contrary, the Proposal marks a shift in policy that EPA has up to this point followed EPA arbitrarily fails to acknowledge this shift, to identify good reasons for the change, or to explain why EPA believes the proposed rule would be an improvement over current mechanisms utilized by EPA to ensure the integrity of EPA's actions.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency. EPA notes that this final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [8750-1682]: Commenter (9227) states that an agency rule is arbitrary and capricious if it "entirely failed to consider an important aspect of the problem." EPA's Proposal completely fails to consider the numerous barriers that currently exist to making underlying data public. As highlighted in OMB and EPA policies, there is an understanding that the worthy goal of ensuring greater transparency of scientific information is in tension with other compelling, competing interests such as privacy and confidentiality. When these two are in tension, existing policies have recognized that this will prevent certain data from being publicly released -- and that agencies still need to be able to use scientific information in these circumstances. Transparency goals should not override the ability of the agency to rely on otherwise valid scientific information as it goes about achieving its core mission. While the Proposal purports to take into account privacy and confidentiality concerns, it appears to do so by either grossly oversimplifying EPA's ability to address these concerns or by deeming all such information unusable -- essentially completely failing to consider the problems of this approach.
OMB Circular A-130 recognizes that the values of openness, transparency, and allowing the free flow of information between the federal government and the public are important values, they must be contextualized. Thus, it cautions: "Promoting openness and interoperability, subject to applicable legal and policy requirements, increases operational efficiencies, reduces costs, improves services, supports mission needs, and increases public access to valuable Federal information." Similarly it states: "The open and efficient exchange of scientific and technical Federal information, subject to applicable security and privacy controls and the proprietary rights of others, fosters excellence in scientific research and effective use of Federal research and development resources." Circular A-130 makes clear that "[p]rotecting an individual's privacy is of utmost importance. The Federal Government shall consider and protect an individual's privacy throughout the information life cycle." It requires that agencies recognize that "Federal information is managed by making information accessible, discoverable, and usable by the public to the extent permitted by law and subject to privacy, security (which includes confidentiality), or other valid restrictions pertaining to access, use, dissemination, and disclosure. . . ."
Further, Circular A-130 requires agencies to "[l]imit the creation, collection, use, processing, storage, maintenance, dissemination, and disclosure of [personally identifiable information] to that which is legally authorized, relevant, and reasonably deemed necessary for the proper performance of agency functions" and "[t]o the extent reasonably practicable. . .reduce all [personally identifiable information] to the minimum necessary for the proper performance of authorized agency functions."
The appendix to the Circular realizes that privacy protections require ongoing progress and:
Emerging technologies and services may continue to shift the ways in which agencies acquire, develop, manage, and use information and technology. As technologies and services continue to change, so will the threat environment. Agency programs must have the capability to identify, respond to, and recover from current threats while protecting their information resources and the privacy of the individuals whose information they maintain.
OMB Memorandum M-14-06 specifically lays out policies intended to help agencies make the most of "administrative data that cannot be made publicly available due to statutory, regulatory, or policy protections," for statistical purposes, including "activities typically characterized as research, evaluation, and analysis, as long as the focus of those activities is on reporting aggregate findings about a group." It notes "[s]ome administrative data can be publicly released, whereas other administrative data cannot be released. . . [and] it is the case that both types of administrative data (public and nonpublic) can be useful for Federal statistical purposes," suggesting agencies should not abandon reliance on data not able to be publicly released.
OMB Memorandum M-11-02 "strongly encourages Federal agencies to engage in coordinated efforts to share high-value data" but notes that in certain cases sharing data will contravene other compelling concerns and that federal agencies need to think about applicable privacy laws, regulations, and policies to "fully protect[] individual privacy" and preserve public trust. Unlike the Proposal, it takes a more nuanced approach recognizing that sharing data is not always appropriate and should only be done "responsibly and appropriately."
OMB recognizes that even when just sharing information among agencies, privacy concerns must be weighed against those benefits that agencies can achieve with sharing data: "Agencies should work together to determine what data sharing opportunities are desirable, feasible, and appropriate. In general, data sharing should only be pursued if the benefits outweigh the costs."
OMB Memorandum M-10-06 also encourages "a plan for timely publication of the underlying data. . . in an open format and as granular as possible, consistent with statutory responsibilities and subject to valid privacy, confidentiality, security, or other restrictions." The memorandum aims to achieve "transparency, participation, and collaboration," recognizing that not making data available does not deter those goals when there are valid concerns and the legitimacy of the data is not otherwise questioned.
EPA's Draft Strategic Data Action Plan Version 1.0 similarly aims to work towards a more open government, and to increase the public's access to high quality data. However, the agency recognizes barriers to this goal, not applying the plan to "data resources containing CBI or sensitive data that are not available for public access." In similarly recognizes that "[i]n order to protect the privacy and security of the public, businesses, and US Government staff and operations, some types of data may be deemed sensitive and will not be made public or published on Data.gov."
The commenter states that these examples all highlight instances where EPA and OMB have recognized that privacy and confidentiality present ongoing concerns that are not easily addressed and that conflict with other aims of federal government. Yet, they recognize that protecting information in these cases is a valid path, and not making data public does not compromise the validity of the findings or conclusions upon which the data is based and should prevent agencies from using those findings, conclusions, and data to inform their work. According to the commenter, the Proposal provides no explanation for why EPA is now changing its view to a conflicting one, making the Proposal arbitrary.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency.
Comment [8750-1697]: Commenter (9227) states that the proposed rule asserts that a broad interest of the current Administration is to "ensure that the data and models underlying scientific studies that are pivotal to. . . regulatory action are available to the public" and to "change agency culture and practices regarding data access so that the scientific justification for regulatory actions is truly available for validation and analysis." However, the Proposal specifies a particular interest and initial focus on "dose response data and models" as evident throughout the preamble and proposed regulatory provisions.
Dose-response studies are a critical element of risk assessments for toxicants including air pollutants. Assessment of a toxicants risks typically proceeds through a four-step process: 1) hazard identification, 2) dose-response assessment, 3) exposure assessment, and 4) risk characterization. Dose-response assessment describes the relationship between exposure to a toxicant and observed effect on human or ecological receptor. EPA provides the following description of dose-response on its website: "Dose-Response Assessment...characterizes the quantitative relationship between chemical exposure and each credible health hazard. These quantitative relationships are then used to derive toxicity values." Dose-response plays a central role in the evaluation of chemical risks as it provides the characterization of the potency or effect size of the toxicant. In other words, dose-response assessment is used to determine the levels of exposure at which adverse effects will occur and thus informs what risk management actions should be taken to protect human and ecological health. Dose-response assessments are commonly used to derive chemical toxicity values. The lower a substance's toxicity value the greater its potency and the less exposure is necessary for an effect to occur.
EPA reveals the underlying motivation behind its interest in transparency of dose response data and models on page eight of the Proposal, where it states:
      In addition, this proposed regulation is designed to increase transparency of the assumptions underlying dose response models. As a case in point, there is growing empirical evidence of non-linearity in the concentration-response function for specific pollutants and health effects. The use of default models, without consideration of alternatives or model uncertainty, can obscure the scientific justification for EPA actions. To be even more transparent about these complex relationships, EPA should give appropriate consideration to high quality studies that explore: A broad class of parametric concentration-response models with a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the exposure range; and spatial heterogeneity. EPA should also incorporate the concept of model uncertainty when needed as a default to optimize low dose risk estimation based on major competing models, including linear, threshold, and U-shaped, J-shaped, and bell-shaped models.
This excerpt raises several troubling and erroneous concepts that are contrary to core scientific tenets and best practices in chemical hazard and risk assessment as discussed extensively in a seminal 2009 report by the National Academies (Academies): Science and Decisions: Advancing Risk Assessment (Science and Decisions). The report was requested and sponsored by EPA's National Center for Environmental Assessment and was developed over a three-year period by a 15-member committee that included state environmental agencies, non-governmental organizations, industry, and academic institutions. The committee was specifically tasked with "developing scientific and technical recommendations for improving risk analysis approaches used by EPA, including providing practical improvements that EPA could make in the near term (2-5 years) and in the longer term (10-20 years)." The report has been cited over 400 times in the scientific literature.
The Proposal fails to discuss these best practices for risk assessment, much less provide any persuasive reason for departing from them. The Proposal provides no support for its assertion that there is "growing empirical evidence" of nonlinearity in dose-response relationships; fails to acknowledge or contend with the National Academies' finding that nonthreshold dose-response relationships are common for toxicants, and should be assumed as a default; fails to discuss the well-known rationales put forward by the National Academies for using default models; and irrationally prioritizes consideration of studies that employ a wide range of dose-response models, without any consideration for whether those alternative dose-response models are appropriate for risk assessment. Alarmingly, the Proposal offers no analysis of how the proposed requirements to consider threshold-response relationships and avoid default models would further the protection of human health and the environment -- and gives no indication that the Agency has considered whether its proposed approach affords appropriate protection for the public in evaluating the risks of dangerous pollutants and toxicants. The proposed requirement is irretrievably arbitrary and unjustified and must be withdrawn.
Response: EPA notes that the final rule only applies to dose-response data underlying pivotal science used in significant regulatory actions.
Comment [8750-1761]: Commenter (9227) states that it is arbitrary and capricious to "`entirely fai[l] to consider an important aspect of the problem' when deciding whether regulation is appropriate." Michigan v. EPA, 135 S. Ct. 2699, 2707 (2015) (quoting State Farm, 463 U.S. at 43). As in Michigan, failure to consider the costs and benefits of a regulation where there is no statutory bar to doing so is arbitrary and capricious.
The proposed rule entirely fails to comply with the requirements of non-arbitrary-and capricious rulemaking because it fails to disclose, much less analyze or consider, any of the costs of the rule; barely discusses and does not analyze or quantify the benefits; does not provide any reasoned explanation of why the benefits of the rule justify its costs; and does not consider potential alternatives. The Proposal's discussion of costs and benefits is a scant two paragraphs (and was apparently not included at all in the version sent to the Office of Management and Budget). The proposed rule begins by conclusory asserting that "EPA believes the benefits of this proposed rule justify the costs." It then briefly discusses the perceived benefits, incorrectly suggesting that the NAS shares EPA's view by citing to a publication that discusses both risks and opportunities of expanding access to research data, and does not discuss at all the costs and benefits of ignoring relevant science in regulatory decision-making. It then merely states that the "action should be implemented in a cost-effective manner," citing vaguely to "recent activities of the scientific community and other federal agencies" without any concrete examples or analysis. The preamble's discussion emphasizes that the Proposal does not compel EPA to make information available where it concludes that doing so is not possible, but omits that if compliance is not possible, EPA will not consider the study, which has its own costs. It then concludes by citing the working paper of the Mercatus Center that baldly asserts that improvements in reproducibility "can be thought of as increasing the net benefits of regulation because they would avoid situations in which costs or benefits are wrongly estimated to occur or in which regulatory costs are imposed without corresponding benefits." Setting aside the lack of substantiation for this assertion, it entirely omits situations in which costs and benefits are wrongly estimated because the relevant science is not used -- and the costs that would be imposed on society if EPA inadequately protects communities from harmful pollution or toxic exposures.
Indeed, the Proposal nowhere discusses its significant costs in either quantitative or qualitative terms, costs that have actually been examined by independent organizations, and that are susceptible to analysis. If the Proposal is truly "designed to provide a mechanism to increase access to" data "in a manner consistent with statutory requirements for protection of privacy and confidentiality of research participants," 83 Fed. Reg. at 18,770, then it will have significant costs. And if, as it appears, the Proposal's true "mechanism" is excluding science from regulatory decision-making, its costs will be even greater in the form of insufficiently protective regulations.
If it were not possible to quantify and monetize any of the costs, which is not the case here as discussed below, EPA would still be required under Executive Order 12866 and the requirements of rational rulemaking to identify and discuss the qualitative costs of this Proposal. It is inherently irrational for an agency to take an action without any consideration of any costs, disadvantages, or negative effects of that action. The qualitative costs of this Proposal include the costs to researchers of actions they must undertake to protect the confidentiality of patient and subject data, as well as to compile and make public their raw data, and the potential loss of subjects (and attendant damage to research efforts and results) due to confidentiality concerns. There are also various costs to the agency of administering the regulation, which include contacting researchers, gathering data, ensuring that patient confidentiality and confidential business information are not disclosed. Additional costs could also be incurred through conducting any additional peer reviews required by proposed section 30.7 and any additional analyses imposed by proposed section 30.6's requirement that "EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions." Most importantly, there are potentially huge costs of regulating without using the relevant science merely because the underlying raw data is not publicly available. If studies supporting a stronger standard are excluded and EPA can therefore only justify a weaker requirement that leaves large numbers of people at risk of health effects from a pollutant, pesticide, or chemical, then this Proposal could impose enormous costs for each insufficiently protective regulation. Yet the Proposal fails even to mention these costs, let alone discuss their scope and significance.
In addition, many of these costs can be quantified and monetized, but EPA has neither attempted to do so nor explained why it could not. For example, EPA has extensive information available to it on what the agency would need to do to implement this Proposal and how much those activities would cost. In fact, EPA already gathered much of this data and provided it to the CBO for use in estimating the costs of a similar (though not identical) proposal from Congress, the HONEST Act. With respect to the Congressional proposal, CBO concluded, just with respect to the costs to EPA, that "based on information from the EPA and other federal agencies, as well as organizations and researchers in the scientific community that publish in peer-reviewed journals," EPA "could spend between a few million dollars per year to more than one hundred million dollars per year ... to ensure that data and other information underlying studies are publicly available in a format sufficient to allow others to substantially reproduce the results of studies." In the 2017 estimate, CBO concluded that "[i]f the EPA continued to rely on as many scientific studies as it has used in recent years ... then CBO estimates that the agency would need to spend at least $100 million dollars per year to upgrade the format and availability of those studies' data," "on average, $10,000 per scientific study." Such costs would cover the costs of "obtaining all the underlying data used in a study, reviewing the data to address any confidentiality concerns, formatting the data for public access, providing access to the computer codes and models used in the study's analysis, and providing descriptions and documentation on how to access the data." Notably, this does not include the cost to researchers to engage in this effort. As Deputy Assistant Administrator Nancy Beck noted, during the development of the Proposal, requiring "a huge amount of data to be submitted to the agency" would "be incredibly burdensome" and "not practical."
Even the Mercatus working paper -- apparently the only thing EPA relied upon in discussing the costs and benefits of the Proposal, 83 Fed. Reg. 18,772 n. 24, notes, with respect to the HONEST Act, that "[t]he cost of providing access to data has been one of the primary concerns about requiring access to data used by the federal government." Far from concluding, as the Proposal suggests, an increase in net benefits from greater reproducibility, the Mercatus working paper simply explained a figure the authors were suggesting could be calculated (the point where net benefits would be positive); the authors do not themselves calculate the benefits, and admit that their "estimates of the benefits of public access to data supporting federal regulatory decisions fall short of proving that the benefits outweigh the associated costs." And while the Mercatus working paper disagrees with CBO's cost estimates, it does not argue that that requiring access to data is cost-less; indeed, it discusses the "costly activities and services that need to be performed," including activities related to "data collection and data accessibility." According to that working paper, data collection requires "correspond[ing] with researchers and publishers to obtain the data, review[ing] the data for confidentiality concerns, format[ting] the data for public access, publicly post[ing] the computer code and models used in each study's analysis, and provid[ing] descriptions and documentation on how to obtain the date." Data accessibility requires "computer processing services to construct and maintain data bases to store study-related information." While the actual calculations put forward by the Mercatus working paper appear faulty (for example, it entirely omits the cost to researchers to compile and make their data public, does not include the costs of ensuring patient privacy is protected, and makes assumptions about the similarity of a chemical manufacturer collecting its own studies and EPA collecting and disseminating information of other researchers), the working paper at least acknowledges that there are costs, something EPA's Proposal completely ignores.
Nor does the proposed rule disclose the cost -- highlighted on the very first page of a NAS report on data access -- that "perceived risks to privacy and confidentiality reduce survey participation," a cost that the NAS explains is "borne out by research." NAS explains that this "threatens the research enterprise itself, because concerns about privacy and confidentiality are among the reasons often given by potential respondents for refusing to participate in surveys, and those concerns have been shown to affect behavior as well. The NAS panel emphasized: "Any confidentiality breach that became known would be likely to heighten such concerns and, correspondingly, reduce survey response rates. Efforts to increase researchers' access to data must, therefore, take into account the need to avoid increasing the actual and perceived risks of confidentiality breaches." The Proposal does not so much as discuss this potential cost.
This confidentiality risk has a further cost: it affects the quality of the data collected. As the NAS explained:
The reason for confidentiality pledges and for stringent procedures to prevent disclosure is that they improve the quality of data collected from individuals, households, and firms. It is essential that respondents believe they can provide accurate, complete information without any fear that the information will be disclosed inappropriately. Indeed, if the information was disclosed, harm might come to an individual respondent.
The Proposal's only acknowledgment of this complex problem and cost is its statement that "EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government." Remarkably, EPA does not cite a single example of these common solutions, citing only vaguely to "examples from the U.S. Department of Health and Human Services, National Institute of Standards and Technology, U.S. Department of Education, and the U.S. Census Bureau" and some hyperlinks not in the Proposal added to the docket almost a month into the comment period. Accordingly, not only does the Proposal include no analysis of these alleged solutions and their costs and benefits, it does not even explain what the solutions are that EPA believes address this concern.
And if EPA complies with the regulation not by spending the money to make data publicly available, and if the research community does not bear those costs itself, see 83 Fed. Reg. at 18,770-71 ("Nothing in the proposed rule compels the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections."), then it appears that EPA would simply ignore studies that do not comply with the regulation. See 83 Fed. Reg. at 18,769 n. 3 ("EPA is proposing to exercise its discretionary authority to establish a policy that would preclude it from using such data in future regulatory actions."). That course of action has its own significant costs, and EPA provides no analysis in the Proposal of the magnitude of studies that it has previously relied upon that it could no longer rely upon in regulating. See 2017 CBO Estimate ("EPA officials have explained to CBO that the agency would implement H.R. 1430 with minimal funding and generally would not disseminate information for the scientific studies that it uses to support covered actions. That approach to implementing the legislation would significantly reduce the number of studies that the agency relies on when issuing or proposing covered actions...."). As the SAB noted in its May 12, 2018 letter, "[t]he proposed rule does not include any assessment of the impact of data restrictions on existing or future regulatory programs. Without access to the restricted data, regulatory programs could become more or less stringent than they otherwise would be, with consequences for both regulatory costs and benefits."
Likewise, EPA has included only a cursory mention of the expected qualitative benefits of the Proposal, with no discussion of the anticipated likelihood, scope, or impact of the suggested benefits, let alone any effort to quantify them, much less monetize them. EPA simply assumes that the Proposal will "improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing an exploration of key data sets" without any analysis or evidence. In fact, as we have explained, the likely outcome of the Proposal is that it will degrade the data and scientific quality of the Agency's actions by ignoring relevant science simply because the underlying data is not publicly available. Moreover, EPA's finding is not consistent with the conclusions of the National Academies, as the Proposal suggests. As also explained above, the NAS report highlighted both the risks and benefits of making data publicly available and nowhere concluded that there were benefits to excluding data from the agency's regulatory decisions simply because the underlying data was not publicly available. Nor does the agency analyze how likely its Proposal is to actually facilitate expanded data sharing, and its main aim appears to be excluding science as it does not actually provide any funding, mechanisms, or best practices for sharing data.
It is more than ironic that EPA claims -- without any data or analysis -- that its Proposal will increase the net benefits of other regulations while it does nothing to actually consider the costs and benefits of the Proposal itself. Moreover, there is no reason to think that excluding relevant science merely because the underlying data is not publicly available would increase the net benefits of a regulation. For example, it appears that under the proposed rule EPA would exclude a peer-reviewed, published study whose conclusion had been reproduced based upon numerous different datasets (and whose underlying data, though not publicly available, had been reevaluated by outside experts), while including a study that had had no peer review, was not published, had no corroborating studies, and had not actually been replicated or reproduced, merely because the underlying data was made publicly available. That is simply not a recipe for more accurate decision-making.
The proposed rule also violates the APA and other statutes' requirements for reasoned decision-making by failing to consider any alternative approaches, much less their costs, here. This is particularly irrational in this context where it appears that many of the benefits sought by EPA could be largely achieved with much less burdensome and costly approaches. A critical element of reasoned decision-making is consideration of alternatives which are congruent with agencies' statutory responsibilities and objectives. Motor Vehicle Mfrs. Ass'n v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 48, 50 (1983) (safety agency acted arbitrarily in failing to consider alternative safety measures after rejecting passive restraints). EPA failed to consider other methods to ensure scientific robustness at the agency. For example, the SAB letter notes that "[t]he proposed rule fails to mention that there are various ways to assess the validity of prior epidemiologic studies without public access to data and analytic methods." The Proposal does not consider any alternatives to ensuring that studies are reliable even where the underlying data cannot be made public because of privacy or other concerns.
Furthermore, by failing to consider costs and benefits, the Proposal contravenes Executive Order 12866. Executive Order 12866 requires agencies to assess the costs and benefits of proposed regulations and propose or adopt a regulation only upon a reasoned determination that the benefits justify the costs. For "significant regulatory actions," like the proposed rule, 83 Fed. Reg. at 18,772, the agency must provide:
      An assessment, including the underlying analysis, of benefits anticipated from the regulatory action (such as, but not limited to, the promotion of the efficient functioning of the economy and private markets, the enhancement of health and safety, the protection of the natural environment, and the elimination or reduction of discrimination or bias) together with, to the extent feasible, a quantification of those benefits;
      An assessment, including the underlying analysis, of costs anticipated from the regulatory action (such as, but not limited to, the direct cost both to the government in administering the regulation and to businesses and others in complying with the regulation, and any adverse effects on the efficient functioning of the economy, private markets (including productivity, employment, and competitiveness), health, safety, and the natural environment), together with, to the extent feasible, a quantification of those costs; and
      An assessment, including the underlying analysis, of costs and benefits of potentially effective and reasonably feasible alternatives to the planned regulation, identified by the agencies or the public (including improving the current regulation and reasonably viable nonregulatory actions), and an explanation why the planned regulatory action is preferable to the identified potential alternatives.
The agency must also make these assessments and analyses "available to the public." Executive Order 13563 reaffirms these principles and requirements, explaining that agencies "must take into account benefits and costs, both quantitative and qualitative."
Agencies are further encouraged to weigh the costs and benefits of developing higher information quality in OMB's Information Quality Guidelines. Costs that the Guidelines encourage agencies to consider include "costs attributable to agency processing effort, respondent burden, maintenance of needed privacy, and assurances of suitable confidentiality." EPA's existing information quality guidelines track the OMB Guidelines closely. EPA's disregard of the Guidelines' recommended weighing costs and benefits further contributes to the arbitrariness of EPA's failure to consider the costs of the Proposal.
The Proposal's failure to analyze and disclose costs and benefits cannot be cured in a final regulation. Should EPA not abandon this misguided Proposal, it must re-propose it after first analyzing its costs (both to public health, to researchers, and to the agency itself) and benefits, and providing the requisite opportunity for public comment on its analysis. As discussed further below in Section VIII.D [of their comment letter], the public cannot meaningfully comment on the proposed rule without understanding the actual costs and benefits of the Proposal, the alternatives EPA considered, and the analyses underlying EPA's assessments.
Response: See response to Comment 6112-4452. Any further cost-benefit analyses associated with implementing this procedural rule will be conducted as appropriate in forthcoming rulemakings.
Comment [8750-1824]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that Exec. Order No. 13563, 76 Fed. Reg. 3821 (Jan. 21, 2011) provides: "Our regulatory system must protect public health, welfare, safety, and our environment while promoting economic growth, innovation, competitiveness, and job creation. It must be based on the best available science."
Executive Order No. 13563 requires agencies to utilize the "best available science" in regulatory actions. This requirement is further encoded in numerous statutes and policies that EPA implements. EPA states in the proposed rule that: "The best available science must serve as the foundation of EPA's regulatory actions." However, as the comments raise more thoroughly, by arbitrarily restricting the scientific studies EPA will consider, this proposed rule will hinder EPA's use of the best available science and therefore violates the command of Exec. Order No. 13563 and other versions of these requirements.
Furthermore, this Executive Order requires agencies to "ensure the objectivity of any scientific and technological information and processes used to support the agency's regulatory actions" consistent with the President's Memorandum for the Heads of Executive Departments and Agencies, ``Scientific Integrity'' (March 9, 2009). As their comments note, however, the proposed rule along with the provision allowing the Administrator to grant discretionary exemptions will harm the objectivity of scientific and technological information and processes at EPA by paving the way for politics, rather than objective scientific criteria, to dictate which scientific studies are considered.
Response: See response to Comment 6163-1114. EPA issued this rulemaking consistent with its obligations under all applicable Executive Orders.
Comment [8750-1826]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter notes that the EPA claims about the proposal that "[b]y better informing the public, the Agency in[sic] enhancing the public's ability to understand and meaningfully participate in the regulatory process." EPA then cites to the Memorandum for the Heads of Executive Departments and Agencies on Scientific Integrity. Not only does the proposal conflict with this memorandum, but it will make it more difficult for the public to meaningfully participate in the regulatory process.
The memorandum sets out a number of actions for agencies to take to ensure scientific integrity. Just one of these factors involves making scientific and technological information publicly available, notably specifying, "Except for information that is properly restricted from disclosure under procedures established in accordance with statute, regulation, Executive Order, or Presidential Memorandum, each agency should make available to the public the scientific or technological findings or conclusions considered or relied on in policy decisions." The memorandum thus supports only making scientific findings and conclusions publicly available, not the data underlying those findings and conclusions. Further, it correctly notes that some information is properly restricted from disclosure. It does not say that the inability to disclose such information should prevent it from being considered by agencies. The memorandum thus provides no support for the notion that agencies should be barred from relying on studies where the underlying data cannot be disclosed. The memorandum's narrow approach to public disclosure should not be taken to support EPA's proposal but rather counsels against the proposal's mandate that all underlying data be made publicly available.
According to the commenter, the EPA's proposal fundamentally conflicts with the heart of the memorandum -- that "[t]he public must be able to trust the science and scientific process informing public policy decisions." To earn this trust, the memorandum declares: "Political officials should not suppress or alter scientific or technological findings and conclusions." By discarding scientific studies where underlying data cannot be made publicly available, this proposal will result in scientific findings being suppressed. By allowing the Administrator to grant exemptions to this policy based on their discretion with no public record or explanation, the proposal allows for the Administrator to pick and choose based on their preference the science informing the agency's actions, eroding the public's trust in the science informing public policy decisions.
The memorandum provides a number of ways in which agencies can ensure scientific integrity which the proposal does not consider including: hiring candidates for science and technology position based on their "knowledge, credentials, experience, and integrity," having in place appropriate rules and procedures to ensure integrity of the scientific process, establishing scientific processes such as peer review and accurately reflecting scientific and technological information, establishing procedures to identify when scientific integrity may be compromised, including establishing whistleblower protections. EPA does not explain why any of these pathways would not serve as a better means of ensuring scientific integrity.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency.
Comment [8750-1828]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that in footnote 3 of the proposal, EPA notes that "courts have at times upheld EPA's use [sic] non-public data in support of its regulatory actions" and cites to Coalition of Battery Recyclers Ass'n v. EPA, 604 F.3d 613, 623 (D.C. Cir. 2010) and American Trucking Ass'ns v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002). These cases indeed held that EPA's prior, longstanding position of relying on scientific studies even when the underlying data could not be made publicly available was reasonable. It is well-established that agencies must acknowledge changes in position and "show that there are good reasons for the new policy." This footnote, the only mention of EPA's previous policy, does not sufficiently acknowledge or explain why EPA is now changing its position.
In American Trucking Ass'ns v. EPA the Court held that the CAA did not require EPA to make public underlying data where EPA relied on the study itself and not the raw data underlying the study. The Court stated that such a requirement "would be impractical and unnecessary." They agreed with EPA's then statement that:
If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment.... Such data are often the property of scientific investigators and are often not readily available because of ... proprietary interests ... or because of [confidentiality] arrangements [with study participants].
In Coalition of Battery Recyclers Ass'n v. EPA, the Court cited American Trucking Ass'ns v. EPA and held, again, that EPA was permitted to rely on studies without making the underlying data public. They noted, "raw data often is unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants." These court cases thus not only upheld EPA's prior practice as permissible, but went on to agree that EPA's prior practice was preferable and necessary in light of these other policy concerns.
The commenter notes that the EPA provides no response to this history, saying only: "Historically, EPA has not consistently observed the policies underlying this proposal. . . . " EPA fails explicitly to recognize that this proposal changes its past policy and provides no justification in light of the compelling opposing points that both EPA and the Courts previously recognized as deterring this approach.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency. EPA notes that neither decision precludes EPA from considering data availability when developing significant regulatory actions or influential scientific information.
Comment [8750-1829]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA claims that the proposal is consistent with Executive Order No. 13777. This Executive Order provides no support for the proposal, and in fact is targeted at eliminating regulations including those that are "unnecessary" and "ineffective," which, as our comments detail, the proposal clearly would be.
This Executive Order creates a Regulatory Reform Task Force and calls for them to identify for repeal, replacement, or modification regulations that among other criteria are inconsistent with the requirements of section 515 of the Treasury and General Government Appropriations Act, 2001 (44 U.S.C. 3516 note), or the guidance issued pursuant to that provision, in particular those regulations that rely in whole or in part on data, information, or methods that are not publicly available or that are insufficiently transparent to meet the standard for reproducibility.
As described in detail in our comments and below, contrary to the inference drawn here in Exec. Order No. 13777, the Data Quality Act and OMB's guidelines issued pursuant to it do not require research data and models to be made publicly available for reproducibility purposes in order for agencies to rely on the scientific findings and conclusions produced using that data.
Executive orders cannot override the statutory requirements that EPA use the best available science or the laws governing administrative procedure including the APA. The proposal's "consistency" with this executive order then cannot serve as a legal basis for EPA to adopt an arbitrary and capricious policy that contravenes these best available science requirements reflected in the statutes EPA administers.
Additionally, Executive Order No. 13777 by its terms requires only the identification of regulations that rely in whole or in part on data not publicly available, it says nothing about precluding agencies from relying on such studies and does not and cannot require agencies to adopt such practices. However, if the proposed rule is to be "consistent" with the executive order then it must also follow section 3(e):
In performing the evaluation described in subsection (d) of this section, each Regulatory Reform Task Force shall seek input and other assistance, as permitted by law, from entities significantly affected by Federal regulations, including State, local, and tribal governments, small businesses, consumers, non-governmental organizations, and trade associations.
There is no evidence that EPA consulted with the many stakeholders impacted by this policy, including the medical or scientific research communities, which have been largely opposed to this policy.
Response: EPA issued this rulemaking consistent with its obligations under all applicable Executive Orders.
This procedural rule does not interpret or apply provisions of the statutes that it administers. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield.
In addition, EPA notes that government and private stakeholders have extensively engaged with this rulemaking. See response to Comment 2157-149.
Comment [8750-1834]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA claims the proposal is consistent with Executive Order No. 13783. However, Executive Order No. 13783 calls for agencies to consider salient information that the proposal has patently ignored. Executive Order No. 13783 calls for agencies to consider the costs and benefits "that are based on the best available science and economics" to ensure sound regulatory decision-making. The proposal provides no analysis of the costs and benefits of implementing this new policy, despite there likely being high costs to making research data public with little evidence of significant benefits achieved from this policy alone. Further, by arbitrarily excluding scientific information that EPA may use in its regulatory analyses, the proposal conflicts with the executive order's command to employ the best available science and economics.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency. See response to Comment 4829-2022 and 6112-4452. EPA issued this rulemaking consistent with its obligations under all applicable executive orders. Regarding cost-benefit analysis, see response to Comment 6112-4452.
Comment [8750-1836]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA wrongly claims that the proposal is "consistent with. . . the focus on transparency in OMB's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies." To say that OMB's Guidelines have a "focus on transparency" that is furthered by EPA's proposal is a gross oversimplification. EPA here appears to suggest that transparency is the highest objective to be achieved, divorced from any consideration of whether transparency hinders or furthers any other goals. The OMB Guidelines, while imposing high standards of quality, objectivity, utility, and integrity of information disseminated by Federal Agencies, recognize the need to implement controls "flexibly, and in a manner appropriate to the nature. . . of the information to be disseminated." They suggest thinking about transparency strategically to further the aims of good government, unlike the proposal, which conflates transparency and quality without consideration of other factors.
As part of ensuring "objectivity" of information these guidelines encourage agencies which disseminate influential scientific, financial, or statistical information, "to include a high degree of transparency about data and methods to facilitate the reproducibility of such information by qualified third parties." However, they emphasize the need to treat certain data differently, due to privacy and confidentiality concerns. While they recommend agencies "identify the sources of the disseminated information" they note that this is "to the extent possible, consistent with confidentiality protections." Importantly, they take great pains to urge agencies not to subject all data to a reproducibility requirement where this could hamper agencies. They require agencies, instead, to consult with "the relevant scientific and technical communities" to identify data that "can practicable [sic] be subjected to a reproducibility requirement, given ethical, feasibility, or confidentiality constraints." There is no indication that EPA consulted with the scientific and technical community, with EPA's own SAB raising concerns about the proposal and finding that "[t]his action merits further review by the SAB." The Guidelines make clear:
Making the data and methods publicly available will assist in determining whether analytic results are reproducible. However, the objectivity standard does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections.
In direct conflict with the reasoning underlying EPA's proposal, the Guidelines specifically provide that it is possible to verify the objectivity of information that cannot be made publicly available through other types of "robustness checks." As an example, they point to the Harvard Six Cities Study, where underlying data could not be made publicly available due to confidentiality concerns, but the raw data was released instead to researchers at the HEI, bound to the same confidentiality requirements as the original researchers, who were able to replicate its results. In contrast, EPA's proposal would not allow for the consideration of this study.
The guidelines also recommend agencies recognize that information quality comes at a cost, and that agencies should weigh the costs and benefits, which EPA has not done in the proposal.
Thus, the proposal completely turns away from OMB's guidelines where OMB "urges caution in the treatment of original and supporting data because it may often be impractical or even impermissible or unethical to apply the reproducibility standard to such data." As the comments discuss further, the proposal rule thus unlawfully conflicts with this flexible approach that prioritizes agencies' ability to use science as set out by OMB under the IQA.
Response: Transparency in addition to quality is important to EPA decision making. See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency.
Comment [8750-1837]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA claims the proposal is consistent with OMB's memorandum on Open Data Policy. This is incorrect, however, as the memorandum supports downstream information processing and dissemination -- not through complete public disclosure without regard to privacy or security -- but through instituting a framework of data collection, formatting, and storage that allows for public dissemination, if possible. Recognizing that not all data can be publicly disclosed, and that such data is still useful, the memorandum declares: "Whether or not particular information can be made public, agencies can apply this framework to all information resources to promote efficiency and produce value."
The proposal is thus inconsistent with the memorandum, which stresses the importance of information stewardship and "review of information for privacy, confidentiality, security, or other restrictions to release." When information cannot be released, the memorandum does not suggest agencies ignore the information or not rely on it for regulatory purposes. It focuses on prescribing agency practices to maximize the downstream usability of data that can be made publicly available, including through "using machine-readable and open formats, data standards, and common core and extensible metadata for all new information creation and collection efforts" as well as "building or modernizing information systems in a way that maximizes interoperability and information accessibility, maintains internal and external data asset inventories, enhances information safeguards, and clarifies information management responsibilities." Thus, while the memorandum centers on how agencies can marginally increase the utility of information they possess for use by the public, the proposal turns this on its head by advocating for discard of otherwise high quality scientific information if the data underlying such information cannot be made publicly available.
OMB stresses that to achieve "open data," agencies should adopt a presumption in favor of openness that is importantly limited by countervailing privacy, confidentiality, security, or other valid restrictions. Thus, agencies are expected to "exercise judgment before publicly distributing data residing in an existing system by weighing the value of openness against the cost of making those data public." The proposal does not at all weigh the costs, to the agency or to the public, of requiring all underlying data to be made publicly available.
While requiring agencies to adopt measures to strengthen privacy protections and data security, the memorandum recognizes serious limitation to data disclosure that EPA completely fails to consider. For example, the memorandum mandates that agencies take into consideration the "mosaic effect," which EPA does not at all acknowledge -- all while making superficial and unsupported statements about how privacy concerns can be easily addressed. The memorandum recognizes and stresses the challenge of responding to this threat, which requires undertaking a "risk-based analysis, often utilizing statistical methods whose parameters can change over time, depending on the nature of the information, the availability of other information, and the technology in place that could facilitate the process of identification." OMB importantly notes this analysis "may affect the amount, type, form, and detail of data released by agencies." According to the commenter, because it ignores these concerns, EPA's proposal is arbitrary and capricious.
Response: EPA notes that the final rule does not require EPA or any party outside of the agency to disclose protected information, nor does it require EPA to exclude studies based on data that cannot be disclosed due to its protected status. See response to Comment 6163-5030 and the final rule preamble at Section III.E.
 Comment [8750-1901]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA purports that the Proposal builds upon "the experience of other federal agencies in this space" but the citations reveal that is simply not the case. To support this statement, EPA provides only a hyperlink to a U.S. Census Bureau website along with vague references to entire executive branch agencies, with no explanation or discussion of which of their policies EPA believes the Proposal is building upon. Without a more specific citation, it is impossible to know which policies EPA is referencing or to respond to them meaningfully.
EPA cites to the U.S. Census Bureau's Federal Statistical Research Data Centers (FSRDC) as an example of use of secure facilities that allow the Census Bureau to provide controlled access to authorized researchers to use restricted-use microdata for statistical purposes only. In order to gain access, researchers must obtain Census Bureau Special Sworn Status by passing a moderate risk background check and swearing to protect respondent confidentiality for life. While this "solution" meets the U.S. Census Bureau's needs by allowing access to confidential information only to researchers whose proposals meet certain criteria, who go through a vetting process, and who agree to protect the information, this is done at a cost -- which EPA has not accounted for --  and would not satisfy EPA's requirement to make data and models "publicly available." Thus, according to the commenter, this example provides no support for the Proposal.
Response: See response to Comment 8750-1837.
Comment [8750-1905]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 10 of the proposal:
      These include policies and recommendations from: the Administrative Conference of the United States' Science in the Administrative Process Project; National Academies' reports on Improving Access to and Confidentiality of Research Data, Expanding Access to Research Data, and Access to Research Data in the 21st Century; the Health Effects Institute; Center for Open Science; members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology; and the Bipartisan Policy Center's Science for Policy Project.
The commenter notes that in footnote 10 of the Proposal, EPA lists a number of organizations whose recommendations and policies the Proposal allegedly took into consideration. In fact, since the Proposal was published, many of these organizations have issued statements opposing the Proposal and contesting EPA's claim that their policies and recommendations endorse the Proposal. In this footnote, EPA provided no hyperlinks or specific citations for which recommendations and policies it was referencing, making it impossible to understand why EPA believed these organizations supported the Proposal or to respond to them. The commenter provides the following specific comments:
I. The Administrative Conference of the United States' Science in the Administrative Process Project
The EPA cites to the Administrative Conference of the United States' Science in the Administrative Process Project -- Recommendation 2013-3: Science in the Administrative Process. Wendy Wagner, sole author of ACUS's final report Science in Regulation: A Study of Agency Decision-Making Approaches and who served on the panel that produced the recommendations strongly opposed the notion that the Proposal builds upon these recommendations, saying: "They don't adopt any of our recommendations, and they go in a direction that's completely opposite, completely different. . . . They don't adopt any of the recommendations of any of the sources they cite. I'm not sure why they cited them."