Response to " Comments of American Chemistry Council, American Forest & Paper Association, American Iron and Steel Institute, American Petroleum Institute, American Public Power Association, and Utility Water Act Group on EPA's Proposed Information Collection Request on Willingness to Pay Survey for § 316( b) Phase III Cooling Water Intake Structures"

DOCKET ID OW 2005 0006 0007 EPA's responses to the " Comments of American Chemistry Council, American Forest & Paper Association, American Iron and Steel Institute, American Petroleum Institute, American Public Power Association, and Utility Water Act Group on EPA's Proposed Information Collection Request on Willingness to Pay Survey for § 316( b) Phase III Cooling Water Intake Structures," the attached comment " Comments on EPA 316( b) ` Willingness to Pay' Survey Materials" from Lawrence Barnthouse, and the attached comment " Evaluation of the EPA's 316( b) Phase III Rule Willingness to Pay Survey: Final Report" from Triangle Economic Research ( TER) are organized to correspond to the individual ordered sections of the comments. These comments are henceforth referred to as the " ACC et al. Comment," " Barnthouse Comment," and " TER Comment," respectively.

Responses to the ACC et al. Comment Introduction and Overview The ACC et al. Comment is summary in nature and draws heavily on the Barnthouse Comment and the TER Comment. In this spirit, EPA offers the following general responses that address the main issues presented in the list of points beginning on page 2 of the comment. Detailed responses to all comments can be found in the Agency's response to the Barnthouse Comment and TER Comment.

The commenter argues that there are several reasons why EPA will not be able to develop reliable estimates of the non use benefits of the 316( b) regulation for Phase III facilities:

" EPA's view of non use values as presented in the November 23, 2004 ICR is conceptually flawed." EPA disagrees with this statement, which does not reflect the wide conceptual acceptance of non use values in the academic literature. As stated by Freeman, " Non use values, like use values, have their basis in the theory of individual preferences and the measurement of welfare changes. According to theory, use and non use values are additive" ( Freeman, 2003).

" The proposed ICR contains numerous flaws that will result in unreliable benefit estimates." Again, EPA disagrees with the commenter. As discussed in the Agency's detailed responses, below, EPA has extensively tested the survey using focus groups and cognitive interviews. Results indicate that the survey is effective at eliciting respondents' opinions about reducing fish losses, and that the survey will produce valid, unbiased, and reliable willingness to pay estimates.

" EPA based its survey on unreliable empirical estimates of the effect of cooling water intake structure regulations on fish populations and fisheries production." In response, EPA first notes that the survey materials are based on the best biological and engineering data available. Second, the stated preference survey will still produce results that are meaningful in the context of the scenarios that are presented to respondents.

The commenter argues that the benefits of regulating cooling water intake structures for recreationally and commercially significant species are already reflected in the EPA's RIA for the proposed rule for Phase III facilities. EPA disagrees with the commenter's statement. As stated by Freeman, " Non use values, like use values, have their basis in the theory of individual preferences and the measurement of welfare changes. According to theory, use and non use values are additive" ( Freeman, 2003). Moreover, as stated by Freeman ( 2003, p. 137), "[ t] he hypothesis of nonuse values has gained wide acceptance among economists working in the field of environmental and resource economics " and the suppression of such values " in natural resource policymaking could lead to serious errors and resource misallocations" ( p. 138). The potential significance of nonuse values as a distinct component of total value  and the requirement that stated preference methodologies are used to measure these values  are also emphasized by the NOAA Blue Ribbon Panel on Contingent Valuation ( Arrow et al. 1993) and EPA's guidelines for benefit cost analysis ( US EPA 2000). The Agency notes that the non use benefits of the proposed regulation are a completely separate category of value from the commercial and recreational welfare effects of the proposed rule. Furthermore, results from the focus groups and cognitive interviews indicated that some individuals hold substantial non use values for protecting fish species with direct uses, in addition to any use values that they have for those species.

The commenter argues that the presentation materials prepared by EPA contain inaccurate statements and invalid comparisons that will lead respondents to believe that the benefits of increased regulation of cooling water intakes would be substantially greater than is actually the case. EPA agrees that certain details of the survey and presentation materials provided in the supporting documentation for the June 9 Federal Register notice required revision. Since issuing the June 9 Federal Register notice, EPA has conducted five focus groups and two cognitive interview sessions to improve the Agency's understanding of the public's perceptions and attitudes concerning fishery resources, to pretest draft survey questions and presentation materials, to test for and reduce potential biases that may be associated with stated preference methodology, and to ensure that both researchers and respondents have similar interpretations of survey language and scenarios. As a result of this extensive pre testing, a number of revisions have been made to the survey that have significantly improved its reliability and reduced its potential for bias. Results from pretests indicate that the presentation materials prepared by EPA do not lead respondents to believe that the benefits of increased regulation of cooling water intakes would be substantially greater than is actually the case.

The commenter argues that because the survey materials are based on unreliable population and fish losses data, the survey results will also be unreliable. In response, EPA first notes that the survey materials are based on the best biological and engineering data available. Second, the stated preference survey will still produce results that are meaningful in the context of the scenarios that are presented to respondents. Different versions of the survey will show a range of different baseline and resource improvement levels, where these levels are chosen to ( almost certainly) bound actual levels. Different respondents will be asked to make choices over all possible policy scenarios where impingement and entrainment reductions range from 0% ( no policy) to 98%. Given that there will almost certainly be some biological uncertainty regarding the specifics of the actual baselines and improvements, the resulting valuation estimates will allow flexibility in estimating WTP for a wide range of different circumstances.

The commenter also argues that the survey is unnecessary because in independently conducted verbal protocol interviews with 15 individuals, respondents did not demonstrate meaningful values for marginal changes in forage fish populations. EPA notes that this finding contradicts the results of the EPA focus groups and cognitive interviews conducted under ICR # 2155.01. These focus groups and interviews provided strong evidence that many individuals value forage fish and are willing to pay to prevent losses of all fish species including forage fish. Participants in the EPA focus groups and interviews cited a variety of motivations for preventing fish losses, including the satisfaction of knowing that the fish exist, the desire to bequeath healthy fish populations to future generations, and the desire to protect the functioning of aquatic ecosystems. Furthermore, EPA notes that the economic literature advises against the use of ad hoc, expert assumptions regarding the magnitude of non use values  particularly assumptions that non use values are trivial. Because values are inherently subjective, Bateman et al. ( 2002, p. 75) emphasize that " it would be wrong for experts to assume that one resource is a perfectly good substitute for another" [ and hence that non use values are trivial or small]. As further noted by Bateman et al. ( 2002, p. 75), " there are, therefore, no easy rules for determining at the outset" whether non use values are likely to be significant or non significant. Based on this clear guidance, EPA believes that empirical analysis should be used to determine the magnitude of non use values in the 316( b) regulation for Phase III facilities. Finally, the Agency notes that TER did not provide complete transcripts for the 15 verbal protocol interviews described in their comments, preventing the Agency from obtaining a complete perspective on the result of this effort.

The commenter argues that the survey does not sufficiently emphasize the uncertainty in the estimates of the fish losses that would be prevented by the proposed policies. EPA points out that debriefing sessions during focus groups and cognitive interviews showed that respondents clearly understood that the ecological changes described in the survey were uncertain. Furthermore, respondents were comfortable making decisions in the presence of this uncertainty. Additionally, EPA has modified the survey and slide show to emphasize the uncertainty of the expected environmental changes. For example, the following statement has been added to the survey: " You will be shown different policy options  with different effects on fish. This is because scientists are still working to determine what the exact effect on fish will be, so it is important to know how you would react to a wide range of possible outcomes. Common sense indicates that preventing the loss of fish eggs and young fish will mean more adult fish in future years, but at this point there is still significant uncertainty regarding the exact size of these future effects." EPA also added debriefing questions to the survey instrument that are designed to identify respondents whose responses are based on incorrect interpretation of the environmental changes described in the survey, including the uncertainty of the expected changes. 1 EPA also notes that tests of survey question formats suggested by TER  incorporating ranges of impacts in survey questions  performed poorly in both TER's own focus group and in EPA's pretests.

1 For detail, see Questions 2, 3, 8, 9, 10, and 11 in the survey instrument provided in Attachment 3, U. S. EPA ( 2005). Finally, the commenter expresses the opinion that the survey should be peer reviewed by an independent panel of experts. EPA agrees with the commenter's opinion, and the Agency plans to convene two peerreview panels. The first panel will review the results of the focus groups, the instrument and the planned survey sampling design, and the proposed willingness to pay estimation methodology, before the survey is fielded. The second peer review panel will review the entire survey process, including EPA's final estimated results for the 316( b) Phase III rulemaking, after the survey is completed.

Responses to the Barnthouse Comment Introduction The introduction to the Barnthouse Comment states that the presentation materials for the 316( b) survey ( 1) overstate both current impacts of impingement and entrainment ( I& E) losses and the biological benefits of reducing those losses, and ( 2) understate the uncertainty inherent in EPA's estimates of the magnitude of those losses. EPA disagrees with these claims. In response to the first point, EPA emphasizes that the estimates of impingement and entrainment losses used in the survey are scientific estimates specific to the policy options valued in this study. The policies considered by EPA in the context of the 316( b) regulations would prevent the loss of a potentially large number of age one equivalent fish, both regionally and nationwide. This change in impingement and entrainment could lead to wide range of biological improvements ( e. g., effects on stock size). In response to the second point, EPA notes that the presentation materials and the survey clearly state that the expected ecological changes associated with impingement and entrainment reductions from the 316( b) regulations are subject to uncertainty. Specifically, EPA added the following statement to the slide show and the matching script: " You will be shown different policy options  with different effects on fish. This is because scientists are still working to determine what the exact effect on fish will be, so it is important to know how you would react to a wide range of possible outcomes. Common sense indicates that preventing the loss of fish eggs and young fish will mean more adult fish in future years, but at this point there is still significant uncertainty regarding the exact size of these future effects." Furthermore, given that there will be some biological uncertainty regarding the specifics of the " actual" baseline conditions and policy impacts, EPA designed the survey as a choice experiment, in which levels of attributes will vary across respondents. The resulting valuation estimates will allow flexibility in estimating WTP for a wide range of different circumstances.

The introduction to the Barnthouse Comment also notes that cooling water intake structures were not listed as major concerns in two recent reports on threats to U. S. marine resources for which new regulatory policies are needed. The commenter then argues that this implies that statements in the survey materials " that entrainment and impingement losses are known to be causing measurable reductions in the abundance of important fish populations, and that reductions in those losses are needed to prevent continuing long term declines," are false. EPA disagrees with this statement. The Preamble to the 316( b) Phase II Rule discusses a number of specific examples of evidence that I& E at CWIS causes significant losses of fish ( 69 FR 41588). For example, at the at the San Onofre Nuclear Generating Station, an estimated 57 tons of fish were killed per year when all units were in operation, and local fish densities decreased significantly. Thus, based on empirical evidence that I& E at CWIS causes significant losses of fish, it is entirely reasonable to list " the use of cooling water in commercial facilities" as a factor that affects fish populations. Moreover, the lack of emphasis in a two reports does not in any way imply that entrainment and impingement losses do not cause measurable reductions in the abundance of important fish populations.

Finally, the introduction points out that the survey materials imply that impingement and entrainment impacts on freshwater species are equally severe as I& E impacts on marine species. EPA disagrees that the presentation materials imply equally severe I& E impacts on marine and freshwater species. The presentation materials clearly emphasize that the effect of impingement and entrainment losses on fish population will vary depending on facility location and that the estimates presented in the slide show discuss average effects across all affected water bodies. ( For more detail see EPA's response to TER Comments on Slide 13).

Specific Comments on the Slide Show The commenter provides a number of comments, organized by slide. EPA's responses follow the same convention:

Slide 8: The commenter asserts that commercial and recreational fishing, pollution, and development of waterfront areas are " among the factors evaluated by the Pew Oceans Commission and the U. S. Commission on Ocean Policy. The reports from these commissions provide ample documentation of their harmful effects. No similar documentation exists for effects of cooling water withdrawals." EPA disagrees with this statement. In the Preamble of the 316( b) Phase II Rule ( 69 FR 41588), under " Examples of Environmental Impacts Caused by Cooling Water Intakes," this passage was included: " At the San Onofre Nuclear Generating Station ( SONGS), in a normal ( non El Nino) year, an estimated 57 tons of fish were killed per year when all units were in operation. The amount lost per year included approximately 350,000 juveniles of white croaker, a popular sport fish; this number represents 33,000 adult equivalents or 3.5 tons of adult fish. In shallow water, densities of queenfish and white croaker decreased 60 percent within one kilometer of SONGS and 35 percent within three kilometers from SONGS as compared to densities prior to facility operations. Densities of local midwater fish decreased 50 to 70 percent within three kilometers of the facility." Thus, based on empirical evidence that I& E at CWIS causes significant losses of fish, it is entirely reasonable to list " the use of cooling water in commercial facilities" as a factor that affects fish populations.

Slide 9: The commenter notes that this slide does not point out that I& E at CWIS is not a primary cause of the general decline most fish stocks. First, EPA notes that the relative importance of various causes of fish loss varies by species. Accordingly, it is misleading to make " general" statements about the relative importance of different causes. For example, while overfishing may be the most important cause of decline for certain highly prized commercial species, it is a negligible cause of decline for most forage fish. Second, EPA's focus groups showed that participants had a basic level of understanding about the importance of various factors that affect fish populations. Most participants ranked pollution and coastal zone development as the main factor affecting fish population, followed by commercial and recreational fishing ( for details, refer to Section 2.2.1 of the focus group report [ Besedin et al., 2005]). EPA also included a debriefing question in the survey that asks survey respondents to rank the effect of various human and natural factors in terms of their importance to the decline in fish populations ( question 2, Attachment 3, U. S. EPA, 2005). The question was intended as a screening tool that would identify those survey respondents who misunderstood the slide show and ranked I& E as a major factor in the decline of fish populations. The commenter also points out that although striped bass are featured prominently on a number of slides, the population of this species has increased significantly in the northeast over the last twenty years. EPA points out that pictures of striped bass are included in the slides show because 1.8 million of age 1 equivalents of striped bass are lost to impingement and entrainment in the Northeast region. Whether eliminating impingement and entrainment losses has any effect on the stock size of this species will be determined by project biologists. The revised version of this slide will include the estimated change ( if any) of the striped bass stock size from reducing impingement and entrainment mortality ( slide 19, Attachment 1, U. S. EPA, 2005). Finally, EPA has added material to the survey and slide show to emphasize that CWIS losses may not be the most significant cause of fish losses in many cases. For example, the slide show script now explicitly states, " It is important to recognize that the effects of fish losses from cooling water use are usually smaller than the effects of commercial fishing. Even though cooling water use is not the largest cause of fish losses in many areas, it has contributed to a measurable decline in some fish stocks, in some locations." This statement is both justified based on scientific evidence, and provides an appropriate understanding that CWIS losses are not, in some cases, the most significant cause of fish decline.

Slide 13: EPA agrees that the effect of impingement and entrainment losses on fish population will vary depending on the baseline condition of the stock size, magnitude of impingement and entrainment losses, and other factors. Therefore, EPA revised slide 13 as follows: the slide shows a map identifying locations of major facilities in the Northeast that withdraw cooling water. The script for Slide 13 states: " Facilities that use cooling water are located in all Northeastern States. This map shows the locations of major facilities in the Northeast that withdraw cooling water in ways that harm fish. Common sense indicates that effects on fish will be greater in areas closer to these facilities, and smaller in areas that are further away. For the sake of simplicity, this presentation discusses average effects across all affected water bodies" ( Attachment 1, U. S. EPA, 2005).

Slide 15: The commenter's point that very few trout are affected by impingement and entrainment is well taken. EPA has removed the photo of the trout from this slide ( slide 15, Attachment 1, U. S. EPA, 2005).

Slide 17: EPA disagrees that the slide show fails to inform survey respondents regarding the proportion of impingement and entrainment losses attributable to forage fish. Slide 16 specifically states that two thirds of impingement and entrainment losses are fish that are not directly used by humans ( slide 16, Attachment 1, U. S. EPA, 2005)

Slide 18: This slide has been removed from the slide show.

Slide 19: EPA has revised this slide to compare impingement and entrainment losses for selected species used by humans with the stock size and commercial and recreational harvest for these species. EPA also added the following statement to the slide show: " There are many billions of fish that are not directly used by humans in Northeast waters. However, the exact size of total fish stocks for these species is not known" ( slides 17 19, Attachment 1, U. S. EPA, 2005)

Slide 21: EPA disagrees with the commenter that no decline in fish population attributable to cooling water withdrawals was documented. In the Preamble of the 316( b) Phase II Rule, under " Examples of Environmental Impacts Caused by Cooling Water Intakes," this passage was included: " At the San Onofre Nuclear Generating Station ( SONGS), in a normal ( non El Nino) year, an estimated 57 tons of fish were killed per year when all units were in operation. The amount lost per year included approximately 350,000 juveniles of white croaker, a popular sport fish; this number represents 33,000 adult equivalents or 3.5 tons of adult fish. In shallow water, densities of queenfish and white croaker decreased 60 percent within one kilometer of SONGS and 35 percent within three kilometers from SONGS as compared to densities prior to facility operations. Densities of local midwater fish decreased 50 to 70 percent within three kilometers of the facility."

Slide 24 25: EPA disagrees with the commenter's interpretation of information presented in Slide 24. This slide simply provides an example of technologies that may be implemented to reduce impingement and entrainment losses. EPA agrees that many facilities already have technologies in place and the proposed policies will not result in further impingement and entrainment reductions at these facilities. To avoid any misunderstanding of the baseline conditions, EPA added the following statement to Slide 25: " Technologies such as these are already in use at 52% of facilities that use cooling water in the US. The proposed policies would require similar technologies at all facilities that use cooling water" ( Attachments 1 and 2, U. S. EPA, 2005). EPA further notes that the survey is to be implemented as a choice experiment survey. The goal of choice experiment methodology is to obtain choice responses covering a wide range of outcomes that bound ( on the upper and lower end) actual possible policy outcomes. Different respondents will be asked to make choices over policy scenarios whose CWIS reductions range from 0% ( no policy) to 98%. This will provide EPA with an opportunity for estimating values for a wide range of potential outcomes.

Slide 31: This slide has been removed from the show.

Slide 42: EPA emphasizes that the proposed survey versions will allow respondents to choose among a wide variety of hypothetical policy options, some with larger and other with very small changes in fish populations and harvest. More generally, the survey is to be implemented as a choice experiment survey, in which levels of attributes will vary across respondents. The experimental design will explicitly allow for variation in baselines and improvements. As long as the range of baseline and improvement levels present in the experimental design bounds the actual levels, the choice experiment survey will be able to estimate appropriate values  even though some respondents will see numbers that are " not right." This is a fundamental property of the choice experiment method. Our goal is to construct survey versions showing a range of different baseline and resource improvement levels, where these levels are chosen to ( almost certainly) bound the " actual" levels. Given that there will almost certainly be some biological uncertainty regarding the specifics of the " actual" baselines and improvements, the resulting valuation estimates will allow flexibility in estimating WTP for a wide range of different circumstances. EPA refers the commenter to the numerous sources in the literature that address the uses and methods of the choice experiment method, including Louviere et al. ( 2000), Bateman et al. ( 2002), and Bennett and Blamey ( 2001).

Slide 44: See EPA's response to TER Comments on Slide 42.

Comments on the Fact Sheet The commenter first states that the fact sheet provided with the survey questionnaire overstates the potential benefits of reducing entrainment and impingement losses. The commenter then lists specific issues associated with the information presented in the fact sheet. EPA notes that the fact sheet was dropped from the final version of the survey. Findings from the focus groups conducted in August September 2005 showed that the slide show and the introductory materials in the survey provided sufficient information to answer survey questions and that the fact sheet was unnecessary. For example, one participant stated: " I found it unnecessary. [ ] I thought I picked out enough from the slide show in order to answer the questions that were being asked" [# 9/ 3 2]. Another stated:

M: You were also handed a fact sheet. Did you use the fact sheet?

R: No, I didn't. No.

M: So basically, what you are telling me, that the only information you used was the slide show.

R: Right, exactly. I was listening to him giving us, you know, what he was reading from, and then looking at the show. It was okay.

M: And you felt that you had enough information to respond to the survey.

R: Right, exactly. [# 9/ 2 11]

Therefore, EPA decided not to include the fact sheet as additional reference material.

Conclusion The commenter states that the informational slide show and fact sheet prepared by EPA contain numerous inaccurate statements and invalid comparisons and that the survey respondents are likely to be misled to believe that the benefits of requiring closed cycle cooling or other advanced intake technologies are substantially greater than is actually the case.

EPA disagrees with this conclusion. First, EPA emphasizes that the estimates of impingement and entrainment losses used in the survey are scientific estimates specific to the policy options valued in this study. The policies considered by EPA in the context of the 316( b) regulations would prevent the loss of a potentially large number of age one equivalent fish, both regionally and nationwide. EPA, however, notes that the estimated reductions in impingement and entrainment could lead to wide range of biological improvements ( e. g., effects on stock size).

In addition, the presentation materials and the survey clearly state that the expected ecological changes associated with impingement and entrainment reductions from the 316( b) regulations are subject to uncertainty. Specifically, EPA added the following statement to the slide show and the matching script: " You will be shown different policy options  with different effects on fish. This is because scientists are still working to determine what the exact effect on fish will be, so it is important to know how you would react to a wide range of possible outcomes. Common sense indicates that preventing the loss of fish eggs and young fish will mean more adult fish in future years, but at this point there is still significant uncertainty regarding the exact size of these future effects."

Given that there will be some biological uncertainty regarding the specifics of the " actual" baselines conditions and policy impacts, EPA designed the survey as a choice experiment, in which levels of attributes will vary across respondents. The proposed survey versions will allow respondents to choose among a wide variety of hypothetical policy options, some with larger and other with very small changes in fish populations and harvest. As long as the range of baseline and improvement levels present in the experimental design bounds the actual levels, the choice experiment survey will be able to estimate appropriate values corresponding to the policy outcomes. The resulting valuation estimates will allow flexibility in estimating WTP for a wide range of different circumstances. Focus groups conducted in August September 2005 showed that focus group participants clearly understood that the 316( b) regulations will have only a small impact on long term fish populations, and that the estimates provided in the survey were uncertain ( Besedin et al., 2005). For example, participants made statements such as: " I don't think a three percent population thing with the fish is a big deal" [# 9/ 8 4], and " It didn't show much either way, to me. From 65 percent to 68 percent, maybe in the big scheme of things three percent is a lot, that I don't know" [# 9/ 10 2].

Responses to the TER Comment 1 Introduction and Overview The introduction to the TER Comment describes EPA's proposed survey, and then provides a summary of later comments. In the spirit of these summary comments, EPA offers the following summarized responses. Details and rationale underlying these responses are provided in later sections of this commentary, corresponding to the appropriate numbered sections of the TER Comment. The primary elements highlighted in this section of the TER Comment, and EPA's summary comments, are found below.

Comment: " Respondents are not valuing marginal changes in forage fish populations. [ ] The pattern of such responses is consistent with other well documented biases ( such as embedding or payment vehicle). [ ] The fact that EPA's survey elicits, in many instances, values other than the nonuse value of forage fish is a fatal flaw." EPA Response: This statement reveals a clear misunderstanding of the purpose of the survey, which is to measure total willingness to pay to prevent losses of both forage and commercial and recreational species. Furthermore, despite the commenter's claims, EPA's pretesting suggests that symbolic biases such as the warm glow effect will not be a major concern. Additionally, the survey instrument provides a means to test for and eliminate responses based on such motives. More detailed responses to this comment are provided in Section 2.1, below.

Comment: " The survey responses reveal a consistent pattern of hypothetical bias." EPA Response: EPA agrees with the commenter that addressing hypothetical bias is an important survey design issue, and the Agency has taken several steps to minimize hypothetical bias. The survey has been explicitly designed to maximize the consequentiality of choice experiment questions, thereby maximizing incentive compatibility, and it includes numerous debriefing questions to assess the rationale underlying the choices made by respondents, and to assess whether hypothetical bias is an issue. EPA also emphasizes that the proposed survey follows the guidelines of the NOAA Blue Ribbon Panel on Contingent Valuation ( Arrow et al. 1993) with regard to mechanisms for reduction of hypothetical bias. Based on the results of EPA's focus groups, EPA believes that these measures have been effective at limiting hypothetical bias in the survey. More detailed responses to this comment are provided in Section 2.2, below.

Comment: " The study results offer little or no evidence that the stated preference format used in this survey  is any less subject to the pervasive biases." EPA Response: EPA's focus groups have shown that the survey respondents are aware of their budget constraints, evaluate the policy choices realistically, and understand the environmental changes they are being asked to value. Furthermore, the referendum survey format allows EPA to evaluate willingness to pay for a range of policy scenarios. More detailed responses to this comment are provided in Section 2.3, below.

Comment: " The survey responses deviate from standard economic principles for measuring benefits and interpreting survey data. [ ] Respondents are answering the questions conditional on the responses given to previous questions, which violates the independence assumption used in the standard statistical analysis." EPA Response: EPA believes that the survey will generate economically consistent responses. Although the resource changes associated with the policies presented in the survey will affect respondents in different ways, the three way factorial survey format  in which each policy is represented as having separate effects on fish losses, long term populations, and commercial and recreational catch rates  is explicitly designed to allow separate calculation of the values that respondents assign to each aspect of resource change, both independently, and though interaction terms. Furthermore, the results of EPA's focus groups revealed that most respondents answered the choice questions separately. More detailed responses to this comment are provided in Section 2.3, below.

Comment: " Respondents' answers are entirely dependent upon the information provided. [ ] Barnthouse ( 2005) shows that the information contained in the EPA survey materials is inaccurate. [ ] Consequently, any responses based on such data are biased and not suitable for inclusion in a RIA." EPA Response: The data presented in the survey represents the best scientific estimates of fish losses and population changes available. The fact that respondents' answers are influenced by this information provided is expected, and is consistent with the academic literature. As the commenter points out, the " NOAA panel identified the inclusion of accurate and complete information in the CV questionnaire as a critical characteristic of a CV survey." More detailed responses to this comment are provided in Section 2.4, below.

Comment: " EPA also has failed to include information concerning the inherent uncertainty of the effects of CWIS on the environment, which further limits the usefulness of the survey responses. The respondents in the verbal protocol survey found it very difficult to answer questions that include uncertain outcomes. Such a finding is supported by the recent Resources for the Future ( RFF) study ( Banzhaf et al. 2004), which found that respondents could not provide answers to questions when uncertainty was present. The RFF solution to this problem was to misrepresent the factual information, which is akin to no solution at all." EPA Response: EPA disagrees with this comment. The survey materials clearly indicate the uncertainty involved in forecasting the impacts of CWIS. EPA's focus groups demonstrated that survey respondents clearly understood that the ecological changes described in the survey were uncertain. Furthermore, most respondents were comfortable making decisions in the presence of this uncertainty. More detailed responses to this comment are provided in Section 2.4, below.

Comment: " Many respondents indicated that they found the survey process to be long, difficult and confusing. Such a finding increases the chances of significant nonresponse bias in the survey. The evident confusion in respondents' answers is yet another source of statistical noise that further lowers the likelihood that this survey would yield useful information for a RIA." EPA Response: EPA has made a number of efforts to simplify the presentation of information and the format of the survey. Results from EPA's focus groups confirm that respondents understood the questions and were able to complete the survey without difficulty. More detailed responses to this comment are provided in Section 2.5, below.

Comment: "  the EPA survey design does not try to determine whether people value protecting all fish from all forms of predation and whether the value of reducing the impacts of CWIS on forage fish is a subset of that broader valuation. At a minimum, this survey presents a classic illustration of the conundrum as to whether respondents have preferences for reducing the effects of CWIS on forage fish or whether such preferences are merely an artifact of the survey process." EPA Response: Results from EPA's focus groups indicate that respondents feel differently about fish losses they perceive as " natural", and fish losses that are caused by humans. Furthermore, the focus groups revealed that many respondents have large values for preventing fish losses; these values are not likely to be an artifact of the survey process.

2 Results of Verbal Protocol Many of the claims presented in the TER Comment are based on the results of a series of verbal protocol interviews conducted by TER. In these interviews, which were held in July 2005, fifteen subjects were asked to " think aloud" while they completed a modified version of EPA's choice experiment survey. A moderator then asked a series of questions intended to determine how subjects processed and reacted to the information and survey questions. The commenter notes that the results of such interviews are qualitative, and appropriately, the TER Comment " does not attempt to provide any statistical analysis of the responses because of the small sample size." EPA emphasizes that the TER Comments are based on interviews with only fifteen participants. Given such a small sample, it is difficult to generalize these findings into the broad claims made by TER. Moreover, the demographic characteristics of the TER sample may not be representative of the general population ( e. g., 2/ 3 of focus group participants are females). This stands in contrast to the extensive testing and improvement of the survey instrument conducted by EPA, which involved twelve focus groups with 120 participants. While EPA nonetheless takes these comments seriously, it believes that comments of greater validity and applicability are generated by the 12 focus groups conducted by the agency.

At various points throughout the TER Comment, the commenter provides " paraphrases of respondents' answers" that support specific claims made by the commenter. Many of these excerpted comments from the TER verbal protocol interviews are of great interest to EPA. However, it is difficult for EPA to evaluate these excerpted comments properly, given that most are provided without relevant contextual information, such as the moderator questions to which respondents were replying, or previous statements made by the same respondents. EPA would greatly appreciate the opportunity to examine the full transcripts from each interview, including moderator questions. In the absence of this complete information, it is difficult to assess the validity of the claims that TER draws from these transcripts  even within the small sample caveats offered by TER.

At the end of the introduction to Section 2, the TER Comment states that " the results from TER's verbal protocol study show that the respondents took the survey seriously. They spent considerable effort to both process the information provided in the slide show and use that information in their attempt to answer the paired comparisons as well as the other survey questions." This result is consistent with the findings from EPA's focus groups. However, the comment continues: "  despite their efforts, a number of fundamental concerns arose which seriously undermine EPA's ability to draw any meaningful or reliable conclusions regarding the nonuse benefits of the Phase III 316( b) Rule based on this survey questionnaire." EPA disputes the accuracy of this statement. As discussed in EPA's detailed responses, below, a number of issues raised in the TER Comment reflect a misunderstanding of the purpose of the survey, which is to evaluate total willingness to pay ( WTP) to reduce losses of all species affected by impingement and entrainment  including forage species as well as commercial and recreational species. Furthermore, the results from the focus groups and cognitive interviews carried out under EPA ICR # 2155.01 contradict many of the findings from the TER verbal protocol interviews. Without access to the full transcripts of the TER interviews, EPA is unable to determine whether this discrepancy is due to differences in the verbal protocol used by TER, the specific form of questioning used by the TER moderator, random variation in attitudes of the small number of participants in the TER interviews, or selective presentation of results by the commenter. Given that EPA conducted twelve focus groups with 120 participants, the Agency prefers to rely on the overall weight of evidence, which suggests that ( 1) respondents understand the information presented, and respond to the survey as they would a real vote, ( 2) the survey is not subject to serious methodological or informational biases, and ( 3) the survey effectively measures respondents' total willingness to pay for reductions in fish losses. Finally, EPA emphasizes that a review of the leading scholarly journals in the environmental economics field ( e. g., Journal of Environmental Economics and Management, Land Economics, American Journal of Agricultural Economics, etc.) clearly reveals that the extent of testing, development and revision given to the survey in question far exceeds that reported for the vast majority of surveys reflected in the peer reviewed literature. Moreover, the information provided to respondents is more extensive than that in most surveys reflected in the literature, and focus groups indicate that respondents are able to process this information appropriately when answering choice experiment questions.

Despite these concerns of EPA with the verbal protocol analysis results selectively reported by TER, the Agency nonetheless takes very seriously the possibility of the reported biases. As a result, the suggestions of the TER Comment were explicitly explored and tested in seven focus groups conducted after the TER Comment was received by the Agency. As noted below, these tests showed that the TER claims, in the opinion of the Agency, are largely unsupported by the weight of the evidence.

2.1 Respondents are not valuing marginal changes in forage fish populations.

This section of the TER Comment begins with the statement: " EPA has developed measures of recreational and commercial fishing values in the regulatory impact analysis for the proposed Phase III rule, so any respondent who is valuing sport or commercial fish and not forage fish would be valuing the wrong commodity." This statement shows the commenter's confusion about the multiple ways in which commercially and recreationally valuable fish may influence the utility of individuals. Specifically, it fails to recognize that there are two legitimate sources of economic value related to these fish: use benefits and non use benefits. As noted in the TER Comment, commercial and recreational anglers benefit from catching these species. However, only a small percentage of the recreationally and commercially valuable fish whose losses would be prevented by the 316( b) regulation will be caught by anglers. The remaining fish would enter individuals' utility functions by providing non use benefits  through their existence  that have little or no relationship to the role of these species in recreational and commercial fisheries. EPA emphasizes the well known tenet of economic theory which states that benefits or costs from multiple direct ( primary) effects on markets should be counted within a benefit cost analysis; this is not double counting, and is indeed appropriate ( Just et al. 1982). Comprehensive estimates of total resource value should include both use and non use values, such that the resulting total value estimates may be compared to total social cost . " Non use values, like use values, have their basis in the theory of individual preferences and the measurement of welfare changes. According to theory, use values and non use values are additive" ( Freeman, 2003, p. 154).

Both the market valuation and revealed preference methods used to estimate benefits of improved commercial and recreational harvest allow EPA to estimate direct use values only. These methods are not suitable for estimating total values ( use and nonuse) of harvested species and therefore would underestimate total value ( Freeman 2003). Both recreational and commercial and forage fish attributes may enter individuals' utility functions directly, providing non use benefits that bear little or no relationship to ( and are not subsumed by) use benefits generated by recreational and commercial fish. EPA refers to Freeman ( 2003), who provides a clear distinction between use benefits ( indirect and direct) and true non use benefits. Finally, EPA notes that nearly all ( 96 percent) of impingement and entrainment losses at CWIS consist of either forage species, or non landed recreational and commercial species that do not have direct uses or, as a result, direct use values. Therefore, EPA believes that ( 1) only a small fraction of potential benefits associated with the 316( b) regulation is quantified in EPA's benefit cost analysis of the proposed 316( b) regulation for the Phase III facilities and ( 2) developing comprehensive quantified benefit estimates for the section 316( b) regulation requires consideration of non use values.

The TER Comment also argues that " in order to empirically measure these nonuse values, survey respondents must be able to conceive of a value of forage fish for its own sake and to be able to separate such values from recreation or commercial use values." This statement reveals a clear misunderstanding of the purpose of the survey, which is to measure total willingness to pay to prevent losses of both forage and commercial and recreational species. The referendum format of the survey allows respondents who do not hold non use values for fish to choose policy options that reflect their preferences; respondents need not value the existence of fish, nor do they need to be able to separate their use and non use values. In fact, the peer reviewed literature is highly skeptical of stated preference surveys that attempt to decompose use and nonuse values, or to measure only nonuse values ( for example, see Cummings and Harrison 1995). This is the rationale behind the estimation of nonuse values as the total willingness to pay ( WTP) of nonusers ( Cummings and Harrison 1995; Johnston et al. 2003). A primary advantage of this approach is the general conceptual acceptance of nonuser values, even among those highly critical of surveys that attempt to isolate nonuse values among the general population ( Cummings and Harrison 1995). In early survey versions tested in focus groups, EPA attempted to assess values for only one subset of fish ( e. g., only forage fish). Such surveys were protested by respondents, and were often received with confusion. For example, respondents had difficulty understanding how regulations designed to prevent fish losses in CWIS could only " target" species with no human uses. As a result of both the above arguments, EPA feels that the current conceptual approach  assessing values for the full range of fish affected by the 316b rule  is both the most appropriate and most practical approach towards stated preference value estimation in this case.

Finally, the Agency notes that these comments are nearly identical to those already addressed by EPA in prior responses to TER. In these prior responses, EPA noted that the TER assertion, " EPA should focus only on the nonuse value of fish," contradicts both the economic literature and its own statement on page 7, " telling respondents to think only about the nonuse part will not work." As noted above, " what matters for policy purposes is total value, regardless of how it is divided between use and non use values" ( Freeman 2003, p. 154). It is generally recognized in the economic literature is preferable to estimate total value of a given resource rather that to estimate components of the total value ( e. g., indirect use vs. non use values) and then aggregate them. EPA further notes that both estimates of use values associated with improved commercial and recreational harvest and the total value of reduced fish losses due to the 316( b) regulation provide valuable information to decision makers.

2.1.1 Respondents are valuing recreationally or commercially caught fish, not forage fish.

The commenter argues that that because participants in TER's verbal protocol study frequently mentioned recreational and commercial species, individuals do not hold significant nonuse values for forage fish. EPA disagrees with this statement. First, EPA points out that the commenter's agument is based on a logical fallacy. Absence of evidence that participants value forage fish is not equivalent to the presence of evidence that participants do not value forage fish. This is particularly relevant here, given that TER has not provided a complete transcript of their focus group results  a full transcript which might well provide evidence of values for forage fish. Second, many participants in the focus groups conducted by EPA stated clearly that they held significant non use values for forage species. For example, when asked what types of fish they were imagining as they filled out the questionnaire, participants gave responses such as: " All of them: little minnows to big whales," " I am more worried about the types of fish that are not directly used by us," and " I think about them as being living creatures." When directly questioned in both written debriefing questions and verbal focus group questions, the majority of respondents indicated that their survey responses reflected values for all fish, including forage fish. EPA notes that one possible explanation for the discrepancy between the results of the TER's verbal protocol interviews and EPA's focus groups is that seven of the fifteen individuals in the TER study stated that they participate in recreational angling. Inclusion of such a disproportionately high percentage of anglers  who presumably hold large use values for species affected by the proposed policies  would naturally result in a higher percentage of responses mentioning recreational species. However, the fundamental fallacy of TERs argument is grounded on the fact that frequent mention of recreational and commercial fish among their respondents in no way implies that values for forage fish are negligible.

2.1.2 Respondents are giving a value for improving the environment, not forage fish.

In this section, the commenter expresses a concern that the survey may measure " the feeling of satisfaction of helping to improve the environment, and not the value of the enhanced fish populations per se. Based on the results of verbal protocols, respondents did report what amounts to warm glow as a reason why they chose particular programs in the paired comparisons." As evidence, the commenter provides several statements made by respondents about their motivations for answering the survey. EPA notes that not all of the statements support the hypothesis of a warm glow effect. For example, the participant who stated that " I have a conscience. Don't want to see anything wiped out," was clearly thinking about fish mortality, not about a general desire to benefit the environment. EPA also notes that the lack of full transcripts from TER's focus group is important here, as the critical issue when assessing symbolic effects is whether these effects contribute to specific answers to stated preference responses, not whether such issues are mentioned casually by respondents in focus group discussions.

More importantly, the evidence from EPA's focus groups and cognitive interviews shows that the warm glow effect is not a significant source of bias in the survey. Many participants specifically stated that their responses to the choice questions were based on the value of preserving fish, not on a general desire to help the environment or to send an environmental message. Some participants felt that fish were worth saving for their own sake. Others felt that preserving fish was important because changes in fish populations can affect aquatic ecosystems, anglers, and fish markets. For example, one participant stated, " You know, if you were telling me you were saving spiders, who cares? You know what I mean? So, it is not just to do the right thing. I think that fish are important."

The commenter continues by quoting one participant who expressed hesitation over paying money to reduce fish losses when there are other important issues that she felt strongly about. The commenter argues that her statements demonstrate how respondents' answers " do not conform to the basic economic assumptions that are required to measure benefits." EPA disagrees with this point. As provided in the TER Comment, the participant's answers do not reveal any inconsistencies. Instead, her responses show a strong awareness of her own budget constraint  a critical point for any stated preference survey. The commenter also cites the following statement about the programs for which the respondent stated that she was willing to pay: " Air pollution, something having to do with forests  not cutting down forests, what else? I would be for polices like this or anything having to do with preserving wildlife or animals or fish and plant life. Education would definitely be a top issue." This comment shows that the respondent is aware of a variety of environmental and social programs that the government may support and that she has well defined preferences for these programs. EPA notes that survey respondents are expected to answer survey questions based on the specific information provided and their own preferences. Thus, the provided excerpts shows that respondents are likely to vote differently for different programs and that their responses do conform to the basic economic assumptions required to measure benefits ( i. e., responses are based on well defined preferences and take the budget constraint into account).

On page 9, the TER Comment states that " respondents are likely to consider that their valuation responses would affect either other fish populations besides those that would be described in the contingent valuation ( CV) survey or other environmental factors such as habitat quality that limit fish populations." The commenter then provides excerpts from statements made by respondents. EPA notes that three of the four statements do not support the commenter's argument:

One participant in the TER verbal protocol interviews stated that: " It's important to protect the ecosystem. Fish are a part of that [ ecosystem]. If you kill all the fish in a lake, the lake is going to change a lot. And other plants or animals in the lake might not be able to survive." This statement clearly demonstrates the participant's understanding that fish are part of a larger aquatic ecosystem, and that fish losses may harm that ecosystem. While the phrase "  kill all the fish in a lake " is inconsistent with the survey scenario, it is not clear from the excerpt whether the respondent actually believed that the proposed policies would have this effect, or whether the respondent was simply using an extreme example to illustrate a point to the moderator.

Another participant stated: " I think it's to determine what we should do as far as saving the fish. We did lose a lot with that oil spill. That one in Alaska, and we had one somewhere else not long ago." This statement provides no evidence that the respondent was imagining that the policies would affect anything except fish populations. Instead, it shows that the respondent is aware of previous situations in which fish have been killed.

The fourth participant stated: " Coastal development in my lifetime has hurt fishing a whole lot. You can't catch fish beside a high rise condominium. Part of fishing is environment ( the fisherman's surroundings). I have a better experience on an island fishing than at Virginia Beach or Myrtle Beach. Now it is row after row of box buildings. You don't want to fish beside the storm water sewer outlet. It degrades the whole experience." This statement reflects the participant's opinion about the negative effects of coastal development, but at no point does the participant indicate that cooling water intake structures are the cause of this problem, or that the proposed survey policies would address coastal development.

Thus, only one of the responses provided in the TER Comment provides evidence that a participant misunderstood the effects of the proposed policies. This is consistent with the results from EPA's focus groups, in which the majority of respondents indicated that they completed the survey questions based solely on the impingement and entrainment changes described in the survey. Although a few respondents indicated that they thought that the proposed policies might also reduce pollution or have other environmental effects, most respondents considered only changes related to fish losses, fish populations, and commercial/ recreational catch rates. For example, when asked what kind of environmental changes would occur as the result of the policies, respondents answered, " Just fish. And the fishing industry; it's not just the fish. But that's about it" (# 8 7), and " I understood it as, this would only, the only effect of this would be to increase the catch for local fisheries" (# 9/ 5 4). Furthermore, EPA has added the following statement to the survey: " Sometimes people taking this survey imagine that the policies being discussed would affect such things as water quality, pollution, or other environmental issues. They will not. The only environmental effect of these regulations is on the quantity of fish in the water." Testing of this statement in later focus groups indicated that it was effective at eliminating any confusion about the results of the proposed policies.

However, while finding that the preponderance of evidence does not point to the substantial " warm glow" biases suggested by TER, EPA does take the possibility seriously. As a result, the revised survey contains numerous debriefing questions to identify explicitly those answering based on symbolic or warm glow considerations. For example, there are four questions in the revised survey ( questions 9, 11, 12 and 13) that are designed, among other things, to identify respondents answering based on symbolic considerations rather than based on a personal value for fish. 2 In addition, the survey slide show materials provide extensive text to focus respondents on specific changes to fish attributes, and to eliminate considerations of broader environmental issues. For example, the slide show script states: " Now, before you answer the survey questions, there are a few very important things that need to be emphasized. First, sometimes people taking this survey imagine that the policies being discussed affect such things as water quality, pollution, or other environmental problems. They will not. The only significant environmental effect of these regulations is on the quantity of fish in the water. There are no other significant effects on the environment." Focus groups and cognitive interviews clearly indicate that this text was effective at minimizing symbolic responses. Hence, although EPA's pretesting suggests that symbolic biases will not be a major concern, the survey instrument provides a means to test for and eliminate responses based on such motives.

2.2 Respondents' answers show substantial hypothetical bias.

This section of the TER Comment begins with a description of the well known problem of hypothetical bias. The comment then states, " Respondents in the verbal protocol survey explicitly recognized this difference between what they said they would do in the survey and what they currently do or would actually do in the future." The comment follows with several statements from participants whose answers seem to indicate potential for hypothetical bias. EPA agrees with the commenter that addressing hypothetical bias is an important survey design issue. Thus, EPA has taken several steps to minimize hypothetical bias:

1. The survey has been explicitly designed to maximize the consequentiality of choice experiment questions, thereby maximizing incentive compatibility ( i. e., reducing strategic and hypothetical biases) ( Carson et al. 1999; Johnston and Joglekar 2005). Elements specifically designed to maximize consequentiality include a] explicit mention of the agency involved, b] explicit mention that this survey is associated with considerations of actual policies that are being considered, c] numerous details provided in the slide show and survey concerning the proposed policies, d] emphasis that some sort of policy WILL be enacted  and that the type of policy enacted will depend in part on survey results. The consequentiality and policy implications of the survey are highlighted in both the survey script and the slide show. In sum, respondents are presented with a situation in which they are very much aware that their survey responses will influence impending policy decisions  which has been shown to ameliorate problems with hypothetical bias in past work. Evidence from focus groups suggests that most 2 For detail, see Questions 9, 10, 11, and 12 in the survey instrument provided in Attachment 3, U. S. EPA ( 2005). respondents feel that the survey is consequential, and that they are responding as if voting in a binding referendum. EPA believes that the emphasis on the budget constraint, the consequentiality of the survey, and realism of the payment mechanism is as extensive as virtually any survey reflected in the peerreviewed literature to date. Moreover, the survey as currently specified clearly meets the guidelines of the NOAA Blue Ribbon Panel on Contingent Valuation ( Arrow et al. 1993) with regards to the emphasis given to the budget constraint. 2. The survey includes numerous debriefing questions to assess whether hypothetical bias is an issue. For example, the survey includes the question: " If Questions 3 5 were asked in a real referendum, how certain would you feel about the answers you provided? ( circle one)". Focus groups indicate that this question is quite successful at identifying respondents for whom hypothetical bias is an issue. Previous research has shown that such " certainty" questions can be used to mitigate hypothetical bias ( Champ et al. 2004). Pretests regarding this question reveal that the majority of respondents feel very certain of their stated preference responses, and indicate that they would answer the same way in a binding referendum  a key means of addressing the criterion validity of stated preference responses ( Vossler and Kerkvliet 2003). 3. The survey also includes debriefing questions that assess the rationale underlying the choices made by respondents ( why they answered the way that they did), and to assess whether symbolic or hypothetical bias issue are present. These questions are detailed in the draft survey instrument.

Because of these measures, EPA does not believe that hypothetical bias will be a significant issue for this survey. This conclusion is supported by the results of EPA's focus groups and cognitive interviews. Participants took the survey questions seriously, as illustrated by the following excerpt from one of the focus groups:

M: Well, when you answered this survey, did you consider that these results might actually be used by the government?

[ voices of assent]

M: Did that affect how you answered the survey?

R: No, I think I'd answer it the same. If this is just an exercise to see how I would react, I'd still answer the same as if the government really used it to create policy. The reason I used it, I would use no matter what, no matter who's going to use the information, yes.

M: But do you feel that the survey will have an impact on actual policies?

R: We hope so.

[ others repeat]

R: It would be a waste of our time if it doesn't. [# 8 8]

Although some participants had mixed opinions about whether their responses would actually affect government policy, most indicated that they would have chosen the same answers if the survey had been part of a real vote. For example, one participant stated, " I take it seriously but I have my doubts whether it will affect government policy" (# 6 16). Even those participants who did not believe that their answers would have any effect said that they expressed their opinion honestly. For example:

M: And I guess the last thing before I let you go is, did you think that  when you were, you know  obviously we're testing the survey today. But were you to get this and take it, would you think that your answers would actually affect anything? Or would this just be a survey, okay, it's another survey? Do you think that your answers would make a difference? Were you thinking that someone would actually make use of the answers?

R: No. R: Not mine, anyway. Maybe, " Okay, his is not what we're looking for." But like you say, there's no wrong or right answers. I was just giving honest answers.

R: I think that it gave you the impression that they will take this seriously the way they said, regarded as a real vote. So it gives you that impression that they are going to use it. [# 10 18]

The TER Comment also argues that " the verbal protocol results revealed that many respondents either did not realize that they could choose neither option ( opt out) or felt some social/ moral responsibility not to do so." This argument is not supported by EPA's observations during the focus groups conducted under ICR # 2155.01. In these focus groups, many respondents indicated that they would not vote for the proposed policies. For example:

R: If I had to make the choice between the three options, that's unfortunately, if I was going to go into a voting booth

M: That's how you'd vote.

R: I would not vote for any of them. [# 9/ 8 8]

The section of the TER Comment concludes with a summary of results from previous studies of hypothetical bias, and a discussion of the magnitude of such bias in the current survey. EPA notes that such discussion is highly speculative. EPA emphasizes that these arguments in the TER Comment represent largely unsupported empirical claims that were clearly and repeatedly addressed within EPA's focus group and survey design process. EPA emphasizes that the peer reviewed literature provides very few examples of surveys provided with survey design and testing as extensive as the survey proposed by EPA for the Phase III 316( b) analysis. Given that the results of EPA's focus groups indicated that most survey respondents answered the questions as honestly as they could  as if they were really voting in a binding referendum  the Agency does not believe that hypothetical bias will significantly affect the results of the current survey. As one participant stated: " I think I'd answer it the same. If this is just an exercise to see how I would react, I'd still answer the same as if the government really used it to create policy. The reason I used it, I would use no matter what, no matter who's going to use the information" [# 8 8].

2.3 Respondents' answers violate key economic assumptions.

In this section of the TER Comment, the commenter argues that " the manner in which the respondents answer the questions must be conceptually consistent with the way EPA intends them to be answered. Any substantial divergence between the respondents' answers and the key economic assumptions used in the statistical analysis of the data will either bias the estimated benefits or create such a large amount of statistical noise that the results are not economically meaningful." While EPA agrees that there should be general correspondence between the survey scenario and the actual 316( b) scenario, exact correspondence is not necessary, and in fact, may not be desirable. As is common in surveys, scenarios must be presented in simplified form to facilitate respondent comprehension, and to encourage respondents to focus on the most important policy characteristics, balancing the desire to provide detailed policy information with respondents' cognitive abilities to simultaneously consider a large number of attributes ( Louviere et al. 2000). Policy attributes that are suppressed include the specific technology used to prevent fish losses and the exact location of cooling water intake structures. As an additional note, EPA emphasizes that the level of " statistical noise" suggested by TER will be reflected in the variance of model parameter estimates. This will influence the statistical significance of WTP measures, but will not, as suggested by TER, render results " not economically meaningful." The confusion between theoretical appropriateness and statistical variance is an additional flaw in TER's reasoning.

The commenter states that " information obtained through the verbal protocol indicates that many of the respondents are valuing vastly different commodities both from what the EPA survey intends and from one another." The commenter then provides several examples of different environmental changes for which respondents in the interviews expressed values, including improvements in commercial and recreational fishing, reductions in losses of fish, and increases in overall fish populations. EPA agrees that it likely that the resource changes associated with the policies presented in the survey will affect respondents in different ways. However, the three way factorial survey format  in which each policy is represented as having separate effects on fish losses, long term populations, and commercial and recreational catch rates  is explicitly designed to allow separate calculation of the values that respondents assign to each aspect of resource change, both independently, and though interaction terms. Thus, EPA believes that opinions expressed by participants in the TER verbal protocol interviews are consistent with the purpose and format of the survey.

TER's comments suggest a lack of understanding of the capacity of choice experiments to isolate the marginal effects of different attributes on stated choices and hence on WTP  a methodology clearly described by numerous sources ( e. g., Louviere et al. 2000). For example, TER states that " some respondents are valuing improvements in recreational and commercial fishing  while others are simply valuing reductions in the number of fish killed." EPA is not at all surprised by these findings  indeed this is the specific reason that the choice experiment survey distinguishes attributes for recreational and commercial fishing and total quantity of fish losses. The survey is specifically designed to address the patterns observed by TER. The fact that TER finds such patterns in conflict with appropriate stated preference value estimation belies a lack of understanding of current methods for choice experiments, reflected in Louviere et al. ( 2000) and Bennett and Blamey ( 2001), among others.

The commenter also argues that the results of the survey should not be used to estimate national WTP to prevent fish losses because ( 1) respondents are thinking about local water resources, and ( 2) respondents to the survey may not actually live near a Phase III facility. This comment reflects a misunderstanding of the survey statistical sampling design. In response to the first point, EPA notes that the survey will allow calculation of different WTP values for respondents in different regions of the country. Such geographic differentiation will allow EPA to control for regional differences in water resources, fish species, and CWIS impacts. In response to the second point, the Agency notes that the survey will be implemented using a national probability sample. Regardless of whether respondents actually live near a Phase III facility, their answers will reveal information about their preferences for saving fish. However, when EPA estimates non use benefits in the context of the 316( b) regulation for Phase III facilities, EPA will estimate non use benefits only for the relevant population of households that are likely to hold non use values for losses attributable to Phase III facilities.

On page 14, the commenter again demonstrates a clear misunderstanding of the purpose of the survey by stating: " Moreover, to the extent that users are expressing a use value, the relevance of such a value to the nonuse value of marginal changes in forage fish populations is questionable, at best." EPA points out that the purpose of the survey is to measure total WTP for the changes discussed in the survey  including both use and non use values. When the results of the survey are used to estimate benefits in the context of the 316( b) regulation, EPA will approximate non use values associated with the policy changes based on total WTP of non users  a method characterized as "  one of the more defensible means of distinguishing use and nonuse values in a stated preference context" ( Johnston et al. 2005). A primary advantage of this approach is the general conceptual acceptance of nonuser values, even among those highly critical of surveys that attempt to isolate nonuse values among the general population ( Cummings and Harrison 1995). In early survey versions tested in focus groups, EPA attempted to assess values for only one subset of fish ( e. g., only forage fish). Such surveys were protested by respondents, and were often received with confusion. For example, respondents had difficulty understanding how regulations designed to prevent fish losses in CWIS could only " target" species with no human uses. As a result of both the above arguments, EPA feels that the current conceptual approach  assessing values for the full range of fish affected by the 316b rule  is both the most appropriate and most practical approach towards stated preference value estimation in this case.

Additional comments by TER in this area suggest additional confusion among the commenter with regard to the appropriate measurement of values in the 316( b) policy case. For example, TER states " even respondents who are not users of the resource may still be valuing recreational and commercial species," as an argument that the valuation proposed by EPA is conceptually flawed. As noted above, however, it is perfectly valid for nonusers to maintain nonuse values for recreational and commercial fish that are in addition to the use values that might be realized by users. That is fact is not noted by the commenter suggests, again, a fundamental flaw in reasoning and understanding of accepted conceptual models for benefit cost analysis ( Boardman et al. 2001).

Finally, the commenter stated that the TER verbal protocol interviews showed that respondents were not answering the questions independently. EPA takes this concern seriously, and has hence explicitly assessed the potential for non independence in focus group. Based on the results of these tests, EPA finds little evidence that this is a serious problem in the current survey. As shown in the following excerpt from the Agency's focus groups and cognitive interviews, most respondents considered the choice questions independently.

M: When you answered these questions three, four, and five, were you answering them separately or did you compare them across

R: No, separately. [# 11/ 2 9]

Moreover, EPA notes that there are numerous well known statistical methods to test for  and if present, account for  the potential for non independence of responses among individual respondents ( e. g., McFadden and Train 2000). Hence, while focus groups provide little evidence of concern in this area, statistical options exist even if unanticipated independence is discovered ex post.

2.4 Respondents' answers are conditioned by inaccurate and misleading information.

The commenter states that the TER verbal protocol interviews show that respondents' answers were informed by, and conditional on, the information provided in the slide show. EPA believes this result is entirely expected, and is consistent with the academic literature. It is well known that information can influence economic values ( Bergstrom and Stoll 1989; Bergstrom et al. 1989; Hoehn and Randall 2002). As the commenter points out, the " NOAA panel identified the inclusion of accurate and complete information in the CV questionnaire as a critical characteristic of a CV survey."

The commenter also states that some verbal protocol respondents felt that the survey was a " marketing effort and sales pitch" designed to convince them to vote for the survey. This finding contradicts the results from EPA's focus groups and cognitive interviews, which demonstrated that most participants felt that the slide show presented objective, unbiased information. Participants felt that the slide " didn't push you either way" (# 10 4) and that the information presented " seemed objective" (# 11/ 7 1). Furthermore, the participants who felt that the slide show did not provide objective information had mixed opinions about which way it was biased. For example, one individual explained that " I think by saying that the cost will be passed on to the consumer, without discussing the benefit other than just more fish, [ it] leaves out the portion that crab prices could go down, cod prices could go down, there would be more abundance of fish" (# 12 16). Another individual gave an opposite opinion: " I can't remember what it said at the end there, that seemed a little bit like, you know, ` We're trying to protect everything'" (# 6 5). Additionally, many of the individuals who felt that the factual information in the slide show was biased revealed through their responses that the factual information presented was startling ( i. e., the number of fish killed was very high), but that the presentation of the materials was not itself slanted. For example, one participant stated, " They push you out there, 1.8 billion or 12.8  they make it sound like it's so much. Like you're killing this amount of fish, you know. And almost anything that you're killing, you know, you don't want to do that" (# 10 4). Finally, some individuals who indicated that the slide show was biased also indicated that this perceived bias did not influence their decisions. For example:

R: I thought they were trying to tell me to vote so they don't use their cooling systems.

M: Okay. Did it actually get you to vote a different way than what you would have?

R: No. (# 11/ 8 1)

On page 16, the TER Comment claims that presenting costs on a monthly basis is deceptive. EPA does not believe that this is true. First, the Agency notes that most consumers have ample experience with monthly payments for items such as rent, mortgages, loans, utility bills, big ticket items, and insurance. Furthermore, to ensure that respondents carefully evaluate the cost of the policies relative to their budget constraints, the survey also shows costs on a yearly basis.

On page 17, the commenter questions EPA's rationale for identifying EPA as the sponsor of the survey, and argues that doing so may affect participants' responses. EPA notes that this choice improves the consequentiality of the survey, as clearly recommended by Carson et al. ( 1999). Respondents are presented with a scenario in which they are told that the results of the survey " will influence real policy changes," and as evidence, EPA is cited as the sponsor of the survey. Results of EPA's focus groups show that this treatment is effective, and that participants took the survey questions seriously. For example, one respondent stated, " If this is just an exercise to see how I would react, I'd still answer the same as if the government really used it to create policy. The reason I used it, I would use no matter what, no matter who's going to use the information, yes." Although the commenter cites a few respondents who felt that identifying the Agency as the survey sponsor biased their responses, the majority of participants in EPA's much more extensive focus groups did not think that the survey was biased.

In the following paragraph, the commenter refers to arguments made in the Barnthouse Comment about the validity of EPA's estimates of fish losses and long term fish population impacts. EPA's responses to these comments are discussed in the Agency's response to the Barnthouse Comment, above.

Finally, the commenter provides a number of excerpts from the TER verbal protocol interviews that demonstrate, in the commenter's opinion, that the survey provides in inadequate treatment of uncertainty. EPA disagrees with this comment. EPA's focus groups demonstrated that survey respondents clearly understood that the ecological changes described in the survey were uncertain. Most respondents were comfortable making decisions in the presence of this uncertainty ( although they were less comfortable about political uncertainties, such as whether their money would be used effectively). As one respondent explained, " It sounds like they're talking from a certain degree of certainty, and they're not putting any figures out there that they can actually prove, but based on theory we know that fish populations are declining and so these are all estimates. And I accept them as estimates" (# 7 7). Another stated, " If they had stated figures as though these figures are it  that I would question" (# 7 7). However, EPA acknowledged that a few participants did express protest when told that biologists were uncertain as to the long term effects of programs on fish populations. For example, one participant stated, " I instantly thought, oh, here we go again! We are going to spend the money and they are going to tell us later on that they were wrong, but we are not going to say that we were wrong. We are just going to do another study." (# 3 17). EPA emphasizes that the slide show clearly states that results of the proposed policies are uncertain: " You will be shown different policy options  with different effects on fish. This is because scientists are still working to determine what the exact effect on fish will be, so it is important to know how you would react to a wide range of possible outcomes. Common sense indicates that preventing the loss of fish eggs and young fish will mean more adult fish in future years, but at this point there is still significant uncertainty regarding the exact size of these future effects."

Beyond these comments, EPA also highlights that many of TER's comments regarding respondents' reactions to uncertainty are based on a version of the survey invented by TER itself  and not the version of the survey developed by EPA. EPA emphasizes the clear and significant differences between the two survey versions, as highlighted in the TER Comment and the Appendices. EPA as a result, finds these results questionable, as they are not based on the survey instrument proposed by EPA, but rather a " straw man" survey designed explicitly by TER. EPA believes that these comments also show the relative success of the EPA survey at addressing issues of uncertainty. Unlike the version of the survey developed by TER, which was met with skepticism and protests by their respondents, the survey versions developed by EPA were found to appropriately capture uncertainty by the vast majority of focus group and cognitive interview respondents. For example, participants made statements such as: " This is what the scientists project. It's not for certain. I knew that anyway from the presentation" [# 11/ 1 4], " It was said in the slide show, that they really don't know. [ ] Scientific speculation, see, so there's a scientific aspect of it" [# 11/ 2 7], and " Even in the slide show, when we heard it and watched it, a lot of these things are estimates. They're not exact sciences. So, basically taking these questions with a grain of salt in that respect as far as the averages and the figures and all that" [# 11/ 7 4].

To address the problem of uncertainty, the comment suggests modifying the survey to present a range of possible effects. EPA agrees this is a valid suggestion, and in fact, in one of the groups of cognitive interviews, EPA tested a version of the survey that did show a range of effects. The respondents were explicitly asked whether the ranges were helpful in understanding the uncertainty of estimates presented in the choice question or whether they were a source of confusion. Seven out of the eight respondents interviewed indicated that that ranges were more confusing, that the original presentation of resource changes was more clear, and that they clearly understood that the ecological changes described in the survey were uncertain. Furthermore, respondents were comfortable making decisions in the presence of this uncertainty. Moreover, and more importantly, the respondents in cognitive interviews showed that different people anchor their responses to different values within the provided range. This indicates that the model suggested by the commenter does not match respondents' behavior, as few of the cognitive interviews showed that respondents were using expected values to determine their responses. For example, some respondents anchored their responses to either upper or lower bound of the presented ranges, while others used a mid point estimate. Still others had no clear understanding of the meaning of the presented ranges. Therefore, the use of range midpoints for estimation ( or any other point for that matter) will result in errors in variables bias ( Greene 2003).

2.5 Respondents have difficulty completing the survey, which could lead to nonresponse bias.

In this section, the commenter makes that argument that the information presented in the survey is complex, which may lead to respondent fatigue and non response bias. This argument is contradicted by observations from EPA's focus groups. Participants made statements about the slide show such as: " I thought it was good. I liked it," " Just about right," and " It seems interesting. Informative." No participants in EPA's focus groups indicated that they would not have completed the survey because it was long or difficult. Furthermore, EPA has made a number of efforts to simplify the presentation of information and the format of the survey. Participants responded positively to these changes. For example, one participant stated: " On option B, looking at it, I saw that the blue was where we were presently, and the purple would be how much more we would increase the population of fish. So that was pretty straightforward. It wasn't very difficult to understand." [# 11/ 7 7]. Another participant stated, " The pie graphs, I could understand them." [# 11/ 4 1].

The commenter also argues that participants failed to understand some of the information presented in the slide show and survey. EPA notes that in any survey, some participants will fail to understand or remember certain pieces of information. However, in EPA's own focus groups, most participants seemed to comprehend and remember information presented in the slide show and survey. Participants displayed a high degree of understanding with regard to fish, fish ecosystem services, and aquatic habitat. Furthermore, no participants indicated that they had trouble completing the survey because they didn't remember or understand information from the slide show. Finally, EPA has included specific questions in the survey to test respondents' recollection and understanding of specific elements of information from the slide show. 3 These questions will assist in the identification of respondents for whom recall of critical information is a concern.

On pages 21 and 22, the commenter discusses reasons why participants in the TER protocol interviews were uncertain about their answers, such as insufficient information or misunderstanding the choice sets. In EPA's focus groups, participants generally felt comfortable with their answers, although some expressed uncertainty. For example, participants made statements such as: " I was pretty certain" (# 7 17), and " I would be very certain that I would answer in a real referendum the way I did it here, if this was the same exact survey that went out to the people, and it was presented this way with the presentation" (# 9/ 7 6). Some participants expressed hesitation: " I wasn't real sure. I don't know that much about it. If I was going to be voting on it I'd do some research on it before I would vote on it" [# 8 6]). Most participants who were unsure of their answers cited a lack of information as the primary reason. These participants often stated that they wanted more details about how the policies would be implemented, particularly 3 For detail, see Questions 2 and 3 of the survey instrument provided in Attachment 3, U. S. EPA ( 2005). what technology would be used (" You would need to go into a whole lot more specifics with the technology you're going to use" [# 10 17]). As discussed previously, EPA has included several debriefing questions in the survey to identify respondents who are very uncertain about their answers.

2.6 Summary The commenter summarizes the major points of the comment as follows:

" The verbal protocol study demonstrates that the stated preference survey proposed by EPA to measure the nonuse value of certain marginal changes in forage fish populations will not produce reliable results." As discussed in the previous sections of this response, most issues identified by the commenter reflect a misunderstanding about the purpose of the survey, or have been addressed in more recent versions of the survey.

" The verbal protocol analysis demonstrates that the study is unnecessary. Respondents do not have a separate meaningful value for marginal changes in forage fish populations. To the extent that respondents care about fish, and perhaps would actually pay to protect fish, they are thinking of recreationally and commercially significant species. The benefits of CWIS on those species are already reflected in EPA's RIA for the Phase III proposed rule." As discussed in EPA's responses above, this conclusion reflects a misunderstanding of the purpose of the survey, which is to measure total use values for reduction in impingement and entrainment of all species  including both forage species and commercial and recreational species.

Appendix A: Description of Verbal Protocols and Respondents First, EPA notes that the TER sample is very small ( 15 participants) to use to draw general conclusions based on the focus group findings. EPA notes that the demographic information provided in Appendix A shows that the TER sample may not be representative of general population. Some demographic groups are overrepresented in the TER sample. For example, 2/ 3 of the participants are females, and half of the participants are recreational anglers.

EPA also emphasizes that the pretest conducted by TER is not necessarily a " clean" pretest of the survey instrument, in that additional materials and questions are presented to respondents prior to viewing the slide show and taking the stated preference survey. This is distinct from the approach used in actual survey implementation, in which respondents are only shown the slide show and then given the survey. For example, as noted by TER, " Respondents completed a set of questions at the start of the interview inquiring about their knowledge of the effects of cooling water intake structures on fish populations. They were also asked to indicate whether or not enough money is spent on other problems in this country and then rated the importance of specific issues, ranging from preserving aquatic life to reducing taxes. A slide show was presented to the respondents that gave background information on the effects of cooling water intakes on fish populations. The slide show also explained the decision that the EPA is trying to make regarding regulations for cooling water intake structures and how the technological advancements might affect the respondent financially. The respondents were then given the SP survey." This divergence between TER's pretest methods and actual survey implementation methods invalidates much of the pretest results with regard to the specific survey developed by EPA. EPA also emphasizes that TER apparently did not test the specific version of the survey proposed by EPA, but rather a " similar" survey. Again, any differences between the survey proposed by EPA and the survey tested by TER will reduce the validity of focus group results with regard to the specific survey instrument proposed by EPA.

Appendix B: Verbal Protocol Material No response is required.

References Arrow, K. , R. Solow, E. Leamer, P. Portney, R. Rander, and H. Schuman. 1993. " Report of the NOAA Panel on Contingent Valuation." Federal Register 58( 10): 4602 4614.

Bateman, I. J., R. T. Carson, B. Day, M. Hanemann, N. Hanley, T. Hett, M. Jones Leee, G. Loomes, S. Mourato, E. Ozdemiroglu, D. W. Pierce, R. Sugden, and J. Swanson. 2002. Economic Valuation with Stated Preference Surveys: A Manual. Northampton, MA: Edward Elgar.

Bennett, J. and R. Blamey, eds. 2001. The Choice Modeling Approach to Environmental Valuation. Northampton, MA: Edward Elgar.

Bergstrom J. C., and J. R. Stoll. 1989. " Application of Experimental Economics Concepts and Precepts to CVM Field Survey Procedures." Western Journal of Agricultural Economics 14( 1): 98 109.

Bergstrom, J. D., J. R. Stoll, and A. Randall. 1989. " Information Effects in Contingent Markets." American Journal of Agricultural Economics 71( 3): 321 323.

Besedin, Elena, Robert Johnston, Matthew Ranson, and Jenny Ahlen. 2005. " Findings from 2005 Focus Groups Conducted Under EPA ICR # 2155.02." Memo to Erik Helm, U. S. EPA/ OW, October 18, 2005.

Boardman, A. E., D. H. Greenberg, A. R. Vining, and D. L. Weimer. 2001. Cost Benefit Analysis: Concepts and Practice, 2nd edition. Upper Saddle River, NJ: Prentice Hall.

Carson R. T., T. Groves, and M. J. Machina. 1999. " Incentive and Informational Properties of Preference Questions." Plenary Address at European Association of Resource and Environmental Economists, Oslo, Norway, 1999.

Champ, P. A., R. Moore, R. C. Bishop. 2004. " Hypothetical Bias: The Mitigating Effects of Certainty Questions and Cheap Talk." Selected Paper prepared for presentation at the American Agricultural Economics Association Annual Meeting, Denver, Colorado, 2004.

Cummings, R. G. and G. W. Harrison. 1995. " The Measurement and Decomposition of Non use Values: A Critical Review." Environmental and Resource Economics 1995( 5): 225 247.

Federal Register. 2004. " National Pollutant Discharge Elimination System  Final Regulations To Establish Requirements for Cooling Water Intake Structures at Phase II Existing Facilities; Final Rule." 69 FR 41588. Freeman III, A. M. 2003. The Measurement of Environmental and Resource Values: Theory and Methods. Washington, DC: Resources for the Future.

Greene, W. H. 2003. Econometric Analysis. 5th ed., Prentice Hall, Upper Saddle River, NJ.

Hoehn, J. P. and A. Randall. 2002. " The Effect of Resource Quality Information on Resource Injury Perceptions and Contingent Values." Resource and Energy Economics 24( ER1 2): 13 31.

Johnston, R. J., and D. P. Joglekar. 2005. " Validating Hypothetical Surveys Using Binding Public Referenda: Implications for Stated Preference Valuation." American Agricultural Economics Association ( AAEA) Annual Meeting, Providence, July 24 27.

Johnston, R. J., J. J. Opaluch, M. J. Mazzotta, and G. Magnusson. 2005. " Who Are Resource Nonusers and What Can They Tell Us About Nonuse Values? Decomposing User and Nonuser Willingness to Pay for Coastal Wetland Restoration." Water Resources Research 41( 7): W07017.

Johnston, R. J., E. Y. Besedin, and R. F. Wardwell. 2003. " Modeling Relationships Between Use and Nonuse Values for Surface Water Quality: A Meta Analysis." Water Resources Research 39( 12): 1363.

Just, R. E., D. L. Hueth, and A. Schmitz. 1982. Applied Welfare Economics and Public Policy. Englewood Cliffs, NJ: Prentice Hall.

Louviere, J. J., D. A. Hensher, and J. D. Swait. 2000. Stated Preference Methods: Analysis and Application. Cambridge, UK: Cambridge University Press.

McFadden, D. and K. Train. 2000. " Mixed Multinomial Logit Models for Discrete Responses." Journal of Applied Econometrics 15( 5): 447 470.

U. S. EPA. 2000. Guidelines for Preparing Economic Analyses. USEPA 240 R 00 003. U. S. EPA, Office of the Administrator, Washington, DC, September 2000.

U. S. EPA. 2005. " Supporting Statement for Information Collection Request for Willingness to Pay Survey for § 316( b) Phase III Cooling Water Intake Structures: Instrument, Pre test, and Implementation."

Vossler, C. A., J. Kerkvliet. 2003. " A Criterion Validity Test of the Contingent Valuation Method: Comparing Hypothetical and Actual Voting Behavior for a Public Referendum." Journal of Environmental Economics and Management 45( 3): 631 649.