And so when you did your examples, you did it in the Ohio watershed.  And we have so many other climates, you know, arid, tropical rain forest out there in the Pacific Northwest.  I think you need to move it around and show that the precipitation or some of these other models work in some of these other climate zones so that you can show that.
                  I did appreciate that you had a hierarchy, and you recognize that having the flow as your preferential use of the data.  And then the stage, and then the precipitation data.  So, you all have been playing with the model, and you know well some of the caveats of it, so I appreciate that.  Maybe having a good hydrologist on your staff would be a good thing.
                  The other thing I would say is, is that each of these situations that you're going to with all these different climate zones, document what you do very well for transparency.  Because, you know, there's going to be so many little nuances; you might have to do something here, but over here you can't do that.
                  And you know, there's no perfect model, as, I think, all the caveats that we've shown.  So, the best you can do is document what you do for the future and move on.  Thank you.
                  DR. ROBERT CHAPIN:  All right.  Ken?
                  DR. KENNETH PORTIER:  Ken Portier.  I wanted to follow up on something Dr. Nowell said.  What we're talking about for the near-complete data is an imputation model for the MTFA and the STFA time series; right?
                  And they proposed using precipitation and stage data to kind of inform that imputation.  But for the near-complete time series, the MTFA changes very little.  So, if the gap of data is pretty small, linear amputation probably works as well as anything else you're going to do.  The bigger issue is the STFA, the Short-Term Flow Anomaly, and capturing that variability and putting that back into the model.
                  I would kind of recommend that you go look at simply looking at the variability in the STFA and imputing just with random numbers, if the period is small enough, and see what happens.  Okay.  You can do an imputation within an imputation, if you like.
                  And I think you're going to find, that may work as well, is using precipitation and streamflow for a lot of the reasons that we've heard.  It becomes a bigger issue when we're doing the sparse data; right?
                  Because if you've got -- you've only got streamflow and you're missing a lot of the streamflow.  Now, it's really important to get the STFA pattern right, and to kind of capture that trend that the MTFA is adding to the model.
                  And as I was listening to the discussion, I'm hearing, well, precipitation doesn't quite have the same trend as the waterflow MTFA.  So, the beta parameter is going to change, and its importance in explaining variability is going to change and likely go down.  And then the precipitation variability, the short-term flow anomaly estimate, is going to be much more variable because precipitation's likely to be more variable.  So, you're adding more variability there and I don't know what that does.
                  I think the point is, you're kind of looking at what you're trying to do with these covariates.  And I would have liked to have seen what the MTFAs and STFAs look like for some of the streamflow data, and then what you replace them with for the precipitation or stage flow data.  So, we could've seen how the different covariates make to that component in the model.  I think that would help the White Paper to explain why it does or does not work.
                  Because I think in some situations, the precipitation data or the stage data really mimics those patterns well.  And in a lot of situations, it doesn't mimic them well at all.
                  DR. ROBERT CHAPIN:  Tom?  Other comments on the use of precipitation in the stream stage data?
                  DR. THOMAS POTTER:  I had a follow-up with something I forgot to say, and I was looking back in my notes here.
                  And I'm going to say that if precipitation is used, again, you know, hopefully, as you go forward we'll get a more defensible position on that that will add confidence to the model output.
                  One of the great challenges with precipitation is deciding what data to use.  I think we all know and appreciate that precipitation can be widely variable across the landscape.  It depends a lot on what part of the country you are.  I'll say that the part of the country that I worked for many years, precipitation is wildly variable, especially during the growing season.
                  There's a very nice paper that was done by one of my former colleagues, David Bosch, who analyzed all of the precipitator and the Little River Experimental Watershed where I worked for many years.  And I think it would be insightful to look at that, particularly if you're going to use precipitator in the southeast.
                  But you really need to explore further, providing guidance into how measured precipitation data is going to be used.  I don't think it's a simple answer and one that you need to spend some time on.
                  And with that said, perhaps it may be useful to look at alternate estimates of precipitation that would operate on a watershed scale or larger scale, such as NEXRAD Radar Data.  I recall that it was suggested that that be used in the Spatial Aquatic Model, SAM.  And I certainly would support that.  I think there's been additional research effort in that area.  I'll provide a citation on that.
                  But I think, again, the whole issue of if you use precipitation, you're going to have to decide on which data to use.  And that matters.
                  DR. ROBERT CHAPIN:  Okay.  And other comments on the --
                  DR. XUYANG ZHANG:  Xuyang Zhang here.  I have two quick points that I want to add.
                  DR. ROBERT CHAPIN:  Go for it.
                  DR. XUYANG ZHANG:  Okay.  In the occasion that the flow data is moving, I think Dr. Baffaut had probably mentioned this.  But I recommend that we could possibly looking to using hydrological models to predict the flow, when the flow data is moving.
                  That sort of takes care of the drawbacks using solely precipitation data, because the landscape processes and the soil process we're taking into consideration, if we use hydrological models.
                  My second point is that if we deem to use precipitation data, I think we should consider irrigation in arid and semi-arid regions.  Coming from California, I know that irrigation is a big driver, and pesticide runoff, especially in arid and semi-arid regions; especially when the use site or the crop is commonly irrigated, using flood and the sprinkler irrigation.
                  You know, case by case scenarios, you know that the use site is an irrigated crop.  I think special attention should be paid to the irrigation water use and perhaps either add that to the precipitation data or incorporate it in some other way.
                  That's it.  Thank you.
                  DR. ROBERT CHAPIN:  Okay.  Thank you.  Other comments about the use of precipitation in stream stage data?
                  Okay.  Let me turn to my EPA colleagues and ask if you have any questions for us for clarification, if you want to go back to some comment that we made.
                  DR. ANNA LOWIT:  I'll ask one was, while the other members of my team huddle down there.
                  Dr. Potter, I've heard you say a couple of things that weren't exactly identical.  In your first round of comments, you had very strong recommendation to not use precipitation.  And then as the conversation went further, with Dr. Portier's suggestion of analysis and some of the other Panelists who were a little bit more tempered in their response, that you also tempered your response with some really nice comments about the weaknesses and some other things to do, and that sort of thing.
                  So I'd just like some clarity on the strength of your response.  Is it the latter that's more in line with the others that's -- we don't really like precipitation, but if that's all you got, just be careful with it?  Which is sort of what I heard from the others; as opposed to, no, don't do it at all.
                  DR. THOMAS POTTER:  I guess I would agree that there is potential for using precipitation data as a surrogate for streamflow.  But there needs to be further work to demonstrate that it's an effective covariate.
                  There seems to be some data described in the White Paper that supports that.  But the assessment is very limited.  And I just wanted to make the point in the outside of my comments that -- you know, and I've heard it here in the room during some of the presentations yesterday -- that there's less confidence in the outcome when precipitation is used for a surrogate side.
                  I make the point that, you know, you need to kind of think about that and what that represents in the overall outcome of the assessment.
                  DR. ANNA LOWIT:  Thank you.
                  DR. ROCHELLE BOHATY:  I think the only point of clarification might be -- I think we only heard one person talk about stage.  I just want to make sure that there was no other comments.  A lot of the discussion was on precipitation.
                  DR. ROBERT CHAPIN:  Anybody want to dip their oar into stage as it were?  No?  Well -- oh, Lisa.
                  DR. LISA NOWELL:  Lisa Nowell.  I agree with everything that Dr. Baffaut said about stage.
                  DR. ROBERT CHAPIN:  You happy?
                  DR. ROCHELLE BOHATY:  Thank you.
                  DR. ROBERT CHAPIN:  Okay.
                  DR. THOMAS POTTER:  This is Tom Potter.  I want to say something about stage.  And that is that in areas of the country where you have flooding, you're rating curve kind of falls apart at that point.
                  And so again, this relates to extreme events which are of concern here.  I think there's some nexus there where you need to look at your data and see whether you have a scenario where you have extreme flooding, the rivers are leaving their banks, and what that looks like in terms of your model outcome.
                  DR. ROBERT CHAPIN:  Okay.  All right.  Excellent.  I think we've had a productive morning.  I have a little after 12:00, so we get almost an hour for lunch.  So we'll be back here at 1:00.
                  Yeah.  Thank you all.  And we'll see you in 55 minutes.

                                    [LUNCH]

                              CHARGE QUESTION 1(d)

                  DR. ROBERT CHAPIN:  All right.  I've got 1:00.  We're going to start, and our EPA friends are here.  All right.  So who gets to read the question?
                  DR. ROCHELLE BOHATY:  Hi, Rochelle here again to read 1(d).  Considering that SEAWAVE-QEX was developed from data from flowing systems and that community drinking water systems use drinking water from a variety of surface water sources, including low- or non-flowing systems, please comment on the utility of using SEAWAVE-QEX for low- or non-flowing systems.  Is the panel aware of a better tool for infilling monitoring data for low- or non-flowing systems?
                  DR. ROBERT CHAPIN:  Excellent.  Sort of the obverse of some of the stuff we've just been discussing.  All right.  And I've been told that Dr. Potter's going to lead off on this response.  Okay.  Just a second.  There'll be a momentary -- everybody take a breath.  And while you're doing that, I'll remind the group that we've been requested to sort of couch our needs and recommendations as to immediate need, mid-term, and long-term.  Okay.  The more we can do that the better, more useful our recommendations will be for our EPA friends.  Dr. Potter, over to you.
                  DR. THOMAS POTTER:  Okay.  Thank you.  Well, I guess I'm responding to this one with a risk assessment hat on, so I'm going to start off with a few comments in that area.  And essentially, they identify something you probably already know that maybe needs to be priority placed on evaluating concentration dynamics in static water systems.  I found a paper by Mossgan (phonetic) et al. that looked at the upper centiles of atrazine concentrations, comparing static and flowing systems.  And indeed, for raw water, the upper centiles for atrazine were always higher.  So there seems to be a potential for peak concentrations being higher in static systems.  In addition, the duration of sustained concentrations is likely much longer.  So that's also a problem in terms of exposure in many ways.
                  I'll say this was indicated in SEAWAVE-QEX assessments with ways for static systems that were analyzed that had -- the waves had very low amplitude and long duration.  I also think that was somewhat problematic in terms of the application of the model.  I think there would be much greater uncertainty associated with that wave shape rather than one that has a much higher amplitude that comes to a peak and follows a decay.
                  I did spend some time, probably too much time, digging around trying to find out just how many community water systems out there are static and flowing.  I did find a presentation that was made at the environmental working group meeting some years ago.  And in that presentation, the authors had looked at the water systems as part of the atrazine monitoring program, and 70 percent of the systems were static.
                  And again, maybe you guys already know all of this.  But that certainly, again, was another indication that, from a risk perspective, these systems should take priority.  And what I'm going to do here, because there are several other folks here with hats on, they're going to talk about the model and hydrology.  So I'll stop here and defer to my colleagues.
                  DR. ROBERT CHAPIN:  All right.  Excellent.  And then, Dr. Miglino?
                  DR. ANDREW MIGLINO:  Okay.  So unlike the previous ones, I have a lot more to say here.  And the first thing is I kind of want to define these terms low- and non-flowing systems because they're not clearly defined in the white paper, and to me, that means there's a lot of variability in what we could be talking about.  So when I'm going to be talking about a low-flow system, I mean it never dries.  It just has a very low flow and may be very sensitive to changes in flow.  Non-flowing I'm going to consider maybe locked, and it may experience an overflow.  But it's a rare event, so it's typically dry.  Okay.
                  So for a low-flow system -- I'll talk about that one first.  I don't really think the model is inappropriate.  I know that was a weird way of saying that, so sorry if I scared you.  It should probably work.  You just have to be very sensitive to those events where flow would suddenly jump.  And I believe in one of the Vecchia papers, or maybe it's in the white paper, there's some discussion that one of those anomaly terms can handle that kind of event.  But regardless, there still needs to be some care in application to those systems because those are short-term, potentially very high concentration events compared to the normal standard scenario.
                  Those events are also probably never going to be sampled or rarely sampled.  So there's even less confidence that the model is treating them appropriately or getting close to those estimates.  So again, care, not that it's inappropriate, just take care.
                  The non-flowing system poses, in my mind, a bunch of additional questions and thoughts.  And I'm not certain that the white paper really addresses whether SEAWAVE is appropriate for those kind of systems to the detail necessary to really make an assessment.  You discuss the anomaly terms and whether their p values are sufficient to show statistical proof that it's reasonable but not really what they are doing, if that makes sense.  If you look into the white paper and Vecchia and kind of try and parse out what these terms are trying to do in a mechanistic way rather than a statistical way, they're described as positive values, meaning that flow is important.  Negative values mean dilution is important over some timespan.
                  Well, for a static system, flow is rarely important.  The input flow is supposed to be captured by the wave phenomenon, and the degradation is also supposed to be captured by the wave phenomenon.  So I'm not really sure what that short-term parameter is doing, other than making it fit.  It's not, again, to say that it's not appropriate to keep in there because we don't want to pull apart the model.  But it's just hard from a mechanistic modeling perspective to determine what's going on.
                  The longer-term flow I think makes at least more sense because in filtration it's these other processes that take years, months, whatever the long-term kind of pieces.  The one real issue I have with the non-flowing system is these terms are now aggregating a lot of other terms that wouldn't be evident in a flowing system.  And what I mean by that are all the longer-term fate processes like slow degradation, absorption, burial, all of those are getting buried into these other terms that are called flow terms.  And those do have an impact in a non-flowing system that you may not exhibit in a flowing system.  Clearly, they make a difference in a flowing system, as well.  But they're going to have much more of an effect where the flow doesn't matter.  Okay.
                  I had written in here that maybe we could explore removing the two anomaly terms and seeing what that's like.  But after this morning's discussion, I'll pull that back.  What I had been thinking is, because that wave term encapsulates so much of the input and mass loss of the pesticide in these systems, restricting it to two waves a year for static waterbody may not be the best restriction.  You may have multiple events much closer to a large runoff event that would be encapsulated by that kind of term.  So that's the discussion on non-flowing systems.  I just think more work needs to be there, and I think that a short-term, near-term issue is it really needs to be shown in more than I have p values that are under my level of significance.
                  As for better tools, I have -- I am not aware of a better tool.  I will say that there are alternative tools that probably cause large headaches, other than things like kriging or things like that, which may be appropriate and could be used.  There are certainly hydrodynamic models that could be used to generate flow values on some kind of scale.  They are certainly going to be a headache to implement.  So it's kind of an effort versus reward there.  And they're also going to be prone to probably significant error.  So with all that said, I don't know of a way forward to get a non-flowing system enough data to simulate properly in SEAWAVE with flow values.  That's all I have.
                  DR. ROBERT CHAPIN:  Okay.  Excellent.  Thank you.  And Dr. Green?
                  DR. TIMOTHY GREEN:  So short-term responses affirming the consensus that no -- the answer is no, we don't actually have an alternative estimation tool to recommend at this time, unfortunately, and yet recognize the importance of surface water reservoirs and short-term recommend further more analysis of more reservoirs and water supply storage systems to guide either different sampling schemes and/or potentially a whole different model.  So this will address your question "Is the SEAWAVE model functional form relevant and maybe optimal for such waterbodies?"
                  Longer-term, I'd like to address something that's a little bit ancillary maybe, but it is a scale issue.  And this actually related to item 2(c).  So this will all be here instead of there.  So it raises this issue that has not been addressed, to my knowledge, in the current evaluations, and it's the issue of watershed scale.  I have -- I think Don has a quick set of slides I want to show you on a reference to Augustus and et al in Environmental Sciences and Technology back in 2004 using monitoring data from the Heidelberg Water Quality Laboratory and the USGS dataset.
                  And those analyses concluded that maximum daily concentrations generally decrease with watershed scale.  This occurs despite the fact that lower percentiles of concentration don't display a similar scaling behavior.  So the scaling behavior with watershed scale is particular to those maximum concentrations, maximum daily.  And the concentration estimated at the edge of field from those extrapolations match the pesticide root zone model by Carcel.  That model was by Carcel in 1998 that matches as well.
                  So this just shows the map, the next slide -- and you've seen this.  This is just to show you that you obviously know these watersheds because you've used these data.  This is a 1996 time series.  Next slide, please.  We can skip this.  This shows a range of chemical attributes, but I meant to take that out.  You're aware of these.  Next slide, please.
                  And then this is the scaling with watershed areas.  So this is actually streamflow, and this is per unit area.  And this just shows that, if you look at that P99.7, that's your daily maximum value.  And you see a clear with log/log relationship again.  The log is important.  So those look flat, but they're not so flat.  Next slide, please.
                  And then this shows a few chemicals: atrazine, metolachlor, metribuzin.  And again, we see that the maximum, or the 99.7 percentile, has a clear scaling relationship.  Lower quantiles don't.  So if we're looking at average values of a year, we may not expect this.  So we need to be aware of watershed scale with all of these estimations, whether it's from a correlation model or, in this case, from data and be aware that your drinking water supply will need to scale according to watershed area.  I think that was the last slide.
                  Oh, there's one more.  This is a normalized concentration.  And again, we see this log/log linear relationship with watershed scale.  So just wanted to point out that I wasn't aware of anything in the white paper supplement.  I might have missed it, but this is very important to your estimation.
                  So based on this knowledge, peak concentrations will likely be lower in rivers of large watersheds than in upstream monitoring locations and contributions from large watersheds to drinking water supply storages.  And reservoirs are expected to follow this kind of input behavior as was talked about.  The cycling within those water bodies will affect the results.  And then again, this does get, as pointed out by your presentation, confounded by multiple contributing sources to these water bodies.  And that can cause complications in time series.  Thank you.
                  DR. ROBERT CHAPIN:  Excellent.  Thank you very much.
                  DR. TIMOTHY GREEN:  That's a longer-term need pointed out.  Let me just comment that that scaling relationship can be included in something shorter term.  I don't know how you would couch that as medium term.  But it's something that doesn't have to involve a very complex model.  Thank you.
                  DR. ROBERT CHAPIN:  That's very helpful.  Thank you.  Okay.  And finally, the last listed person for this is Dr. Klaper.
                  DR. REBECCA KLAPER:  Hi.  My answer I think is the same as what I'm hearing -- is that, no, I don't think you can use this model for doing reservoirs.  I think maybe low --
                  DR. ROBERT CHAPIN:  Sorry, Rebecca.  Could you move the microphone back?  Thank you.
                  DR. REBECCA KLAPER:  Just in our experience in trying to model chemical contaminants in a semi-enclosed system, the flow models don't work.  To make the contaminant estimation within basically a reservoir, you have to do different hydrodynamic modelling in 3D space, and it's not the same as the stream almost point source moving down in one direction.  Modelling that is done in this kind of an application where -- and the inputs from the stream may be modeled as far as what's going into the reservoir at any period of time using maybe this existing example.  But then what happens in the residence time, et cetera?  The breakdown, the movement within that reservoir wouldn't be estimated within this type of model.
                  The characteristics of the reservoir, including the underlying geology that determines the reservoir, the hydrological inputs such as streams, rivers, whatever's going in there, the bathymetry are necessary to determine the potential movement of a plume and what that plug of water is doing once it goes into that system and then how it ends up averaging over a period of time.  Surface water monitoring stations could potentially be used as estimation inputs into the model.  However, linear model estimating source and discharge downstream wouldn't be appropriate.  As much as a fore-creaking could estimate the distribution of a plume across the waterbody, if done, multiple measurements have been made over space, which, again, is not necessarily common for a lot of waterbodies.  The draw down rate due to the outflow from the drinking water intake would also have to be considered, as well as any other kind of outputs from that system, as it would affect contaminant migration in the system from the source.  And the source load would have to be calculated as an input rather than strictly the point measurements taken in this pesticide model.
                  The EPA already uses several hydrodynamic models it seems from looking at your webpages.  There's the environmental fluid dynamics code, which looks at a bunch of different characteristics in lakes and estuaries and another one that is the -- you have inputs from the NARS estimations, inputs on lakes from lake surveys, which could inform parameters going into such a hydrodynamic model of a watershed into a reservoir.  There are also other publications out there and groups that have estimated things in estuaries and in reservoir systems, the CE-QUAL-W2 model originally by Huntinger and Buchak but now is most used -- I found a paper by Jesnak (phonetic) et al, which I can put the reference in our documents, from 2016 where they used it to look at modelling over several different watersheds.  And they have a website talking about their model use, which is going across many different watersheds.  So it might not answer, necessarily, the predictability of -- not the predictability but trying to estimate unknown when you don't have the data.  But it seems that maybe estimating one watershed from some of the characteristics from another would be better in this type of a model, instead of using that linear model.
                  I'll also note that in the absence of the bathymetry data there are several people that have used other types of estimations, so they kind of figure out the characteristics of that watershed.  Hollister and Milstead have used GIS to estimate the lake volume, the maximum depth, and using GIS layers of the shoreline in order to figure out what the lake looks like in the absence of that kind of information.  So it can be used within the hydrodynamic models of the lakes in order to give you some information in the absence of that data.
                  For low-flowing systems, I do believe that it's possible, if you're talking about a stream or a wide river that's not moving very quickly, that the SEAWAVE could potentially be useful.  But in those reservoir systems, my personal opinion is that it cannot.
                  DR. ROBERT CHAPIN:  And I was just going to look over here to see if Dr. Portier had his hand/card up.  Dr. Ken?
                  DR. KENNETH PORTIER:  Sorry.  This is Ken Portier.  I wrote two paragraphs, four lines.  So on the issue of a jump, I think the STFA might catch that because you'd get negative residuals and positive residuals.  So you'd see that.  Clearly, you'd see it in the diagnostics that that's happening.
                  The other thing, it's not clear what happens to the model in particular to the estimated model parameters when there's no flow in the middle of the season.  So you've got a stream that has intermittent flow, and it just stops.  So the seasonal WAVE says it's supposed to be up here, concentrations up here, but you have no data and no flow to predict concentrations -- and in particular what happens when flow starts again, and you get a pulse.  So you've been storing up this pesticide, and now it's released.  Absolutely the model won't pick that up.  But again, the diagnostics are going to show you that the model's not fitting for that.  It's going to be obvious.
                  I would think a short-term -- I put down an intermediate-term task for EPA could be to identify some of these complete monitoring sites that have this kind of no-flow problem and just document what happens to the model when you apply it to that scenario, the full dataset, because I really can't see in my mind the impact of three weeks of no flow in the middle of the season on the fit of this model.  You're going to put in zeros.  The MTFA's going to do something.  STFA's going to have no variability.  The trend is going to be affected, so I think it's a nice little task that could be done in a couple of days and documented.
                  DR. REBECCA KLAPER:  Can I just ask a clarifying question?  If the system isn't flowing, that means there's no water going in.  So how would the pesticide concentration increase?
                  DR. KENNETH PORTIER:  I'm saying it won't be until there's flow.  But I'm saying pesticides sitting there.  My worry is the pulse that occurs when it starts to flow.
                  DR. REBECCA KLAPER:  But then that would be captured in their flow, right?  So the stream volume would increase, and then it would pulse.
                  DR. KENNETH PORTIER:  Yeah.  So you might --
                  DR. REBECCA KLAPER:  Which gets captured in some of their -- you see the pulses going on in their system.
                  DR. KENNETH PORTIER:  Right.  But I don't think, for example, the short-term flow anomaly would really capture all of that.  You'd be missing that big peak.  And then that big peak really should be affecting the maximum, and you might not catch it.
But again, I'm not sure.  There's a lot of uncertainty here, and I think what they need to do is demonstrate it for some examples.
But the way, in one of the meetings, the 2011 panel or the 2012, there was a whole day talking about low flow streams.  I remember Nebraska, during the middle of the pesticide season and all these little streams with no water in it.  And what do you do with it?  And I don't think they came to a conclusion either, what to do with it.
                  DR. ROBERT CHAPIN:  Other comments about how to handle low-flow systems or other tools for dealing with that?  All right.  I'll turn back to our EPA colleagues and ask if there are any clarifying questions you have of the panel to clarify answers that we've given.
                  MR. CHARLES PECK:  Hi, this is Chuck Peck.  I just wanted to clarify, with the hydrodynamic models that you talk about, are you suggesting that we use that information to infill monitoring data?  Or are you suggesting we use it to predict what's going on in a particular system?
                  DR. REBECCA KLAPER:  That's a good question.  I don't know if I know the answer.  They're designed to be more of a predictive tool than to definitively identify a hole in the data where they were going to put a different number.  But since it is a predictive tool and what you're trying to do is predict exposure, it seems like it would be relevant -- more relevant than a model that doesn't fit that type of hydrodynamic system.  Did you have something else you wanted to say?
                  MR. CHARLES PECK:  Thank you.
                  DR. ROBERT CHAPIN:  Other questions from our EPA friends?
                  DR. ROCHELLE BOHATY:  We're good.  Thank you.
                  DR. ROBERT CHAPIN:  All right.  Question 1(e).  Bingo.
CHARGE QUESTION 1(e)
                  DR. ROCHELLE BOHATY:  1(e), in Section 6.6. of the white paper describes the utility of SEAWAVE-QEX in the context of designing a surface water monitoring program.  Please comment on EPA's conclusions regarding how SEAWAVE-QEX can be used to optimize a surface water monitoring program design in order to use monitoring data in pesticide drinking water assessments.
                  DR. ROBERT CHAPIN:  Mr. Councell?
                  MR. TERRY COUNCELL:  All right.  Thank you.  All right.  So our colleague from Washington State did an awesome job in presentation on using your SEAWAVE-QEX model and how he modified his monitoring program, so you have an excellent source right there who's working with you already.  So I wouldn't hesitate to reach out to him for advice.  But you have this model, and you have some data requirements for it.  Yes, use those three years of data.  You want a USG staging station nearby that measures flow.  You want all the past use data.  Put all those requirements in.
                  The one thing I would say to EPA is put all this down and go do some outreach to a lot of these programs that are doing monitoring.  If they can slightly modify their programs and get you data as well, you have a win-win situation.  And I think they would.  The last I looked, the cost of doing pesticide analysis was about $800 to collect and analyze a sample.  So it's a fair amount of investment.  So if a program can get dual use out of it, they're going to.  So I would do that.  I would outreach to all these state programs and universities that are doing pesticide monitoring.
                  The other thing I would suggest you do is somewhat what FDA did with their ELEXNET.  They took all the FERN, the Food Emergency Response Network labs, and they created this database called ELEXNET.  And they asked them to put all their microbial testing data and chemical data into that.  So maybe EPA could create a database of water monitoring pesticide data in which these various state and federal and local agencies could input that data in.  If we do that sooner than later, you might have some data by the time you implement your SEAWAVE-QEX model and using it in your Riggs assessment.  So that would be my suggestions.  EPA would have to do a significant amount of QCing of all that data but look what source you have for minimal input and cost.  That's all I have.
                  DR. ROBERT CHAPIN:  Thanks very much.  Lisa, you're up next.
                  DR. LISA NOWELL:  That Section 6.6. does have a lot of useful information of features of a surface water monitoring program, so I'm just going to focus on those conclusions relative to how the SEAWAVE-QEX would obtain to that model.  And as Dr. Councell said, there are basic data requirements for SEAWAVE-QEX, so it makes sense if you can incorporate those into a monitoring program.  I had a little bit of -- there was a statement in your conclusion, I think.
                  One concluding sentence is that, if surface water monitoring data are insufficient for use in drinking water assessments and modeling indicates estimated drinking water concentrations exceed the drinking water level of concern, then development of a well-designed, targeted surface water monitoring program may be an option to refine uncertainties in the drinking water assessment for those areas that are identified to potentially have concentration.  So I was wondering does this mean that you have the ability to request that a monitoring program be set up?  Is that what we're talking about, or are you talking about sort of incidental use of existing data and the kind of tweaks that we were hearing about?
                  MR. DANA SPATZ:  We do have the authority to ask registrants for pesticide monitoring data.
                  DR. LISA NOWELL:  Okay.  That obviously would make a difference.  So the data requirements were pretty clear-cut.  So I think it makes sense to follow the number of years of sampling and the per sample-able censorship.  Of course, that you don't know until you get your data back, but you can make some estimations.  And presumably, if you're talking about a designed or targeted monitoring program, you're talking about a chemical that is frequently detected, at least in the area of concern.  For a targeted program like that, it's very reasonable to develop a sampling frequency that would be designed specifically for that pesticide, so that makes a lot of sense.  And all the arguments are spelled out in this section very nicely about why that might be necessary.
                  In terms of making use of USGS data or any other external surface water monitoring program data, I just want to point out that, usually, we're using broad spectrum methods.  So we're analyzing not for one compound but a lot of them.  So we're going to make any decisions on sampling frequency.  It's all or nothing.  So we would either have to go with the most demanding chemical or, if we were trying to meet that specific objective, we might not have as much flexibility, I guess.
                  You have a table in there, 6.1 Components for Surface Water Monitoring Program, which has a lot of information in it.  I found the format a little odd in that there was a section on considerations and then one on program elements and one on additional.  And I thought there was some redundancy.  I just thought that might be more useful if things were separated a little bit and the elements of monitoring program, say that particular table was just listing the elements of monitoring program.  But I thought your discussion following those was excellent.
                  Oh, and yeah.  A very important point that you make is that the analytical methods have to be sufficiently sensitive to detect and quantify pesticide concentrations in water that you have to reach biologically significant thresholds and also that the QC has to be adequate.  And I'm sure that's one of your stumbling blocks in terms of putting data together from various sources.
                  DR. ROBERT CHAPIN:  Alrighty.  Dr. Kennedy?  Make sure you're close enough to the microphone, please, sir.  Thank you.
                  DR. IAN KENNEDY:  I'll just make a few more comments, particularly on section 6.6.
                  DR. ROBERT CHAPIN:  So let me just remind you that we need volume for the transcribers so that they can capture it later.  So that's right.  Pull that sucker in there.
                  DR. IAN KENNEDY:  On page 99 it says, "If monitoring data are collected for use in a model such as SEAWAVE-QEX, daily sampling is not necessary as the tool can provide estimates of these values."  And of course, that's probably true.  But you don't necessarily want to be designing your sampling program too closely to the requirements of SEAWAVE-QEX because something's going to go wrong.  At some point, you have to account for Murphy's Law.  And in generally, I think it's always better to have more samples rather than fewer.
                  Another thing is, of course, that you only evaluated this on five chemicals, one of which didn't work, and a limited number of sites.  So if you can ever have a sampling program that would let you expand that evaluation, go for it, especially if you can do it in different regions with different chemical properties and such.  And related, the SEAWAVE-QEX requires three years of data.  But as there was some discussion yesterday, sometimes years can be very different from one another, and it might be that you have one particularly anomalous year.
                  So I would suggest at least having one extra year just to account for that sort of thing.  And because it wasn't really mentioned, although it was mentioned earlier in the comments, of course you want to do it at a site that has your flow measurement.
                  DR. ROBERT CHAPIN:  Thank you.  And now we'll return to Dr. Nowell as she switches hats.
                  DR. LISA NOWELL:  Not sure which hat I was wearing.  Was that water quality I was addressing first, supposedly?
                  DR. ROBERT CHAPIN:  Ostensibly, and now it's environmental risk assessor.
                  DR. LISA NOWELL:  I can't really separate them.  Sorry.
                  DR. ROBERT CHAPIN:  Okay.  That's fine.
                  DR. LISA NOWELL:  I did think of one other thing to say, though, that a lot of times we're really particularly interested in those compounds that are intermittent, sporadically used, not often high detection frequencies, or at least not necessarily high detection frequencies, but biologically very toxic both to humans and aquatic life.  So we're particularly interested in tracking those.  And it does seem, although you did a good job -- the model did a good job with chlorpyrifos, which would be in that category I would say, and carbaryl.  It does raise an issue that a lot of the compounds of interest we would most want to predict the at maximum concentrations might be the ones we might have the most difficulty fitting the model.  I don't have a solution for that.
                  DR. ROBERT CHAPIN:  Okay.  Cliff?
                  DR. CLIFFORD WEISEL:  I also thought this is a good section.  I also, as I mentioned the other day, I liked what was in that table because it's good guidance for some of the thoughts.  And I appreciated what was done from the group in Washington.  But when I read this, one of the things it's saying is can you currently use the SEAWAVE to help optimize any number of monitoring programs, is what I'm reading this charge question.
                  If I'm interpreting it correctly, we have to remember what we said was the limitations we currently have with this system.  Limitations are we have it only for limited compounds, which was mentioned.  So if you're going to new compounds, I don't know that you can do that yet until you can really demonstrate that the model and the parameters that you set are relative to that compounds.  The same thing if you're going to other watersheds that have different systems.  We heard climate differences that may have that.  Again, you can't use what you currently have to predict those until you actually have enough data to say that that works in those.  And that's limitations.
                  So the action is you need to define what you currently have and what you conditions you can currently do it under.  Take a look where you're meeting that.  And if you're not, what is it that you can do to do it?  And maybe you can do more water ranges and therefore give not exactly what the people do in a water monitoring zone but say, "Okay.  We think you're in this range.  Maybe double it just to get us enough data to see are things going to work."  So you'd get that.
                  In a system that's much friendlier to what you have, I think you then can go ahead and do a lot better and give them finer systems.  And I certainly agree with the comment that this is a current evaluation -- like any time I use modeling that gives me measurements, I say, "Okay.  That's my minimum."  That's not what I do.  I do plus a little extra just to make sure because modeling and measurements go from one to the other and build upon each other.  So keep that in mind.
                  DR. VERONICA BERROCAL:  Can I add something?  This is Dr. Berrocal, and I wanted to breakdown your comments, Dr. Weisel.  So actually, you raise a good point in terms of we have discussed in the previous hours about the fact that, when SEAWAVE was used on some sample data, the performance, at least for the long-term concentration was not what we were hoping.  And there were also some limitation in terms of the short-term exposure.  So I think that all those comments that we made at that point could actually be used to design a monitoring sampling scheme for assessment so that, when you guys would be implementing SEAWAVE-QEX on this monitoring data, you actually will get the similar performance that Dr. Vecchia got in his paper.
                  DR. KENNETH PORTIER:  I'm kind of responding to our chair saying we should be thinking about what EPA might be able to do in the intermediate- or long-term.  Somebody said more samples are good.  Well, while more samples are good, a design concern is what's the value of a new sample?  So the white paper reports on a small study that looked at increasing sample size and concluded that diminishing returns are greater than ten samples per year.
                  In the intermediate term, EPA could perform -- what happened here?  EPA could perform a simulation add-on to the sparse data study to estimate the value of getting on additional fill-in data point, say mid-period add-in, added to the middle of the season to answer the question does this added $800 point -- does this added data point substantially reduce the width of the 80 percent confidence interval?  Does it significantly decrease the percent underestimation in situations where underestimation is an issue, for example, in the 365-day long-term maximum?
                  I think a little study like this adds a way of arguing with people who do monitoring, like state what's the value of having you go out in the middle of the season and get one more sample?  And you can come back and say, "Wow, a confidence interval is reduced 30 percent with that one sample."  To me, that's a good return on investing that effort of improving, not asking for five-day sampling for the whole year.  You might be able to get one more sample if you can convince them of that.  And you need this simulation study to do that.
                  DR. ROBERT CHAPIN:  All right.  Other comments about the use of -- yup, Tom.
                  DR. THOMAS POTTER:  Dr. Portier got the wheels rolling in my brain here, so thank you, Ken.  Hopefully it's going to be positive -- the outcome anyway.  But this motivates what Dr. Portier was saying about value of sampling.  I think always we have to ask the question what about extreme hydrologic events and whether or not we need to bear to invest the time, energy, and treasure necessary to capture them.  They don't necessarily fit well into the SEAWAVE-QEX framework.  But certainly, I think they are potentially, again, depending upon the time of year that they happen, very high value samples.
                  DR. ROBERT CHAPIN:  All right.  Other comments on the use of --?
                  DR. REBECCA KLAPER:  From what I'm reading and remembering from the document, it seems that --
                  DR. ROBERT CHAPIN:  Make sure you're close enough to the --
                  DR. REBECCA KLAPER:  The monitoring that you're talking about here would be monitoring after you decide that the SEAWAVE model actually says that there's a concern.  So then you can go back and get other samples where maybe the model is giving you questionable results.  It's not like you're talking about designing a whole new monitoring program based on the fact that you have now this model that you want to use for lots of different pesticides.  Is that correct?
                  MR. DANA SPATZ:  I think we're talking about both.
                  DR. REBECCA KLAPER:  Okay.  Then I'll just say that it seems most valuable for the first one of those where you found something where you think it might be potentially harmful.  You've gone all the way to this part of the process, and then you say, "Well, we don't really have enough samples in these particular kinds of watersheds, et cetera," but not necessarily for developing a wholescale watershed monitoring program for everything.
                  DR. CLIFFORD WEISEL:  I think the utilization for the second actually is potentially very strong.  I don't think you're there yet.  If you had something that allowed you, I think that would be a big bonus because then that saves a lot of time, money, and measurement, which is what we're always trying to do with it, too.  It's probably what I'd say mid- to long-term; whereas, certainly what you currently have is the short-term for the first one.
                  DR. ROBERT CHAPIN:  All right.
                  DR. VERONICA BERROCAL:  So I think actually I read this question as, given that the goal long-term would be to try to use SEAWAVE-QEX to characterize short-term and long-term exposure based on the fact that we do not have access to daily pesticide concentration data, what would be the best monitoring design that we could have that would allow us to use SEAWAVE-QEX and get these estimates of short-term and long-term concentration that we actually can predict?  Right now, it seems what they have shown in the white paper with the subsampling every seven days, every 14 days is that using SEAWAVE-QEX when you only have samples every seven days or every 14 days, you were not able to actually characterize that long-term concentration well.  And with the short-term, it's hit and miss depending on where you are and which pesticide you're looking at.
                  So I think the question here is how can we incorporate what we have learned from the actual study that we have done on SEAWAVE-QEX to design a better monitoring program?  So I think actually what Dr. Ken -- can I call you just Ken?  What Ken said is that you could actually do an evaluation, as you've done in the white paper, where you change the sampling frequency, instead of seven days every three days, see what happens when you implement SEAWAVE-QEX to characterize long-term and short-term concentrations, see if it improves over what you have with the seven day and 14 day sampling frequency, and then use that information to maybe push for a different monitoring design.
                  DR. ROBERT CHAPIN:  Better the sound of loon than crickets I suppose.  Let's see.  So other comments about this question from the panel?  All right.  Hearing none, I'll turn to our friends at the EPA and say do you have any clarifying questions for us?  Did we answer this question for you clearly enough?
                  MR. DANA SPATZ:  Thank you.
                  DR. ROBERT CHAPIN:  I'll take that as a yes.  Okay.  My agenda says lunch.  Let's go to -- sorry.  1(f), forward on the --
                  DR. ROCHELLE BOHATY:  I promise I wasn't ignoring you on the last one.  I was reading over my notes to make sure we had covered everything.
                  DR. ROBERT CHAPIN:  We want to capture everything we can at the right time, so take your time.  It's all good.
CHARGE QUESTION 1(f)
                  DR. ROCHELLE BOHATY:  Okay.  So I(f), during EPA's evaluation of SEAWAVE-QEX, we used the following criteria in running the model: 3 years of data, 12 samples per year, and greater than 30 percent of the samples were detections.  However, Vecchia (2018) notes that there is flexibility around the data requirements for input into SEAWAVE-QEX, provided that the diagnostic plots indicate that the model assumptions are fulfilled.  Please comment on any data characteristics, such as sample frequency and timing within and across years, that should be considered when exercising flexibility in the data requirements.
                  DR. ROBERT CHAPIN: Data flexibility, do we think we can allow them, all right, and how do they know?  Dr. Berrocal?
                  DR. VERONICA BERROCAL:  Yes.  Simulation results by Dr. Vecchia indicated that, if the modeling assumptions underlying SEAWAVE-QEX are satisfied, then bias of 80 percent of bounds are smaller when centering rate is 30 percent compared to 70 percent.  Additionally, greater sampling frequency is also associated with the smaller bias and smaller 80 percent outer bounds, particularly in relationship to the correlation in the data.  In other words, if the pesticide concentration data as residual temporal correlation, once the effect of stream flow or precipitation, since anatomy and fluctuation of the streamflow both at the monthly time scale and at the daily time scale, then it is important that the pesticide concentration is sampled frequently enough to capture these residual temporal correlation.  Instead, all the other correlation in the water-pesticide concentration data is already explained by these factor, seasonality, streamflow, and anomalies, then frequent samples are not necessary.
                  Even though the SEAWAVE-QEX model does incorporate seasonal correlated errors, it also meets the possibility of a very small, if not mild, concentration temporal scale.  Subject matter, hydrological, toxicology, et cetera, knowledge may be useful in understanding whether it is likely that, for a given site, pesticide concentration data will still be correlated even after having already accounted for seasonality, streamflow, and medium to short range streamflow anomalies.  That informing a user or assessor whether pesticide data with a higher sampling frequency is needed to reliably generate daily chemograph.   So I am not one of these people.  I don't have that expertise, so I just think that you should talk with somebody that has that expertise.
                  To modeling assumption of which SEAWAVE-QEX model relies on, the assumption anomaly distributed residuals, and there is no measurement error.  If the data have been collected over several years and all the less reliable instruments were used in the first part of the data series, it might be preferable that older years were not analyzed jointly with more recent pesticide concentration data.  Histograms of log daily pesticide concentration data potentially even stratified by season, since the SEAWAVE-QEX model assumes different level of variability in the residuals and in the data by season, should be generated prior to feeding a SEAWAVE-QEX model to determine whether the normative assumption is appropriate for the data.  If the assumption of normality or log daily pesticide concentration data is not stratified, a SEAWAVE-QEX model might not be appropriate for the data.  Particularly, the SEAWAVE-QEX derived chemograph might fail to capture the highly daily pesticide concentration if the histogram in the log concentration data appears to be very skewed.
                  And then I had another note about how in the white paper it was shown that it is possible the pesticide concentration data over multiple years might lead to plus two WAVE model feat during and that's supported by the data.  Hence, when pesticide concentration data is measured over a few number of seasons over multiple years, the SEAWAVE-QEX model might be only applied to partial records provided via the recommendation in terms of data frequency until the size are satisfied.
                  DR. KENNETH PORTIER:  My comments fall into two categories.  The first category is just the suggestion on how to improve the diagnostic plots to determine adequacy of fit.  And then I have a few more suggestions that basically are around the theme what can go wrong, even if the minimum dataset conditions are met.  So on the first one, SEAWAVE-QEX assumes that the normalized residuals are approximately normally distributed with mean zero and variance equal to one.  The SOP suggests examining plots of the normalized residuals with time of year to assess adequacy of this assumption.  With these plots, the advice is to look for obvious seasonality remaining in either the mean trend or changes in spread over time.
                  I also suggest that this plot -- it also suggests that this plot will show obvious non-normality, so for example skewedness, outliers, and so forth.  A better diagnostic for these latter conditions is to examine non-normality through a normal quantile plot.  And I'll provide a reference to that.  Normal quantal quantile plots would show clearly the percent of censoring in the data and how much of the censored residuals have been imputed and how well the imputation is doing.
                  And on the recommended plot of normalized residuals over time, I'd suggest maybe adding a low S smooth curve to give the viewer an indication of what kind of trend are in the residuals over time.  So it'd be a lot clear to see any trends, which shouldn't be there, but a smooth would at least give you an indication of what's happening.  And there's a capability to do a similar kind of plot on variability, so you can kind of do a smooth variance plot at the bottom that would show how much the variance may be changing over time.  And those are minor additions that are easily add in all.
                  So what can go wrong, even if the minimum dataset conditions are met?  So under the minimum data requirements, one would expect to see at least 36 monitoring data points with at least 11 data points not below detection.  So what happens if one of the three years has no uncensored values?  So all the non-detects occur in a year.  Well, now you only really have two years' worth of data.  You may have 36 data points, but one whole year is useless.  So if this happens in the first or the last year, the year can be dropped from consideration.  But at that point, the data falls below minimum data requirements.
                  But what happens if this is in the middle year?  So you've got data at the beginning and data at the end and nothing in the middle.  And if you think that's weird, you can go look.  There are some datasets in there that have whole years' worth of no detects in the middle of a period of three to five to seven years.  And the model happily goes through and pitstop.
                  My belief is that this likely also indicates a site with data below minimums.  And note that to ignore this situation and fit SEAWAVE-QEX model is to later find residual by time plots showing heterogeneity of residuals over time.  So it's going to show up in the diagnostics.  This suggests that the minimum 30 percent detection rate kind of should apply to every year being modeled, not just the whole kind of period of record.  And some of this is discussed in Section 6.2.1 of the SOP.  So there is some recognition within EPA that this can happen, and it should be looked at carefully.
                  Notice that the SEAWAVE-QEX SOP suggests a more stringent minimal dataset.  In Section 6.1 of the SOP, "Providing Advice on Handling Poorly Fitted Data," one finds the following: "If more than three years of data are being analyzed, do at least the first and/or last year of data individually fulfill the SEAWAVE-QEX requirements of minimum samples and detection rates?  If the first and last year of data being analyzed does not fulfill the SEAWAVE-QEX requirements, then remove any non-compliant first or last years from the analysis.  The first and last year of the datasets are anchor years for the model, so it's important for them to meet the minimum requirements, if not having higher detection frequencies, while maintaining a full dataset that meets minimal requirement."  I like that.  I fully think that's a great recommendation.
                  Another issue not addressed in the minimal dataset conditions is that detections should offer some indication of seasonal pattern.  In Section 6.1 of the SOP, the question is asked "Do the chemical detections appear to be random with little or no seasonal pattern?"  Clearly where the detections are found along the timeline is important to the success of the model timing.  So two non-detects or detects, three non-detects and detect, two non-detects is not going to be as useful as three non-detects, four detects, and four non-detects in a 12 dataset.  So the timing is something that needs to be looked at.  I don't know if any of the diagnostics really help you look at that, except maybe the first one, the summary of the fit.
                  The white paper and the SOP both mention alternating years of analysis and the benefit of using in season monitoring data for sites.  But it's not clear how a partial sampling season is to be defined.  The SOP mentions a partial sampling season as one having less than or equal to six months and suggests using best judgement in the designation of the season.  It would seem important to the selection of the best seasonal wave form for there to be non-detects at the beginning and at the end of the sampling season.  But how many should there be on either side I think needs to be determined.
                  With 12 data points per season and only three to four detects, say, located in the middle of the season, that leaves about four non-detects at the beginning and four at the end, if it's balanced.  Well, what happens if you only have one non-detect at the beginning and all the rest of the non-detects are at the end or the opposite?  I think that may have an impact on how the models fit.  But in my mind, I don't see how that's going to work.  I think you're going to have to play around with that to see what that means.
                  The point is that it seems clear that the spread of monitoring data across the use season within a year, and across all years, and the location of detects within and among years is critical to obtaining acceptable results from SEAWAVE-QEX model.  And I think this comment also ties into the last comment I made about the extra datapoint.  Sometimes putting one more detect in the middle of the season is going to make a big difference in how well it estimates the maximum or tightens up on the confidence interval.  That's my comment.
                  DR. ROBERT CHAPIN:  I'm going to break with tradition and ask do you want to ask any questions about that?  I'm not sure I followed it all.
                  DR. KENNETH PORTIER:  I'm looking at Dr. Hartless.  She's my anchor.  And when she's kind of like doing this, I'm like okay.  I've got it.  If she does this, I have to try over.
                  DR. ROBERT CHAPIN:  If Dr. Hartless is happy, I'm happy.  All right.  Dr. Miglino?
                  DR. ROCHELLE BOHATY:  Just one clarifying question.  I thought I heard in there that you thought that we should have these minimum requirements for each year of data.  But then you read the quote from the paper that said just in anchor years, so some clarification there would be helpful.
                  DR. KENNETH PORTIER:  That quote was from your SOP, and kind of the SOP is giving some advice, and I think it's very good advice, to look at your data and say, if you don't have an adequate year at the beginning and a year at the end that anchors your fit, you're going to have problems.  You don't even have to look too hard.  You're going to have to adjust your period of record, maybe eliminating that first year if it only has one or two detects and they're at the end of the season.  It's not going to add to the model fit, and it may actually detract from the model fit.  But that's not my rec.  I was just quoting from your SOP that that works.
                  I like the idea of having it every year, but I think it may be unreasonable to hold to that tightly.  That's maybe a stretch goal.  I don't think it's a requirement.  Because I think the model works, even when you have a little bit of data in the middle.  I get a little nervous when there's a whole year of no-detects in the middle of five years, and you wonder what went wrong and is that real data or are those really indication of something.  And I don't know what happens if you pull that year out and just stick the years back together and assume that year never happened.  2009, nobody wants to think about that.
                  DR. ROCHELLE BOHATY:  Thank you.  That was helpful.
                  DR. ROBERT CHAPIN:  All right.  Now.
                  DR. ANDREW MIGLINO:  So I'm going to take this from the perspective of a modeler who's happy that he has two data points to work with, and clearly we have a lot more here.  So I think you've done a very good job in the white paper of outlining minimum data requirements.  Those are more stringent then what's in Vecchia 2018, which just says six points with 30 percent detect.  So again, I think much better perspective.  There's one quote in Vecchia, though, that I think should be kept in mind.  Let me make sure I have it.  "Model verification can be especially difficult when a large percentage of data, more than 50 percent, are censored," which is clearly a higher bar.
                  I'm just thinking of myself fitting this model, looking at those seasonal wave patterns and seeing 50 percent of the data as non-detects with made up concentrations that I'm now saying, "Oh, they fit in there."  And of course they do because they're supposed to fit in there when I've made up the data to fit.  So from a modeling perspective, I'm not saying we need to go up to 50 percent.  I'm just saying there needs to be some kind of bright line in this SOP that says, "Here's what you do when you have 30 percent detect.  Here's how you handle this.  Here's the important features to look at in these more extreme cases where data is low."
                  I think I had one other thing in here.  The other thing that I had been thinking of -- and I don't know if it's possible.  I don't think it is.  I suspect it isn't -- is everything we've been talking about is tied to calendar years.  And if we can adjust those to maybe incorporate the very beginning of a season through  a year from that season, you may be able to include more data.  I don't know.  I haven't looked at the data that deeply.  But you may be able to kind of throw out the first four months of the year that had nothing, and then suddenly start picking up data rich years in future years.  And that may be an opportunity.  Again, I don't know if the way that the model is coded and the way that it's developed ties it to a true calendar year.  But water years, farm years, whatever you want to call it may be more fruitful for larger data sets.  I think that's all I had.
                  DR. ROBERT CHAPIN:  Mr. Councell.
                  MR. TERRY COUNCELL:  I really don't have anything to add to the discussion other than to reiterate what Dr. Nowell said earlier -- that for many of the compounds, finding 30 percent detects that are going to make it to that tier four assessment, you might find trouble finding that many detects in there.  That's just part of the monitoring and how frequently these pesticides of concern show up.  Other than that, everybody else -- I agree with their comments.
                  DR. ROBERT CHAPIN:  Great.  Dr. Yang?
                  DR. RAYMOND YANG:  My comments are for mid- to long-term for your benefit.  May we have Dr. Ken Portier's first slide this morning with the model?  I need that because I need to explain what are my thoughts.
                  DR. ROBERT CHAPIN:  The answer is yeah, in a minute.  Sure, go ahead.  Ray, we're going to get a comment from Cliff.
                  DR. CLIFFORD WEISEL:  Two things, one is the SOP I thought was very, very good.  I found it very helpful.  It helped me understand really what's going on.  That said that SOP was put together with probably very clear set of examples that you could use that SOP.  I've heard several times that the most difficult thing about using this is interpreting those plots, diagnostic plot.  So my advice is, if you're going to do this, set up a training program for who you think might be using it because my feeling is, once this gets out there, there will be people that don't necessarily have the background that you do that are going to attempt it.  And it could end up getting into a point that it's not being used properly, and that will bring the program down a step.
                  DR. RAYMOND YANG:  Ready for me?  My explanation also will tell you why I'm so obsessed with model, that you have to have the model.  These are the models.  What I will do, I will ask you to look at the right panel, that particular equation, but also consult your left panel, the explanation of the terms.  I'm going to talk this way, looking at -- Tim gave me three pages of notes and so on.
                  So this is the SEAWAVE-QEX model.  If you look at the model, look at the left-hand side.  Forget about log.  For my purpose, don't worry about log.  C as a function of T, that is the concentration of pesticide of interest.  What is the concentration?
                  Now, that concentration is dependent on the righthand side.  Now, let's look at the righthand side.  B0, B1, and so on and so forth, all this beta, forget about that because, at the risk of offending statisticians here, these are all flux factors.  That's the reason why component statistician or mathematician could fit any curve.  So forget about all the beta.  Forget about first term and last term.  Last term is the arrow.  The arrow is dependent on all the rest of the terms.
                  So now you have four terms.  So w as a function of t, that is seasonal wave.  And what is A sub mt?  A sub mt is dependent on MTFA, which is mid-term anomaly.  So it's a variable.  And AST is short-term anomaly.  So these are the variables.  And T minus tM is also a variable because what are they?  t is the time reported --
                  DR. ROBERT CHAPIN:  Ray, excuse me.  Can you get to your suggestion?
                  DR. RAYMOND YANG:  Yeah.  I have to do this explanation.  Otherwise, you don't know what I'm doing.  T is the time recorded in decimal years.  And Tn is the midpoint of the time interval being analyzed.  So these four terms will determine your concentration.  It's very important, okay?
                  And when I look at this, because all of these things are variables, therefore, over the years, you're going to have a probability distribution.  And therefore, this morning I mentioned that you need to consider Bayesian approach and multichain Monte Carlo because Bayesian approach has to take care of all of your available data, including some of the data right now you rejected.  And there was one public comment -- written comment, Crop Life America specifically concerned about large unused data.  These unused data could be used in the Bayesian approach as posterior distribution formulation.
                  So what you know right now is prior.  When you have new information added, you formulate a posterior distribution for your parameters.  Now, where does multichain Monte Carlo come in?  It assesses all the probability distribution, taking into consideration variability and uncertainty at the same time.  And it also takes into consideration the covariate of interactions among parameters.  If you use those two in combination, then you don't have to worry about I only have three years of data -- I only have 12 samples in a year and so on because you can scientifically, validly, using Monte Carlo simulation to sample from the probability distribution to reach 1,000 sets of data if you want to, a million set.  So this is very important for you to consider.  Thank you.
                  DR. ROBERT CHAPIN:  Thank you, Dr. Yang.   Other comments on -- go for it.
                  DR. VERONICA BERROCAL:  So I want to respond to Dr. Yang.  I am actually a Bayesian statistician, and I do appreciate you sticking out for Bayesian statistics, typically doesn't happen in more -- I don't know -- outside of academia.  Typically, it's hard to convince people of using Bayesian methods.  I do agree with you that there is a lot of uncertainty in some of the data sources that go into that regression equation.  But I do want to acknowledge that EPA has been doing some sort of Monte Carlo simulations when they generated those concentration -- those chemograph.  So the ideas are there.  Maybe they're not doing a fully Bayesian modeling approach.  But they are simulating daily chemograph by changing the values of the residual ones.  So you could increase the uncertainty by also simulating the parameters from the posterior distribution, which would be accounting for all sorts of uncertainty.  But I don't know if this is going to be too much of a big ask, given that we already mentioned that maybe SEAWAVE-QEX is already hard to implement as it is for people that are not experts with the model.  So in principle, I agree with you.  I just don't know whether it's feasible in practice.
                  DR. RAYMOND YANG:  Well, we use this particular model as a basis for this discussion.  This has been going on since 2007.  At the beginning, it's always difficult.  I'll tell you right now.  If you get used to what I'm saying in terms of applying Bayesian approach and using multichain Monte Carlo, you're going to get used to it, just like you get used to a SEAWAVE-QEX.
                  DR. ROBERT CHAPIN:  Okay.  All right.  Anyone -- any other comments from the panel?
                  DR. THOMAS POTTER:  I just wanted to come back to the idea -- or bring forth the idea that actual use patterns of active ingredients can inform a lot in terms of what your data looks like and what you're phased with, whether you have sporadic use or there's consistent use from year to year.  And I think from a risk assessment perspective, you'll always want to have that in mind as you go in and start exploring the data.  And that's going to guide, I think, a lot of your thinking.
                  DR. ROBERT CHAPIN:  Comments from any other panel members?  Okay.  And we'll turn to our EPA colleagues and ask if you have clarifying questions for us.
                  DR. ROCHELLE BOHATY:  We're good.  Thank you.
                  DR. ROBERT CHAPIN:  Life is good.  All right.  So it's 2:15.  What I'd like to do is we'll keep working up until 2:30, and then, wherever we are in responding to the next charge question, we'll stop and take a ten-minute break and then come back.  Okay.  So question two, 2(a).

CHARGE QUESTION 2

                  DR. ROCHELLE BOHATY:  Okay.  Question two, while SEAWAVE-QEX provides a way to estimate daily pesticide concentrations from non-daily surface water monitoring data, for many pesticides there are not sufficient monitoring data to use SEAWAVE-QEX.  This is because the data are too highly censored or there are not enough samples per year or across years.  Therefore, EPA would like to have an alternative approach to estimate drinking water concentrations from non-daily pesticide surface water monitoring data.
                  Previous Scientific Advisory Panels supported the development of sampling bias factors to derive bounds on pesticide concentrations in surface water.  EPA used SEAWAVE-QEX to develop daily pesticide chemographs from infrequent surface water monitoring data.  These chemographs were used to generate short-term sampling bias factors for acute exposure durations of concern, utilizing the methods supported by past Scientific Advisory Panels for sites with varying attributes across the contiguous United States.
                  Building upon that previous work, EPA evaluated different sampling strategies and imputation techniques to develop short-term sampling bias factors and concluded that using a random sampling strategy with a log-linear imputation is suitable for deriving short-term sampling bias factors.  In addition, EPA developed a new method for developing long-term sampling bias factors for chronic and cancer exposure durations of concern.  EPA concluded that for pesticides with chronic and/or cancer endpoints, that as few as four samples per year can be used to estimate a 365-day average concentration for use in drinking water assessments.

CHARGE QUESTION 2 (a)

                  2(a), please comment on the use of SEAWAVE-QEX estimated daily pesticide concentrations, including the underlying data requirements, to expand the data available to derive pesticide-specific sampling bias factors.  Please comment on EPA's optimization of the short-term sampling bias factors, derived using a random sampling strategy with log-linear imputation, to estimate the range of potential concentrations not measured between sampling events.  Please also comment on the long-term sampling bias factor approach.
                  DR. ROBERT CHAPIN:  Now both short-term and long-term comments focus here.  Dr. Berrocal?
                  DR. VERONICA BERROCAL:  Yes.  So I'm going to anticipate that this is going to be a long response.  I will not talk so much on 2(b), but you have to listen to my voice for this one.  So first off, I would like to state that EPA's decision to derive sampling bias factor for both short-term and long-term is responsive to recommendations made in previous SAP meetings, which invited EPA to make more use of migrant data rather than model data for drinking water assessment.  So in light of that, I support EPA's effort in developing short- and long-term bias sampling factor.
                  But both short- and long-term bias sampling factor are derived by computing a ratio where the numerator is a summary statistics of the daily pesticide concentration data representing a short-term or a long-term pesticide concentration obtained by using the SEAWAVE-QEX model.  And the denominator is the fifth percentile of a summary statistics obtained using a different declaration function, not the one in the SEAWAVE-QEX model, which is used to fill in missing data in a subsample series.  Given definition of the sampling bias factors, there are many components that affect the magnitude and the distribution of the sampling bias factor.
                  Looking to each of them, first, I would like to note that, from the white paper, it was not clear how the bootstrap data was sampled using the varying sampling strategy.  The white paper seemed to indicate a procedure that would include first sampling the first date for the daily concentration data and then perform a sequential sampling of all the other data with constraints to fall within a certain sampling interval, either seven, 14, 28 days and so forth.  However, yesterday's presentation seemed to indicate that instead the sampling procedure used was more of a systematic sample where just the first day was selected at random from a set of possible dates.  And then all the other dates were derived systematically every seven, 14, 21 days and so forth.  So this was not clear to me.  I am still very confused of what strategy did you use.
                  Although it might seem very pedantic to the extent where sampling procedure was used, the devil is in the details here.  And the type of sampling scheme used in conjunction with the form of interpolation function used to in fill the bootstrap data will influence the magnitude of the summary statistics that go in the denominator for the sampling bias factor calculation.  Specifically, my main concern initially regarded mostly the fact that the bootstrap data was interpolated with a function that was either a step or a lean or a log leaner that did not allow for the imputed daily pesticide concentration to be higher than any of the sample data.
                  Again, even though this might seem quite irrelevant, it does matter since, with the current choices, the daily interpolated pesticide concentration would lead to summary statistics that, one, will always be smaller than the maximum volume average computed just using the seven day and the 14 day bootstrap data in the case where the sampling frequency is -- in the case of a 20 day volume average.  And two, it will always be smaller or equal to the summary statistic derived using the bootstrap data if the step-wise function is used to infill the missing data.  So the denominator, the choice of sampling, the choice of interpolation function that you're using is going to affect the magnitude of the summary statistic that goes in the denominator of the sampling bias factor.
                  The repetition of the bootstrapping procedure multiple times does allow for variability in the summary statistics that gets generated from the interpolated data.  However, as a result of the method used for imputation, I believe that the short-term sampling bias factor might tend to be larger more over than it would be if other interpolation functions were used, in particular, interpolation functions that would allow for the imputed data to be larger than what is sampled.  In light of that, the tendency of short, small sampling bias factor to have smaller, medium, and more concentrated distribution as the average in time increases or as the sampling frequency decreases is expected to me.  Similarly, it is expected that the histogram of the short sampling bias factor did show an extremely wide skewed distribution.
                  Since the sampling bias factors are used to derive an upper level prediction limit, the fact that the sampling bias factor might tend to be larger than what they should actually be -- it would just translate in higher upper level prediction limits, which might not be an issue as we will be averaging on the protective side.  The fact that there was no substantial difference between a random sampling strategy versus a stratified sampling strategy is no surprise to me mostly because of a point I already raise, which is that the imputed, interpolated data will never be larger than the bootstrap data using the chosen interpolating functions.  Additionally, given that the stratified sample strategy is a minimal of every seven or every14 days, the sample pesticide concentration varies would very likely be independent.  The fact that the log leaning interpolation works better than the linear function or stabilized function is also surprising, given that the log linear function seems to be closer to what is expected behavior of daily pesticide concentration, at least according to the SEAWAVE-QEX model.  Okay.  We're getting close to the end.
                  For the long-term sampling bias factor approach, no biometric form was used to infill the bootstrap data.  I could not find information in the white paper about the distribution of the long-term sampling bias factor.  My only concern is, with the use of the formula for the sum of error of the mean used to compute the average -- for the sum of error of the mean and raw pesticide concentration.  While it is true that the sample data is very likely to be independent since you're sampling four, six, eight, or ten points at maximum, the true daily pesticide concentration data is correlated.  This means that the standard error used to compute the upper limit of the 90 percent prediction interval for the mean is likely too small, and it does not capture the sampling variability in the estimated annual mean pesticide concentration.  As a result, this means that the upper limit of the 90 percent prediction intervals are smaller than they should be, making the long-term sampling bias factor larger than they should be.
                  Again, as for the short-term sampling bias factor, larger long-term sampling bias factor will add on the protective side.  That means there will be a higher titer error or higher false positive, as it was mentioned yesterday during a public comment.  However, personally, I don't find this to be an issue given that the sampling bias factors are usually mostly for screening and moving from Tier 3 to Tier 4.
                  Finally, as I mentioned, the sampling bias factor is made of denominator and numerator.  For the use of SEAWAVE-QEX for the numerator in the sampling bias factor, both in the short-term and long-term, I do believe that, given that the SEAWAVE-QEX performs quite well in characterizing the distribution of both short-term and long-term pesticide exposure when the data is quite dense and it satisfies the criteria, at least in the Vecchia paper, I don't see any problem in using SEAWAVE-QEX to compute the numerator term in the bias sampling factor.
                  DR. KENNETH PORTIER:  I wish I had this better written.  Now, you're going to see where I start to fall apart here because these are kind of a bunch of thoughts, and I'm not sure I have them all properly organized.  So on the optimization of the short-term SBS, Section 4.4. describes a simulation study which examined the impact of four factors: exposure duration, sampling frequency, sampling strategy, and interpolation methods.  On the ability of SEAWAVE-QEX chemographs, that is time series of daily chemical concentrations to produce short-term one day, four-day, 21 day rolling average maxima SBFs with acceptable properties.
                  So just right off the bat, there's a lot going on in this simulation.  You've got four factors.  Plus you've got three rolling actions.  This is why it kind of gets complicated to make a coherent statement on stuff.
                  So acceptable is operationalized by looking for combinations of these factors that minimized the fiftieth percentile and ninetieth percentile of the root mean square errors from the simulations.  So a general linear mixed model assuming log normal error was used to estimate these data across sites and years and in particular to identify the best interpolation method and sampling methodology to be used in further development of SBFs.  And this is all in the supplemental file, USGS sampling bias factor evaluation, short-term SBF analysis.
                  While it's not clear that a log normal distribution is appropriate for the distribution of the median RMSE, I wasn't quite sure that that's the right distribution.  It's probably acceptable for the ninetieth percentile of the RMSE.  One's an extreme value, and one's a central location parameter.  The central location parameter is more likely to have a normal distribution or something like a normal or T distribution; whereas, an upper extreme, like a ninetieth percentile, is likely to have a skewed log normal distribution.
                  So instead of -- in addition, instead of separating out year and site by year effects, it might have been better to think in terms of a year within site covariance term.  And I'm going to talk about why this is.  And I made a note to myself.  Boy, if I had time, I would go into the CSV file that they gave us and run some QQ plots to see if the distributions did look lognormal.  It's one of those things that you just run out of time to do.
                  So what should the analysis model be for this simulation experiment?  With 15 realizations per condition for every effect -- for this condition, of course every effect will be significant.  This is an experiment with a lot of residual sample size.  So statistically testing for effects, everything's going to be P of 0.0001.  So you don't kind of want to be looking at statistical tech -- you want to be looking at the size of the estimates of the effect and see where there's big effects versus small effects.
                  There are 108 sites, three sampling protocols, three interpolation methods, and two sampling intervals.  Analysis was done by sampling interval.  So you only have 108 sites, three sampling protocols, and three interpolation methods.  By the way, another aside, the SASS code has no comment, nada.  This is a no-no.  You've got to at least add comments if you want us to look at this stuff.
                  So the residual is the variation among realizations within site year by sampling by interpolation methods for each sampling interval.  One can think of year within site as a random effect, and site as a random effect.  So the random statement for site -- I would have thought the random statement for site should have an identity structure.  But the random statement for year within site have something like a correlation structure, like an AR1 or a CS structure.
                  So your analysis model is a little too simple for the experiment.  The model assumes random variation among realizations within site year is constant across the site.  But you don't show any statistics to convince me that that's the right assumption to make.  So I think having a slightly more flexible correlation structure would have helped.  And doing the analysis this way give you some idea of how important is site to site variability and how important is year within site variability and how variability is year within site variability is it for these situations, like interpolation.
                  So the white paper concludes on page 66 the results suggested the SBFs developed with estimated daily concentrations from SEAWAVE-QEX compared well to SBFs developed from measure daily data.  In addition, the analysis lead to the decision to use the log linear interpolation method to develop site specific SBFs.  Both random sampling and stratified sampling methodologies could be used because there was, in a sense, very little effect difference between those two things.
                  Another aside, there's an error in the short-term SBF, SBFA sites Chem1.CSV file.  You get halfway down, and it's summary data instead of the individual data.  I was looking at that file.  So when one scrolls down the 14-day sample interval, one sees the summary statistics from SBFA Summary Chem1.CSV instead of the by year summaries seen in the seven-day sample interval.  So you gave us good data to look at variability in the seven-day sampling interval, but somebody copied the wrong data to the 14-day interval.  So I can't comment on that one.
                  I'm sorry this is very technical.  But the white paper also included on page 66 "In order to summarize the SBFs for each site, EPA employed a two-step summarization process that calculated the median SBF across chemographs for each year and then calculated the median SBFs across years."  One can see the need for this two-step process by viewing the wide range of estimates produced across the hundred simulated samples within each year.  And that's where you need to go to that summary table that has problems with the 14 days.  When you look through it, you see, oh, yeah.  You need to look at medians by medians.
                  In that supplemental file, you see that for most of this two-stage estimate demonstrates little year to year variability in medians.  But for some sites, for example CSIMS0242354750 -- and I'll put that in the report -- there still remains much year to year variability with a range of 100-fold for the one-day SBFs, fivefold for the P50 -- I'm sorry, the SBF P50, and 5X for the 90-day SBF P50.  So there's still sites that have a lot of variability in here.
                  The approach was also applied to static systems in the monitoring program.  The white paper indicated that each static fit produced issues, for example, broad wave systems, positive and negative correlations with MTFA, empirical correlograms lower than fitted correlogram functions, et cetera, that were captured during examination of the diagnostics plot.  So this experiment didn't work for the static systems as well.
                  So each of these sites have a lot of observations where the high fraction that detects for most of these, the estimated max is very close to the observed max, as should happen.  I think Dr. Berrocal mentioned that.  It's also clear from the diagnostic plots provided in the supplemental files that fitting and appropriate seasonal wave model is at the heart of the estimation issues.  And part of the difficulty is that these static systems have active management and results that active management -- and the results show that active management is evident in the diagnostics plots.
                  So I concluded, looking at this, that the study and the associated analysis looks to be properly done, and conclusions are reasonable, given the results of the analysis.  And it's clear from the comments that each and every fitting of the SEAWAVE-QEX needs to be examined for proper fit.  This is clearly not an apply-it-and-move-on process.
                  So on the long-term sampling bias factors, which is the second part of the question, right?  So in Section 4.2, the white paper states that the creation of long-term SBFs is not that different from the short-term SBFs and that SBF approach is likely to produce better results when applied to sparse monitoring data because, one, the influence of missing data -- and this is a quote.  "The influence of missing data is believed to have less of an impact on longer term concentrations that shorter-term concentrations." And two, "Interpolation between sampling concentrations has minimal impact on estimating a long-term average."
                  While this may be true, it's likely that when monitoring samples are taken during the use season -- when monitoring samples are taken during the use season can have a large impact on the average, which in turn will have a large impact on an SBF estimated long-term average annual concentration.  So we're back to the same question of where are the detects in the year, and that has a big impact, especially on the long-term estimation.
                  So on the experiment described in Section 4.5 where it produced sparse monitoring datasets with four, six, eight, or ten samples per set, using random or stratified protocols.  And they looked at the fifth percentile of the 10,000, ninetieth percentile upper confidence limit for the mean.  EPA concluded that results suggested SBFs developed with estimated daily chemo -- daily concentrations derived from (ifible) compared well to SBFs developed from measured data.
                  So I wasn't -- in the writeup it was confusing.  You had to keep reading it to say, well, where was SEAWAVE used, and why did you make this statement?  Because you've got monitoring data which has a little bit of missing values.  And you use SEAWAVE to fill that in, and then you went back and looked at -- compared those two again.
                  Bob, do you want to take a break at this point and let me finish summarizing or move on and I'll come back at the end?
                  DR. ROBERT CHAPIN:  I was thinking that taking a break might be good.  Being able to summarize --
                  DR. KENNETH PORTIER:  That's what I'm trying to do.
                  DR. ROBERT CHAPIN:  Summarizing your comments and suggests rather than summarizing what they did will be good.  So let's take a ten minutes.  Look at your watch.  Ten minutes from now we'll be back.  Thank you.

                                             [BREAK]

                  DR. ROBERT CHAPIN:  All right.  Here we go.  Okay.  Let me see.  Ken, shall we go to somebody else?
                  DR. KENNETH PORTIER:  I'm going to summarize real quickly.  I wrote three summary statements.
                  DR. ROBERT CHAPIN:  Awesome.  I love it when you talk like that.  Let me just make sure that our EPA friends are ready to receive the words of wisdom.
                  DR. KENNETH PORTIER:  They're looking at the error file that I suggested they look at.  So in summary, I think the study was performed properly, but a slightly different statistical analysis model should be considered.  Secondly, the conclusions from the analysis are probably correct, but they need a better, read clearer, discuss and summarization.  I think part of my problem is I'm trying to understand what they actually concluded.  And most of the time, I'm saying, okay, if I read this correctly, they're right.  But I'm not always sure I'm reading it correctly.  And the third thing is there does not seem to be that much difference in the findings between the short-term SBFs and long-term SBF properties and performance.  So I think that's kind of my conclusion on this.
                  DR. ROBERT CHAPIN:  That's helpful.  Thank you.  Okay.  All right.  We're done with the statistical end of the table.  We'll move down to Dr. Potter.
                  DR. THOMAS POTTER:  I have found my risk assessment hat.  So let me get closer to the mic here.  As noted, just some statements of the obvious here, but sample bias factor has been a topic of discussion for a long time.  I feel like somewhat of an innocent stepping into this at this point since I haven't been part of that dialogue.  I will say I think from reading the tea leaves here that there is a consensus that they may be a valuable tool for risk assessment.  And some samples were shown, particularly in Tier 3, which seem perfectly logical and I think would be a valuable next step, if it's not already being implemented.
                  With that said, a number of folks have raised concerns about SBF favoring false positives over false negatives.  From a health perspective, or a health protective perspective, maybe that bias towards false positives seems reasonable in a sense of misclassifying or missing a health exposure that would be a real concern.  So again, that will be an ongoing concern I think by some.  But perhaps it's not an exceptionally of a concern.
                  And then it was observed, and has been observed, and I think sort of the motivation of this question here today is that sampling bias factors can be really uncertain, particularly when you have not much data.  So I look at SEAWAVE-QEX and, to some degree, that can solve the problem.  And indeed, looking at the way the model was applied, it all seems reasonable to me.  I think from a risk perspective this is a really neat way and elegant way to get at this particular parameter, the sampling bias factor.
                  Ultimately, I believe its best use as a tool for risk assessment is screening, and I'm getting the feeling that that's the perspective on the Agency.  We'll perhaps get to that later on in our discussion of, I think, it's Question 4 when we talk a little bit more about the whole risk assessment process.  But as a screening tool, it appears to be quite valuable.
                  My concern in the analysis that's gone on here, kind of again, some of it's going to be discussed later on.  But if we have enough data to calculate -- to use SEAWAVE-QEX to calculate the sample bias factor, I'm not sure what the game is, other than that you may be able to screen out selected sites for further analysis in Tier 4.  Ultimately, the real challenge, I think, with SBF is that we have problems that they don't regress well.  So it's hard to extrapolate across chemicals.  And I'm not sure there's a good solution for that.  We'll talk a little bit about that later on.
                  So I'll simply say, from a risk assessment perspective, I think sampling bias factors are a very powerful and useful tool.  You can screen your data, particularly at Tier 3.  And that can serve to minimize efforts or at least direct efforts in areas where you have a real water quality concern in terms of exposure.  And that'll optimize resources and hopefully get the job done.
                  DR. ROBERT CHAPIN:  Dr. Klaper?
                  DR. REBECCA KLAPER:  I think overall I was really impressed with how well the models ended up fitting the data and anticipated peak concentrations, which seems to be the goal -- is to figure out high concentrations in particular areas, not necessarily daily concentration.  So that the bioconcentration factor put that boundary on where you thought that the top limit would be and helped to give a fudge factor to the data that I think is necessary based on  concurring with comments from other panels that have happened -- so overall, I think it's a really good tool and that the sampling factor bias calculations are really useful and seem good.  The one problem that I think that I voiced before is that I think that it would be good to keep in mind about how climate change is causing extreme variability in systems and that when you go to calculate the upper end of pesticide concentrations and variability over time that we haven't necessarily seen the variability evaluated as far as estimating how that will impact the high concentration and this bias factor analysis.  But otherwise, I was very impressed with how well the model stuck to the data.
                  DR. ROBERT CHAPIN:  All right.  And Dr. Sadd?
                  DR. JAMES SADD:  I agree with the panel that you've demonstrated the SBFs have value in some cases for screening at Tier 3.  And I also agree with Dr. Berrocal's explanation in the optimization analysis that your decision to use a deterministic interpolation method limits the ability of the model to identify extreme values.  I know that you recognize that, and you also recognize that the range of values for SBFs are wide.  And you're concerned about that and the additional uncertainty in the data.
                  That would be particularly true if you were going to apply it to non-flowing systems or small rivers or those with rapid fluctuations of discharge.  So if additional testing analysis and new datasets as possible, which it seems like it might be, then it might help resolve some of these reasons and explain the wide range of SBF values in some cases and maybe offer the opportunity to conduct testing with different pesticides.  I don't have anything else to add regarding the long-term SBFs.
                  DR. ROBERT CHAPIN:  Excellent.  Thank you.  Comments from other members of the panel on these SBFs for short-term and long-term approaches?  All right.  I turn to my EPA colleagues and ask if you want to ask clarifying questions of the panel.
                  DR. ROCHELLE BOHATY:  I think we're good.  Thank you.
                  DR. ROBERT CHAPIN:  You can have another third 20 seconds to discuss among yourselves if you need to.  We have a point of clarification.
                  DR. CHRISTINE HARTLESS:  Thank you.  I wanted to address some of the earlier comments made by Dr. Berrocal about whether or not the use of the 14 day and the seven-day sampling intervals and clarify that that pieces of that was for just evaluation of SEAWAVE itself.  We used a different sampling methodology for evaluation of the sampling bias factors.  So obviously in our white paper and/or the presentations, the distinction between those wasn't quite made clear enough.  So we will have to work to make sure that we can do a better job of clarifying those differences -- clarifying what we've done.
                  DR. ROBERT CHAPIN:  Great.
                  DR. XUYANG ZHANG:  Hi.  This is Xuyang Zhang here, I have a brief comment.
                  DR. ROBERT CHAPIN:  Go for it.
                  DR. XUYANG ZHANG:  I just want to second Dr. Portier's comment on a better statistical analysis on the results, especially on the site to site variability.  I was trying to explore optional reasons why the regression models did not work well.  And I think a better understanding of the site to site variability and the magnitude and the factors that are contributing to the differences on the outside would be instrumental in performing the regression analysis.  So I just think that, in the future, if EPA could do a little bit more in that regard, that might be helpful.  That's it.
                  DR. ROBERT CHAPIN:  All right.  Good.  Thank you very much.  So we're good around the table.  Anything else from the EPA folks on this question?  You're good?  Shall we move on to 2(b)?
CHARGE QUESTION 2(b)
                  DR. ROCHELLE BOHATY:  2(b), please discuss the strengths and weaknesses of developing site-specific sampling bias factors using a percentile, for example, the median across realizations and the median across years, from SEAWAVE-QEX estimated daily pesticide concentrations.  Describe the utility of this approach for use in pesticide drinking water assessments as highlighted in the attached drinking water assessment case studies.
                  DR. ROBERT CHAPIN:  Just because we haven't given him enough work to do already, we're going to start off with Dr. Portier.  All right.  So we're going to do strengths and weaknesses of site-specific SBFs.
                  DR. KENNETH PORTIER:  Pass.  No, actually I don't have a lot to say on this.  Using the medians of the medians makes practical sense, and I think it makes statistical sense.  Because of the way SBFs are conceptualized as a ratio of the true value divided by a statistic derived from subsampled monitoring realization, any statistic could be used in the denominator, and any statistic could be used to describe the kind of average effect across all the realizations.  And the median makes a lot of sense, because of all the percentile estimates, the median has some of the better statistical properties.  It's more likely to be symmetrically distributed and skewed.  It's easier to estimate its P50 value.  So that's one of the strengths of what you're doing.
                  And I was sitting there saying, "A weakness of the use of the median?"  And I'm thinking I can't think any reasons why I'd not use the median for this kind of thing.  Again, the whole point is, to a certain extent, any statistic would work.  You want a statistic that has good properties, and I think the median's there.
                  And I may have been one of those that recommended that in 2012, say "Why don't you use the median?"  I didn't go back and look at it, but he may have been using an average.  I said, "Oh, no.  That's too --"  For example, the average is very susceptible to the extreme values.  So if you get one bad value, the average could bounce all around, but the median doesn't care.
                  DR. ROBERT CHAPIN:  There's statistics that don't care?
                  DR. KENNETH PORTIER:  So I kind of say, in the end, the best justification is that it works when tested with best available data, which is what's presented in the white paper.  So one of the argues for why it works is because it works.  It has good properties.
                  "Describe the utility of this approach for use in pesticide drinking water assessments as highlighted in the drinking water assessment case studies."  And I think I'm going to punt on that one and let the risk assessors and others address that.  And Veronica here's going to be really upset with me because she really wanted me to talk for 35 minutes, and now she's got to say something.
                  DR. VERONICA BERROCAL:  That is true.  I didn't have much to say on this Question 2(b).  So that's why I asked that he goes first, and I was hoping he would talk forever.  But once I hope then he doesn't do it.  So yes, I don't have anything else to add besides what Dr. Portier already said.
                  DR. ROBERT CHAPIN:  I love it when you talk like that.  Let's see.  So that leads us up to Dr. Sadd.
                  DR. JAMES SADD:  So the site-specific sample bias factor can be valuable in some cases to inform an existing sampling program as it goes forward to make it more effective or in deciding a sampling plan at sites where they are sufficiently comparable to the one where the SBF was developed.  EPA recognizes that SBFs at a given site can vary substantially from year to year.  And they made the choice to select the median as the SBF specific to that site.  That's a reasonable decision to address a difficult problem.  But I can see that EPA also recognizes that that's not the optimal long-term solution.
                  Another potential negative aspect to site-specific SBFs is the reasonable expectation that in some places there will be environmental changes associated with changes in land use within the watershed or perhaps even changing climate conditions that might add to the year to year variation and eventually render a site-specific SBF less and less effective over time.  And I recommend the EPA consider a further analysis to better understand the year to year variability and the reasons for it.
                  DR. ROBERT CHAPIN:  You think that ought to be short-term, mid-term, or long-term reanalysis?
                  DR. JAMES SADD:  Well, I think mid- and long-term probably because that's where the changes really would be most pressing or most effective.
                  DR. ROBERT CHAPIN:  Good.  Excellent.  Thank you.  All right.  And Dr. P?
                  DR. THOMAS POTTER:  I guess that's me.  Thomas Potter.  Well, I'll echo much of what's been said already.  I think that it's a reasonable approach.  It's actually a very conservative approach compared to the alternatives, which are applying across sites and in other factors.  We certainly saw the need for it in some of the analysis that was done, which showed there are large variations between some of the sites that were evaluated with the SEAWAVE-QEX data.  So yeah.
                  It seems reasonable.  I think your data analysis was excellent, and it's a tool that can be valuable.  Although, again, I'm concerned about how limited it might be if indeed you can't extrapolate between chemicals.  So again, we're going to get to that.  But I think that's really the discouraging part.  And again, maybe there's a path forward and we'll talk about that.
                  DR. ROBERT CHAPIN:  Okay.  Thank you very much.  Comments from other panel members on Question 2(b)?  Okay.  Seeing none, I'll look to my EPA colleagues and ask if you want any clarifying questions of us?
                  DR. ROCHELLE BOHATY:  I think we're good.  Thank you.
                  DR. ROBERT CHAPIN:  Cool.  Tick.  Okay.  Turn the page over.  Question 2(c).

CHARGE QUESTION 2(c)

                  DR. ROCHELLE BOHATY:  2(c), please comment on the utility of using the maximum short-term and the median long-term sampling bias factor for the four pesticides, that is atrazine, carbaryl, chlorpyrifos, and fipronil, to estimate upper-bound concentrations of other pesticides in surface water for which either sampling bias factors cannot be or have not been derived.  In addition to using these values as a screen, EPA proposes an option to select sampling bias factors for an individual pesticide, either atrazine, carbaryl, chlorpyrifos, or fipronil, based on other defining attributes, such as environmental fate properties, use profile, flow rate, basin size, waterbody type, and/or land use.  Please comment on what factors EPA should consider when selecting an alternative sampling bias factor for estimating upper-bound pesticide concentrations on a national or regional scale.
                  DR. ROBERT CHAPIN:  Another simple question.  Dr. Potter?
                  DR. THOMAS POTTER:  Yeah.  That's a tough one.  We've alluded to this several times, and of course you folks made, I think, a valuable presentation yesterday on this topic.  The challenge, again, sampling -- if you could extrapolate between chemicals, sampling bias factors would be an extremely powerful tool, rather than a moderately effective tool, from my perspective.  So I'm not sure there's an effective path forward on this.  But one thing I've been kind of thinking about, and perhaps there's some way to dig into this, is to look at the shapes of the waves because I think, to some degree, they could be governed by chemical properties -- sort of relates to the idea of field dissipation processes that are governing that wave shape in addition to hydrology.
                  So if there's an agreement in wave shape, you could look at some parameter associated with wave shape, such as amplitude or width if it's a relatively symmetric one -- width at half-height.  It's sort of analogous to what one does in a chromatographic analysis in chemistry.  You're looking at, from a quality perspective, analyzing peak shape to see if it fits the criteria.  And again, you can compare similarity across things that can be tailing factors that could be built in.
                  But perhaps that's a path forward is to really actually go back to the model and look at what the waves are and do some kind of comparison between peak shape to see if there's some uniformity there between certain types of chemicals.  And then therefore, there's seems to be some potential for extension of the SBF factors calculated for the four chemicals to others that would be of interest.  So I offer that as a thought.
                  What else do I have to say on this?  I had one other point.  No, I'll stay right there.  I think again, peak shape -- oh, I know what I wanted to say.  This probably is really going to dig things out.  But let me ask this question because it comes back to yesterday.  I thought I heard you say yesterday that you did a regression with terrestrial field dissipation half-lives.  Is that correct?  And I thought, wow, that's a great idea.
                  So I still think it's a great idea.  I believe you saw some relationship.  And it would stand to reason that terrestrial field dissipation might be an interesting parameter because, again, it's integrating a lot of processes that are going out there in the landscape.  I was thinking, yeah, finally, there's some real, possible, valuable use for TFD other than as a screening tool.  So I commend you on that and recommend that you look into that in more detail.
                  DR. ROBERT CHAPIN:  Excellent.  Thank you very much.  Dr. Sadd?
                  DR. JAMES SADD:  Using SBFs in this way would provide EPA with a potentially valuable tool, I agree, if you can pull it off.  But applying them to other pesticides introduces more uncertainty and potential bias because of differences in the properties of other pesticides and their behavior and fate and the environment as compared to the four pesticides for which SBFs have been carefully evaluated.  Using the other attributes as mentioned in the question to inform -- such as a substitution of -- has a potential to introduce even greater error and greater bias.  And their likely effects probably vary substantially, perhaps more than pesticide to pesticide properties variation.  And if they were going to be -- and some of them are going to be not quantitative, such as waterbody type and possibly some land use characteristics.
                  So this would require subjectivity in applying them in this context.  If EPA plans to use them in this way, I would just suggest that clear rules for how they would be used to inform a decision on SBF use should be developed and justified, preferably through testing.
                  DR. ROBERT CHAPIN:  Let's see.  Dr. Baffaut?
                  DR. CLAIRE BAFFAUT:  So I'm going to address the second part of this question.  I think here in a sense EPA is asking how the vulnerability of land could be introduced in the analysis and in the selection of the SBF.  And I carefully looked at the list of attributes listed in Table 11.2.  I couldn't find any that should not be there, so keep those.
                  But in addition to those, I could think of a few:  for example, the dominate hydrologic soil group of the land likely to be treated with a pesticide of interest, the similarities between land cover to which pesticide is applied -- that would be the percentage of area in the watershed, as well as the type and management of the land covered.  Try to be more specific in ag land, perhaps land and the crops that are relevant to the pesticides of interest -- the slope of the land likely to be treated with that pesticide.  The biochemical properties, we've talked about it -- mean annual temperature, just because it may affect product degradation; rainfall amount or intensity; shape of the watershed --  I think you had some of those variables that reflect that -- the presence of subsurface artificial drainage.  You do have the presence of a restrictive layer in that list, but the presence of subsurface artificial drainage, or any kind of drainage, even if it's surface drainage, would be important.
                  Then I would like to add that this question of land vulnerability -- there's been a lot of research I'd say in the last ten years, and several groups have come up with different indices.  So I'm going to list three that may be interesting.  The Cornell group, and that's Tamue Stenuis (phonetic), came up with a topographic index.  And they have different versions of it.  But it was directed at land likely to be saturated when it drains.  And it was in a setting that was characterized by permeable land or permeable soils on top of a restrictive layer.
                  In the Midwest, my group, we have come up with a conductivity claypan index.  When you compare with a topographic index, it's actually very similar.  And then NRCS came up recently with something called the Soil Vulnerability Index.  The interesting thing is all those three index use very similar soil properties.  And the other thing is those -- in the equations that you present in Section 11, the factors are additive.  All those indices that I'm talking about, the primers are -- or the indices are multiplicative.  So the different variables are multiplied by each other or divided.  But it's multiplication over fractions, not an addition.  And it has proven to be effective at predicting vulnerability.  And that's -- I've said everything I wanted to say.
                  DR. ROBERT CHAPIN:  All right then.  Dr. Green?
                  DR. TIMOTHY GREEN:  Thank you.  Those are great comments.  I'll be brief.  First, to reiterate what I said in 1(d), I think you can use scaling information of watershed area with the sample bias factors.  Tomorrow, I think in 3B we'll look at regarding quantitative watershed characteristics, which may support some of Dr. Baffaut's ideas in terms of how you would use watershed characteristics as inputs.  And then I didn't think of this, but thanks Dr. Potter for ideas about leveraging chemograph characteristics, so essentially using the response as another way of characterizing the different chemicals.  That's all I have.
                  DR. ROBERT CHAPIN:  You can just shuffle that microphone boom right over next to Dr. Nowell.
                  DR. LISA NOWELL:  I agree with the comments previously stated.  And I was thinking about the multiple regression.  I guess this is going to come up with a later question in three.  So EPA concluded at the end of the regression experiment that additional research was needed before the SBFs could be confidently predicted using watershed characteristics, and that seemed quite clear.  But it seemed like it was promising and that it'd be worth continuing to work on this.  So if you can't explain the variability of a same compound in different watersheds, it's going to be harder still to explain across different compounds.
                  So I was think that, as a goal, we could use the SBFs to try to understand more about the sources of variability.  And I think the regressions so far were oriented -- they were predictive models.  You were trying to use basic GIS data that you could get for any site.  So a lot of the factors that we know are going to affect those concentrations, like flow and sources use within the watershed, would be much harder or even impossible to get generically.  But there might be -- I think you could rationalize a need to develop -- to work on an explanatory model rather than a predictive model, just to try to get a handle on the factors that are affecting the variability and then use that maybe as a guide to try again with a predictive model.  And some of these more data intensive variables, like pesticide use or flow, to the extent -- or surrogates for these would be worth trying, and scale, of course, watershed scale.
                  Again, we've only got four chemicals.  Although, really I think you could count metolachlor as five.  I don't know if you have others.  But you used metolachlor in this as a way of evaluating your SBFs.  But it looks like you have data for metolachlor, as well as the other four, for both the NCWQR sites and presumably the USGS sites as well.  I don't know if you've done that yet.  But anyway, that would be five.  But that's still not really enough to develop a regression model among compounds.  So I'm hoping that you'll be able to continue the effort, expand the number of compounds that you can work with.  And then maybe a ration might be feasible.
                  DR. ROBERT CHAPIN:  All right.  And for this question, we get to end as we began.  Dr. Potter is down here as the last with his aquatic modeler hat on.  Would you like to add anything else to your earlier comments?
                  DR. THOMAS POTTER:  Well, I'm thinking that there's only one other thing that I would add, and it's not necessarily connected to my earlier comments.  But it was when I was listening to Dr. Baffaut talking about watershed properties.  One thing that would be very useful to look at is the relative distribution of base flow and storm flow.  I think those are key parameters related to chemical transport.  And there may be some watershed where you can get that information readily available.  There's a very nice paper that was out, again by -- I keep on promoting a former colleague -- Dave Bosch, and I will give you the citation on that.  And they talk quite a bit about the importance of base flow and contaminate transport, stormflow, and how to separate those out.  I think that might be something that would be very useful as a predictive tool.
                  DR. ROBERT CHAPIN:  Comments from any untapped panel -- go ahead?
                  DR. LISA NOWELL:  I just forgot I just wanted to make one further comment.  I was thinking about the univariate relation between -- was it the terrestrial field dissipation?  And it looked to me like fipronil might be driving that because it has a notably lower SBF, and it's also by far the most persistent.  So again, more compounds with  more varied properties will help with that.
                  DR. CLIFFORD WEISEL:  So I agree completely with everyone saying you need more compounds.  The question is how you get that data.  And I'm looking to my colleagues to see if this is at all something to think about when you talk about environmental fate.  There's a lot of work done -- we're not looking at the watershed but rather absorption of the pesticides into soils and how long they last, which is related, I guess, to the terrestrial -- that each company has to do it to understand it.  I wonder if that might be a source of data just to look at how different pesticides might go around and then combine that solubility information just to see whether that may extend some of the underlying principles because if you don't have enough data you can't do the complete statistical analysis.  So you have to meld the mechanistic with the statistical.  And then they'd just be another source of thinking.
                  DR. REBECCA KLAPER:  I'll just say that the compounds that do bind to soil more heavily still end up in the water column when they wash up on land.  But they still could behave differently as far as transport goes and such.  So it's apples and oranges.  It's very useful information.  It tells you the chemicals are different and can behave different in an aquatic environment.  But it's not easy to translate the octanol water coefficient to some behavior in a water system, unfortunately.  It would be nice if it did.
                  DR. ROBERT CHAPIN:  Other untapped, unheard comments from the panel on this question?  All right.  None yet?  Okay.  Anything that we can clarify for our EPA friends?
                  DR. ROCHELLE BOHATY:  Yes.  So I just want to make sure that I think I heard consensus on using the four chemicals as a screen, but there was some reluctance on identifying one of those individually for application to another pesticide.  Is that correct?  And if not, can you please clarify?
                  DR. ROBERT CHAPIN:  Where did that come from?  Jim, is that --?
                  DR. JAMES SADD:  I agree with what you just said.
                  DR. ROCHELLE BOHATY:  Is that the consensus?  Is there any --?
                  DR. REBECCA KLAPER:  I would agree with that, too, but it's very difficult to extrapolate from one to the other at this point.  It's very difficult to, right now, with these four chemicals and given the limitations of the variability in their chemical primers, to say that you could extrapolate from one of these to everything or everything with that category.  But it is good for the screening purposes and for the examples that you've given to demonstrate that they were given enough data.
                  DR. ROCHELLE BOHATY:  Great.  Thank you.
                  DR. ROBERT CHAPIN:  Okay.  Check.  D, 2(d).  Here we go.

CHARGE QUESTION 2(d)

                  DR. ROCHELLE BOHATY:  All right.  2(d), Rochelle here again.  In Chapter 4.6, EPA concludes that sampling bias factors are a reasonable tool for increasing the amount of available monitoring data that can be used as a quantitative measure of exposure in pesticide drinking water assessments beyond those meeting the SEAWAVE criteria.  Considering the answers to questions 2(a) through 2(c), please comment on this conclusion.
                  DR. ROBERT CHAPIN:  Okay.  So we have Dr. Sadd.
                  DR. JAMES SADD:  I'll just say briefly that I agree with this conclusion, and I agree with it with consideration of the limitations and the issues of concern that have been discussed earlier in the panel's answers to Questions 2(b) and 2(c).
                  DR. ROBERT CHAPIN:  Short and sweet.  Dr. Potter?
                  DR. THOMAS POTTER:  Yes.
                  DR. ROBERT CHAPIN:  Shorter and sweeter.  They're clinking glasses down at the end of the table.  Did we have any clarifying questions from the EPA?
                  DR. ROCHELLE BOHATY:  No, that was very clear.  Thank you.
                  DR. ROBERT CHAPIN:  So let me just see if any other panelists have comments on this question.  All right.  I think we've finally hit everybody's tolerance limit for long, involved answers.  All right.  So are we done with 2(d)?  No questions from you guys?  Do you have anything to follow up?
                  DR. ROCHELLE BOHATY:  No questions.  Thank you.
                  DR. ROBERT CHAPIN:  2(e).

CHARGE QUESTION 2(e)