All right.  We're going to have a case study now and we're going to have Dr. White from the EPA give us a walk-through of that pesticide with short-term exposure with considerations.
                  Katrina White.

DRINKING WATER ASSESSMENT CASE STUDY   -   A PESTICIDE WITH SHORT-TERM EXPOSURE CONSIDERATION

                  DR. KATRINA WHITE:  It helps if you turn the microphone on.
                  Good afternoon.  My name is Katrina White and I'm a senior scientist with the Environmental Fate and Effects Division.  I'll be talking with you today about the case studies.  Jessica Joyce and Chuck Peck also contributed to the case study work.
                  First, I wanted to go over an outline of our presentation.  I will start by talking about the purpose of the case studies and then discuss our approach and the methods used in the case studies.  Next, I will discuss the results and conclusions of the case study with short-term exposure considerations.  Jessica Joyce will then discuss the case study with long-term exposure considerations, and finally, Mrs. Joyce will also discuss the implications for drinking water assessments and lessons learned from the case studies.
                  Throughout the talks, for the short-term exposure considerations case study, we refer to the pesticide as Chem 1 and for the long-term exposure consideration case study, we refer to the pesticide as Chem 2.
                  We conducted the case studies to provide the panel with a practical example of how EPA conducts drinking water assessments using the framework and tiering process and to provide an example of the use of SEAWAVE-QEX in short-term sampling bias factors in our fighting drinking water assessments.  We also wanted to provide an understanding of the monitoring data available for analysis and how it can be used in risk assessment.  Finally, the case studies provide examples of the use of weight of evidence approach for assessing and integrating surface water monitoring data into drinking water assessments.
                  On the next slide, I will talk about how these case studies relate to the framework of conducting drinking water assessments.  Recall that Dr. Bohaty presented the framework for conducting drinking water assessments earlier today.  One of the charge questions relates to the clarity and organization of the framework and one reason the case studies are provided is to help understand the framework.
                  Recall that the framework presents a tiered process where at lower tiers you utilize simple inputs and conservative outputs and move to more detailed and realistic estimates of exposure at higher tiers.  As you move into higher tiers, the analysis becomes more regionally specific.
                  In the case studies, we focus on Tier 3 and Tier 4 where new methods are provided.  In Tier 3, we propose applying sampling bias factors calculated for four other pesticides to monitoring results without daily sampling to count for the temporal uncertainty in the data, that is, not having daily sampling frequency.
                  In Tier 4, we introduce the use of SEAWAVE-QEX to estimate daily concentrations from data with non-daily sampling.  This provides an exposure estimate for the site with enough data to run SEAWAVE-QEX and chemographs to calculate pesticide-specific sampling bias factors that can be applied to data that is not robust enough for running SEAWAVE-QEX.  Finally, a weight of evidence approach was developed to better understand the spatial relevance of our monitoring.
                  For the case studies, we selected one pesticide with acute toxicity endpoint of concern and another with a chronic and cancer toxicity endpoint of concern.  We selected pesticides that required refinements at higher tiers and pesticides that had a monitoring data set available where SEAWAVE-QEX and sampling bias factors could be applied.  We wanted the case studies to be illustrative of the process but not too complicated as there are a lot of materials supporting the SAP.
                  To simplify the case study, we selected a subset of use patterns to evaluate and focused on the tiers of the assessment where the new tools we present here today are utilized, Tier 3 and Tier 4.  In Tier 4 of the drinking water assessments, we focused on two regions.  The case studies do not represent complete drinking water assessments and while the case studies are mostly based on a true story, some items were altered for simplicity.
                  Now I will change gears and focus on the short-term exposure case study.  The first step in the framework is understanding the background as part of scoping for a pesticide to determine what will be needed for a drinking water assessment.
                  Chem 1 is an insecticide applied to residential areas, apples, citrus, and soybeans with most usage occurring in residential areas.  It has an acute drinking water level of concern of 34.5 micrograms per liter and previous drinking water assessments required higher tiered assessments.  Chem 1 has a lot of monitoring data available; however, data has non-daily sampling frequency and is unlikely to capture the peak concentration that occurred at the site.
                  In previous drinking water assessments, we characterized these monitoring data but could not rely on it quantitatively because it was not likely to capture those peak concentrations.  So today I'm going to walk you through how we can use SEAWAVE-QEX and sampling bias factors to quantify the uncertainty in this monitoring data, to use it quantitatively in the exposure assessment.  It is important to consider that while we have a lot of data, the majority of it is not robust enough that we can quantify the amount of uncertainty in the data.  We can only apply the tools we present today to about 25 percent of the monitoring data for Chem 1.
                  Based on the environmental fate properties, Chem 1 is moderately mobile and not persistent in water.  Scoping indicated that the need for at least a Tier 3 exposure assessment.  At this tier, modeling is refined with the pesticide and water calculator and monitoring data are adjusting with sampling bias factors calculated for four other pesticides to account for the uncertainty in the data.  As the sampling bias factors are not specific to the pesticide of interest, this is considered a screen.
                  First, I will discuss the Tier 3 PWC modeling results.  PWC is pesticide in water calculator.  In Tier 3, the pesticide and water calculator is used to estimate potential Chem 1 concentrations in surface water.  We modeled typical application rates and numbers of applications where usage of Chem 1 is reported for citrus, apples, and soybeans.  The estimated drinking water concentrations are on the Y-axis and the use sites are on the X-axis.  Results indicated that estimated drinking water concentrations may exceed the drinking water level of concern for all crops evaluated.
                  Recall that Dr. Bohaty presented that we have more regional specificity as you move to the higher tiers in the assessment.  In Tier 3, monitoring data are analyzed for each HUC-02 region.  To account for uncertainty in the monitoring data with non-daily sampling, measured concentrations are adjusted using sampling bias factors.
                  So let me walk you through the process of the Tier 3 monitoring data analysis.  First, we filter the data by HUC-02 region.  We consider the site years with detections with at least 13 samples per year in the analysis because these are the data where we can quantify the uncertainty and apply sampling bias factors.
                  Next, we categorize the data based on the number of samples collected per year.  Recall that Dr. Hartless explained that sampling bias factors are calculated for different sampling frequencies which were translated into a sample number per year, such as 13 to 16 samples per year and so on.
                  Finally, measured concentrations are multiplied by the appropriate sample bias factor determined based on the sample number category and averaging period, that is, daily, 4-day, or 21-day average, to provide a concentration in which we assume that about 95 percent of the time the concentration would be below that value for the site year.
                  We use the sampling bias factor calculated for the four USGS pesticides presented by Dr. Hartless to estimate a daily average concentration.  Monitoring analysis in Tier 3 is considered a screen as the Chem 1 specific sampling bias factors may be different.
                  The results of the monitoring data with sampling bias factor adjusted daily average concentrations are graphed here.  The sampling bias factor adjusted daily concentrations are on the Y-axis in log scale and separated by region on the X-axis and compared to the drinking water level of concern, the red line, to focus the assessment on areas of the country where there could be potential concern.
                  Looking at this analysis, all HUCs with adjusted concentrations above the red line would move to Tier 4.  Those below the red line would not necessarily be passing because there may be too much uncertainty in available monitoring data to rely on the monitoring data.  For example, if there are very limited number of site years with data in the region, these regions would rely primarily on modeling; however, if adjusted concentrations were below the red line and there was enough relevant data, that region would not move to Tier 4.  For this case study, 15 regions exceed the drinking water level of concern and would move to Tier 4.
                  In the short-term case study, we evaluated HUC-02 Regions 3 and 17 to simplify the case study.  These regions have high usage and sampling bias factor adjusted concentrations exceeded the drinking water level of concern.  Tier 4 requires the highest level of refinement.  Uncertainty in monitoring data is characterized using SEAWAVE-QEX in Chem 1 sampling bias factors.  Then a weight of evidence analysis is conducted.
                  So let's go through the monitoring analysis process in Tier 4.  The first step is to run SEAWAVE-QEX for the sites that meet the criteria that Dr. Hafner discussed earlier.  Next, the chemographs developed using SEAWAVE-QEX are used to calculate sampling bias factors.  As we only have a few sites where we could run SEAWAVE-QEX, results from all sites across the country are used to calculate the sampling bias factors, even if the site is in a region that did not move to Tier 4.  Here the assumed variability of across a region is assumed to be similar to the range of available data.  After calculating the sampling bias factors, they are applied to the monitoring results for site years with at least 13 samples per year for regions that move to Tier 4.  Finally, a weight of evidence analysis is conducted for the regions and sites with predicted drinking water level of concern exceedances.  This helps to understand the relationship of monitoring to usage in drinking water and whether monitoring data are available in vulnerable areas.
                  Of the roughly 8,000 sites with available monitoring data, only 32 sites, the pink dots on the map, could be evaluated using SEAWAVE-QEX.  Fortunately, most SEAWAVE-QEX sites are located in use areas and in drinking water watersheds.  There are two sites were SEAWAVE-QEX predicted daily concentrations are above the drinking water level of concern shown with the blue dots on the map.  One is in northern Virginia in an urban area and one in Alabama.
                  Next, I will talk about how we determined whether SEAWAVE-QEX predicted a drinking water level of concern exceedance for a site.
                  Recall we are using SEAWAVE-QEX to capture the peak concentrations that could be missed due to infrequent sampling.  Here is an example of the diagnostic plots for one site.  We show the drinking water level of concern using a dashed red line and the blue boxes are the uncertainty around the maximum estimated concentration across chemographs.  When the blue boxes are above the drinking water level of concern, there is a potential for exceedances at the site.
                  For example, in 1999, the blue box is completely above the drinking water level of concern.  We have more confidence that this site might have a drinking water level of concern exceedance.  For 1997 and 1998, there is some potential for a drinking water level of concern exceedance, but it is less certain on whether it is likely to occur.  Finally for 2000, an exceedance was not predicted.  The site was determined to have an unlikely but possible drinking water level of concern exceedance.
                  SEAWAVE-QEX results are used to calculate Chem 1-specific sampling bias factors.  The approach to determine one sampling bias factor for each site is to take the median across chemographs and the median across years.  This figure shows the short-term sampling bias factors on the Y-axis and each sample number category on the X-axis.  After calculating the Chem 1-specific sampling bias factor for the short-term case study, we apply the maximum sampling bias factor across sites to the monitoring results at other sites with at least 13 samples collected per year.
                  As expected for Chem 1, the higher the number of samples collected at the site, the lower the sampling bias factor.  Also note that some sampling bias factors are large, up to 288, indicating a lot of uncertainty in the monitoring data without daily sampling.
                  Now we will move to the weight of an evidence analysis for Region 17 as an example.  Here we are looking at a map of the monitoring data with apple growing areas in red and where about 170,000 pounds of Chem 1 is applied per year.  Drinking water watersheds are outlined in blue.  Of the 55 sites with data, the 6 SEAWAVE-QEX sites shown in pink in the region did not predict a drinking water level of concern exceedance; however, 8 sampling bias factor adjusted concentrations shown in green exceeded the drinking water level of concern in the region.  Those sites were at high usage areas and we relevant to drinking water.  Based on this information, there's high confidence in potential drinking water level of concern exceedances in the area.  As some measured concentrations were just below the drinking water level of concern.  The tools we have presented today allow us to quantify our uncertainty in the available data and draw this conclusion.
                  Today I walked you through how the SEAWAVE-QEX and sampling bias factors were used to quantify the uncertainty in monitoring data.  The first step was to run SEAWAVE-QEX and we were able to do that for 32 sites.  In step two, we calculated the Chem 1-specific sampling bias factors which ranged from 2.3 to 288.  This highlights that we can have high uncertainty in monitoring results with infrequent sampling.
                  Calculation of the sampling bias factors allowed us to consider data from 55 sites in Region 17.  When applying the sampling bias factors in step three, eight sites in Region 17 had a drinking water level of concern exceedance and some with little uncertainty.  When conducting the weight of evidence analysis, the sites were exceedances were in high usage areas and at sites that were relevant to drinking water.  Quantifying uncertainty in the available monitoring data provided us with confidence that there could be potential drinking water level of concern exceedances in Region 17.
                  Finally, I wanted to talk about some of what we learned about available monitoring data from this case study.  For Chem 1, we started out with a lot of data, but we could only use a small portion of that data with the tools we have developed to quantify the uncertainty in the data.  Across the United States of the 9,000 sites, we could use SEAWAVE-QEX for 32 sites and could apply sampling bias factors to results from 900 sites.
                  Another important item we found is that while the majority of data is in the most uncertain data category, there is too much data to discount these data but too much uncertainty to use these data quantitatively.  These data are considered with the understanding that these values may underestimate the potential for exposure.
                  We are asking the panel to provide feedback on how we used SEAWAVE-QEX in sampling bias factors in the case studies.  Now I will take clarifying questions on the short-term case study.
                  DR. RAYMOND YANG:  Ray Yang.  Could you go back to slide 152?  All right.  So we have these crops.  Let me just you a soybean as example because it covers most of the Midwest.  You say these are all above, all exceeded the DWLC.  Now, are these different chemicals combined or you're talking about one chemical, say atrazine, okay?
                  DR. KATRINA WHITE:  So this slide is one we did pesticide in water calculator modeling for one pesticide, and we --
                  DR. RAYMOND YANG:  Just one pesticide?
                  Now am I correct, for any of this crop, it is likely that multiple pesticides are used?  In other words, we could be dealing with chemical mixture and if, let's just say soybean, we use three pesticide.  Does EPA do each of them separately and then use additivity to consider possible interaction or what do you do?
                  DR. KATRINA WHITE:  So I think we had a few questions there, so I'll start with the first one.  I think that there are multiple pesticides used on a single crop and we do expect that.
                  I think the second one was, do we estimate exposure and consider exposure for mixtures in the dietary assessment?  And so for most chemicals, we consider them individually and do not do any type of additive analysis except there is a subcategory of assessments where we would consider, for chemicals that have a similar mode of action they would do what is considered a cumulative assessment.  So for the organophosphates, for example, they would do a different -- they would consider combined exposure for those and those would be a very specific assessment.  But those are outside of the scope of what we're doing today.
                  DR. RAYMOND YANG:  Thank you.
                  DR. ROBERT CHAPIN:  Claire.
                  DR. CLAIRE BAFFAUT:  This is Claire Baffaut.  Can you go back to slide 158?  All right and in Bullet 2 here, you say that there -- you assume that the variability across the nation is similar to the variability across the region.  I think I missed it, but could you explain a little bit how you used that statement?
                  DR. KATRINA WHITE:  So let me go to a map somewhere.  So we have 32 pink dots on that map, and we need to estimate exposure for all of those HUC-02 regions.  And if you go to any one individual region, there's a very limited number of pink dots.  And so we didn't think that we could only consider the data from one region to estimate exposure for that region because there are so few number of dots.  And that really simplified things.  And there is quite a bit of variate.  There are so many factors that influence exposure.  There can be a wide variability across a region and across the United States.  I think so for precipitation and HUC-17, I think Rochelle has often presented an example where precipitation is going to be very different across HUC-17.
                  Did that answer your question?
                  DR. CLAIRE BAFFAUT:  Yes.  So to rephrase, you're using those 32 dots to calculate the range of sampling bias factors?
                  DR. KATRINA WHITE:  And using all of those and applying it to data and looking at a regional analysis.
                  DR. ROBERT CHAPIN:  Tom.
                  DR. THOMAS POTTER:  I'm still a little bit confused about Tier 3 so I'm going to go back to that.  When you applied the sampling bias factors for the four pesticides where you generated data using SEAWAVE-QEX, how did you decide which value to use for your compound-specific analysis for Chem 1?
                  DR. KATRINA WHITE:  So for Chem 1, and this is a case study example.  This doesn't necessarily mean this is what we would do.  It's just an example of how it could be done.  For Chem 1 for the short-term, we took the median of the median to get one SBF for each site.  And in the White Paper, they calculated SBFs, one value for each site for four chemicals, 100 sites for atrazine, 30 for the rest, and we took the maximum.  In this graph, the lower end of the bar is the minimum for across those SBFs in the White Paper, and the max is that highest value for the region times the maximum SBF.  Across all chemicals, all sites.
                  DR. THOMAS POTTER:  And the assumption I guess is that this is protected in from a regulatory perspective.
                  This is Tom again.  Again, the assumption is that that approach is protected from a regulatory perspective?
                  DR. KATRINA WHITE:  That would be the assumption, yes.
                  DR. ROBERT CHAPIN:  Okay.  Ran through that gauntlet line.  Okay, excellent.  Thank you very much.
                  All right.  And Ms. Joyce for the longer-term exposure considerations, a case study.

DRINKING WATER ASSESSMENT CASE STUDY  -  A PESTICIDE WITH LONG-TERM EXPOSURE CONSIDERATIONS

                  MS. JESSICA JOYCE:  Good afternoon.  I might want to get us on the right slide.  All right.
                  Again, my name is Jessica Joyce and I am presenting the second drinking water assessment case study which highlights the quantitative use of monitoring data for developing long-term exposure estimates.  As with the acute case study, the first step in our drinking water assessment is scoping.  I will begin with some background information on the pesticide.
                  We are calling this pesticide of interest Chem 2 which is applied primarily to weeds and citrus and other orchards which is depicted by orange and yellow on the pie chart.  The chronic and cancer drinking water level of concern for Chem 2 is eight micrograms per liter.  Chem 2 has a lot of non-daily monitoring data.  It may indicate drinking water level of concern exceedances though the majority of the data are not robust enough that we can quantify the amount of uncertainty in the data.  Therefore, based on previous modeling and preliminary evaluation of the monitoring data, a higher tiered assessment where tools, such as SEAWAVE-QEX, and sampling bias factors will be needed to reduce the uncertainty in the monitoring data.
                  Based on the pesticide fate properties, Chem 2 is moderately mobile and persistent in water and soil.  Scoping indicating the need for at least a Tier 3 exposure assessment.  At this tier, modeling is refined with a pesticide and water calculator and monitoring data are adjusted with sampling bias factors calculated for four other pesticides, the chlorpyrifos, atrazine, fipronil, and carbaryl, the USGS pesticides, to account for uncertainty in the data.  This is considered a screen as the sampling bias factors are not specific to Chem 2.
                  First, I will discuss the Tier 3 modeling results.  This is using the pesticide in water calculator and is used to estimate potential Chem 2 concentrations in surface water.  The estimated drinking water concentrations are on the Y-axis and the various use sites are on the X-axis.  Tier 3 modeling indicates that both the maximum 1- and 10 year annual concentration and the 30 year annual average concentration exceed the drinking water level of concern of eight micrograms per liter, that red line, for all uses indicating the need for more refinement.
                  Now I will discuss the monitoring data.  At Tier 3, monitoring data are refined on a regional basis.  Pictured here are the hydrologic unit code 2 regions across the country.  To address the uncertainty and non-daily monitoring data, EPA could use the long-term sampling bias factor derived for the four pesticides from the USGS data.
                  EPA applied sampling bias factors to the monitoring data in this case study by first filtering the monitoring sites by region.  We consider site years with detections with at least four samples per year in the analysis because these are the data where we can quantify uncertainty and apply sampling bias factors.
                  Second, we categorize the data based on the number of samples collected per year.  Recall that Mr. Peck explained that sampling bias factors are calculated for different sampling frequencies which were translated into a sample number per year.
                  Lastly, we adjust the measured concentrations by multiplying by the appropriate sampling bias factor, using factors developed from the four USGS pesticides.  This ultimately provides a concentration in which we assume that 95 percent of the time the adjusted concentration would be below that value for a site.  This step is considered a screen as the Chem 2 specific sampling bias factors may be different.
                  The results of the monitoring data with sampling bias factor adjusted annual concentrations are graphed here.  The sampling bias factor adjusted annual concentrations are on the Y-axis in log scale and separated by region on the X-axis and compared to the drinking water level of concern, the red line, to focus the assessment on areas of the country where there could be potential concern.  This figure indicates that there are potential exceedances in the drinking water level of concern, those that are above the red line in Regions 3, 15, 17, and 18; however, values that fall below the red line may not necessarily be passing if there's too much uncertainty in the monitoring data, such as when there are only a few years of site data.  Since these four regions exceed the drinking water level water of concern, they would move on to Tier 4.
                  The following conclusions can be made at Tier 3:  refined model estimated concentrations still exceed the drinking water level of concern in most regions for all uses.  The sampling bias factor adjusted concentrations are higher than the level of concern in Regions 3, 15, 17, and 18; therefore, these regions would move on to Tier 4.
                  Now onto Tier 4.  Tier 4 requires the highest level of refinement.  For this case study, EPA focused on use of SEAWAVE-QEX, pesticide-specific sampling bias factors, and a weight of evidence analysis.  In Tier 4, the first step of processing the monitoring data is to run SEAWAVE-QEX for the sites that met the criteria that Dr. Hafner discussed earlier.
                  Second, the chemographs developing using SEAWAVE-QEX were used to calculate Chem 2 specific sampling bias factors.  Since there were only a few sites where we could run SEAWAVE-QEX, results from all the sites across the country were used with the assumption that variability across the nation is similar to the variability across a region.  After calculating the sampling bias factors, they are then applied to the monitoring results in relevant regions that move to Tier 4 and had at least four samples per year to develop annual and 30-year averages.
                  Finally, a weight of evidence analysis is conducted.  This aids in understanding the relationship of monitoring data to usage locations and drinking water watersheds.
                  The first step in processing the monitoring data at Tier 4 is to rune SEAWAVE-QEX.  As a reminder, it's designed to capture higher end concentrations with less than daily sampling.  Of the roughly 6,000 sites with monitoring data for Chem 2, only a fraction of these sites, 27 noted by the pink dots, had the data requirements to run SEAWAVE-QEX to generate chemographs and annual averages.  After running SEAWAVE-QEX for these sites, EPA determined that the annual average concentrations from these sties did not exceed the drinking water level of concern.
                  Next, we explored pesticide-specific sampling bias factors.  These bias factors allow for more monitoring data to be used than with SEAWAVE-QEX alone and are calculated using the SEAWAVE-QEX chemographs.  This figure demonstrates the long-term sampling bias factors on the Y-axis and each sample number category, four to five, six to seven, and so on, on the X-axis.  The approach to determine one sampling bias factor for each site is to take the median across chemographs and the median across years.  And then the median sampling bias factor across sites was selected for use.  As seen in the figure, the higher the number of samples collected out of site, greater than or equal to 10 for example, the lower the sampling bias factor or uncertainty.
                  Next, EPA conducted a weight of evidence approach to qualitatively evaluate spatial relevancy of monitoring sites to drinking water sources.  This weight of evidence analysis focuses where monitoring data overlap with potential use areas in drinking water watersheds.  For the purposes of this presentation, only Region 18, which covers the state of California, is discussed.
                  This is a map of Chem 2 monitoring data with citrus growing areas in yellow and other orchards in red.  Of the 229 site years of data, three sites exceed the drinking water level of concern which is depicted by the green circles, two of which overlap in the top left-hand corner and one down below.  All three of these sites are located in a drinking water watershed that contain a large area of citrus and other orchards where potential use could occur.  Based on this information, there is high confidence in potential exceedances of the drinking water level of concern in the area.
                  In conclusion, when modeling suggests risk concern and in lieu of daily monitoring data, the first step is to fun SEAWAVE-QEX for sites that meet the criteria.  In this case study, only 27 sites could be run, and none had drinking water level of concern exceedances.  In step two, we calculated Chem 2 specific sampling bias factors which ranged from 1.1 to 7.  In step three, we applied pesticide-specific sampling bias factors to site years with at least four samples per year which allowed us to consider data from 154 sites in Region 18.  In step four, when conducting the weight of evidence analysis, the sites with exceedances were in high usage areas and at sites that were relevant to drinking water.
                  Ultimately, quantifying the uncertainty in the available monitoring data provided us with confidence that there could be potential drinking water level of concern exceedances in Region 18.  But before we switch gears, I will follow up with what we have learned about the available monitoring data from this case study.
                  On a national level, there are a lot of monitoring data for Chem 2, over 6,000 sites, and only 27 of them met the criteria to run SEAWAVE-QEX and about 1,500 sites could be adjusted with sampling bias factors.  The majority of the sites with detections of Chem 2 had less than four samples per year as noted by the 70 percent portion of the pie chart.  This is too much data to discount but has too much uncertainty to use quantitatively which highlights the importance of considering all relevant monitoring data.
                  Now on to lessons learned from both case studies.  For drinking water assessments conducted for pesticides with short-term and long-term exposure concerns and a lot of non-daily monitoring data, EPA considers SEAWAVE-QEX a useful tool to address uncertainty and non-daily monitoring data for pesticides with seasonal applications.  But the number of sites nationally that meet the minimum data requirements are limited and are far fewer on a regional basis.
                  SEAWAVE-QEX allowed for the calculation of short-term and long-term sampling bias factors.  Application of sampling bias factors permitted the quantitative use of much more data than previously could be used in drinking water assessments or by using SEAWAVE-QEX alone.
                  A weight of evidence approach can be used in absence of a quantitative approach to address spatial uncertainty in the available data.  Additional monitoring data will be needed to reduce the uncertainty associated with available surface water monitoring data.
                  And this concludes the presentation on the case studies, and I will now take any clarifying questions.
                  DR. IAN KENNEDY:  Ian Kennedy.  Did you have any non-detects and if so, how did you deal with those?  Did they count in, say, the minimum of four samples?
                  MR. CHARLES PECK:  So when you're looking at the monitoring data and the application of the bias factors, when we took the average we used the detection, the limit of detection that was provided with the monitoring data.  If we're looking at SEAWAVE-QEX and calculating the average, it was basically the values that were provided in the chemographs.  So if they were below the limit of detection, we didn't go back and correct for the limit of detection.
                  DR. IAN KENNEDY:  SEAWAVE-QEX allows for non-detects, correct?
                  MR. CHARLES PECK:  So SEAWAVE -- yes.  What it does is if it estimates that a value is going to be below the limit of detection, it pulls a random value from that down.
                  DR. IAN KENNEDY:  Thanks.
                  DR. ROBERT CHAPIN:  Cliff.
                  DR. CLIFFORD WEISEL:  Cliff Weisel.  If I heard you correctly, when you're doing Tier 3, you said you used some sort of estimate of factor with the recognition there was a screening system.  But how do you decide what factor to use because if you really underestimate it, you can get them all below your screening saying everything is good but might be wrong.  I'm sure you've figured that out, but I'm just --
                  MR. CHARLES PECK:  So in Tier 3, again, we go back to the four chemicals that we had in the USGS data set and we used the maximum concentration that was detected, or the maximum annual concentration that was derived in a HUC-02 and applied the minimum and the maximum sampling bias factors that we derived for the for chemicals to sort of get a bound of, were we above or below the DWLOC.  And I agree, in some cases we may be below, but at this point, we're considering that the values that we derived for those four chemicals would be protected enough and we're looking for your feedback on whether you agree with that or whether, you know, we need to derive more of them.
                  DR. CLIFFORD WEISEL:  Do you have any sense as to what are the properties of the chemicals that most drive these bias factors?  And I don't know, and so that's how I help make the decision whether the range you have are right or what you should be looking for.
                  MR. CHARLES PECK:  So we tried to do the regression analysis with the fate properties.  It didn't quite work.  We do know the properties in general of the chemicals, the four chemicals that we looked, and that is kind of one of the questions we would pose to the panel is, you know, looking at those properties and looking at the properties of the chemical we're assessing, would it be more prudent if a chemical, let's say, binds to soil and sediment if we used the bias factor for that as opposed to one that may be more soluble.
                  DR. ROCHELLE BOHATY:  So to add on to that, if you go back to the sampling bias factor presentations, we had a graph by each of the chemicals were represented by a different bar and it showed the distribution.  You could generally see, for example, that insecticides had higher sampling bias factors.  You would expect that from the use profile.  And then you also need to consider the dissipation of those and generally the more persistent the chemical is the lower the bias factor is as well.  So the non-persistent chemicals would have higher sampling bias factors.  So if you had an insecticide that was less persistent, you're likely to have a much higher sampling bias factor.
                  DR. CLIFFORD WEISEL:  So if I heard you right, what category -- usage is one thing to consider.  Are the physical chemical properties another is persistent in the environment might be a third.  You have a beautiful table and I appreciate -- I think it was Table 6.1 or something like that, which is very important that might be something that we can looking at to help, but that's going to have to go back.  Thank you.
                  DR. ROBERT CHAPIN:  Ray.
                  DR. RAYMOND YANG:  Ray Yang.  Thanks for your presentation.  I have two questions.  First one, you have four example chemicals.  Now, as a toxicologist, of the four, I would have worried the most about chlorpyrifos.  That's because of my understanding of toxicology of these chemicals and so on.  Even if it's lower in terms of monitoring results, okay, does EPA take that sort of factor into consideration, you know, toxicity?
                  DR. ROCHELLE BOHATY:  So all of the information you're hearing about today is just in the context of developing and evaluating the models, except for the two case studies.  And so as they presented the way we would take into account the toxicity, it would be represented by the benchmark of the drinking water level of concern that was used in each one of those case studies.  But again, these methods and tools could be used in other pesticide risk assessments.
                  DR. RAYMOND YANG:  All right.  Second question, it's a follow-up of what I asked this morning in terms of the distribution of tier 1 to Tier 4 and so on.  So we are almost at the end of 2019, okay.  So in the year of 2019, how many chemicals EPA checked up to Tier 4 assessment?
                  DR. ROCHELLE BOHATY:  So today we have done a high level Tier 3-like assessment where we've delved into monitoring data to this kind of level and it's only been for two pesticides just to give you context.  It's not to say that it wouldn't apply to other pesticides.  That's why we're here today is to bring these methods forward before we start using them for other pesticides.
                  DR. RAYMOND YANG:  So 1 out of 150 or 100?
                  DR. ROCHELLE BOHATY:  Two so far.
                  DR. RAYMOND YANG:  Two.  All right.  Thanks.
                  DR. ROBERT CHAPIN:  Tom.
                  DR. THOMAS POTTER:  Hey, this is Tom Potter.  Thank you for those two case study presentations.  They certainly clarified a lot of issues for me and I thought they were very well do so again, thanks for doing that.
                  One thing I've been wondering about in the Tier 4 analysis is -- and I say this because I've sat on the panel that first looked at SAM some years ago, so I'd like to know where SAM is hiding, and if indeed, he's going to become part of the Tier 4 process.  That's the Spatial Aquatic Model for those of you who we're on the panel or familiar with it.
                  And then the follow-up is in the weight of evidence approach.  When you look at, for example, I think you had a HUC-17 was your target area, do you do a detailed analysis of use and use amounts and use profile, timing application and that sort of thing to further strengthen your insights into whether or not you have a DWLOC exceedance or not, et cetera?  So does that come into play?
                  DR. ROCHELLE BOHATY:  I can take the first question.  If you look in the framework in the Tier 4 section, we do mention that's where the Spatial Aquatic Model would be integrated into the framework.  Tier 4.  Rochelle Bohaty.  I didn't mention my name.  I'm sorry.
                  DR. KATRINA WHITE:  This is Katrina White.  I think you definitely would -- well, we would look more into the usage data and how it related to the monitoring and timing of when the monitoring occurred and that was presented by Dr. Hartless when she talked about the weight of evidence analysis.
                  We do -- most of the usage data that we use is only available at the state level.  So for HUC-17 in this case study, there's a high usage on orchards.  So in 170,000 pounds, we know there's usage in that area and we have a pretty good idea of that.  But if the percent crop treated might be a lot lower and we only had it available on a state level, we might not know if there's really specific usage in the watershed and the timing of the applications we often might not know the exact timing.  So to the extent that we can and that we know when something would be applied, we would consider that, but we don't always know when and where exact applications are occurring.
                  DR. THOMAS POTTER:  Can I just follow up?  This is Tom Potter again.  I think going back to that SAM assessment we did we spent several days talking about those issues, and I think some good ideas were put on the table as to how to approach timing of application.  We've heard earlier about use data being available nationally through USGS or the state of California and their program so, you know, I think there's a path forward there and I certainly would like to see the agency take it.
                  DR. ROBERT CHAPIN:  Dr. Rodgers.
                  DR. JOHN RODGERS JR.:  John Rodgers.  Very good job on the case studies.  I'd like to ask about is there is a similar effort going on the DWLOC, drinking water LOC side of the equation because these have got to come together some day or is all the effort going on the exposure side?
                  DR. KATRINA WHITE:  I'm not a dietary assessor and I wish Anna were here, but they do have a tiered assessment for the dietary assessment on the food as well.  And I don't have a good understanding of the type of refinement work they're doing at this time.  I'm not sure if someone else does.
                  DR. JOHN RODGERS JR.:  And I guess to follow that up, the basis for that question is the effort in terms of risk assessment, you know, you boil it down to one number.  We've taken some 90th percentiles here and some max values here and then you multiply those and then -- I mean, we do that in risk assessment all the time -- and we worry about compounding conservativism because these values are often linked correlative or whatever.  And so you worry about losing realism and I just wanted to know if an equal effort is being put into -- if we're going to do this ratio thing, or cook it down to a ratio where we've got a number that we've pulled from a model or pulled from monitoring data and a number that we're using to indicate potential for harm in terms of a DWLOC, how much confidence do we have in those numbers?  It seems to me like you need equal effort.
                  DR. KATRINA WHITE:  So they start at a higher tier assessment where they would use some simpler exposure estimates on the dietary side.  And then they'll move to measured concentrations in the food and they'll look at what -- conservative assumptions of what people are consuming and move to more specific assumptions on what people consume.  So there is that refinement and moving to a higher realism in the dietary side as well.
                  Did anyone have something to add?
                  DR. ROCHELLE BOHATY:  I was just going to add that earlier when Anna mentioned -- she talked about the HED, the Health Effects Division also has a tiered process that tries to mirror what we're doing in our framework.
                  DR. JOHN RODGERS JR.:  Very good.  I was also curious about the weight of evidence and use of that term because usually other branches use the term and they argue about whether you've got lines of evidence and whether there's a weight of evidence call.  And I realize that that's not necessarily a quantitative interpretation in this situation and that's fine.  But could this analysis be trumped if I went to a drinking water system and pulled a sample and measured a pesticide concentration in excessive of the drinking water LOC that the drinking water intake or at a customer tap?
                  DR. KATRINA WHITE:  I think that would be a yes.  That's a solid line of evidence that we might have a concern, right?
                  DR. JOHN RODGERS JR.:  I just think I would consider the weight of evidence for a hierarchy of data.  I mean, in one case, you've got real data coming in from the field and in another case you've got estimates from a model and in maybe a higher tiered case or a higher line of evidence, you've got, it's in my cup.  I'm drinking it.
                  So, I was thinking about that as I went through all of this and, you know, ultimately that's the call when we start pulling those samples.  Because the drinking water intake samples that I've pulled, a lot of them they're not in Lodec (phonetic) system so they're not well mixed.  They are stratified quite often.  The pesticides are not uniformly distributed at all.  You could certainly hit some sometime or miss some completely sometime and you know they're there.  You just can't find them, and you say, what kind of situation is that?
                  But we have EPA-approved aquatic pesticides that people like me try go out and try to track.  I mean, we know where they're going, we know when they're going in the water.  We want to know whether they get the drinking water or not so we kind of got to jump on things a little bit.  We know how much went out there and so on.
                  So I just was curious about the analysis that you did in those case studies in terms of the juxtaposition of sampling versus where the drinking waters' intake were and that was very enlightening.
                  DR. KATRINA WHITE:  One of the things we considered the case studies was the time of travel based on the flow and you could also consider the fate properties in water.  So with the persistent pesticide, you could think about the time of travel in relation to that.
                  DR. JOHN RODGERS JR.:  It's very useful.  Thank you.
                  DR. ROBERT CHAPIN:  Dr. Nowell.
                  DR. LISA NOWELL:  Lisa Nowell.  Reiterate, thanks for the great presentations today.  They've all been really helpful in trying to process all of the reading I've been doing over the last while.
                  I have a question, I guess, thinking about the use of usage data or use data.  That's all you have in the case studies or in the weight of evidence.  Can you tell me a little more what you have to work with in terms of non-agricultural applications?  Do you have any market survey data or are you going with labels or you have to go back to the states for that?
                  DR. KATRINA WHITE:  So nationally its generally -- we consider the reliable at the state level.  It's based on market survey data.  And then in California, we have a robust status set for licensed applicators.  And then some states we might be able to ask for more specific usage data and they might -- some other states have some data that they could share with us, but we'd have to -- it's not necessarily available readily in the database.
                  For non-agricultural usage, there are some surveys for that, but it's not as -- they aren't as robust as they are for the agricultural side.  And sometimes we might ask for some sales data or some more specific information from registrants, but that's much more difficult to get and much more uncertainty in that data.
                  DR. LISA NOWELL:  May I ask a follow-up?  So Lisa Nowell again.
                  So would that be part of the weight of evidence stage basically?  And I'd love to be reminded about your case studies.  You had, I think, Chem 1 had substantial residential use and Chem 2 did not.  How did you factor those in separately in the case studies?
                  DR. KATRINA WHITE:  So for Chem 1, there was estimated about 3.8 million pounds applied in residential areas and about, I think -- I'm going to get the numbers wrong -- about 100,000 pounds -- well, about 700,000 pounds, I think, applied in ag.  So most of the usage was in residential areas.  And what we consider is that we have a lot of uncertain in the usage data and so it's hard to say where that 3.8 million pounds is being applied and we didn't have that information available in the case study.
                  DR. ROBERT CHAPIN:  Dr. Yang.
                  DR. RAYMOND YANG:  Ray Yang.  Quick question --
                  DR. KATRINA WHITE:  Can I do a little bit more follow-up?  For Chem 1, the sampling, there's much higher detection frequency in urban areas and all but two of the sites, SEAWAVE sites, where we had detections were -- they had less than 20 percent ag in the watershed.  So the usage data and detection frequency for Chem 1 help to tell a story.
                  DR. RAYMOND YANG:  All right.  I'm going back to my question.  Quick question about -- does it require fair amount of competency in computer programming for a scientist to run this SEAWAVE-QEX modeling?
                  DR. SARAH HAFNER:  No, it's pretty simple and straightforward to run.  It's the interpretation of the diagnostic plots that takes some work.
                  DR. RAYMOND YANG:  Thank you.
                  DR. ROBERT CHAPIN:  Okay.
                  DR. SARAH HAFNER:  And also the data preparation takes time, but it's not difficult.
                  DR. RAYMOND YANG:  Is it programmed in R -- Ray Young again.
                  DR. SARAH HAFNER:  Yes.
                  DR. RAYMOND YANG:  Okay.  Thanks.
                  DR. ROBERT CHAPIN:  All right.  We have Dr. Bischof.  Mr. Bischof you get the Job award for patience today.
                  All right.  So we're going to move to a surface water monitoring program from Washington.
                  DR. ROCHELLE BOHATY:  Okay.  I'm going to start.  This is Rochelle Bohaty back to introduce Mr. Matthew Bischof with the Natural Resources Assessment Section of the Washington State Department of Agriculture.  We've been working with Mr. Bischof for more than two years.  He's been an integral part of our team contributing primarily to the evaluation of SEAWAVE-QEX using alternative covariates as well as the development of SEAWAVE-QEX standard operating procedures.
                  In his role at Washington State Department of Agriculture, Mr. Bischof monitors for pesticides in eastern Washington and he's going to provide you with an overview of the state's monitoring program as an example.  His presentation is not an endorsement that all monitoring programs need or should resemble their program, but we invited him here to present as he is part of our team and he offers a unique technical perspective having used the tools you heard about today and offers a practical experience with the conduct of surface water monitoring sampling and program design.

DEVELOPMENT OF SURFACE WATER MONITORING PROGRAM

                  MR. MATTHEW BISCHOF:  Thank you, Rochelle.  Thanks for the opportunity to speak here.
                  So you've just heard about two pesticide case studies that demonstrated the use of SEAWAVE-QEX.
                  Can everybody hear me fine?  Oh, sorry.  Okay.  There.
                  All right.  So the case studies demonstrated the use of SEAWAVE-QEX in the sampling bias factors in drinking water assessments.  I will now present how these tools could be use in the design of the surface water monitoring program.
                  So on this slide, I've outlined my presentation beginning with the purpose of my involvement followed by describing an example monitoring program.  I will then broaden the discussion to general considerations when designing a surface water monitoring program and how SEAWAVE-QEX and sampling bias factors can be used to help overcome monitoring program challenges.
                  So my role has been to provide technical guidance on the conduct of surface water monitoring programs as well as support the evaluation work on SEAWAVE-QEX.  To this end, I will describe Washington State's surface water monitoring program as an example of an ongoing pesticide monitoring program.  I will then step back to highlight important surface water monitoring program considerations.  I will also bring forward practical limitations in the conduct of a monitoring program and how SEAWAVE-QEX and sampling bias factors can help augment these challenges as well as highlight the utility of these tools for Washington State.
                  Washington State Department of Agriculture Pesticide Monitoring Program consists of ambient monitoring during the typical pesticide use season.  Sampling occurs weekly or biweekly from March to August or into September/October, depending on the year, resources, as well as watershed.
                  So this slide shows a map of Washington state with the 2019 monitoring sites as red dots including nine sites in eastern Washington and seven sites in western Washington.  The green overlay on the map represents agricultural crop land.  There's also influence from urban and mixed-use areas depending on the sampling site.
                  Now our sampling site locations are driven by some Salmonid co-occurrence.  We are currently monitoring for about 150 different analytes, and this includes pesticides as well as some transformation products.  So note the lack of monitoring sites in far eastern Washington.  The goal of our program is to add additional monitoring sites further east but is currently limited by budget.
                  SEAWAVE-QEX has influenced our program by identifying ways we can improve our program.  The Washington State Department of Agriculture uses a tiered approach to determine sampling frequency depending on its detected pesticide concentrations.  While this tiered approach is different from the one you heard about in the drinking water assessment framework, the concept is the same, to conserve and best allocate resources.
                  The Tier 1 when a new site is added to the program, the site is sampled every other work during the application season for three years.  The site is moved to Tier 2 sampling if at least one detection is greater than Washington State Department of Agriculture benchmark which is based off of an EPA or other water quality benchmark with a safety factor applied to it.
                  At Tier 2, samples are collected weekly during the time period when higher concentrations are expected.  Tier 3 sampling is dependent on program budget and happens when best management practices are implemented.  So 2-3 monitoring is used to assess the effectiveness of best management practices.
                  Now as a result of using SEAWAVE-QEX, some sites have been established as long-term monitoring sites and are in agriculturally important regions.  The site is sampled weekly and may be monitored for 10 or more years for the purpose of identifying pesticide trends and changes in pesticide use or agricultural practices.
                  Stepping back from Washington State's program, there are several factors to consider when designing a surface water monitoring program that will influence if, when, and how the data can be used by the conducting organization as well as other organizations.  Listed on this slide are a few considerations that are important when developing a monitoring program.
                  There are different ways to collect samples, composite, grab, or passive.  The most common method is a grab sample; however, a collection of composite samples is increasing due to reduction in costs associated with sample automation.  A composite sample is where multiple samples are collected throughout a specific time period, for example, daily or weekly and combined to develop an average concentration over that duration.
                  Site characteristics that may influence when or how often samples need to be collected include precipitation, soil vulnerability, use profile, and land practices such as irrigation.  The relevance of the sampling site to areas where you're trying to protect or represent also needs to be considered.
                  The number of analytes needs to be thought through.  Often this needs to be a balance decision-making progress that considers resources as well as available methods and detection limits.  It is important for a program to document quality assurance and quality control protocols along with the analytical methods.  It is also important to make sure the detection limit is below the level of concern.
                  There are several considerations when determining the ideal sample frequency, as I mentioned previously.  Site characteristics play a role as does the exposure duration of concern.  For example, one or more -- for example, more intensive sampling may be needed for pesticides with acute or short-term durations of concern, for example, 1-, 4- and 21-day average compared to chronic or long-term durations of concern such as 365-day average.  So when considering sample timing, it is important to think about application timing and environmental conditions that impact runoff.
                  In some cases, the sampling window needs to extend beyond the application season because, for example, the rainfall event necessary to derive runoff does not occur until much later in the year compared to the application season.
                  So monitoring programs need to target use areas that could impact source drinking water.  Furthermore, programs are also more robust when all residues of toxicological concern are considered.  For example, if the parent pesticide is expected to transform quickly in the environment to a more persistent and mobile transformation product, having data for the transformation product is imperative.
                  Other considerations for monitoring programs include when the samples are collected compared to the seasonal pesticide occurrence window in collecting data for other covariates such as flow.  Taken all together, the sum of these factors can quickly snowball into an intensive sampling program that agencies or organizations may not have the resources to do; however, the SEAWAVE-QEX can be used to reduce the burden of intensive sampling at sites of interest between interpolating between sparse sampling to estimate daily pesticide concentrations.  In cases where SEAWAVE-QEX cannot be used, sampling bias factors can also help inform monitoring decisions.
                  First, I'm going to discuss how SEAWAVE-QEX can be used to help inform the design of surface water monitoring program and there are a few things to consider in a program design when planning to supplement the data using SEAWAVE-QEX.  One consideration is that SEAWAVE-QEX requires at least three years of monitoring at a given site, although non-consecutive years is acceptable if the timeframe and the covariate data matches the pesticide data; however, more years of data will help capture the range in variability and weather end also use patterns.
                  If you have many years of data, it may be useful to split the data for SEAWAVE-QEX to get better model fit.  As described in the SEAWAVE-QEX standard operating procedure, it is important for the book end or anchor years to have robust data.
                  The minimum number of samples that need to be collected for use in SEAWAVE-QEX each year and the detection rate will likely vary by pesticide properties, use profile among other things, and therefore needs to be reviewed periodically.  Factors that can influence these decisions include the likelihood of the pesticide to run off versus absorb to soil, the persistence of the pesticide in soil, and water and the residues of concern.
                  SEAWAVE-QEX generally performs better when the measure samples represent more of the year rather than collecting, for example, 12 or more samples within a short timeframe; however, for some pesticides, the detection rate will drop too low when sampling outside of the usage season or too long after precipitation events.  Therefore, if the sampling program is planning to use SEAWAVE-QEX on a specific pesticide then the properties will need to be carefully considered.  If a non-targeted approach is taken and the sample number is limited to the minimum, that is about 12 samples, then focusing the sampling to the use seasons and likely increases the number of pesticides from those samples that may be usable in SEAWAVE-QEX.
                  SEAWAVE-QEX requires samples that are on a daily time-step, like the covariate data, which could be accomplished with either grab samples or daily composite samples; however, composite samples over more than a day would not be suitable for SEAWAVE-QEX.  Additionally, while passive samplers are a useful monitoring tool, they are not recommended to be use in SEAWAVE-QEX currently.
                  Similarly, the flashiness of a waterbody, and that is a waterbody having rapid or short-lasting changes in streamflow following runoff, needs to be factored in how frequently a site is sampled so that these sites are sampled more frequently or that there is flexibility in the sampling routine, such as possible precipitation event focused sampling.  This becomes more important for site and pesticides that have a significant correlation between concentration in the short-term flow anomaly when running SEAWAVE-QEX.
                  The last output figure of the SEAWAVE-QEX diagnostic plots provides a correlation time scale, or CTS.  This parameter can be used as a rough gage as a relative comparison of the duration of time which concentrations are correlated.  This can inform the sample frequency necessary for a site and pesticide combination.  This parameter captures all the variables that impact the concentration of the pesticide over time at a given site, for example, flow an environmental and transport properties.
                  Pesticide use patterns should also influence the timing of sampling whether there is consistent use season or sporadic use driven by past pressure.  This includes considering the use sites, for example, urban uses compared to agricultural uses.  Also, to use SEAWAVE-QEX, a daily covariate needs to be collected.  As we saw in the evaluation of SEAWAVE-QEX, streamflow is the preferable covariate for use followed by staged then precipitation, noting that precipitation might be needed for certain systems where streamflow and stage aren't appropriate.
                  The covariate needs to be daily during the time period that the pesticide samples are taken but are not required for the full year.  For example, if pesticide sampling takes place from April to October, streamflow does not need to be collected after October; however, for the midterm flow anomaly to be calculated, the covariate data needs to be collected 30 days prior to the first sampling event.
                  So one of the biggest challenges for adopting SEAWAVE-QEX for some programs is collecting continuous flow data.  Continuous flow data is expensive and time consuming to collect and ensure accuracy.  While streamflow is preferred, if necessary, collecting stream stage may be a cheaper alternative for use in SEAWAVE-QEX if the covariate would otherwise not be collected.
                  Additionally, data from USGS data stations or other agency stations can be leveraged when possible.  Other challenges with flow data are that it may not be the best predictor of pesticide concentrations at all sites, as not all sites exhibit a natural flow behavior, and may include periods of back flow or no flow.  In these situations and other covariates of just precipitation maybe an option or better.
                  As with all daily field sampling, there are unavoidable challenges, such as equipment malfunctions or natural causes, like freezing streams that prevent a complete daily record of streamflow and may result in that streamflow record needing to be infilled before it can be used in SEAWAVE-QEX.
                  Now, there are various methods for infilling occasional missing flow data for use in SEAWAVE-QEX.  One is a method used by SEAWAVE-QEX on flow data pulled from USGS gage stations from the R package waterData.  The figure on this slide shows an example of infilling flow using a waterData package.
                  Flow in cubic feet per second is shown on the Y-axis and the record day is shown in the X-axis.  You can see here at the yellow arrow is an example of one segment of the incomplete flow input that was infilled using the waterData program.
                  Now, when using USGS flow data in SEAWAVE-QEX, the model automatically pulls the data and interpolates missing flow data using this package.  However, if you're using flow data that you have collected yourself, you must interpolate the missing days with a method, such as waterData before it can be used in SEAWAVE-QEX.
                  Additionally, use of this package keeps the flow data infilling consistent across covariate data sources so that external covariate data and USGS gage station data are treated similarly.
                  So these next couple of slides highlight the work we did in identifying SEAWAVE-QEX alternative covariates to streamflow.
                  This slide is showing SEAWAVE-QEX seasonal wave fits to the same pesticide data using two different covariates, streamflow and stream stage, as shown on the left and right of the slide, respectively.
                  The site used to produce these seasonal waves is a stream located in western Washington, so on the wet side of the state.  That exhibits a natural flow pattern.  There is some irrigation used in the watershed that contributes to the stream source from ground water or pulled from a creek.  But in general, it follows a natural flow pattern.
                  So the monitoring site has a gage station that provides continuance streamflow and stream stage data.  We compared these covariates in SEAWAVE-QEX with the monitoring data and found that stream stage is a good surrogate for streamflow.
                  The results from these simulations were evaluated as Dr. Hafner described in her presentation.  However, for simplicity, these figures illustrate the similarities in the model selected by SEAWAVE-QEX.  Where both covariates is shown by the similar wave shapes and the position of the season shown in gray in the two figures.
                  We also looked at measured precipitation as a covariate compared with streamflow.  And the results for streamflow are shown in the figure on the left and results, the measured precipitation are shown on the right.
                  The site used for this comparison to make the seasonal waves shown here is in eastern Washington.  It's on the dry side of the state.  And so the site is influenced by irrigated agriculture and sourced by canals and laterals.
                  This site does not follow a natural flow pattern.  Precipitation used for the covariate comparison here came from agweather.net, which is a weather station maintained by Washington State University.  Precipitation data was collected from four weather stations within this watershed. Precipitation data from these sites was used to calculate daily average watershed precipitation and considering the type of travel.
                  Again, comparing measured precipitation to streamflow in SEAWAVE-QEX showed that measured precipitation may be a suitable covariate to use when streamflow is not available or if it is not appropriate given the hydrology of the site.
                  In this example, while the seasonal wave looks similar in both plots shown, when you look at the other diagnostic plots, precipitation is a better covariate compared to flow for this site.
                  Although SEAWAVE-QEX can be a valuable tool for leveraging resources, it may not always be possible to collect enough samples to meet the minimum requirements for the model at every site.
                  Additionally, even if you meet the data requirements, some pesticides may be more difficult to fit using SEAWAVE-QEX.  In these cases, sampling bias factors can be used to make more use of the data.  Sampling bias factors can be used as a screen, requiring fewer years of data to determine which sites may require more intensive marketing.  If there is a sampling bias factor concentration that exceeds the level of exposure concern.
                  When using sampling bias factors inform a monitoring program, the pesticide and the associated duration of exposure certainly needs to be considered.  The pesticide of interest has a longer duration of exposure concern, less sampling may be possible if sampling bias factors are considered.  This strategy could also be used if the program wanted to increase the number of sampling sites.
                  There are many benefits to incorporating SEAWAVE-QEX in sampling bias factors into monitoring programs to better understand real-world exposure levels without intensive sampling.
                  SEAWAVE-QEX can be used to estimate concentrations occurring between infrequent sampling events, that is nondaily.  This is particularly useful for pesticides with short-term exposure durations where the sample frequency needs to be high, yet is resource prohibited.
                  Sampling bias factors can be used to help understand the range of uncertainty in available data for infrequent sampling.  Sampling bias factors can also be used to gage the added value in increasing the sample frequency at a site.
                  Taken together, these tools can be used to optimize monitoring programs.  For example, in some cases, fewer samples need to be collected, and as a result, resources can be reallocated to increase the number of sites that are monitored.
                  Washington State Department of Agriculture is considering this approach for the purpose of expanding our monitoring program further east in Washington.
                  In addition, these tools can be used to assess the added value in increasing the frequency of monitoring at a given site.
                  SEAWAVE-QEX can also be used for trend analysis to leverage the data we have and gain insights into what is happening in the environment and what outreach is to growers is effective.  These tools can increase the utility of the data beyond the organization, collects the data as well as for use by other organizations.
                  All right.  Questions?
                  DR. KENNETH PORTIER:  One of the things I've been thinking about is why the regression between the site characteristics and the SBFs have such low R².  One of the ways to increase, at least your competence in the regression would be to sample more in the extremes, to increase the range of the characteristics you look at.  And that kind of tells me that you'd need to go to site characteristics sites that you aren't currently monitoring, because they may not be the sites where you expect to see a lot of action, right.  Right now, you're design is places where you know you have a lot of ag use.  You have stream flow.  You put a monitoring station.  So one of the take homes from this whole modeling process might be requiring you to go somewhere else.
                  Is that something that a state would look at, like, Washington, say, I don't know I'm going to put a million dollars in a marginal site over here?  I'm just wondering.
                  MR. MATTHEW BISCHOF:  Yeah.  A million dollars, that'd be nice.  Well, the thing is, we do also devote a lot of time and energy into outreach to growers because they don't know what's in the water unless we're sampling and then we're able to tell them and explain it to them.
                  So we look for watersheds where we're likely going to see pesticides where outreach can be effective.  So those are the sites where we can make a difference and that's where we put our value, I guess.  I mean, it would be nice to monitor in beautiful mountain streams too, but the biggest difference we can make is at sites where we're likely going to see pesticides.
                  MR. KENNETH PORTIER:  Yeah, but sometimes knowledge is not gained by going where you currently are, right.  Sometimes you have to go where you aren't.  And I know that's a hard communication issue, but I think I hear the answer to your question.  The answer is no.
                  MR. MATTHEW BISCHOF:  Yeah.  Yeah.  Good question though.  Thanks.
                  DR. CLAIRE BAFFAUT:  Hi.  This is Claire Baffaut.  So I have three questions, actually.  Is there -- so you talked about missing flow rate, missing discharge values.  Is there a way to deal with streams that go dry?
                  MR. MATTHEW BISCHOF:  Well, I guess none of our streams really go dry.  I mean, then you just don't have samples to collect, I guess.
                  I mean, we do partial years, but then you run into the issue of, you know, not having enough samples to fulfill the requirements of SEAWAVE-QEX.
                  DR. CLAIRE BAFFAUT:  Sure, but I can think of several streams that, you know, every year maybe in August or September, they'll go dry for two weeks.
                  MR. MATTHEW BISCHOF:  Oh, okay.
                  DR. CLAIRE BAFFAUT:  And then it rains again, and it flows again.
                  MR. MATTHEW BISCHOF:  Yeah.  Yeah, that's interesting to think about.  I can't think of -- we don't have any sites that do go dry.
                  DR. KENNETH PORTIER:  This is Ken Portier.  The 2012 atrazine study, we spent at least a day and a half talking about dry run streams and what you can and can't do with them.  And of course, the answer is you can't do much.  I mean, you run out of water.
                  MR. MATTHEW BISCHOF:  Yeah.
                  DR. CLAIRE BAFFAUT:  Okay.  So my second question was have you thought about running SEAWAVE-QEX for wet years together or drier years together?
                  MR. MATTHEW BISCHOF:  No.  I have not.  But that can be a good idea to, I mean, for further investigation because especially in the west, we have had periods of dryness.
                  DR. CLAIRE BAFFAUT:  Yeah.  I also think that it might change the way the pesticides are applied and when they are applied.  So it might actually change the wave itself.
                  MR. MATTHEW BISCHOF:  Yeah.  In 2015, we had a very particularly dry -- well, it was a warm winter, but we had near normal precipitation but no snowpack.  And so snowpack in Washington is relied upon as basically a reservoir for feeding the streams.  And so we had very little influence on those streams and we were expecting to see some differences in pesticide detections, but it seems like -- it seems like we didn't -- there wasn't a big contrast from year to year, with looking at 2015 compared to others.
                  But in terms of looking at chunks of years separately, wet versus dry, I haven't done that, but that could be worth looking into.
                  DR. CLAIRE BAFFAUT:  Finally, my last question is you or anybody else has anyone used SEAWAVE-QEX for fall applied pesticides, like pesticides that are applied now, basically, October or November?
                  DR. SARAH HAFNER:  So the only one that we have or the only example that we have was cyanazine, which is in the White Paper briefly.  And it's a poor example.  Not because it's fall applied, but just as a chemical it had a summer peak and a fall peak.  But other than that, we haven't looked at fall applications, specifically.
                  DR. ROBERT CHAPIN:  Tom.
                  DR. THOMAS POTTER:  Well, in southern Georgia and northern Florida, we actually have some rivers that go underground at certain times of year, but I'm not going to ask you how to deal with those.
                  MR. MATTHEW BISCHOF:  Thank you.
                  DR. THOMAS POTTER:  What I am going to ask you is that I've been digging through the White Paper and over several days and I still have not been able to find where you take into account time of travel in the context of precipitation data.  Is that -- can you guide me to that, or can you tell me, you know, what it is verbally?
                  DR. SARAH HAFNER:  If you give me a moment, I could give you the page number.
                  DR. THOMAS POTTER:  Yeah.
                  DR. ROBERT CHAPIN:  Do you have a specific question about that?
                  DR. THOMAS POTTER:  Yeah, we do.  Yeah, what is it?  I mean, what's the time of travel?  I mean, there are -- actually, there is another approach to using precipitation data in SEAWAVE-Q that was used by Johnson et al in 2011.  And they actually combined flow and precipitation with the wave regression and seemed to get very good data fits.  They looked at Washington state data, a lot of California data and I'll read the citation into the question and answer period tomorrow.  But I just think there may be a lot of other ways to think about this.  And so my specific question is, how did you handle this -- and I'll go back -- I need to go back, I guess if it's in the White Papers while I'm here I'll dig into it tonight and we'll look at it.
                  But I also wanted to explore the broader question, you know, other ways of incorporating precipitation into the model, including combining it with flow to get better data fits.
                  DR. SARAH HAFNER:  So section 9.1.3 discusses the model precipitation and there's a figure in 9.3 that talks about the time of travel.
                  DR. THOMAS POTTER:  Okay.  Thank you.
                  DR. ROBERT CHAPIN:  Ray.
                  DR. RAYMOND YANG:  Excuse me.  Ray Yang.  Thanks very much for your presentation.  Thank you also for collaborating with EPA.  I have a question.  This is a two-part question.  I'm asking you as well as EPA colleagues, so you don't have to answer if you don't want to, okay.
                  We all know the state of California is having multiple, multiple wildfires, and in large area and some are encroaching upon crop land, okay.  Combustion of pesticide residues from my perspective, is extreme oxidation reaction, okay.  Therefore, new chemicals might be synthesized.
                  Now, in a situation like this, and this will happen again and again, okay.  Part 1, how are you going to deal with this in terms of monitoring?  Part 2 is SEAWAVE-QEX flexible enough such that the parameter could be modified to accommodate this type of scenario?
                  DR. ANNA LOWIT:  Sorry.  I was gone for an hour, guys.  I do apologize for being away from the table for an hour.
                  DR. ROBERT CHAPIN:  That's all right.  we had a wild party, and somebody has to clean up the room.
                  DR. ANNA LOWIT:  I know you did.  I heard it was fabulous and I missed a lot of important things.  So that is a practical question and one that I know a lot of people put some thought into.  But I think it's important that we don't -- regulatory science is about doing the best we can at a moment in time with what we have in front of us.  And we've brought you a White Paper with a lot of thought into it and a lot of hard work.
                  And going down the rabbit hole of what if this and what if that, and you know what if a meteor comes in and the world explodes.  And I'm being a little bit facetious and I apologize for that, but the fire will change the fate properties, which will fundamentally change how we assess a chemical.  There's just fundamental changes in a situation like that, that really have nothing to do with SEAWAVE.  That would really fundamentally change how we might evaluate the fate properties, whether it's volatilization and movement in the air.
                  There's a lot of other things that go with these fires that would, in terms of air quality and everything else.  So although, that's a very thoughtful question, I think we would respectfully ask that we maybe stick to the questions that we've asked.
                  DR. CLIFFORD WEISEL:  This is Cliff Weisel.  Again, thank you for your presentation.  And this is probably due to my lack of knowledge.  When we're looking at the precipitation versus streamflow, one of the fundamental properties you need to understand precipitation is to make sure the watershed that you are defining only goes to the stream that you're thinking about and not others within the state of Washington or elsewhere.  How good do you really know that?  And how variable might it be?  What might you miss looking at that from a water -- just a water balance system?
                  MR. MATTHEW BISCHOF:  So I use precipitation data from four weather stations within this watershed, which is four weather stations is quite a few within a watershed.  You're not always going to find that many.
                  DR. CLIFFORD WEISEL:  Now, that's to say how good you know the rainfall.  My question is a little different.  How good do you know the watershed for a stream, which is a simpler, might be a simpler question?  Do you really know for this stream, this is my entire watershed going into it and it doesn't go to another one, or there might be another area that -- if you know 90 percent then that tells me a lot.  If there are times that it's maybe you're 70 percent sure of it, that has a quite a bit of uncertainty to precipitation as to this.  And maybe the Washington is not the right answer to look at that.
                  DR. REBECCA KLAPER:  Doesn't the USGS has a pretty good map of all the surface watersheds, anyway.  Groundwater is a little bit more difficult, but --
                  DR. LISA NOWELL:  Yeah, and there are definitely watersheds where there's big water transfers.  You know, in the arid west, you're going to get a lot of trouble setting a drainage area.  So there definitely could be complications in some areas.  I'm sure Matt knows about that more than I do on that.
                  MR. MATTHEW BISCHOF:  Yeah, so for this site where I did look at precipitation, this one particular site it receives excess canal water periodically, so irrigation water is delivered through canals.  And this one, particular, some of the canals it can have a six day time in travel, so you'll have growers periodically taking water from this canal and they may have to -- they need to fulfill the needs for the growers.  And so a grower might need water and where they're pulling water to feed into the canal, way up at the top, the grower may not have time to wait for that additional water to come through.  So they have different reservoirs throughout that also feed into the canal.
                  So periodically there's excess canal water and that will spill into this one stream that I was looking at that doesn't -- and that's why it doesn't exhibit a natural flow pattern because of excess irrigation water that feeds into it, so you can have a spike in the hydrograph and it's not due to a rain fall event or anything.  It's just excess canal water that's feeding into it.
                  DR. CLIFFORD WEISEL:  Any idea how that might affect your ability to predict what the pesticide concentrations in the water system are, either by stream or by the precipitation?
                  MR. MATTHEW BISCHOF:  Well, it certainly makes it a little more difficult to identify a relationship between the hydrograph and the chemical because this canal water, it feeds growers that are both conventional and organic.  So you don't really want -- you don't want chemicals in this canal water.
                  And so which is why sometimes you see a negative relationship between pesticide concentration and the hydrograph.  So and in the case that I showed, you actually saw a positive relationship between the precipitation with the time in travel incorporated into it and that chemical.
                  So yeah, in irrigated areas, you are going to have a lot of complexities with hydrology of some of these sites.
                  DR. ROBERT CHAPIN:  Dr. Zhang?
                  DR. XUYANG ZHANG:  Hi.  Yeah, I second that.  I want to comment.  I think the same thing occurred in California, especially in the central valley.  Like the San Joaquin River watershed because of all this canals and the direction of the water flow doesn't really follow the elevation.  So that adds more complexity to probably the StreamCat data set that EPA uses and the regression analysis of sampling bias factors, I think because the NHD watershed boundary may not be accurate.  Especially in areas with very low relief, very flat areas.  And the boundary of the watershed doesn't really -- what really follow the elevation.  So that's just a comment.
                  And I have another question regarding the case study.  So you used to extrapolate the sampling bias factors based on a few sites that can apply with SEAWAVE-QEX model.  And then use that sampling bias factor to extrapolate to the rest of monitoring sites within the region, right?
                  So I think there's a large uncertainty involved in this extrapolation and I wanted to understand more about that.  And do you know the range of variability among different sites in the USGS dataset that you have calculated the sampling bias factor using the four pesticides?  What's the range of variability among different sites?
                  DR. KATRINA WHITE:  This is Katrina.  I think we agree that there's uncertainty that we're taking sampling bias factors taken for some specific sits and applying it to other sites.
                  And I'm going to ask Christine or Chuck who did the sampling bias factor work if they want to talk about some of the variation in the sampling bias factors they calculated for the USGS chemicals.
                  DR. CHRISTINE HARTLESS:  So this is Christine Hartless.  So certainly when we're doing the estimating the sampling bias factors either for the short term or for the long term, there was quite a bit of variability in those values that were estimated both within a site year, where you're looking amongst those realizations within a site year as well as across sites.  Or excuse me, across years within a site and as well as across sites.  And you could see that in some of the graphs that we showed earlier today.  And it goes into much more detail in the White Paper.
                  And I think --
                  DR. XUYANG ZHANG:  I was trying to -- yeah, I was trying to see if there's like a number showing the magnitude of differences.  But I searched through the White Paper.  I couldn't find any information about that.  If like for example 10, 10 fold or 100 fold in difference?
                  DR. CHRISTINE HARTLESS:  I think it depends upon whether or not you're talking about the variation across, say all realizations or not.  In the White Paper, probably for the short-term sampling bias factors, a place where you would find that, would be where we looked at the mixed model analysis and there should be some summary statistics in there looking at the -- that would summarize what the variability is for those different variance parameters.
                  DR. ANNA LOWIT:  So this is Anna Lowit.  This seems like a question; it might be helpful if we just looked in the paper overnight and follow up first thing in the morning to give a more accurate answer to the question.  If that's okay with the Chair.
                  DR. XUYANG ZHANG:  Okay.  That will be fine.
                  DR. ROBERT CHAPIN:  Perfect.
                  DR. XUYANG ZHANG:  Yes.  Thank you.
                  DR. ROCHELLE BOHATY:  If there is a specific question -- this is Rochelle Bohaty -- in the White Paper.  If you let us know the page that you're looking at, that would be helpful.
                  DR. KENNETH PORTIER:  This is Ken Portier.  You might also look in the supplemental files under the USGS regression results is where you can see the SAS results for the mixed-model analysis, that Dr. Hartless was talking about.  So you can actually find those estimates there.
                  DR. XUYANG ZHANG:  Do you have the page number?
                  DR. KENNETH PORTIER:  No.  It's in the supplemental files.  So you go to supplemental, USGS, regression, and somewhere in there is the SAS program and then the SAS output.
                  DR. ROBERT CHAPIN:  It's why God made a glass of wine.  Sit down with a glass of wine and your computer.
                  DR. KENNETH PORTIER:  this tells you how much I've spent, time.
                  DR. ROBERT CHAPIN:  I am completely impressed.
                  All right.  We're going to move on.  All right.  Let's see, Dr. Bohaty, can you wrap us up for this afternoon?  This impressive series of presentations?

SUMMARY AND WRAP-UP

                  DR. ROCHELLE BOHATY:  Hi.  Rochelle Bohaty here again.  So I want to thank you all for listening to all of our presentations.  And I did have several slides that I planned to previously give, but I'm going to skip those for the sake of time.  And also, I think our active listening skills, at least mine, are starting to wear thin a little bit.
                  But I want to quickly wrap up.  So you heard about a couple of tools, sampling bias factors, SEAWAVE-QEX as well as a weight of evidence that we can add to our toolbox.  And that these would just be another couple or few tools to add into that toolbox among others that we already have and use on an ongoing basis in our drinking water assessment.
                  We're trying to do our best with the tools and the data we have available to make use of the available surface water monitoring data.  And as Dr. Lowit mentioned earlier, we're in the later stages of our 15-year review cycle.  As such, we need some practical feedback from you folks on how we can best integrate the tools that we have and make the best use of the available data that we have.
                  We do have several challenging pesticides remaining that we need to conduct drinking water assessments for as part of that review process.
                  So aside from that, we're interested -- that's a short-term need -- but we're also interested in your recommendations on what we can consider on a long-term basis.
                  That can be including improving the methods and the tools that we use in our drinking water assessments, as well as recommendations you have on improving monitoring data so that we have better monitoring data to use in our drinking water assessments in the future.
                  So with that, if you have any clarifying questions, I can take those.
                  DR. ROBERT CHAPIN:  No clarifying questions.  Okay.  That's -- sorry, that's a rabbit hold we've been down already today.  So we're done with clarifying questions.
                  Okay.  So the -- I'm going to use the chairman's prerogative and say, break, shmake, we're late.  We're going to move to public comments.  If you need a bio break, take it yourself.  We're going to move to public comments.
                  Dr. Lowit, I think you're sitting in the seat that is going to be occupied by some of the --
                  DR. ANNA LOWIT:  Yes, I will happily move back again.
                  DR. ROBERT CHAPIN:  There you go.
                  Okay so --
                  DR. ANNA LOWIT:  So should our -- normally, our whole team vacates so the public commenters come up.
                  DR. ROBERT CHAPIN:  Okay.
                  DR. ANNA LOWIT:  So I think that's what we'll do.  Our whole team will vacate.
                  DR. ROBERT CHAPIN:  Okay.
                  DR. ANNA LOWIT:  And they'll just sit here.
                  DR. ROBERT CHAPIN:  Cool.

                                    [BREAK]

                  DR. ROBERT CHAPIN:  Okay.  So the first person up is going to be Dr. Perkins from Waterborne Environmental.  And as soon as the space is available, you can work yourself to the table.  I want to thank our EPA colleagues for number one, a buttload of work, making the White Paper and all the stuff that got sent us out to us.  And then you're calm, collected, presentations today covering a lot of ground.  I said break shmake.  Daniel, all right.  So Dr. Perkins.
                  So as soon as your slides appear, as if by magic, then you can -- there we go.  Cool.  Sweet.  Technology is wonderful when it works.  All right.  The microphone is yours.
PUBLIC COMMENTS

                  DR. DANIEL PERKINS:   All right.  Thank you very much.  My name is Daniel Perkins and I hold a Ph.D. from the University of Florida, Gainesville.  I spent my professional career as a regulatory scientist for two different patricide registrants and now as senior consultant at Waterborne Environmental.
                  We recognize the tremendous effort and thought that has been dedicated to the means by which monitoring data will be included in the FIFRA Drinking Water Assessment as well as to the risk assessment process, overall.
                  We submit that the work presented by the agency falls into two categories.  First, how to interpret the value of monitoring data in order to quantitatively use it in the risk assessment.  And two, how that value compares with other lines of evidence towards risk base decision making.
                  I will address these two categories specifically.  Related to the first category, how to interpret the value of monitoring data.  Several comments will be provided.
                  First, tools such as SEAWAVE-QEX were presented in the EPA docket materials to demonstrate that upper end concentrations for monitoring data could be estimated by SEAWAVE-QEX.  More information could be provided to define the utility of SEAWAVE-QEX as related to prediction on certainty associated with different monitoring data sampling characteristics and design, given that many analysis we've seen today have been termed preliminary.
                  Additionally, some data required by SEAWAVE-QEX may unnecessarily prohibit its application to some monitoring data, specifically related to streamflow requirements.  In our limited SEAWAVE-QEX testing on relatively high variability concentration time series data sets.  Results suggest that the source of streamflow data is less important than including streamflow at all.
                  In fact, the SEAWAVE-QEX requirement to include streamflow data as a covariate may be unnecessary in some systems.  This may create opportunities to use alternative time series imputation techniques for which streamflow is not required or available, such as ordinary and universal kriging as published in the literature on this topic.
                  It remains to be seen if the addition of covariates that are not colinear with flow might improve prediction accuracy.  However, the effort required to explore additional covariates that would replace streamflow is perhaps not cost effective considering that the highest potential payoff might only apply to an unclear number of relatively sparse monitoring data sets.
                  We also support further testing of SEAWAVE-QEX be completed toward its ability to adequately represent temporal trends associated with reservoirs.  Reservoirs are relatively common with approximately 25 percent of all flowing waters having a reservoir.  And likely exhibiting hydrodynamic behavior similar to a lake a pond rather than a flowing stream.
                  We urge further SEAWAVE-QEX model performance evaluation against concentration data that reservoir, measured at reservoirs that represent drinking water sources.  In relatively large data sets that contain both nontargeted and targeted monitoring data, the upper percentile concentrations are nearly always represented by targeted monitoring programs in highly vulnerable areas.
                  Additional guidance on the amount, type, and quality of data that would be acceptable for decision making would be helpful as there is likely a point at which there may be a limited benefit of even long term targeted monitoring.
                  While this SAP focuses on monitoring data specifically, we'd like to better understand the degree of detail and complexity that HED or the Human Effects Division will be able to accommodate in higher tier aggregate risk assessments.  It is clear that EFED will generate exposure estimates across a range of data quality and processing techniques.
                  It seems as though, HED may need to update risk assessment techniques in their higher tier assessments.  But this was not clear in the information provided to date.  We support the idea that all monitoring data be used to its potential and the characterization of drinking water exposure, as well as in the aggregate human health risk assessment.
                  Beyond statistical and interpolation tools to support the use of monitoring data and risk assessment, other approaches are available and can even offer additional information beyond imputation techniques presented by the Agency for this SAP.
                  Specifically, physically based modeling can be used to identify the value of monitoring data and support monitoring data through more technical calibration or traditional calibration and validation models that account for dynamic and static physical processes.
                  Physically based modeling approaches also offer the advantage of not only characterizing the value of monitoring data within its temporal bounds, but also simultaneously providing a framework to estimate long term exposure profiles.  They are especially useful when applied to high vulnerability areas that are protective of other lower risk areas.
                  Second, related to the second category, more specific guidance on the prioritization schemes of different information and data sources would be helpful in the weight of evidence approach.
                  It seems obvious that the highest certainty data would be ranked to have the most weight in a weight of evidence framework.  Thus it is our opinion that high certainty monitoring data in context with acceptable uncertainty, should supersede scenario based modeling assessments for risk decision making.
                  Furthermore, data obtained from targeted monitoring specifically designed to address a regulatory concern should be the highest priority in the weight of evidence framework and if designed properly, characterize the highest vulnerability in use areas that can be representative and protective of other geographies.
                  Finally, nontargeted monitoring data may still be weighted or ranked in the context of the standard EPA scenario modeling and sample bias factor adjusted monitoring values, monitoring data.
                  And we think that the weighting of information that is considered in the weight of evidence approach should be an explicit parallel topic.  Thank you.
                  DR. ROBERT CHAPIN:  Excellent.  Thank you very much.
                  Okay.  Dr. Mosquin from RTI.
                  DR. PAUL MOSQUIN: Thank you.  Oh it's the first slide.  Okay.  All right.  Before I discuss any issues related to bias factors and modeling, it's worth considering other methods which take advantage of available data to characterize extremes outside your concentration profiles.
                  For example, our method, which is based on survey sampling techniques estimates medians of distributions of site year percentiles using most of the available data, including site years where there is insufficient data for a direct estimate.
                  For example, the method could use either of the data sets depicted here where sampling is too infrequent to estimate either the 90th or the 99th percentile reliably.  Analysis such as these may be useful at earlier tiers in the assessment processes as they can use much more of the available monitoring data.
                  Moving on to bias factors.  The next slide provides a simple definition of the bias factor.  Given the sampling distribution for theta, an estimator for a target quantity theta, the bias factor theta over c if the c is the 5th percentile of the sampling distribution. The product bias factor in theta then provides the limit of a upper one-sided 95 percent confidence interval for theta.
                  Alternatively to put it another way, on your repeated sampling, which is not possible in practice, the successive interval is zero through the first bias factor adjusted value and then zero through the second bias factor adjusted value and so on, will contain theta 95 percent of the time, both for nominal and actual coverage.
                  And important point here is that this is the definition of a random interval, not a plane estimate.  Of course, exact calculation of the sampling bias factor requires the sampling distribution to be known, which is not true in practice.
                  In practice, the true bias factor is unknown and must be estimated using other sources of information.
                  This gets to our first point.  Although sometimes correctly describes as interval estimates, the White Paper also describes bias factor adjusted values using point estimate terminology.
                  For example, point estimates, protective estimates, or estimate with confidence.  Although suggestive of point estimation, this sampling bias factor adjusted values can be more than an order of magnitude larger than a simple point estimate, which can confuse readers.
                  Furthermore, interval estimates, if considered as point estimates have sample size dependent bias, decreasing with n.  For example, using an upper 95 percent normal interval for the mean, we have u -- if we were to take that as an estimator for the mean -- is equal to x-bar, plus a bias depending upon n.
                  The existence of sample size dependent bias is suppressed when bias factor adjusted value represented as a single number.  What is n?  Is it five?  Is it 150?  Or a bias factor adjusted value is a varying n are summarized as a group.  For example, the bending of samples of size 26 to 51.  Thus if numerical values are to be reported, we recommend that the following standard of statistical practice, bias factor adjusted values be stated as interval estimates.  That you provide both point and interval estimates, both should be useful in an assessment.
                  Also, any associated decision role for screening should be stated in terms of the interval and not a protective estimate.  A terminology that we think should be avoided as it is unnecessary.
                  In addition, if presented as interval estimates, we note that actual coverage may be poor for the four reasons listed here.  If also accompanied by a point estimate, we note that the analogous method for creating a point estimate of multiplicative factors, as we called it, was shown in our earlier study to be outperformed in many cases by log-linear interpolation of the sample data.
                  For these reasons, we only recommend the use of bias factor adjusted values for screening at this time.
                  If bias factor adjusted values are used for screening, certain sampling design and target quantity combinations may have extremely high false positive rates.  Because the current White Paper approach uses a maximum of medians of bias factors, it is expected that false positive rates are higher than in our earlier paper.
                  In separate work, we found the White Paper approach to be approximately equivalent to using the 90th percentile of a reference set of bias factors, so you calculate a set of bias factors and then based upon that set of bias factors, you choose some value which you're going to apply to the sample data as your sample bias factor.
                  And in previous works is that value has been the mean or the median and the method that's been proposed in the White Paper, according to this analysis, which is not provided, is -- we have found it to be approximately equivalent to the 90th percentile of that reference set.
                  To illustrate a scenario of many false positives, the X-axis of the figure gives true target quantity values for a test set of 46 site years.  The Y-axis gives bias factor adjusted values.
                  For a given site year target quantity value, all 90 possible bias factor adjusted values are arising from the systematic every 90 days sampling design are plotted.
                  The vertical and horizontal lines indicate the decision boundary of ten parts per billion, which divides the region into four screening outcomes.  A false positive region for true negatives.  A false negative region for true positives, and two correct result regions.
                  Clearly in this case, the false positive rate is very high.  More generally, the next figure gives false positive and false negative rates for varying values of the decision boundary for annual maximum using the same test set of 46 site years.  The X-axis provides the possible decision boundary values and the Y-axis, the overall false positive or false negative rates.  Plotted points are true target quantity values in the test set.  So those are the black dots in the lower left of the plot.
                  Notably, there can be very high false positive rate within the distribution of true values and this rate may persist well above the distribution of true values.  We also see that infrequent sampling such as every 90 days, tends to have higher false positive rates.
                  And note that this approach, which was described in detail on the right hand side because I only had five minutes, is -- does not depend in any way on the SEAWAVE-QEX estimation.  That's been eliminated here because we're using EMP data, which is everyday sampling within the summary so you can calculate what you think are these true bias factor values exactly more or less.
                  The next figure shows the same plot for the maximum 21 day rolling average.  False positive rates are somewhat lower, but still high within the distribution of true values and can be persistent beyond.
                  Finally, the last plot shows the results for the annual mean.  Again, there are high false positive rates within the range of the data extending beyond the extreme outlier at about nine parts per billion.
                  I note that these false positive rates must have contribution for more than the extreme site year.  And that is that if you see the plotted values beyond nine parts per billion, since those false positive rates are greater than 1 over 46, I believe then that indicates there's contribution from some of the other sites, which are down much further in the distribution.
                  The next slide --
                  DR. ROBERT CHAPIN:  Okay.  So you're well over your five, so can you bring it to a close, please?
                  DR. PAUL MOSQUIN:  Yeah.  I'm actually getting there right now.
                  DR. ROBERT CHAPIN:  Excellent.
                  DR. PAUL MOSQUIN:  Although I -- all right.  My next two slides provide examples from the acute and chronic case studies showing positive screens to be more likely, i.e., likely false positives in smaller sample sizes.
                  I'll skip that one.
                  Two other concerns, which should be investigated further are the correlation of the log bias factor and the log target quantity identified in our earlier study and the impact of moderate to high level censoring on bias factoring calculations.
                  So here is the summary slide to emphasizes some of these points.  And I'll just mention the first of these.  The terminology, protective estimate, confounds point and interval estimation and is not necessary for the interpretation of bias factor-adjusted values, i.e. interval estimates, or their use for screening.  And furthermore, the terminology of bias factor is somewhat unfortunate.  It arises, I think, from the downward bias of the standard estimators which is say, for example, linear interpolation is bias downwards, and as a result, there is a desire to correct for that bias.  So quantity was created which is the bias factor, and that corrects for the bias.  But it goes even further to provide you with a confidence interval.
                  And it would be much clearer, maybe to refer to something like this, than say, an interval factor, which would help for the use and interpretation since it does provide an interval estimate and that's the end of the time.  Thanks.
                  DR. ROBERT CHAPIN:  Excellent.  Thank you very much.
                  All right.  Dr. Aldworth, you have five minutes.
                  DR. JEREMY ALDWORTH:  Thank you.  I'm just trying to see how this thing work.  Okay.  Oh, how does this thing work.
                  DR. ROBERT CHAPIN:  It's still coming up.  The magic needs to happen up here and then you'll have it.
                  DR. JEREMY ALDWORTH:  Sure.
                  Okay.  So I'm going to comment on the SEAWAVE-QEX model, which I found to be a very interesting model and in fact, I think it gets the modeling fundamentals pretty much right, at least tested on the atrazine data that we have looked at.
                  For example, it applies a log transformation to your chemical concentrations and to atrazine data that works because for models like this, they really work better if you have normal data, so some sort of transformation that does that is good.
                  It assumes that noise is zero and we have actually investigated that at atrazine data, and we have found that the noise is approximately zero.  We did this looking at a semivariogram analysis.  And the nuggets is just about zero.  And we also did an independent measurement or analysis of replicated data in the lab.  And that kind of bears that out.
                  And having zero noise is potentially very good for prediction.  And by prediction I mean infilling missing days in the time series.  And the third thing is that we want to see -- serial correlation is always present in atrazine data at any rate.  And if you include that in the model, that's all, it's going to prove your prediction.
                  So those modeling fundamentals with sort CA QH (phonetic), I see those as strengths.  I see those as strengths.
                  Now, getting to the fixed trends, there may be a couple of questions like the linear trend term, which is always a useful term but what if the trend is nonlinear over several years?
                  The White Paper then suggests that you split the analysis into multiple time frames.  But that might be unnecessarily restrictive.  What if that renders those time frames just not analyzable and it's also less efficient?  There are other ways to do -- to work with nonlinear trends.
                  And so are some of the seasonal wave term.  What if that pattern varies across the years?  Or what if chemicals are used sporadically?  In fact, in one of the case studies, if you read the document, it says the usage is sporadic.  So there are a couple of questions about some of those fixed terms.
                  And the one that might cause a bit of concern is streamflow.  There are actually two terms.  There's median term flow anomaly and short term flow anomaly.
                  Now, in usage periods, flow and concentration may be positively correlated and that's a good thing for the model.  But in non-usage periods, they may be negatively correlated.  So how would you square that in the same model?  That could be an issue.
                  We have also found that short term flow anomaly is right skewed and that may cause overpredictions of extrema, which is what you are interested in.  And when you're working in the log scale, those overpredictions may not be that serious.  But you want to back transform to the original scale, and it blows up those overpredictions.  We have seen that.
                  I'm going to go on to the next slide.  Paul and I have been working on something similar.  We call it the universal kriging model.  But it's very similar to what is done in SEAWAVE.  We use the same fundamental modeling assumptions, log transformation of the concentrations, virtually zero noise and we apply serial correlation very similar to what is done in SEAWAVE.
                  The real differences are in the fixed terms.  We have much simpler fixed terms.  We just have a linear and quadratic trend terms, and for trend and seasonality and for short term flow anomaly, the problems we saw with skewness, we saw thought there may be a better approach.
                  We came up with the Box-Cox transformation of the flow term that basically removes the right skewness.  It makes it symmetric.  And that, we found, reduces the overpredictions of extrema.
                  Our much simpler model can be fit to a single year, as opposed to having to have three years.  So it's more useful when data limited or if you want to focus on a particular year of interest.  Maybe there's just a year where there's a concern.
                  We can also fit it to the usage season only.  So we don't have the complications of having to fit the same model to the non-usage season as well.
                  DR. ROBERT CHAPIN:  Okay.  So you're at -- you're past five minutes now okay.  So you're past five minutes.  So can you come to your last slide please?
                  DR. JEREMY ALDWORTH:  Okay.
                  DR. ROBERT CHAPIN:  Please and thank you.
                  DR. JEREMY ALDWORTH:  I am going to come to the second last slide and end here.  This how serial correlation is determined.  Okay, in the SEAWAVE-QEX model, it's determined by correlogram analysis applied to residuals after the SEAWAVE fixed terms have been modeled, which is good.
                  But the residuals of censored data in the model, I think are incorrectly determined.  In equation 9 in Vecchia's paper, he says the residual of censored data is the log of the censoring limit which is typically a constant, minus the fitted value.  So those censored residuals mimic the structure of the model, not the serial correlation of the data.
                  And these residuals are then fed into -- mixed in with the uncensored residuals and they're contaminate the correlation and the correlogram analysis.  And that worsens as the censoring rate increases all the way up to 70 percent.  I would recommend that you don't use censored residuals if they're going to be calculated like that.  Thank you.
                  DR. ROBERT CHAPIN:  Excellent.  Thank you very much.
                  Okay.  And our last public commenter today is Dr. Basu from CropLife America.
                  So you need to punch the little thing and get a little red light.
                  DR. MANOJIT BASU:  Okay.  All set now.  Good afternoon and thank you for the opportunity to provide oral comments at the US EPA Scientific Advisory Panel.  I'm Manojit Basu, managing director of Science Policy at CropLife America.
                  CropLife America is the national trade association that represents the manufacturers, formulators, and distributors of the wildlife crop protection and biotechnology products used by farmers, ranches, landowners, and home gardeners.  CropLife America encourages farming practices and supports environmental policies that are based on sound science and best practices in that respect and maintain U.S. farmers' ability to grow healthy and abundant food, feed, and fill.
                  CropLife America appreciates the progress made by the agency to develop tier drinking water risk assessment and inclusion of drinking water quality monitoring data for risk characterization.  CropLife America has provided extensive technical comments to the charged questions for this scientific advisory panel meeting.  I would like to request the SAP to consider the following three issues as they review the SEAWAVE-QEX and public responses to the charged questions.
                  First, developing a standard operating procedure for specifically identifying the conditions, situations, or decision points, for which modeling and monitoring refinements would be acceptable.
                  Second, providing more clarity on the application and use of monitoring data.  Post sampling bias factors for risk assessment.
                  And third, for clarity of communication, CropLife America recommends that bias factor, adjusted values be reported as interval estimates, together with associated point estimates.
                  This will allow a better understanding of the uncertainty associated with using sampling bias factors.  And the estimated statistics would be communicated clearly.
                  Once again, I'd like to thank the Agency and the SAP for your time this afternoon.  Thank you.
                  DR. ROBERT CHAPIN:  Bless you.  All right.  Okay.  Let me congratulate the panel for staying awake and engaged and focused through a significantly challenging topic, at least for me.  Both my neurons were challenged by it.
                  Let's see, so it is after 5.  We're not going to tackle the first question today.  We'll start off with that first thing in the morning.  Let's -- I hear moans of disappointment to my left, which I am going to resolutely ignore.
                  Let's see, we'll start off at 9.  Feel free to engage with other respondents to the questions that you've been assigned or have to the degree that you have energy for tonight.  And let me see, we've been told that there will be pastries in the breakout room.  Do we know when those pastries show up?  First thing in the morning.  Is that 9:00?  Earlier than that, 8:00.  Some indeterminate time there might be breakfast down there.  We don't know.
                  I think that's all.  I think that's all from here.  Okay.  You're done.  Thank you very much and thanks to our EPA presenters.  Nicely done.

                        [MEETING ADJOURNED FOR THE DAY]

                          OPENING OF MEETING - DAY 2

                  MS. TAMUE GIBSON:  Good morning.  We'd like to welcome everyone again for day two.  And thank each one of you for participating in today's meeting for the Approaches for Quantitative Use of Service Water Monitoring Data in Pesticide Drinking Water Assessments.  Thank you.  I want to, at this time, turn the meeting over to our Chair, Dr. Chapin.

                            PREVIOUS DAY FOLLOW UP

                  DR. ROBERT CHAPIN:  All right.  Thank you, Ms. Gibson.  Good morning, everyone.  Thanks for showing up fully caffeinated and ready to go.  Let's see.  So, today, we're going to start to answer the questions that the EPA has posed to us.  We get to the real meat of why we're here.
                  Let me just remind you that these microphones are directional; so if you've got the mic pointed at you and you turn this way, then all of the sudden the mic's not working very well.  It's helpful, not as much for the people in the room but for the transcripts, to be able to keep talking into the -- or make sure the microphone is in front of you as you turn your head.  That's number one.  Identify yourself for the transcript.
                  Let me also make the point to everyone that if you're -- if the point that you want to make for one of your answers has been made already, you can say, yeah, I second Dr. Nowell's comment about this, and this, and this.  Just -- that's perfectly fine.  And that gives our EPA colleagues a sense of how many people are sort of holding up their hands saying, yes, I support this concept.
                  And finally, our EPA friends are going to be using this methodology or a variance thereof in the very near future.  And so, what they would be happy to have from us, what they would really like to have from us, is a sense of the stuff that's really a must-do before this rolls out and starts to be fired in anger, versus the nice to have longer term stuff.
                  If we could maybe spend less time focusing on the nice-to-haves, and more time and be clear about the things that really need to happen before they roll it up and pull the trigger on the go.  That would be helpful for them.
                  Having said all that, let me just ask the Panel, are there any questions before we get started?  Okay.  All right; then let's do this.  So, the format is that they read the question, read the lead-in to the question and then read the question.  And then we'll turn to the various discipline leads and associates.  And I think I was asked to start off the charge question responses for this first one with -- yeah?
                  DR. REBECCA KLAPER:  Are the EPA folks going to be here the whole time that we're answering questions?
                  DR. ROBERT CHAPIN:  Yes.
                  DR. REBECCA KLAPER:  Okay.  To ask -- answer questions about the question basically?
                  DR. ROBERT CHAPIN:  Yeah, that's right.
                  DR. REBECCA KLAPER:  Okay.
                  DR. ROBERT CHAPIN:  And so what'll happen is -- excuse me -- after we give our responses, I'm going to turn to them and say, did we answer your question?
                  DR. REBECCA KLAPER:  Okay.
                  DR. ROBERT CHAPIN:  And they'll -- there will be a huddled conversation.  And then they'll look at us and go either, no, it's fine, or they'll turn to someone and say, could you please go over what you said about blah-blah-blah?
                  DR. REBECCA KLAPER:  Okay.
                  DR. ROBERT CHAPIN:  So we are here to answer their questions.  We don't necessarily -- we don't have the time to engage in a discussion with them; but if there's a clarification, something that you want, they're here to answer that for you.
                  DR. REBECCA KLAPER:  Okay.
                  DR. ROBERT CHAPIN:  Does that help?
                  DR. REBECCA KLAPER:  Yeah; thanks.
                  DR. ROBERT CHAPIN:  Cool.
                  Okay.  We ready?  All right.  I'll turn it over to our EPA colleagues to lead us into question number 1.  Who gets that lucky task?  All right.

                               CHARGE QUESTION 1

                  DR. ROCHELLE BOHATY:  All right.  Good morning, everyone.  Rochelle Bohaty here.  So we'll start with Charge 1.  And I will read the background.
                  Because of the sporadic nature of pesticide concentrations in surface water, monitoring programs with limited sampling frequency often do not provide a reliable estimate of the range of pesticide concentrations relevant to cancer and noncancer durations of toxic logical concern typically considered in pesticide human health risk assessments, for example, 1, 4, 21, and 365 days.
                  The Scientific Advisory Panels in 2010, 2011, and 2012 suggested that EPA look into use of a seasonal wave regression model, SEAWAVE-Q, developed by USGS to help interpret surface water monitoring data by generating daily pesticide concentration kymographs.
                  SEAWAVE-QEX, a modified version of SEAWAVE-Q is designed to estimate extreme, that is peak or daily average pesticide concentrations using streamflow as a covariate.  EPA evaluated SEAWAVE-QEX using high-frequency surface water monitoring data and streamflow data and concluded that SEAWAVE-QEX is a suitable tool for estimating pesticide concentrations for non-sample days, so that upper-end pesticide concentrations may be estimated.
                  EPA also evaluated alternatives to using streamflow, that is precipitation and stream stage, as a covariate.  This was done because daily streamflow data are not always complete or readily available and may be seasonal; that is periods of the year will not have flow record.
                  This would allow for sampling sites located in low-flow or non-flow systems where flow would not be a suitable covariate.

                             CHARGE QUESTION 1(a)

                  Charge 1(a):  Please discuss the strengths and weaknesses of using SEAWAVE-QEX to estimate short-term, that is 1-, 4-, 21-day average pesticide concentrations as well as longer-term, that is 365-day, pesticide concentrations.
                  DR. ROBERT CHAPIN:  Okay.  So we'll turn to the statisticians.  And who wants to go first?  Okay.  Dr. Portier.
                  DR. KENNETH PORTIER:  Thank you.  Good morning.  I asked to go first, I don't know why, except maybe I'm prepared this morning.
                  In answering this question, I felt I had to look under the hood.  I know in the presentation yesterday, there was a little bit of discussion, like, well, we wanted you to kind of get the gestalt of the model, rather than get into the details.  But I think if you ask car enthusiasts to a car show and don't open the hood, you're asking for trouble.
                  You have statisticians, hydrologists, and modelers and you haven't shown the model to them so that we can kind of talk in a little bit more than generality.
                  It's maybe a little thin, but I extracted this out of the Vecchia paper.  And I wanted to go through and just talk a little bit this morning about what SEAWAVE-Q and then SEAWAVE-QEX actually are doing in a little bit more detail, so that we can then talk about the different components.
                  SEAWAVE-QEX is a parametric regression model involving four explanatory variables that account for background pesticide concentrations, for a seasonality of pesticide use, for correlations with two components that together describe variation and streamflow about its mean.  And it also accounts for temporal autocorrelations.
                  And then this is discussed in Appendix A of the -- excuse me -- of the minutes, I think of the 2012 SAP.  Fitting the model is a five-step process involving 17 tasks that are implemented as 15 functions in R.  And all of this is described in the Vecchia 2018, USDOI USGS Scientific Investigations Report.
                  So, just quickly looking at this equation you see that they're fitting Log concentration to an overall average, which is this b0 or beta zero.  You have a parameter, b1, which is a weighting function against this W(t) form.