U.S. EPA FEDERAL INSECTICIDE, FUNGICIDE, AND RODENTICIDE ACT

                     SCIENTIFIC ADVISORY PANEL (FIFRA SAP)
                                PUBLIC MEETING

PEER REVIEW FOR THE APPROACHES FOR QUANTITATIVE USE OF SURFACE WATER MONITORING DATA IN PESTICIDE DRINKING WATER ASSESSMENTS

                  DOCKET NUMBER: EPA - HQ - OPP - 2019 - 0417
             TSCA SACC WEBSITE http://www.epa.gov/tsca-peer-review

                       HOLIDAY INN ROSSLYN AT KEY BRIDGE
                             1900 FORT MYER DRIVE
                              ARLINGTON, VA 22209
                             November 19-21, 2019

FIFRA SAP Chair

ROBERT E. CHAPIN, PH.D.
FORMER SENIOR RESEARCH FELLOW (RETIRED)
PFIZER GLOBAL RESEARCH AND DEVELOPMENT
GROTON, CONNECTICUT
DESIGNATED FEDERAL OFFICIAL

TAMUE L. GIBSON, M.S.
BIOLOGIST AND DESIGNATED FEDERAL OFFICIAL FIFRA SCIENTIFIC ADVISORY PANEL
OFFICE OF SCIENCE COORDINATION AND POLICY
U.S. ENVIRONMENTAL PROTECTION AGENCY
FIFRA SCIENTIFIC ADVISORY PANEL MEMBERS

GEORGE B. CORCORAN, PH.D.
CHAIRMAN AND PROFESSOR
DEPARTMENT OF PHARMACEUTICAL SCIENCES
WAYNE STATE UNIVERSITY
SONYA K. SOBRIAN, PH.D.
DEPARTMENT OF PHARMACOLOGY
HOWARD UNIVERSITY COLLEGE OF MEDICINE
CLIFFORD P. WEISEL, PH.D.
PROFESSOR
RUTGERS UNIVERSITY
RAYMOND S.H. YANG, PH.D.
PROFESSOR (EMERITUS)
COLLEGE OF VETERINARY MEDICINE AND BIOMEDICAL SCIENCES
COLORADO STATE UNIVERSITY
FQPA SCIENCE REVIEW BOARD MEMBERS

CLAIRE BAFFAUT, PH.D.
RESEARCH HYDROLOGIST
UNITED STATES DEPARTMENT OF AGRICULTURE
AGRICULTURAL RESEARCH SERVICE
CROPPING SYSTEM AND WATER QUALITY RESEARCH UNIT
VERONICA BERROCAL, PH.D.
ASSOCIATE PROFESSOR
DEPARTMENT OF STATISTICS
UNIVERSITY OF CALIFORNIA
TERRY COUNCELL, M.S.
COORDINATOR
TOTAL DIET STUDY
UNITED STATES FOOD AND DRUG ADMINISTRATION
TIMOTHY GREEN, PH.D.
RESEARCH AGRICULTURAL ENGINEER/HYDROLOGIST
UNITED STATES DEPARTMENT OF AGRICULTURE
AGRICULTURAL RESEARCH SERVICE
CENTER FOR AGRICULTURAL RESOURCES RESEARCH
IAN KENNEDY, PH.D.
SENIOR EVALUATION OFFICER
HEALTH CANADA
PEST MANAGEMENT REGULATORY AGENCY
REBECCA KLAPER, PH.D.
PROFESSOR
SCHOOL OF FRESHWATER SCIENCES
UNIVERSITY OF WISCONSIN-MILWAUKEE
ANDREW MIGLINO, PH.D.
PHYSICAL SCIENTIST
UNITED STATES FOOD AND DRUG ADMINISTRATION
CENTER FOR VETERINARY DRUGS
LISA NOWELL, PH.D.
RESEARCH CHEMIST
UNITED STATES GEOLOGICAL SURVEY
THOMAS POTTER, PH.D.
PRESIDENT AND PRINCIPAL SCIENTIST
INNOVATIVE SYSTEMS FOR EDUCATION AND ENVIRONMENTAL TECHNOLOGY
KENNETH PORTIER, PH.D. (RETIRED)
(FORMERLY OF THE NATIONAL CANCER SOCIETY)
CONSULTING BIOSTATISTICIAN
JOHN RODGERS, JR., PH.D.
PROFESSOR OF AQUATIC TOXICOLOGY AND ECOTOXICOLOGY
CLEMSON UNIVERSITY
JAMES SADD, PH.D.
PROFESSOR OF ENVIRONMENTAL SCIENCE
DEPARTMENT OF GEOLOGY AND ENVIRONMENTAL SCIENCE
OCCIDENTAL COLLEGE
XUYANG ZHANG, PH.D.
SENIOR ENVIRONMENTAL SCIENTIST (SPECIALIST)
CALIFORNIA DEPARTMENT OF PESTICIDE REGULATION
ENVIRONMENTAL MONITORING BRANCH

PRESENTERS

MARIETTA ECHEVERRIA, M.S.
DIRECTOR
OCSPP/OPP/EFED

DANA SPATZ, M.S.
BRANCH CHIEF, OCSPP/OPP/EFED
ROCHELLE BOHATY, PH.D.
SENIOR CHEMIST
OCSPP/OPP/EFED/ERB3
SARAH HAFNER, PH.D.
CHEMIST,
OCSPP/OPP/EFED
ALDO VECCHIA, PH.D.,
STATISTICIAN EMERITUS (RETIRED)
U.S. GEOLOGICAL SURVEY
MATTHEW BISCHOF, M.S.
NATURAL RESOURCE SCIENTIST

            WASHINGTON STATE DEPARTMENT OF AGRICULTURE

CHRISTINE HARTLESS, PH.D.
BIOLOGIST, OCSPP/OPP/EFED
CHARLES PECK, M.S.,
SENIOR ENVIRONMENTAL
ENGINEER, OCSPP/OPP/EFED
JAMES HOOK, M.S.
ECOLOGIST, OCSPP/OPP/EFED
KATRINA WHITE, PH.D.
SENIOR BIOLOGIST, OCSPP/OPP/EFED
JESSICA JOYCE, M.S.,
PHYSICAL SCIENTIST, OCSPP/OPP/EFED
PUBLIC COMMENTERS

DANIEL PERKINS, PH.D
WATERBORNE ENVIRONMENTAL, INC.
PAUL MOSQUIN, PH.D
RTI INTERNATIONAL
JEREMEY ALDWORTH, PH.D.
RTI INTERNATIONAL
MANOJIT BASU, PH.D.
CROPLIFE AMERICA

TABLE OF CONTENTS
OPENING OF MEETING	6
INTRODUCTION OF PANEL MEMBERS	11
INTRODUCTION AND WELCOME	18
OPP TECHNICAL PRESENTATION  -  OVERVIEW OF THE APPROACHES FOR QUANTITATIVE USE OF SURFACE WATER MONITORING DATA IN DRINKING WATER ASSESSMENTS	23
FRAMEWORK FOR CONDUCTING PESTICIDE DRINKING WATER ASSESSMENTS FOR SURFACE WATER	37
ANALYZING AND INTERPRETING SURFACE WATER PESTICIDE MONITORING DATA	57
EVALUATION OF SEAWAVE-QEX AS AN IMPUTATION TECHNIQUE FOR ESTIMATING DAILY PESTICIDE CONCENTRATIONS FROM PESTICIDE MONITORING DATA PART 1	70
EVALUATION OF SEAWAVE-QEX AS AN IMPUTATION TECHNIQUE FOR ESTIMATING DAILY PESTICIDE CONCENTRATIONS FROM PESTICIDE MONITORING DATA PART 2	113
DEVELOPMENT AND EVALUATION OF A SAMPLING BIAS FACTOR PROGRAM PART 1: SHORT-TERM	142
DEVELOPMENT AND EVALUATION OF A SAMPLING BIAS FACTOR PROGRAM PART 2: LONG-TERM	186
WATERSHED EXTRAPOLATION AND WEIGHT-OF-EVIDENCE APPROACH	198
DRINKING WATER ASSESSMENT CASE STUDY   -   A PESTICIDE WITH SHORT-TERM EXPOSURE CONSIDERATION	220
DRINKING WATER ASSESSMENT CASE STUDY  -  A PESTICIDE WITH LONG-TERM EXPOSURE CONSIDERATIONS	238
DEVELOPMENT OF SURFACE WATER MONITORING PROGRAM	263
SUMMARY AND WRAP-UP	296
PUBLIC COMMENTS	299
OPENING OF MEETING - DAY 2	322
PREVIOUS DAY FOLLOW UP	322
CHARGE QUESTION 1	325
CHARGE QUESTION 1(a)	327
CHARGE QUESTION 1(b)	395
CHARGE QUESTION 1(c)	422
CHARGE QUESTION 1(d)	450
CHARGE QUESTION 1(e)	468
CHARGE QUESTION 1(f)	484
CHARGE QUESTION 2	504
CHARGE QUESTION 2 (a)	505
CHARGE QUESTION 2(b)	529
CHARGE QUESTION 2(c)	534
CHARGE QUESTION 2(d)	547
CHARGE QUESTION 2(e)	549
CHARGE QUESTION 3	558
CHARGE QUESTION 3(a)	559
OPENING OF MEETING - DAY 3	589
PREVIOUS DAY FOLLOW-UP	589
CHARGE QUESTION 3(b)	607
CHARGE QUESTION 4	624
CHARGE QUESTION 4(a)	625
CHARGE QUESTION 4(b) - CASE STUDY 1	653
CHARGE QUESTION 4(c) - CASE STUDY 2	671

                              OPENING OF MEETING

                  MS. TAMUE GIBSON:  Good morning and thank you everyone.  I am Tamue Gibson and I'll be serving as the designated federal official for the Federal Insecticide and Rodenticide Act Scientific Advisory Panel for this meeting.
                  I want to thank our chair, Dr. Chapin, for agreeing to serve as the chair for FIFRA SAP meeting.  I also want to thank both the members of the panel and the public for attending this important meeting.  We appreciate your time and effort of the panel members in preparing for this meeting and taking account of your busy schedule.
                  In addition, I want to thank the EPA's Office of Pesticide Programs and my colleagues and staff at EPA for their very hard work in preparing for this important review of the approaches for quantitative use of surface water monitoring data and pesticide drinking water assessments.  You all should be commended in the amount of effort that you all had taken to prepare for this meeting.
                  By way of background, the FIFRA SAP is a federal advisory committee that provides independent scientific peer review and advice to the agency on pesticide and pesticide-related issues regarding the impact proposed regulatory actions on human health and the environment.  The FIFRA SAP only provides advice and recommendations to EPA.  Decision-making and implementation authority remains with the agency.
                  The FIFRA SAP consists of seven members which is called the permanent panel members.  The expertise of these members is augmented through the science review board which at ad hoc expert nominees that are considered and selected on an as-needed basis.  For the present meeting, five ad hoc -- excuse me.  The ad hoc expert peer reviewers have been added and we appreciate the contributions of these experts.
                  As the DFO for this meeting, I serve as a liaison between the FIFRA SAP and the agency.  I am also responsible for insurant provisions of the Federal Advisory Committee Act are met.  The Federal Advisory Committee Act of 1972 established a system that governs the creation, operation, and termination of executive branch advisory committees.  FIFRA SAP meetings are subject to all FACCO requirements.  These include an open meeting, timely public notice of meetings, and document availability which is provided via the Office of Pesticide Programs public docket at www.regulations.gov.
                  As a designated federal official for this meeting, a critical responsibility is to work with appropriate agency officials to ensure that all appropriate ethics regulations are satisfied.  In that capacity, panel members receive training on provisions of the federal conflict interest laws.  In addition, each participant has filled a standard government financial disclosure report.  I, along with our deputy ethics official for the Office of Science Coordination and Policy and in consultation with the Office of General Counsel, have reviewed these reports to ensure that all ethics requirements are met.  A sample copy of this form is available on the FIFRA SAP website.  The address for the website is noted on the meeting agenda.
                  The FIFRA SAP will review challenging scientific issues over the next few days.  We do have a full agenda and meeting times are approximate.  Thus, we may not keep to exact times as noted due to panel discussions and public comments.  We strive to ensure adequate time for agency presentations, public comments, and panel deliberations.
                  For all presenters, panel members, and public commenters, I would like to ask that you please identify yourselves and speak directly into the microphone provided since this meeting is being webcasted, recorded, and transcribed.  Copies of all EPA presentation materials and written public comments are available in the public docket at regulations.gov.  Any copies of presentations given during this meeting by public commenters will be available in the public docket within the next week.
                  Members of the panel are encouraged to fully consider all written and all public comments submitted for this meeting.  For members of the public that have not preregistered for public comments, please notify either myself or another member of the FIFRA SAP staff if you're interested in making a comment.
                  At this time, the agenda has been set; however, as we move through the proceedings, if time allows, we may be able to accommodate additional brief comments of five minutes or less.
                  As I mentioned previously, there is a public docket for this meeting.  All background materials, questions posed by the panel, by the agency, and other documents related to this meeting are available in the public docket.  Some documents are also available on the EPA FIFRA SAP website.  Again, the docket number and website are noted on the meeting agenda.
                  For members of the press, EPA media relations staff are available to answer your questions about this meeting.  Please address all questions to Robert Daguillard, EPA media contact.  His number is<PHONE_NUMBER>.
                  At the conclusion of the meeting, the FIFRA SAP will prepare a report as a response to questions by the Agency, background materials, presentations and public comments.  The final report also serves as meeting minutes.  We anticipate the final report and meeting minutes will be completed in approximately 90 days after the meeting.
                  Again, I wish to thank the panel for your avid participation, the program office for your diligent work, and my colleagues at the Office of Science Coordination and Policy and the public for your attendance.  I'm looking forward to both the Office of Pesticide Programs presentation and at this time, I turn the meeting over to our chair, Dr. Chapin.
                  DR. ROBERT CHAPIN:  I know his names' here someplace.  Thank you, Dr. Gibson.
                  All right.  So my name is Bob Chapin.  Welcome everyone.  Thank you for being here.
                  So at this point, let's go around the room and get our -- introduce the panelists and our EPA colleagues and with one phrase of our expertise and background and what brings us to the table.
                  My name is Bob Chapin.  I do developmental and reproductive toxicology and I spent my career at the National Toxicology Program and at Pfizer.

INTRODUCTION OF PANEL MEMBERS

                  DR. SONYA SOBRIAN:  Good morning.  I'm Sonya Sobrian.  I'm at the Howard University College of Medicine Department of Pharmacology.  My expertise is developmental neuropharmacology and toxicology.
                  DR. CLIFFORD WEISEL:  I'm Cliff Weisel from the Environmental and Occupational Health Science Institute at Rutgers University.  I work on exposure science.
                  DR. RAYMOND YANG:  Good morning.  I'm Ray Yang, retired professor from Colorado State University in toxicology and cancer biology.  And my interest -- research interest is in toxicology of chemical mixture.  As a result of that, I'm interested in biologically based computer modeling.  Thank you.
                  DR. ANDREW MIGLINO:  Hello.  My name is Andrew Miglino.  I work at the U.S. Food and Drug Administration Center for Veterinary Medicine.  I do environmental fate modeling of pesticides and veterinary drugs.
                  DR. REBECCA KLAPER:  My name is Rebecca Klaper.  I'm a professor at the School of Freshwater Sciences at University of Wisconsin-Milwaukee.  I study -- I'm an ecotoxicologist and I study the effects of emerging contaminants on freshwater organisms and also look at the presence and distribution of emerging contaminants in freshwater systems.
                  DR. GEORGE B. CORCORAN:  Good morning.  My name is George Corcoran.  I'm a professor of pharmaceutical sciences in the Eugene Applebaum College of Pharmacy and Health Sciences of Wayne State University in Detroit.  My area of specialty is chemical and drug toxicities.  I'm a toxicologist and my organ of greatest interest is the liver, but other organs as well.
                  DR. JOHN RODGERS JR.:  Good morning.  My name is John Rodgers.  I'm at Clemson University in South Carolina, professor of ecotoxicology.  And the area I'm interested in is fate and effects as well as the toxicology of materials as they come from our surfaces, our ag surfaces.
                  DR. LISA NOWELL:  Hello.  My name is Lisa Nowell.  I'm with the U.S. Geological Survey.  I'm a research chemist.  I've worked with the National Water Quality Assessment Program or project since the beginning of NAWQA actually.  And I've specialized in pesticides and other contaminants, monitoring and assessment, in particular implications for human health and ecological health.
                  DR. TIMOTHY GREEN:  Hi.  I'm Tim Green.  I'm with the USDA Agricultural Research Service in Fort Collins, Colorado.  I'm a hydrologist working in special variability and modeling.
                  MR. CHARLES PECK:  Good morning.  My name is Chuck Peck.  I'm a senior fate scientist with the Office of Pesticide Programs, Environmental Fate and Effects Division.  I deal with aquatic modeling as well as spray drift and volatility.
                  DR. CHRISTINE HARTLESS:  Good morning.  My name is Christine Hartless.  I'm a biologist in the Office of Pesticide Programs, the Environmental Fate and Effects Division.  I specialist in ecotoxicology and statistics with a smattering of working in lots of different areas of need within the Environmental Fate and Effects Division.
                  MR. MATTHEW BISCHOF:  Good morning.  I'm Matthew Bischof with the Washington State Department of Agriculture.  My background is in aquatic ecology and I've been monitoring for pesticides in surface water in the state of Washington since 2013.  I've been involved with SEAWAVE-QEX and I'm also bringing to the table experience in the field -- field research as well.
                  DR. SARAH HAFNER:  My name is Sarah Hafner.  I'm at the EPA.  I'm a chemist in the Environmental Fate and Effects Division and I've been with the Agency since 2017.
                  DR. ROCHELLE BOHATY:  Hi.  Good morning everyone.  I'm Rochelle Bohaty.  I'm a senior chemist in the Environmental Fate and Effects Division.
                  MR. DANA SPATZ:  My name is Dana Spatz.  I'm a branch chief in the Environmental Fate and Effects Division.  I manage a group of scientists who conduct environmental fates assessments, ecological risk assessments, drinking water exposure assessments.
                  MS. MARIETTA ECHEVERRIA:  Good morning.  I'm Marietta Echeverria.  I'm the director of the Environmental Fate and Effects Division of the Office of Pesticide Programs EPA.
                  DR. ANNA LOWIT:  Good morning.  My name is Anna Lowit.  By training, I'm a toxicologist, but for the last several years, I've had the honor of being the science advisor in the EPA's Office of Pesticide Programs.
                  MS. JESSICA JOYCE:  Good morning.  I'm Jessica Joyce.  I am with the Environmental Fate and Effects Division at EPA as a physical scientist.
                  DR. KATRINA WHITE:  Katrina White, fate scientist, Environmental Fate and Effects Division.
                  DR. THOMAS POTTER:  Can you hear me from here?  Good morning.  I'm Thomas Potter.  I'm currently an independent consultant.  My background and training is in environmental chemistry.  Prior to my work as a consultant, I served for 20 years as a research chemist with the Agricultural Research Service in Tifton, Georgia, the Southeast Watershed Laboratory where I led a series of programs looking at pesticide, environmental fate, monitoring, et cetera.
                  DR. JAMES SADD:  Hello.  I'm James Sadd, Professor of Environmental Science at Occidental College in Los Angeles and a research associate at the University of Southern California Program for Environmental and Regional Equity.  My particular expertise is in GIS and other GS spatial approaches and spatial statistics in addressing questions of environmental justice.
                  MR. TERRY COUNCELL:  I'm Terry Councell and in former agencies that I've worked for, I did a lot of water quality monitoring in community water systems for pesticides and worked as a hydrologist for the U.S. Geological Survey.
                  DR. IAN KENNEDY:  I'm Ian Kennedy.  I work for the Pest Management Regulatory Agency in Canada and I specialize in environmental fate and modeling of pesticides.
                  DR. CLAIRE BAFFAUT:  Good morning.  I'm Claire Baffaut.  I work for the USDA Agriculture Research Service in Columbia, Missouri.  I'm a hydrologist.  I do modeling and monitoring.  Thank you.
                  DR. VERONICA BERROCAL:  Hello.  My name is Veronica Berrocal.  I'm an associate professor in the Department of Statistics at University of California Irvine.  My expertise is in spatial statistics and statistical modeling of environmental exposure and environmental health.
                  DR. KENNETH PORTIER:  Good morning.  I'm Ken Portier, retired biostatician with 40 years' experience in agriculture environment, environmental health, and public health.
                  DR. ROBERT CHAPIN:  And we have Dr. Zhang on the phone.
                  DR. XUYANG ZHANG:  Good morning.  This is Xuyang Zhang.  I'm a senior environmental scientist with California Department of Pesticide Regulation, Environmental Monitoring Branch.  My area of specialty is pesticide transport and fate modeling, and now effective management of pesticide monitoring data and GIS.
                  DR. ROBERT CHAPIN:  Okay.  Thank you all.  An august group.  I'm looking forward to learning a bunch in the next couple days.
                  All right.  So I think we're ready to go.  We've got -- the largest part of today is going to be presentations from our EPA colleagues about sort of setting the stage for the questions they're going to be asking us.  We have some bio-breaks built in, but when -- if you have your own sense of urgency, I encourage you to address that.  Just do what needs to be done.
                  Let's see.  So I guess without -- and I'll just remind us that we need to -- because the meeting is being recorded and then transcribed, please get very close to the microphone and speak into it and just give your name.  You don't need your affiliation and your expertise and all that stuff, just your name.  That helps with the transcription.
                  So with that, I think we'll turn it over to Dr. Echeverria and she can have at it.

INTRODUCTION AND WELCOME

                  MS. MARIETTA ECHEVERRIA:  Great.  Good morning.  My name's Marietta Echeverria and I am the director of the Environmental Fate and Effects Division of the Office of Pesticides Programs.  It is my honor to welcome you to this consultation of the FIFRA Scientific Advisory Panel.
                  The Environmental Fate and Effects Division, or EFED like we like to call ourselves, is charged with conducting ecological risk assessments and drinking water assessments which feed into a human health risk assessment for conventional pesticides as part of the registration and registration review programs.  Our work provides critical information to agency decision-makers ensuring that pesticides, when used as directed, meet safety standards and to not cause unreasonable adverse effects to human health or the environment.
                  I want to welcome and thank you.  I want to thank Dr. Chapin, the chair, for this meeting.  I also want to thank the permanent panel members as well as the ad hoc panel members for being here today.  I thank you for the work that you've done in preparation for this meeting, and I thank you for the input and advice that you'll provide throughout our time together over the next couple of days.
                  I also want to thank the public for participating in this process.  This is a meeting that's open to the public and I'm looking forward the public comments that we will get at the end of the day.
                  I also want to thank my good friend, Ms. Tamue Gibson, the designated federal official.  I've known Tamue for a very long time and it has been a pleasure working with you in this role.  I thank you, Tamue, and I thank the SAP staff for all the work that they've done organizing this meeting and for serving as the liaison between the agency and the panel.
                  The feedback we receive as part of the scientific advisory panel is very important as EPA moves forward with integrating surface water monitoring data in pesticide risk assessments.  We've been working with considerable determination to advance our methods for estimating pesticide concentrations in drinking water and the tools and approaches that we are presenting today are related to our goal, a goal that is shared by many of our stakeholders which is to make better use of surface water monitoring data quantitatively in our drinking water exposure assessments.
                  There's a full agenda today and you will be hearing a series of presentations from my team on how we conduct drinking water assessments using a tiered approach, the use of predictive models in monitoring data to estimate exposure, and how new tools and approaches, specifically SEAWAVE-QEX and sampling bias factors can improve our ability to use surface water monitoring data in refined assessments.  We will also present two case studies that illustrate the use of these new tools in the context of an actual assessment.
                  I want to thank my team for the excellent work they have done in preparing for this meeting and in particular, for developing the White Paper, the drinking water assessment framework document, and the SEAWAVE-QEX SOP and the case studies, all of which are in the docket for this meeting.
                  At this time, I am very happy to introduce the team that is presenting today as follows:  First, we have Mr. Dana Spatz here on my right.  Dana has served as the management lead for this project.  Dana will kick things off with an overview of the approaches that you'll hear about throughout the day, followed by Dr. Rochelle Bohaty to Dana's right.  Dr. Bohaty has been the technical lead for this project.  She will present on the framework followed by a presentation on the interpretation of monitoring data.
                  Next, we will have Dr. Sarah Hafner next to Rochelle.  Sarah is going to present on our evaluation of SEAWAVE-QEX, followed by two presentations from Dr. Christine Hartless and Chuck Peck, all the way to my right over here.  Both of whom are presenting on sampling bias factors.
                  Following the sampling bias factors presentations, Dr. Hartless with make another presentation on our efforts to extrapolate across watersheds and a weight of evidence approach for evaluating relevancy.  Then we will have Dr. Katrina White, all the way over here on the left, and Ms. Jessica Joyce who will present the two case studies.
                  And finally, we are very fortunate to have Dr. Matthew Bischof over here next to Sarah, I believe, representing Washington State Department of Agriculture.  Matthew brings an important perspective to this meeting.  He has on the ground experience and he will be discussing Washington State's program followed by the work he did in collaboration with the EPA team on the use of the tools in designing monitoring programs generally.
                  I also want to take a minute here to thank some additional contributors, specifically Dr. Aldo Vecchia from the USGS who developed the SEAWAVE-QEX model.  Dr. Vecchia is participating with us over the phone today.  I also have Dr. Anna Lowit, the OPP's senior science advisor and Ms. Jan Matuszko over here, the acting EFED deputy director, both of whom provided excellent guidance and support to this team as they were preparing for this meeting.
                  I want to thank Mr. Trip Hook who I believe is participating with us over the phone who provided GIS assistance and to the many other staff members in EFED and OPP, some of whom are here today who provided input towards this efforts.  Again, welcome.  We look forward to your input, your advice, and the discussion.  Thank you.
                  I'll turn it over to Dana now, I believe.

OPP TECHNICAL PRESENTATION  -  OVERVIEW OF THE APPROACHES FOR QUANTITATIVE USE OF SURFACE WATER MONITORING DATA IN DRINKING WATER ASSESSMENTS

                  MR. DANA SPATZ:  Good morning.  My name is Dana Spatz and I'm a branch chief in the Environmental Fate and Effects Division where I manage a group of scientists that conduct ecological risk assessments, drinking water exposure assessments.  I led the team of scientists that will be presenting here today.
                  I'm going to provide you with some background and context to the project you will be hearing about this morning, what brought us to this point, why this effort is important for evaluating the potential exposure to pesticides in drinking water --
                  PARTICIPANT:  A little louder please.
                  MR. DANA SPATZ:  -- recommendations from previous FIFRA SAP panels, and the great value the meeting this week with bring going forward.
                  The Office of Pesticide Programs regulates the distribution, sale, and use of pesticides and establishes maximum levels for pesticide residues in food.  The Federal Insecticide, Fungicide, and Rodenticide Act or FIFRA, provides the federal regulation of pesticide distribution sale and use.  With some limited exceptions, all pesticides distributed or sold in the United States must be registered by EPA.
                  Before EPA may register a pesticide under FIFRA, the applicant must show, among other things, that using the pesticide according to label directions will not generally cause unreasonable adverse effects on the environment.  FIFRA defines the term "unreasonable adverse effects on the environment" to mean, one, any unreasonable risk to man or the environment, taking into account the economic, social, and environmental costs and benefits of the use of any pesticide or two, a human dietary risk from residues that result from a use of a pesticide in or on any food inconsistent with the standard under Section 408 of the Federal Food, Drug, and Cosmetic Act.
                  Section 408 of the Federal Food, Drug, and Cosmetic Act, or FFDCA, authorizes EPA to set tolerances or maximum residue limits for pesticide residues on food.  Under the FFDCA, EPA must determine that aggregate exposure to the pesticide residues is safe, that is, there is a reasonable certainty of no harm from aggregate exposure to the pesticide before issuing a tolerance.  As part of this determination, EPA must consider aggregate risk from exposure to a pesticide from multiple sources, food, water, residential, and other non-occupational sources.
                  The Agency has developed approaches for assessing drinking water exposure and for refining those assessments to meet its obligations to assess aggregate exposure to the pesticide chemical residue under the FFDCA.
                  The Office of Pesticide Programs conducts roughly 100 to 150 drinking water assessments per year.  These assessments are done to support various regulatory actions including when a new pesticide is first submitted to EPA for registration, when subsequent new uses are requested to be registered, and in registration review where all registered pesticides are reevaluated every 15 years.
                  As you will hear about later, the Office of Pesticide Programs uses a tiered process for conducting drinking water assessments.  The level of effort needed to complete a drinking water assessment varies for each case and can range from a few days to several months depending on the exposure and toxicity profile, the number of uses considered, and previous work completed for the pesticide under evaluation.
                  Concentrations of pesticides in both surface water and ground water sources are provided in drinking water assessments.  The concentration estimate recommended for use in the human health risk assessment considers both model estimated and measured pesticide concentrations.  The results are then integrated into the human health dietary risk assessment.  This can be done using a single estimated concentration and a deterministic assessment or distribution of concentrations for probabilistic assessment.
                  Most drinking water assessments conducted by the Office of Pesticide Programs have relied primarily on model estimated concentrations while monitoring data, if available, was summarized and used to characterize the potential for contamination of drinking water.  The reasons why monitoring data have not been used quantitatively include inadequate sampling frequency, limited number of years sampled at a site, and geographical scope of sampling compared with the pesticide use pattern.
                  While there have been a few cases where monitoring data was considered robust enough to use quantitatively in drinking water assessments, most stakeholders and the Office of Pesticide Programs would like to make better use of the available monitoring data and additional monitoring programs going forward.  Together with the sophisticated surface water modeling tools currently in use, the Office of Pesticide Programs wants to use surface water monitoring data to estimate pesticide concentrations in drinking water to best reflect actual exposure, increasing confidence in risk conclusions.
                  We believe the recently developed tools you are going to hear about today can be used to estimate pesticide concentrations between sampling events in available monitoring data to address uncertainty in having non-daily sampling data.  In addition, we will present a weight of evidence approach we believe can be used to describe potential exposure across the landscape to help address uncertainty in having limited geographical representation.
                  When reviewing the methods presented here, it is important to consider that models must be of appropriate quality and rigor for risk assessment and risk management applications; that is, we do not want a complex model to use as a screen at lower tiers.  It's just not practical when conducting -- when considering the workload of 100 to 150 drinking water assessments each year.
                  Before models are implemented for use, the Office of Pesticide Programs requests input and peer review from the scientific community to make sure we got it right.  Models used in a regulatory process need to be publicly available including the underlying code so that anyone can check our work.  And finally, we need a way to easily update the models we use to incorporate evolving risk assessment methodology.
                  The Office of Pesticide Programs has developed and implemented tools and methods to estimate pesticide concentrations for use in drinking water assessments.  These tools and methods have undergone incremental improvements over the last 20 plus years, specifically 18 different scientific advisory panel meetings have focused on tools and methods specific to the conduct of drinking water assessments.  This includes seven on the use of monitoring data in pesticide risk assessments.  These panels of experts have included representatives from academia, industry, and government over the last two decades.
                  The most notable scientific advisory panel meetings on this topic include those related to atrazine where they discuss methods for estimating exposure between measured samples.  Generally, the SAPs supported the concept of sampling bias factors and encouraged exploring the USGS tool SEAWAVE for estimating exposure between sampling events and for developing additional daily pesticide chemographs for further development of bias factors.
                  Here's a quote from an SAP meeting in 2010 on the need for frequent sampling if one wants to use surface water monitoring data quantitatively in a drinking water assessment.  The concept behind this important recommendation has driven much of the work we are presenting here today, especially with the evaluation of the SEAWAVE-QEX model, particularly for pesticides with acute or short term toxicity endpoints of concern, the sampling frequency must be often enough to be able to compare exposure values with toxicity levels of concern.
                  The first bullet on this slide emphasizes the idea that it may not be appropriate to combine monitoring results from one site with those of another as each individual community water system site is unique, thus pooling of data across sites can lead to erroneous conclusions.
                  In 2012, the Scientific Advisory Panel supported EPA's work in developing sampling bias factors and encouraged use to incorporate as much additional monitoring data as possible before attempting to correlate or predict sampling bias factors with watershed characteristics.  We followed up on this recommendation and will be presenting the work later this morning.
                  The SAPs in 2011 and 2012 agreed with EPA that the use of statistical time series models, such as the SEAWAVE-Q model, that use a covariate such stream flow may help to overcome the limited sampling frequency issue characteristic of many surface water monitoring programs.  Other methods, such as kriging, were explored and discussed but the Scientific Advisory Panel favored SEAWAVE-Q as being more practical for use in EPA drinking water assessments.
                  The development of USGS -- the development by USGS of SEAWAVE-QEX and our evaluation of the statistical model, which we will go into detail later today, was in direct response to the recommendations made by earlier scientific advisory panels.
                  So as you've seen, it's been several years that EPA has been looking at ways in which to make more use of surface water monitoring data in its risk assessments and there is a long history of Scientific Advisory Panel meetings that have gotten use to where we are today.  What you've seen in the White Paper and what you will hear presented today, we believe, follows up on the valuable input we've received over the years from the scientific advisory panel and our stakeholders.  We are grateful for all of those who have provided recommendations and we have taken those recommendations seriously and incorporated then into the work presented here.
                  The feedback we receive this week with help inform several important and far reaching decisions the Office of Pesticide Programs will need to make when it comes to the future of using surface water monitoring data quantitatively in drinking water assessments.  These include minimum data quality and quantity criteria, design of future monitoring programs, and future steps in method development.
                  Thank you for serving in this FIFRA Scientific Advisory Panel and we look forward to a vigorous and insightful discussion.  It's important that we get this right as we continue the work to provide growers tools to help produce a healthy food supply while at the same time, protecting our drinking water.  Thank you.
                  DR. ROBERT CHAPIN:  Any questions from the panel?
                  DR. CLAIRE BAFFAUT:  You mentioned that currently the way that the drinking water assessments are conducted are based on predictive models.  And I'm asking this question because in the document that we've read, there's several instances where it stated that the SEAWAVE-QEX would improve on current methods, but I -- could you explain a little more what those current predictive models are and how they're used?
                  MR. CHARLES PECK:  So this is Chuck Peck with EFED.
                  So currently in our drinking water modeling, we use a model that we call the pesticide and water calculator.  It looks at different fate and chemical parameters, degradation in soil, degradation in water.  It looks at different watershed characteristics, runoff parameters, as well as meteorological data that could be representative of different areas of the country.  And we use that information combined with label information, label rates, retreatment intervals, the maximum amount that can be applied in a particular year, to develop a -- what we call an estimated drinking water concentration in modeling.
                  Again, this looks at maximum label rates, minimum retreatment intervals, and what we're trying to do with the SEAWAVE-QEX model is to take information, monitoring data that's provided on the ground by different sampling agencies, as well as covariate data to try and inform those estimates that we develop using our pesticide and water calculator model.
                  DR. THOMAS POTTER:  Hello.  This is Thomas Potter.  I guess I'm going to have to stand or lean for this.  I have a follow-up question to that, and it came to mind as I was reading through the documents is, you know, how often do you run into the circumstance where the models don't give you appropriate answer and when you need to move to another tier or another level such as in using tools like SEAWAVE-QEX or whatever?
                  MR. DANA SPATZ:  So it's not that the model gives an inappropriate answer, it's that it's a screening level assessment that as we go through the tiers it becomes more and more refined using more and more real world data.  Generally, we're at Tier 2 using the pesticide and water calculator.  If we still have risks of concern, we move to Tier 3 where we'll be using typical application rates and more site-specific information.  And what we're looking to do with monitoring data is start looking at that more intensely using sampling bias factors at Tier 3 and then into Tier 4 if we still need to refine.
                  DR. THOMAS POTTER:  Can I follow up with that?  Yeah, perhaps I misspoke when I said inappropriate.  I said it didn't necessarily answer the question of whether or not there was a risk.  But what I wanted to get at is, you know, and I'm trying to figure out how critical it is to have this tool, SEAWAVE-QEX, in your risk assessment tool kit and that relates to basically coming down to registration decisions and how often do you think something like SEAWAVE-QEX would be critical in terms of coming up with an answer in terms of a particular registration of an active ingredient?
                  MR. DANA SPATZ:  We've estimated approximately 10 percent of the chemicals would benefit from going through a higher tier assessment.
                  DR. THOMAS POTTER:  Okay.  Good.  Thanks.  Exactly what I'm looking for.
                  DR. ROBERT CHAPIN: Okay.  Next up.
                  DR. TIMOTHY GREEN:  May I ask a quick question?
                  DR. ROBERT CHAPIN:  A quick question, just for clarification.
                  DR. TIMOTHY GREEN:  Okay.  Just -- well, I'm sorry.  This is historical context.  There was a lag from 2000 to 2007 then almost annual meetings and then haven't had another one of these since 2012.  I just wondered if there was a reason.
                  MR. DANA SPATZ:  There was a lot of focus on atrazine at the time and we were using atrazine because of the robust monitoring data to help develop the tools and with other priorities and things.  We've been working on it for a number of years and it's, you know, mature enough to bring to you today.
                  DR. ROBERT CHAPIN:  Okay.  Dr. Bohaty.

FRAMEWORK FOR CONDUCTING PESTICIDE DRINKING WATER ASSESSMENTS FOR SURFACE WATER

                  DR. ROCHELLE BOHATY:  Hi.  Good morning.  Again, my name is Rochelle Bohaty.  I'm a senior chemist in the Environmental Fate and Effects Division and I will present the drinking water assessment framework used by the Office of Pesticide Programs to conduct drinking water assessments for pesticides in surface water.  I think this presentation will help address some of these questions that we had just a few minutes ago.
                  So during my presentation you will hear the purpose of documenting the process we use to conduct drinking water assessments.  In addition, I will provide you with an overview of the framework including a high-level summary of each assessment tier.
                  The purpose of the drinking water assessment framework is to document standard and longstanding practices for conducting drinking water assessments by creating a clear and concise yet comprehensive document that describes the tiered process.  In addition, we hope the document will increase consistency across assessments and foster better cross-division coordination within the Office of Pesticide Programs.
                  The document also provides transparency to stakeholders on the conduct of drinking water assessments.  Lastly, the document integrates new methods for interpreting surface water monitoring data that you will hear about today that will help address spatial and temporal uncertainties in available surface water monitoring data.
                  As Mr. Spatz mentioned in his presentation, the Office of Pesticide Programs conducts roughly 100 to 150 drinking water assessments per year.  As such, the Office of Pesticide Programs has come up with a tiered assessment approach to prioritize resources.  Lower tiers are easy to use, require simple input, and produce conservative output.  Higher tiers require more input and produce more detailed and realistic output.
                  The drinking water assessment refinement process or framework consists of four tiers and a scoping phase.  I want to point out that the refinement process doesn't necessarily mean that concentrations go down but that we have considered more data and have a better understanding of the values increasing our confidence in the assessment conclusions.  Progression through the tiering process only proceeds until concentrations are below the level of concern or a risk management decision can be made.  We don't continue to refine because we can or the data support refinement.  We only refine until we can stop; that is meaning a risk management decision can be made.
                  Determining the need to advance through the tiers is done by comparing the estimated pesticide concentrations to a drinking water level of concern, also sometimes referred to as a drinking water level of comparison or a DWLOC.  Simply put, a drinking water level of concern is used as a benchmark of how much space in the risk cup is available for residues in drinking water for a pesticide under evaluation.
                  EPA developed the concept of a risk cup to facilitate risk refinement when considering aggregate human health risk to a pesticide.  The risk cup is the total exposure considering multiple pathways or routes of exposure including through food, shown on this image in blue, residential shown in orange in this figure, and water, the remaining top third of the cup in this example, allowed for a pesticide considering its toxicity and the required toxicity safety factors.  Exposures exceeding the risk cup are of potential concern.
                  When drinking water assessments are completed for pesticides, both model-estimated as well as measured concentrations are considered; however, it should be noted that these two data sources can tell you very different things.  For example, monitoring data reflect typical use while modeling can reflect maximum label use rates, which are often different than the typically used rates.
                  The aggregated exposure assessments can be deterministic, using a point estimate or a probabilistic assessment by using the entire distribution of possible pesticide concentrations as mentioned earlier.  Drinking water assessments are completed on different spatial scales from national to regional or sub-regional levels, depending on the pesticide use under evaluation.
                  Most assessments are completed on a national scale; that is, one high end estimate covers the entire country for all use sites.  However, regional or sub-regional scale assessments are conducted when needed.
                  As this figure shows for a national scale assessment, generally a Tier 1 or a Tier 2 level assessment provides a single upper end pesticide concentration as a starting point.  This could be based on solubility or a model-estimated concentration.  This value is expected to rarely, if ever, be exceeded.  The estimate doesn't represent a concentration that everyone across the country is exposed or an average concentration across the country, but rather a pesticide concentration that could occur in a vulnerable pesticide use area.
                  If necessary, the assessment may be refined to a regional scale drinking water assessment.  In doing this, the United States is subdivided into smaller areas, typically water resource regions or hydrologic unit code 2 to focus on areas where the pesticide concentrations may be higher than the drinking water level of concern.  This can be done as part of a Tier 2 or Tier 3 level assessment.
                  A sub-regional scale drinking water assessment provides generally the highest spatial resolution of all assessment scales, further zooms in on vulnerable pesticide use locations where the estimated pesticide concentrations may be higher than the drinking water level of concern.  These spatially refined assessments occur at the Tier 3 or the Tier 4 level assessment.
                  I'm going to briefly walk you through the tiered assessment process beginning with scoping.  Scoping is the first step in any drinking water assessment.  It is also a process that is done in consultation with the entire pesticide team that includes members from several divisions within the office of pesticide programs.  This is the step where we gather all the information we have about a pesticide to determine the level of effort necessary to conduct the drinking water assessment.  For example, this is when the health effects division provides the Environmental Fate and Effects Division the drinking water level of concern.
                  Scoping also occurs through the tiered assessment process.  So for example, the assessment team will scope out next steps following each tier and skipped tiers as appropriate.
                  A tier 1 drinking water assessment provides bounds on potential pesticide concentrations in drinking water.  A tier 1 assessment is not intended to present actual drinking water exposure concentrations.  Tier 1 assessments are used to screen out pesticides having low risk potential or identify pesticides that require additional refinement.
                  Since it is not feasible to directly measure pesticide concentrations for every location and time for a pesticide of interest, EPA uses data from one or more localized sites and scenarios from across the country to develop conclusions about pesticide concentrations across the landscape.  At Tier 2, this includes the use of aquatic models to derive upper bound pesticide concentrations based on maximum label rates and for vulnerable pesticide application locations.
                  We know that concentrations of pesticides are a local or regional phenomenon, but temporal variation based on pesticide characteristics, use, and environmental conditions; however, use of the upper bound estimate is expected to provide a protective measure of all uses across the United States.  Monitoring data are summarized at a very high level for characterization and generally, only to make sure that an important exposure pathway is not missed using modeling.
                  Tier 2 assessments are designed to identify pesticides with low potential for risk from residues in drinking water on a national basis.  Most of the drinking water assessments completed by the Office of Pesticides are Tier 2 level assessments.
                  Tier 3 assessments are less common and require more resources.  For example, usage data, that is data on the actual use of the pesticide, can be considered and is often done so on a regional basis.  Often pesticides are not applied at the maximum single or maximum yearly application rate as specified on the label.  This ultimately impacts the real concentrations of these pesticides in drinking water.  Tier 3 is used to further identify pesticides in corresponding uses or use sites where exposure in drinking water is not a concern and may be done on a more localized spatial scale, such as on a water resource region or HUC-2 basis.
                  Tier 3 is where EPA is looking to introduce methods presented here as a first step in estimated pesticide concentrations using surface water monitoring data.  This includes use of sampling bias factors from four pesticides as a screen to identify pesticides that exposure in drinking water is not likely a concern.  You will learn more about sampling bias factors in upcoming presentations by Dr. Hartless and Mr. Peck.
                  DR. ROBERT CHAPIN:  Dr. B., quick question.
                  DR. VERONICA BERROCAL:  Yes.  Dr. Berrocal here.  I just had a quick clarification.  So no data is used in tier 1?  No modeled or no monitoring data is used for tier 1?
                  DR. ROCHELLE BOHATY:  So currently we don't have a Tier 2 model and so we usually go to a higher screen at Tier 2, but we could use data such as solubility as a tier 1 screen to see if there is a potential concern.
                  Tier 4 level assessments provide the highest temporal and spatial resolution with watershed and scale-based concentrations.  Tier 4 assessments utilize both model estimated concentrations as well as monitoring data to determine pesticide concentrations in drinking water.  It further defines regions or areas of the country with a high degree of confidence where there may be a risk concern versus where risks to human health from pesticide residues in drinking water is unlikely.
                  Tier 4 is also where we are looking to introduce methods presented in the White Paper to estimate pesticide concentrations from available surface water monitoring data.  This includes use of SEAWAVE-QEX to estimate daily pesticide concentrations at specific locations that can be used directly in drinking water assessments to estimate pesticide concentrations in drinking water.
                  You will learn more about SEAWAVE-QEX in upcoming presentations by Dr. Hafner.  In addition, the estimated SEAWAVE-QEX chemographs can be used in the development of pesticide-specific sampling bias factors.
                  Finally, in Tier 4, we would use a weight of evidence approach to determine the relevancy of available monitoring data to source drinking water.
                  To demonstrate the drinking water assessment framework, Dr. White and Ms. Joyce will be providing presentations on two drinking water assessment case studies illustrating how EPA conducts drinking water assessments for pesticides.  These case studies will focus on showing how sampling bias factors, SEAWAVE-QEX, and a weight of evidence approach can be used to quantitatively incorporate surface water monitoring data into drinking water assessments.  In addition, these case studies serve to provide a better understanding of the monitoring data typically available for pesticide drinking water assessments including the quality and the quantity of the data.
                  We as you, the panel, to think about the case studies in light of the drinking water assessment and implementation of the tools for making better use of available surface water monitoring data.
                  With that, I can take any questions you have before I move on to my second talk.
                  DR. KENNETH PORTIER:  So I was thinking of our risk cup and Tier 2 where you kind of use one number for the drinking water and I guess you're thinking in terms of a maximum kind of drinking water exposure concentration.  Does that concept follow over to the residential and food exposures at the same time?  So is that cup a real conservative estimate, you know, the maximum potential food concentration, the maximum potential residential exposure, the maximum surface water?
                  DR. ANNA LOWIT:  Anna Lowit.  So the Health Effects Division, who is sister division to EFED that sits here who conducts the dietary, the foot assessment and actually does what we call aggregate, to add up the food, the water, and the residential when it does occur, and we do have a standard tiering process for the food assessment that's largely parallel to what you see from EFED where we began with very high end assumptions like 100 percent of the crop is treated around the United States, whereas in higher tier we'll do a percent crop treated for example.  Or we may start with the type of residues we use in the more screening level assessments as opposed to a later probabilistic assessment we would use while monitoring data at the point of sale.
                  For dietary, we have a nearly identical side-by-side process so that the goal in time is to get them more aligned so a Tier 4 across the divisions match.  It's a work in progress, but we do have conceptionally similar situation.
                  DR. KENNETH PORTIER:  So when you get to something like Tier 4, you're really dealing with three time series and looking at coinciding or noncoinciding exposures where food exposure may be down or water exposures might be high before the crop is gathered and goes into commercial use, so you're water might drop but your food consumption going -- are you at that level of complexity?
                  DR. ANNA LOWIT:  Most of the time, not.  So in most of our single chemical risk assessments, we make assumptions on the food side of things that there is very little annual changes in variability in food.  We know with Thanksgiving coming, a lot of people will eat turkey and cranberry and things like that, but for most of the year, most food consumption programs have demonstrated that people eat berries year-round.
                  And keep in mind, we also get food monitoring data from imports, so USDA and FDA and others are monitoring the food throughout the year.  People eat strawberries year-round, even if their only grown in certain parts of the country in certain times of the year.  So we generally assume for most of the year that food is a constant from a time point of view.
                  There are situations, like a couple of our very complicated accumulative risk assessments, which you sat on a few of those panels I think, where we have done a far more sophisticated time course of food consumption.  Because when you add up 30 organophosphates in a single risk assessment, you have to throw every tool known to risk assessors at that to get a realistic estimate.  But in most cases, we don't.
                  DR. ROBERT CHAPIN:  Ray.
                  DR. RAYMOND YANG:  Ray Yang.  Thank you very much for your presentation.
                  I have three little questions for clarification okay.  You good folks do 100 to 150 assessments a year of which, what is the percentage of tier 1, 2, 3, 4, roughly?
                  MR. DANA SPATZ:  Obviously, it would depend on which chemicals we're looking at in that year, but the vast majority would be a Tier 2.  Some would go into Tier 3, but the vast majority is Tier 2.
                  DR. RAYMOND YANG:  All right.  All right.  Second question, a follow-up to Tom's question, do you consider inhalation and dermal exposure at all?
                  DR. ANNA LOWIT:  The short answer is yes.  So if you go back to the risk cup, there is that, I think it's pink or orange segment that would be considered residential.  So our Health Effects Division also conducts residential and occupational exposure assessment where we do evaluate workers in the field who may be handling pesticides, either they're spraying or their picking fruit for example, and those may have inhalation or dermal contact.
                  We also conduct assessments of children and adults who apply pesticides around their homes, to pets, ant baits in the house, turf uses, the whole gamut, mosquito use.  We do public health spraying assessments.  So yes, we are counting for those routes.  But the focus of this meeting is just on that drinking water piece from the risk cup.  Not on how it's all put together, but the tools of how we want to refine them.
                  And I think one comment, I think a few of you are having questions about the frequency that we're going to need to use these tools.  So it may not be that often as a percent basis, but keep in mind that when we're going to need to use these tools are also some of the most important tools to our growers and are some of the most heavily used pesticides in the country.  So although there's frequency that may be not very often, when we need these tools, they're extremely important to us.  And as we get in the last couple of years of this 15-year review cycle for registration review, some of our most complicated assessments are yet to come in the next two years where we would really like to apply these tools to these really complicated cases.
                  DR. RAYMOND YANG:  Thank you.  Third question, from your presentation I gathered the modeling started to come in at level 2, Tier 2; am I correct?
                  DR. ROCHELLE BOHATY:  So historically, we did have a tier 1 model, but as of the last few years, we stopped using that tier 1 model because often times when we went to Tier 2, the concentrations were higher and so we needed to come up with a better screen at the tier 1 level.  So currently we don't start modeling until Tier 2.
                  DR. RAYMOND YANG:  Okay.  So when you start to do modeling, are you dealing with SEAWAVE?
                  DR. ROCHELLE BOHATY:  No.  So where we're proposing to integrate SEAWAVE would be in a Tier 4 level assessment.
                  DR. RAYMOND YANG:  Okay.  Thank you.
                  DR. ROBERT CHAPIN:  One more question would be okay.  Yep.
                  DR. VERONICA BERROCAL:  This is Dr. Berrocal.  I was just confused about this drinking water level of concern.  If I'm looking at these pictures, it seems to me that everything is basically estimated or extrapolated.  What is the food exposure, there is a natural exposure, so is there any uncertainty that is considered when this drinking water level of concern is established?
                  DR. ANNA LOWIT:  So the drinking water level concern as it notes there is a benchmark.  In our lower tier assessments, that benchmark is a deterministic benchmark.  So literally it's just some simple math of the hazard assessment point of departure accounted for the uncertainty factors defines the size of that.  So the size of an organophosphate risk cup is actually quite small, whereas something like glyphosate, the risk cup is actually quite large because as determined by the size of that point of departure and the number of uncertainty factors.
                  And so the rest of it's just math.  The amount of food that we have calculated in the tier 1 or Tier 2 assessment plus those determined as residential and it's essentially just a subtraction exercise.  What's available left.
                  As we move up to the higher tiers and the more complicated Tier 3s and the Tier 4s, we're doing more distributional probabilistic assessments, so the distributions of the water are brought into the distribution of the food, so the mathematics become a lot more complicated.
                  We don't, however, in our risk assessments, provide error bars on margins of exposure, things like that when we provide the assessments to the risk managers.  They will provide a probabilistic distribution of different exposures of different percentiles, for example, but not error bars on those.
                  DR. VERONICA BERROCAL:  So, follow-up question.  I guess my curiosity, or my misunderstanding is whether this drinking water level of concern, you are using it to determine which pesticide will go to Tier 2, right?  So is that right?
                  DR. ANNA LOWIT:  Yeah.  So for example, let's say, one of the -- I think one of the case studies that you'll see later this afternoon demonstrated in lower tiers that the food had been calculated.  There's residential exposures to one of the examples that we're showing.  So if you essentially in a lower tier subtract out what's left for water, that becomes a DWLOC and so as they do modeling, you'll see in some of the graphs that they'll show later that those model estimates exceed that level, that we crossed that benchmark.  When we cross that benchmark, that's where it began to trigger the conversations between the Health Effects Division and EFED on where is the quickest and fastest to do refinements to get back within the risk cup.  And then if we make -- often, you know, as we move in the latter parts of the reg review, we're going to be -- both divisions are going to be making heavy refinements on a number of chemicals to do probabilistic assessments at these higher tiers.  So it becomes far more complex.  It's not necessarily even a linear approach because the divisions are working together to be efficient, because we have to get through 100 a year, so we have to be efficient with how we use our time.
                  DR. ROBERT CHAPIN:  Tom.
                  DR. THOMAS POTTER:  Tom Potter.  I just had the quick question, where are safety factors applied?
                  DR. ANNA LOWIT:  So generally, the workflow for risk assessment is that the uncertainty factors are applied through the hazard assessment, the hazard characterization level.  So a point of departure is selected from toxicology studies generally, and in most cases, the Pesticide Office uses animal studies with the standard 10X safety factor for intraspecies and interspecies.
                  We generally do two different types of risk metrics.  One is a reference dose approach and one is a margin of exposure.  Essentially, they're inverses of the other, but the uncertainty factors come from that hazard assessment side.
                  DR. ROBERT CHAPIN:  All right, Dr. B., back to you.
                  DR. ROCHELLE BOHATY:  Thanks.  Before I start my next presentation, I want to provide a little bit more context to make sure that you guys are following the process in terms of modeling.  So the current modeling, or the pesticide water calculator as Chuck mentioned earlier, it doesn't consider monitoring data.  It relies on environmental fate data that's submitted to the agency that's conducted in a laboratory.  And so based on the tier that we're at, we can make those, the input parameters into the model more or less conservative.  But I just wanted to say, because that's very different than SEAWAVE where you would actually input like measured concentration data.
                  Does that introduce any other follow-up clarifying questions?

ANALYZING AND INTERPRETING SURFACE WATER PESTICIDE MONITORING DATA

                  DR. ROCHELLE BOHATY:  Okay.  So next I'm going to talk about how the Office of Pesticide Programs analyzes and interprets surface water monitoring data.  During this presentation, I will provide you with some background information to provide context around the drinking water assessments conducted by the Office of Pesticide Programs.  This includes a summary of drinking water sources as well as factors that drive pesticide concentrations in surface water.  I will also highlight data considerations and finish with providing an overview of the methods that you will hear about today.
                  So more than 282 million people, that's 87 percent of the total population, receive drinking water via community water systems.  In 2015, the total amount of drinking water derived from surface water was about 61 percent of all drinking water.  Surface water sources of drinking water include rivers, streams, lakes, and impoundments such as reservoirs.  Community water systems rely on surface water are distributed across the 21 major water resource regions located in the United States, 18 of which are in the contiguous states.
                  This figure highlights how the community drinking water systems vary across the landscape; that is, including watershed size, percent agriculture, and we know how environmental conditions vary across the landscape.
                  This figure begins to illustrate how drinking water is a local phenomenon.  No one is drinking an average pesticide concentration derived from across the landscape and there are many situations where the water the person is drinking is coming from the same location.  On the short-term basis, for example, someone could go to home, work, or school, and still be consuming drinking water from the same source; however over longer periods of time, for example, someone may live in the same community over time whereas other folks may choose to move or travel elsewhere.
                  Further complicating things is the fact that pesticide concentrations vary across the landscape for reasons other than previously described.  Factors driving pesticide concentrations in surface water include use patterns, that is the application rate, the retreatment, the timing, the practices that's used by the farmer, and those drive things like runoff and drift.  Physical chemical properties such as volatilization and solubility, environmental fate properties including transformation, sorption, or leaching, and waterbody characteristics including size, type, and flow, and soil and rainfall characteristics also play a part.  These factors need to be kept in mind when evaluating monitoring data.
                  Monitoring data are often available from several sources including federal, state, and academic, as well as other sources such as registrant submitted data.  Often the design and the implementation of the programs vary dramatically and as such are designed to be used differently.  Furthermore, the quality of the data also varies.
                  Often we are criticized for the fact that there is 30,000 or 50,000 samples, for example, available for a given pesticide when we're conducting a drinking water assessment, and yet we only use the data for characterization purposes.  This is due to the uncertainty and the temporal and spatial occurrence of the sampling.  For example, there are many sites where only one sample has ever been collected or one sample was collected once per year and maybe for only a few years.
                  Therefore, temporal and spatial information needs to be considered including pesticide usage, varying environmental conditions across the landscape and across time, as well as the program design to better understand the relevancy of the data before integrating it into a drinking water assessment.
                  To give you a better idea of the impact of sample frequency, this figure here presents a real pesticide chemograph, that is, daily pesticide concentrations measured from grab sample.  These data are shown with the black dashes.  The Y-axis is concentration while the sampling date is show on the X-axis.  The orange dots illustrate a hypothetical sampling program that occurs every 14 days.  It is important to note that often we rarely have access to data sampled this frequently.
                  You can see that these hypothetical samples all miss the peak or the highest measured concentrations of the original daily chemograph shown with the black dashes.  As a result, these subsample data would generally underestimate the real exposure.  The less frequent the sampling becomes, the more and more likely you are to miss peak concentrations.
                  We have done this same type of analysis with samples coinciding with peak measured concentrations and while you capture the one-day peak concentrations, you can underestimate as well as overestimate concentrations depending on the concentration duration of interest.  To address the uncertainty and non-daily sampling data, we need a robust method for estimating concentrations between sampling events such that we don't bias the estimates compared to the real concentrations.
                  As important as being able to estimate concentrations between sample events is the ability to estimate concentrations across space, or at least understand how concentrations change across space.  Use and environmental characteristics are major drivers.  This figure showing total precipitation across the United States is one example to show how various parameters affecting pesticide concentrations in surface water change across the landscape.  Going with this example, exposure estimates in one location may be completely different that another location.
                  One could simply be due to the difference in rainfall, or it may be less direct as the difference in pest pressure or the corresponding pesticide applications due to rainfall.  In this figure, the darker brown shading indicates the dryer the area while the darker turquois, the wetter the area.  Conditions such as precipitation can vary on large scale, that is across the country, or on a much smaller scale.  For exampling, looking at the state of Washington or Orgon, you have the rain shadow from the Cascade Mountains.  The precipitation varies as much within this HUC-2 region as it does across the country.
                  So you can see how these parameters impacting pesticide concentrations in surface water can quickly become complicated when trying to understand how it impact pesticide concentrations in surface water.
                  Two prior FIFRA Scientific Advisory Panels recommended that EPA consider SEAWAVE-Q, a USGS model that fits a regression to pesticide concentration data from using stream flow samples to assess variability in the trends, to impute pesticide concentrations with less than daily sampling.  The panel, however, had concerns that peak concentrations may not have been captured by SEAWAVE-Q.
                  USGS made modifications to SEAWAVE-Q so that peak concentrations could be captured and recently, in 2018, released a new model called SEAWAVE-QEX.  This model imputes concentrations using a daily covariate such as flow.  It is specifically designed to capture higher end concentrations.  It produces multiple equally probably pesticide chemographs as shown on the figure, the different colored lines.  The Y-axis in this figure is concentration with the sampling here shown on the X-axis.  You can see that this tool is able to account for the seasonal nature of pesticide occurrence.  See the repeating peak exposure periods on the figure.  Stay tuned for more in depth description as well as the evaluation of SEAWAVE-QEX in Dr. Hafner's presentation.
                  Although many of you may be familiar with the concept of sampling bias factors, I am going to give you a basic definition.  A sampling bias factor is a multiplier for monitoring data to account for the uncertainty associated with non-daily sample frequency.
                  As you saw on the earlier slide, a reduction in sample frequency can have a substantial impact on the various summary statistics calculated from the data, depending on the number of samples and when sampling occurs.  A sampling bias factor can be applied to a summary statistic such as an annual or a 21-day rolling average concentration from less than daily monitoring data to ensure that at least some percentage of the time the sampling bias factor adjusted concentrations is equal to or higher than true unknown concentrations.  Currently, the sampling bias factors are developed such that the adjusted values will cover the true unknown maximum concentration 95 percent of the time.
                  While you will hear that SEAWAVE is a useful tool, it does have minimal data requirements as previously mentioned.  Often available surface water monitoring data are not suitable for use in SEAWAVE-QEX.  As such, sample bias factors are needed to best integrate these data into drinking water assessments.
                  The method used to develop short-term sampling bias factors was previously presented to a scientific advisory panel while the method used to develop long-term sampling bias factors has not.  You will hear more about the development and evaluation of sampling bias factors in presentations later today by Dr. Hartless and Mr. Peck.
                  There are several different sources of surface water monitoring of data available to the Office of Pesticide Programs and it's important to remind you that not all data are created equal.  For the work you will hear about today, we selected and used several different sources of monitoring data which I have listed on this slide including the National Center for Water Quality Research, commonly referred to as Heidelberg University, the U.S. Geological Survey, Washington State Department of Agriculture, the Atrazine Monitoring Program, the Atrazine Ecological Monitoring Program.
                  These programs have different study designs including sample frequency, the number of years sampled, the relevancy to pesticide use and source drinking water, and site characteristics.  Each program is summarized in the White Paper in more detail and you will hear more about these data and how they were used in our work as we go through the next several presentations.
                  On this slide, I have displayed a flow chart to help orient you with the workflow process for the development and evaluation of the tools you're to hear about today.  This chart, which you will see in the next several presentations, illustrates how the different methods that will be presented fit together in the overall process.  This chart also follows the outline of the White Paper.  I should note though, that this process is different than how we propose to implement the use of the tools in our tiered assessment approach as I described in my previous presentation.
                  This chart begins with non-daily pesticide concentration data and covariate data such as flow.  These data are processed through SEAWAVE-QEX, producing daily pesticide concentration data or chemographs.  These estimated concentrations, or if data are available on a daily time scale, you can input these into the sampling bias factor programs, to develop either short or long-term sampling bias factors.  In the end, regression analysis was completed for sampling bias factors once developed.
                  I can take any of your questions.

                  DR. THOMAS POTTER:  Tom Potter here.  Can you expand a little bit more on the regression analysis of the sample bias factors, or are we going to hear about that later?
                  DR. ROCHELLE BOHATY:  Yeah, you will hear about that later.
                  DR. THOMAS POTTER:  Okay.
                  DR.  CLIFFORD WEISEL:  Cliff Weisel.  So, you indicated that not all monitoring data can be used, which I certainly understand.  But do you have a specific set of criteria that you can share with us to make a decision, or is it you look at them and put your best guess at that point?
                  DR. ROCHELLE BOHATY:  So, in the White Paper we did put forth some data quality -- or data quantity information for you guys to weigh in on.  Both for the use of SEAWAVE-QEX as well as the use of sampling bias factors.  Up until now, it's been based on the assessor's best judgement.
                  DR. VERONICA BERROCAL:  Dr. Berrocal here.  Can you go one slide back?  So, does it exist, pesticide data that is measured daily?  My understanding from reading the White Paper was that that data does not exist.
                  DR. ROCHELLE BOHATY:  So, there are a few pesticides where we do have daily data, it's just not many.  Atrazine is one example.  There are a few other chemicals where we do have daily pesticide data, but it's not extensive; which is why we're looking for methods to be able to expand.
                  DR. CLAIRE BAFFAUT:  Claire Baffaut here.  Do I understand correctly that if the best -- if you have daily pesticide data, when does the sampling bias factor equal 1?  I mean, why do we have a sampling bias factor if we have daily data?
                  DR. ROCHELLE BOHATY:  Yeah, so if you had daily data, you would use it, but I think this question is probably better addressed in the sampling bias factors when they talk about the development of it.  But you could develop a bias factor based on that to be applied elsewhere where you wouldn't have the daily data.
                  DR. RAYMOND YANG:  Just a quick clarification as long as we have this -- oh, I'm Ray Yang.  As long as we have this on the screen, now, in your presentation, you mentioned something about probabilistic assessment, okay.  Is it somewhere here or later on in the risk assessment process?
                  DR. ROCHELLE BOHATY:  So the probabilistic component that we've talked about now has been more in the context of how we would use the data in the human health risk assessment.
                  DR. RAYMOND YANG:  Later?
                  DR. ROCHELLE BOHATY:  After we give the number to our sister division.
                  DR. RAYMOND YANG:  Okay.  Thank you.
                  DR. ROBERT CHAPIN:  We good?
                  DR. JAMES SADD:  James Sadd.  You mentioned or you introduced the idea of the covariates and how they would be used.  One of the problems with the monitoring data is that it is only available on the level or on the scale or on the temporal relation that is available.  How about the covariate data?  Does that also vary where you don't have enough or the proper covariate data in order to produce a sampling bias factor?
                  DR. ROCHELLE BOHATY:  That's correct.
                  DR. ROBERT CHAPIN:  Okay.  So I've got a little break now; is that right?  Are we -- okay.  Let's take 10 minutes so we logged at 10:25 so let's meet at 10:35.  We'll reconvene in 10 minutes.  Thank you.
                                    [BREAK]

                  DR. ROBERT CHAPIN:  Okay.  We're going to resume now with presentations from Dr. Hafner and Dr. Bohaty, right?  Am I right?  Okay.
                  DR. SARAH HAFNER:  I don't think the slide is working.

EVALUATION OF SEAWAVE-QEX AS AN IMPUTATION TECHNIQUE FOR ESTIMATING DAILY PESTICIDE CONCENTRATIONS FROM PESTICIDE MONITORING DATA PART 1

                  DR. SARAH HAFNER:  Good morning.  My name is Sarah Hafner and I'm a chemist with the Environmental Fate and Effects Division.  This morning I'll be talking about the evaluation of SEAWAVE-QEX which we've broken into two presentations to allow for clarifying questions on the methods before discussing the results.
                  So this is a brief outline of the first presentation on the SEAWAVE-QEX.  First I'll cover some of the background of the model and the objectives of the evaluation.  Next, I'll give a brief overview of some of the key aspects of the model, and finally, I'll discuss the data preparation and methods we use for evaluating the model.
                  So if you recall this process overview from Rochelle Bohaty's presentation, the use of SEAWAVE-QEX is a critical step in a large project investigating tools for using monitoring data quantitatively in risk assessment which combines non-daily measured pesticide data with a daily covariate like streamflow to estimate daily pesticide concentrations.
                  In addition to proposing the use of SEAWAVE-QEX for direct estimates of pesticide concentrations in drinking water assessments, SEAWAVE-QEX-generated chemographs are being used to developing sampling bias factors for pesticides without daily monitoring data, a process which will be discussed more in later presentations by Christine Hartless and Chuck Peck.
                  Daily pesticide concentrations are needed for estimating short-term exposures and to calculate sampling bias factors.  As a result, the use of monitoring data quantitatively through daily monitored concentrations has been limited to a few chemicals.  To account for uncertainty from infrequent sampling, previous SAPs in 2011 and 2012 recommended that EPA consider the model SEAWAVE-Q to interpolate or fill in daily pesticide concentrations for data sets with less than daily sampling; however, those SAPs expressed concerns that SEAWAVE-Q may not adequately estimate peak concentrations for direct use in short-term exposure assessments.
                  In response, the USGS released a new time series regression model called SEAWAVE-QEX in 2018.  Like SEAWAVE-Q this model interpolates sparse pesticide concentrations using a daily covariate such as streamflow; however, this model differs from SEAWAVE-Q in that it is designed to capture higher end concentrations.  Additionally, SEAWAVE-QEX can produce multiple equally probable estimates of the daily concentrations.  With each chemograph constrained by the measured input data.
                  It is important to note that this input data must meet some minimum requirements to get a proper fit by the model.  The model generally needs at least three years of sampling, at least 12 samples per year, and a detection frequency greater than 30 percent.  There is some flexibility in these requirements based on a condition such as chemical properties and sampling structure as discussed in the publication which is released along with SEAWAVE-QEX in 2018 and we are looking for feedback on data characteristics to consider when using flexibility in these minimum requirements.
                  The broad objectives of this evaluation were to determine if SEAWAVE-QEX can produce daily concentrations with reasonable accuracy such that the peak concentrations are not underestimated or grossly overestimated and if so, to determine how to appropriately adapt the tool to our drinking water assessment process.
                  SEAWAVE-QEX consists of a simple code that's run in R with several modifications that are available to adjust for poor model runs.  Like SEAWAVE-Q, there are two inputs to SEAWAVE-QEX, measured pesticide data and continuous streamflow data.  When USGS gate stations are used from streamflow, the user only needs to supply the pesticide data as SEAWAVE-QEX will download UGSG streamflow data based on the gate station number.
                  The two most important output files produced by the model for risk assessors are the diagnostic plots for evaluating the model fit and the file of daily concentrations or chemographs.  SEAWAVE-QEX can produce as many chemographs as the user specifies with the default value being 100 chemographs.
                  Diagnostic plots are used to determine if the model fits appropriately which is essential to having confidence in the estimated concentrations.  If not, some default input parameters in the model can be changed to improve fit.  If it is determined the model fit well and no further changes are needed, then the chemographs can be used.  In addition to these files, the model provides documentation of the model fitting parameters for each run.
                  To understand the basics of how the model works --
                  Question?
                  DR. VERONICA BERROCAL:  Yes, Dr. Berrocal here.  So I just wanted to ask about the pesticide data.  Is that like an average daily concentration or it is what referring to the White Paper graph samples which I think is just referring to a specific time point.
                  DR. SARAH HAFNER:  Usually they're graph samples.
                  DR. VERONICA BERROCAL:  So the pesticide data is basically specific to a given time point while the streamflow data is some sort of measurement that is representative of the entire day?
                  DR. SARAH HAFNER:  Yes.
                  DR. VERONICA BERROCAL:  So they're not on the same time scale, those two measurements?
                  DR. SARAH HAFNER:  Yes.
                  DR. VERONICA BERROCAL:  Okay.
                  DR. SARAH HAFNER:  Okay.  To understand the basics of how the model works, it's helpful to understand how SEAWAVE-QEX uses the covariate data.  The covariate is used to calculate midterm and short-term flow anomalies which are unitless measures of the variability in the covariate, in this example, streamflow.
                  The midterm flow anomaly represents seasonal variability and is calculated from the streamflow data for the proceeding 30 days.  An example is given in this figure which shows the seasonal variation in streamflow over the years of a SEAWAVE-QEX simulation.
                  On the other hand, the short-term flow anomaly captures high frequency variability and flow as can be seen in the figure on this slide in comparison to the midterm flow anomaly from the previous slide.  As a result in flashy systems that have a quick response to precipitation events, the short-term flow anomaly will tend to be positively skewed compared to the midterm flow anomaly.  The use of both the midterm and short-term flow anomalies in the model allows for estimation of concentrations based on seasonality as well as estimations of upper end concentrations to capture all variability of pesticide concentrations.
                  So over the next few slides, I'm going to walk through a few examples of diagnostic plots including examples of acceptable model fits as well as an example where more work is needed.  This figure is an example of one of the diagnostic plots generated by the model and shows just one of the chemographs in gray with concentrations in log-based ten scale on the Y-axis and years of the simulation on the X-axis.  The daily concentration chemographs produced by SEAWAVE-QEX are constrained by the measured input data which are shown here as red circles.
                  DR. KENNETH PORTIER:  Excuse me.  On this chart, does the graph for this one realization, does it go right through the red dots?
                  DR. SARAH HAFNER:  Yes.
                  DR. KENNETH PORTIER:  In other words, it doesn't predict the uncertainty of that dot.  It goes right to that sample value, right?
                  DR. SARAH HAFNER:  Yes.  They do.
                  DR. KENNETH PORTIER:  Okay.
                  DR. SARAH HAFNER:  The -- every chemograph is constrained by the measured input values.
                  DR. VERONICA BERROCAL:  Dr. Berrocal here.  So what is -- I was think -- I was confusing when reading the White Paper was the censored value or the censoring limit.  There was no explanation of what the censoring limit is.  Is the same as limit of detection, or what is the definition?
                  DR. SARAH HAFNER:  We do talk about that in the standard operating procedure in a little bit more detail, but it is the limit of detection that you input into your measured data.  So it could be a limit of detection that comes with the data that you download.
                  The horizontal red line in this graph represents the censoring limit that was specified in the input data.  The blue line within each blue box shows the estimated maximum concentrations for each year which is the average of the maximum concentration from each simulation for that year.  The blue boxes are 80 percent error bounds on this average so that smaller blue boxes indicate more confidence in the estimated maximum concentration for that year across all chemographs and large blue boxes indicate more variability in the estimated maximum concentration across chemographs.
                  DR. VERONICA BERROCAL:  Sorry to be annoying.  So can I know how those blue boxes are calculated?  Is that like the 10th and 90th percentile of the condition of simulated values?  Or is it using normal approximation to compute those confidence bands?  So you have 100 simulations.  You compute the average maximum of all this simulation that you think 10 lowest and the 90th highest and that's how you compute your bands?
                  DR. SARAH HAFNER:  I believe it is the 10th and 90th percentiles.
                  DR. VERONICA BERROCAL:  Okay.
                  DR. SARAH HAFNER:  For use in risk assessment, a line indicating the drinking water level of concern can also be added to the plot as shown in this example.  The relationship between the drinking water level of concern in the blue boxes, that is, the error bounds on the estimated maximum, can help risk-assessors determine the likelihood that a site might be a risk concern.  For example, if the error bounds are small and overlap with the drinking water level of concern, or if the drinking water level of concern is lower than the estimated maximum concentrations, there's high confidence that the site poses risk to exceeding the level of concern.  This concept will be discussed in a later presentation on the short-term case study by Katrina White.
                  As the name implies, SEAWAVE-QEX fits the data to a seasonal wave model.  This diagnostic plot shows an example of a seasonal wave model fit to the data with all of the years of data in the simulation compressed to one calendar year so that the seasonality of concentrations can be seen.  In this example, the season of pesticide occurrence is from about May to early July which is highlighted in gray shading.  The red dots are, again, the measured input data and open red circles are censored data points.  Note that the Y-axis is, again, in log-based ten scale.  The black line is the seasonal wave fit to the data and the dashed lines represent the seasonal standard deviations around that wave.  In this example, the data are contained within the bounds of the seasonal standard deviations indicating that the model assumptions are verified for this plot.
                   There are several key features of the diagnostic plots that are important for having confidence in the estimate chemographs, and there are choices that our user can made to improve the model fit as needed.  We have documented these in a standard operating procedure for using SEAWAVE-QEX.
                  One feature to highlight is the presence of a secondary season without data to support it, demonstrated in the figure here.  There is a setting in SEAWAVE-QEX that will restrict the peak for a partial record to make sure that the peak occurs during the sampling season and essentially limit the model to selecting a single season instead of two.  Generally, this is recommended when all of the data for the time period being analyzed falls within a six-month window.  For example, the figure here shows all of the data from the years 2009 to 2014 in one calendar year and all of the sampling occurred between early March and early September which is shaded in blue.  And this indicates that a partial record restriction may need to be applied.
                  In the figure on the right, the partial record restriction has been added to the model and a single wave is selected.  While this setting can be preselected based on the data structure, it may not be practical for risk assessment to predetermine the sample structure on a site by site basis, especially given the short amount of time available to process the data for risk assessment.
                  Therefore, we have only recommended rerunning the model with this setting when the diagnostic plots indicate a problem with the default so that sites can be run in batch and only rerun select sites with a partial record restriction when a two-wave model is selected without data in the second wave.
                  DR. CLAIRE BAFFAUT:  Claire Baffaut here.  Just a small clarification, is this what you called optimization in the White Paper?
                  DR. SARAH HAFNER:  I'm not sure if we --
                  DR. CLAIRE BAFFAUT:  There is several instances, for example, all the SBF discussions, there's a caveat that says that the model was not optimized.
                  DR. SARAH HAFNER:  Yes.
                  DR. CLAIRE BAFFAUT:  Is that what you're talking about?
                  DR. ROCHELLE BOHATY:  So the optimization that you're referring to in the White Paper is specific to the sampling bias factors and not to SEAWAVE-QEX.
                  DR. CLAIRE BAFFAUT:  So, no.
                  DR. ROCHELLE BOHATY:  So we'll be talking about the sampling bias factors --
                  DR. CLAIRE BAFFAUT:  Okay
                  DR. ROCHELLE BOHATY:  -- in a couple of presentations.
                  DR. CLAIRE BAFFAUT:  All right.
                  DR. SARAH HAFNER:  And that optimization will be discussed separately.
                  So now that we've done a high-level review of the basics of SEAWAVE-QEX, we'll move into how we evaluated the model.  The evaluation focused on three parts to ensure that the model was robust enough for the types of data sets that we anticipate in countering in risk assessment.
                  First, we evaluated the accuracy of the model using both streamflow and estimated precipitation as covariates by subsampling near daily measured concentrations running the subsample data sets in SEAWAVE-QEX and comparing the SEAWAVE-QEX estimates to the original measured data.
                  Second, we evaluated the use of alternative covariates besides streamflow, qualitatively, by comparing how the flow anomalies were generated and used in the model.  These covariates included estimated precipitation, measured precipitation, and stream stage or height.
                  Third, we evaluated the use of SEAWAVE-QEX with low to no flow systems by viewing the model fits in the diagnostic plots.
                  Accuracy was assessed in a quantitative way by comparing SEAWAVE-QEX estimates to measured data which required preparing the data before use.  These treated data sets were used for determining how much SEAWAVE-QEX underestimates concentrations as well as for evaluating the accuracy of point estimates and distributions of SEAWAVE-QEX estimates using both flow and model precipitation as covariates.
                  Data sets used for another analyses, like the alternative waterbody analysis, did not have daily monitoring and so a qualitative approach was taken by evaluating the diagnostic plots only.  Those data sets were not subsampled before use.
                  For the quantitative analyses, near daily measured data was subsampled to create new data sets with infrequent sampling more typical of what is observed in water monitoring programs.  These subsamples were also meant to test close to the lower bounds of the SEAWAVE-QEX requirements for sample number by only pulling 12 measured samples per year.  Each subsample data set had a random seed start date early in the year with samples pulled every 7 or 14 days until 12 samples were reached for the year.  Then this process was repeated for the following years.  For example, for the seven day stratified samples, this results in data sets with 12 samples per year with all 12 samples occurring within a 12-week window within each year; however, across years, the total number of months over which sampling occurs will vary because of the random start dates for each year.
                  This subsampling process was repeated five times per data set to obtain replicates for combination of variables including pesticide site, covariate, and subsampling strategy.  These include atrazine and metolachlor evaluated with flow and precipitation at four sites for the NCWQR data.
                  The figures on this slide are one example of subsampling a measured data set and one site with a seven-day stratified sampling strategy where the gray circles represent concentrations and the red circles are censored data points.  Note that in the full measured data set on the left, concentrations range up to 90 micrograms per liter, but many of these higher concentrations are removed in the subsampling process leaving the maximum concentration of the subsample data set at about 30 micrograms per liter.  By using only 12 samples per year in the subsample data sets, we're not only evaluating how well SEAWAVE-QEX can fit a model to sparse data, but also if it can capture the peak.
                  DR. KENNETH PORTIER:  So, can I ask a clarifying question?
                  DR. SARAH HAFNER:  Yes.
                  DR. KENNETH PORTIER:  So you have a random start seed, right, and then in your stratified sample, say you're doing seven-day subsamples, you just start with that seed and do a systemic sample every seven days?  You take the value from the time series that date and that concentration.  You do that till you get 12 or 13 points and that's one of your five replicants; is that correct?  Or,
                  DR. SARAH HAFNER:  Or --
                  DR. KENNETH PORTIER:  Or you take those seven days and you take a random day, and do you snap to seven days.  Which one -- because it's not clear in the White Paper for this experiment exactly which one of those you did.
                  DR. SARAH HAFNER:  We did not use snap for this.  We just did the random start date and did every seven days until we hit 12.  But the next year has a different random start date.
                  DR. KENNETH PORTIER:  And you chose your start date from the beginning of sampling to the midpoint of the season; is that correct?  I mean, it was kind of a wide range of almost two months from which you could draw your starting date from; is that correct?
                  DR. SARAH HAFNER:  Yeah.
                  DR. KENNETH PORTIER:  Yeah, I think it is.
                  And then what's the justification for five replicates?  Just five.  Why didn't you do 50 or 10?  I just wondered.
                  DR. ROCHELLE BOHATY:  Yeah, it was time.
                  DR. SARAH HAFNER:  Time, resources.
                  DR. ROBERT CHAPIN:  Quick question.  Tom.
                  DR. THOMAS POTTER:  Tom Potter.  I heard you say that you used both precipitation and flow as covariates.  Were they used in a combined form or were they used independently?
                  DR. SARAH HAFNER:  They were independent.
                  DR. THOMAS POTTER:  And do you have separate breakout of how well precipitation performed as opposed to flow?
                  DR. SARAH HAFNER:  Yeah, they're discussed later.
                  DR. THOMAS POTTER:  Okay.
                  DR. CLAIRE BAFFAUT:  Claire Baffaut.  You said --
                  DR. ROBERT CHAPIN:  Claire, could you get this a little closer to you please?  Thank you.
                  DR. CLAIRE BAFFAUT:  You said at the beginning of your presentation that the goal was to evaluate the model and you said whether it was simulating the extreme concentration with reasonable accuracy which then you defined as not being underestimated or not grossly overestimated.  Do you have any criteria to decide what's underestimated or overestimated?
                  DR. SARAH HAFNER:  We discuss that a little bit later in this presentation.
                  DR. CLAIRE BAFFAUT:  Okay.  I seem to --
                  DR. ROBERT CHAPIN:  One last thing and that's it.
                  MS. TAMUE GIBSON:  I apologize for those members of the public that are joining us online, if you could please mute your lines.  We are getting feedback from discussions or conversations.  So please mute your lines at this time.  Thank you.
                  DR. ROBERT CHAPIN:  Back to you.
                  DR. XUYANG ZHANG:  Hi.  This is Xuyang Zhang.  I have a question about you subsampling.
                  DR. SARAH HAFNER:  Right.
                  DR. XUYANG ZHANG:  So you mentioned that the beginning date was always somewhere in the spring, right?
                  MS. TAMUE GIBSON:  Dr. Zhang, we could not hear you.  Could you please speak and repeat your statement please?
                  DR. XUYANG ZHANG:  Okay.  So can you hear me now?
                  MS. TAMUE GIBSON:  Just a little bit louder.
                  DR. XUYANG ZHANG:  Okay.  I think I might have reduced the volume of my line.  Yeah, if you could increase my volume.
                  MS. TAMUE GIBSON:  Yes, go right ahead?
                  DR. XUYANG ZHANG:  Hello?
                  MS. TAMUE GIBSON:  Okay.
                  DR. XUYANG ZHANG:  So my question is regarding the subsampling.  So the beginning date is always somewhere between January and March in the springtime; is that right?
                  DR. SARAH HAFNER:  Yes.
                  DR. XUYANG ZHANG:  Okay.  So, even with the 14-day sampling strategy, does that mean that the latest month in the year that can be sampled is somewhere around October -- I mean, September and August.  And the samples that occurred later than that, say October, November, and December were not -- appear in the subsampling.  Is that right?
                  MR. CHARLES PECK:  So this is Chuck Peck.  I'm the one that developed the random sort of sampling routine.  What we designed it to do was take a look at the data that were available, the set that we had, split it in half and sort of pick a start date, a random seed from that and then systematically either do 7 day or 14-day sampling.  So typically in a lot of cases, yes, the March is when it started, but it could have moved anywhere to later in the summertime, April, May, June sort of timeframe at which point, if you were doing a 14-day, it would then shift some of your sampling to later in the calendar year.
                  DR. XUYANG ZHANG:  Okay.  Thank you.
                  DR. VERONICA BERROCAL:  Dr. Berrocal.  I have a question about -- so in the model, in the SEAWAVE-QEX, the residuals are modeled to be temporarily correlated and there is that correlation time scale factor.  Did you calculate the correlation times scale factor before deciding on 7 and 14 day sampling strategy?  I'm wondering because if the correlation time scale was small then by doing this 7 or 14-day sampling strategy, you're basically making your data independent when in reality it is temporally correlated.
                  DR. SARAH HAFNER:  No, we didn't calculate it beforehand.  Okay.
                  The sampling process resulted in this data structure for each combination of variables.  For example, for one chemical at one site using flow as a covariate, there are five data sets using seven-day stratified sampling.  These five new subsample data sets are then run in SEAWAVE-QEX to estimate daily concentrations resulting in 100 chemographs for each subsample data set.
                  The SEAWAVE-QEX estimates from the subsample data sets were first briefly evaluated to determine how frequently and how much the model underestimated the measured concentrations in order to address concerns from previous SAPs that SEAWAVE-Q underestimates peak concentrations.  It is important that we have confidence that our drinking water concentration estimates are not underestimating exposure as we are charged with ensuring reasonable certainty of no harm from these concentrations.
                  For each subsample data set, the maximum 1 for 21 and 365-day average was determined across all 100 SEAWAVE-QEX estimated chemographs for each year.  For the one-day average, this would be the maximum concentration that SEAWAVE-QEX estimated in any of the 100 chemographs for a given year in a data set.  These estimated concentrations were then compared against the associated maximum average from the daily measured data set that was not subsampled.  The frequency of underestimation was evaluated by calculating the number of subsampled data sets that underestimated as well as the number of years within each SEAWAVE-QEX run that underestimated.
                  For example, in this diagram, there are five subsample data sets at a site using a seven-day stratified sampling strategy.  For each subsample data set, there are six years of data which totals 30 years of data across the five subsamples.  Only three of these years underestimated the maximum concentration for that year which was equated to a 10 percent rate of underestimation for those data.
                  Next, summary statistics were derived from the SEAWAVE-QEX estimates to compare against measured concentrations to determine which might be appropriate to use as a point estimate in a deterministic risk assessment.  A single percentile can't be used since there are 100 SEAWAVE-QEX chemographs for every data set.  Instead, summary statistics were developed by first deriving a percentile for each of the 100 chemographs.  In this example, the 99th percentile concentration was selected for each year of each chemograph.
                  Another summary statistic was taken across those one-hundred 99th percentiles.  In this case, it is the maximum of the 99th percentile values.  These summary statistics were then compared to the maximum measured concentration for each year.
                  For probabilistic assessment, the entire distribution of concentrations would be used.  To evaluate our ability to use SEAWAVE-QEX in this capacity, we've compared distributions of measured concentrations to all 100 SEAWAVE-QEX chemographs from the five subsample data sets for each chemical, site, and covariate combination; however, the SEAWAVE-QEX distributions needed to be truncated for a fair comparison since the measured data was not completely daily.
                  The figures here show this process over one year of samples.  First, we started with the full near daily measured data set from which 12 samples were taken for that year shown in yellow.  That subsample data set was interpolated using SEAWAVE-QEX to produce daily chemographs which all included the measured input data.  For the final comparison of distributions, the 12 input measured concentrations from each year are removed and concentrations from the SEAWAVE-QEX chemographs from the same days as the original measured data set are compared.
                  I should note that SEAWAVE-QEX is designed to capture the statistical properties of the data and not to predict concentrations on a given date.  For example, estimated concentrations from June 5th should not be compared with measured concentration from June 5th; however, for the purposes of thinning data and creating and comparing distributions of concentrations, using the same dates is considered acceptable.
                  And this concludes the methods part of our SEAWAVE-QEX presentation.  And I'll pause now for any clarifying questions.
                  DR. KENNETH PORTIER:  So this is two times series, right?  So how did you compare them?  What's -- did you just take the difference and look at maximum difference or integrated difference or --
                  DR. SARAH HAFNER:  I'll show that in the next presentation, but visual mostly.
                  DR. ROBERT CHAPIN:  Right.  Ray Yang.
                  DR. RAYMOND YANG:  Thank you for your presentation.  My first question for you is really a reflection of my ignorance of this area, okay.  So be patient with me.  Basically, what I would like to do is for you and your colleagues to educate me so I can do a better job, okay.
                  When I think about a model, I think about either a statistical model which could be a first order or higher order polynomial equation, okay, which has coefficient becomes parameter and so on and so forth.  Or it could be compartmental modeling which in this case, could be ecologically-based compartmental modeling whereby all the factors influencing the flow and exposure rates and so on and so forth would be taken into consideration then each compartment will have parameters and probably mass balance differential equations and then you do modeling, okay.
                  So far and throughout the White Paper, I don't see anything like that of a model.  Model is a nebulous seeing behind so I don't understand what's going on, all right.  So are we going to have somebody actually talk about a model so we know which factor, which parameter is involved in this process, that process and so on so we can make assessments?  That's question number one.
                  MR. DANA SPATZ:  So the compartmental model you're describing describes the pesticide and water calculator, the Tier 2 modeling that we do where what Sarah's talking about, the SEAWAVE-QEX model, is the statistical tool.  Today we're not focusing so much on the pesticide and water calculator compartment model.
                  DR. RAYMOND YANG:  So it is a polynomial equation for the model.  Am I correct or --
                  DR. CHRISTINE HARTLESS:  You take the idea of the polynomial regression equation that you're thinking of and add several layers of complication to it and then you get the SEAWAVE-QEX model that's buried inside that R-code.  But yes, it's much more of a statistical model than what you're thinking of in terms of ecological or compartmental models.
                  DR. RAYMOND YANG:  And do we -- will we discuss that or is there anywhere that you're going to let us see something along that line?  In other words, it seems to me this SEAWAVE now is something in the background.  Trust me, it is working.
                  DR. ROBERT CHAPIN:  Well, you're going to see some examples, right.  We're going to see some case studies in a while.  Do you want to see the code?
                  DR. RAYMOND YANG:  Well, case study -- I looked through the case study.  I didn't see anything either.
                  DR. ROBERT CHAPIN:  See the code, is that what you want?
                  DR. RAYMOND YANG:  Well, the code is derived from the model.  Model is representing a concept.
                  DR. ROBERT CHAPIN:  It makes the model.
                  DR. VERONICA BERROCAL:  There is a document with actually the model that -- yes.
                  DR. RAYMOND YANG:  Okay.  Thank you.  Thank you.
                  DR. VERONICA BERROCAL:  There is a document with the model, the Vecchia 2018 USGS paper as the model, the equation and --
                  DR. RAYMOND YANG:  Okay, okay.
                  DR. VERONICA BERROCAL:  Yes.
                  DR. RAYMOND YANG:  Now let me ask you a less troublesome question, okay.  In your presentation, there was a slide that says SEAWAVE-Q, the earlier panel considered the estimate was too low.  I assume you folks have updated and correct that area and so on with QEX, all right?  And I look at some of the graphics.  They look beautiful, very nice.  You know, you --
                  DR. ROBERT CHAPIN:  What's your question, Ray?
                  DR. RAYMOND YANG:  The question, the question is Crop Life America.  This public written statement.  They say EPA's estimate mostly too high.  So what's going on?  Are we talking about differences of opinion or different interpretation or what?
                  DR. ROCHELLE BOHATY:  So Rochelle Bohaty here.  I'll take that question.
                  So I think it's probably a little bit of both, but I think to help better inform this discussion I think Sarah's presentation on the results and how we interpreted the data would be useful.
                  DR. RAYMOND YANG:  All right.
                  DR. VERONICA BERROCAL:  Yes.  I have a clarifying question on the statement that you made just at the end of your presentation.  You said that the SEAWAVE-QEX model is not the -- should not be used to generate estimates of pesticide concentration for a given day.  I'm just confused about this statement because the model is set up where your response variable is the daily concentration and the covariate is again something that refers to the day, whether it is the streamflow or precipitation.  So it seems to me -- and then the results of the fact that there is -- it was recounted for temporal correlation.  So I don't understand why the output or the estimated concentrated of the SEAWAVE-QEX model gives for a given day cannot be interpreted as what is the estimated concentration on that day.  Can you clarify why you made that statement?
                  DR. SARAH HAFNER:  That's how it's been explained to us by the model developer, that it was not designed to predict a specific concentration for a date, but to encompass the properties so it can give these distributions of concentrations.  So it's -- you know, you're getting equally probable chemographs for each date, but it's not designed to give you a concentration for a specific date to compare it to measured concentrations.
                  DR. ROBERT CHAPIN:  Yeah, just get close to the mic please.
                  DR. IAN KENNEDY:  So correct me if I --
                  DR. ROBERT CHAPIN:  And identify yourself for folks.
                  DR. IAN KENNEDY:  Oh, Ian Kennedy.
                  DR. ROBERT CHAPIN:  Thank you.
                  DR. IAN KENNEDY:  Right.  What you do basically is you take the input data, you run the SEAWAVE-QEX, and it gives you some parameters for model and then does generally 100 possible outcomes of that or 100 chemographs for each year of the run.  And then, according to what you have a couple of slides before, you're taking the 99th percentile of each of those runs and taking the maximum of that; is that correct?
                  DR. ROBERT CHAPIN:  So is there a question there?
                  DR. IAN KENNEDY:  My question is if that's what's going on, how repeatable is that?  If I did it with another hundred runs, because you've got some random number generator in there somewhere that is generating the differences between these hundred runs and so you're sampling from distribution and then looking at the maximum of those samples.  I'm just wondering if there's some issue with repeatability?
                  DR. SARAH HAFNER:  So generally if you take the same data set and run it again, you will get different chemographs, but it should be fit to the same model.  So you may not get -- you won't get exactly the same number when you develop this summary statistic, but it should be very similar.
                  DR. IAN KENNEDY:  How does that work in the regulatory context then if you generate one number and someone else can do exactly the same thing and generate another number, even if the numbers are reasonably close?
                  DR. VERONICA BERROCAL:  Can I try and explain here?  So when the model is fit, the standard deviation of this residual is estimated.  And so, every simulation that is done is basically, yes, simulating random numbers, but it's bounded so that these random numbers have the standard deviation that is estimated from the data.
                  So they are not -- they are taking this percentile and basically characterizing the distribution.  It's not the actual value.  So even if you repeat this experiment many times, it's true that the residuals will not be the same, but the standard deviation of these residuals will be the same because the data that you're feeding to the -- that you're using to feed the model is fixed.  It doesn't change.
                  DR. IAN KENNEDY:  Right.  I think my question is more one of -- is that a sufficient number to be sure that those differences are small enough that you won't get some issue when someone else does it and gets a different value?
                  DR. VERONICA BERROCAL:  Well, then the question may be would be is 100 chemographs enough.  Or to make sure that this percentile is estimated accurately versus should you do 1,000 chemographs or 10,0000 chemographs?  So, here you're going into how to quantify uncertainty in the percentile of a distribution; but I don't think that the procedure in itself is invalid.  This is what we're doing statistically, Monte Carlo simulations.
                  DR. IAN KENNEDY:  Yeah.
                  DR. ROBERT CHAPIN:  Okay.  Dr. Sadd.
                  DR. JAMES SADD:  Jim Sadd.  A couple of slides back you showed some measured concentrations as points and then the estimated chemograph.  And the chemograph -- a little farther back please.  One more maybe, one more.  Yes, right there.
                  And the chemograph actually shows concentrations that are higher than the measured samples.  And I understand that you can't measure the peak, so that is something that's estimated from the covariates.  So if you have a covariate, say it's precipitation, in an arid climate that it's contribution to a waterbody or to a river is going to be very much influenced by infiltration and that could vary seasonally because of, you know, soil moisture and soil dryness.  And in a humid climate, it's going to be contribution to the river from groundwater flow and there's going to be a lag there too.  So are those sorts of covariates included in consideration of precipitation?  What I'm trying to say is it's not just the rainfall, it's how the water actually gets into the waterbody and that is complicated by those other factors.
                  DR. ROCHELLE BOHATY:  So I think the best answer is that we're exploring for -- we would explore for different sites the best covariate in that work and you'll hear some more in some of the subsequent presentations, particularly by Mr. Bischof on how he looked at other covariates because his sites didn't work so well using flow as a predictor for the concentrations.
                  DR. THOMAS POTTER:  Tom Potter here.  I need to use -- ask my same question earlier and you can -- maybe you're going to have the same answer which is I'm going to hear about it later.  But my question is, you did a systematic evaluation of the two covariates, precipitation and flow.  That's what I understand to be the case and so are we going to hear that later, or is this embedded in this analysis here?
                  DR. SARAH HAFNER:  You will hear about precipitation in this -- the results part of this.
                  DR. THOMAS POTTER:  This -- your current presentation?
                  DR. SARAH HAFNER:  Presentation.
                  DR. THOMAS POTTER:  Okay, sorry.  I just needed to get --
                  DR. SARAH HAFNER:  It's still, yeah.
                  DR. THOMAS POTTER:  I needed to be reassured.  Thank you.
                  DR. ROBERT CHAPIN:  Okay.  We good?  Any other --
                  DR. REBECCA KLAPER:  I had a couple questions.  This is Rebecca Klaper.  Dr. Hafner, are there -- so you guys were talking about the Tier 2 --
                  DR. ROBERT CHAPIN:  Rebecca, I'm sorry.  Could you move the mic so it's -- thank you.
                  DR. REBECCA KLAPER:  Better?  You were talking about the Tier 2 evaluation and all of the considerations you make in your model looking at potential pesticide affects including all the properties of the chemicals.  Do you -- have you -- I know you've given us two examples, and I'm not totally clear what those are because they're mystery chemicals, but does it make a difference or have you evaluated other chemicals based on those chemical properties to see how well the models fit based on the properties of the pesticide whether it's more water soluble or not or whether it absorbs to soil or not or those kinds of things?  That was my first question.
                  DR. ROCHELLE BOHATY:  So I think the extent that we have done that is when we use the USGS chemicals and we ran them through SEAWAVE-QEX and evaluated the output plots for that.  We won't specifically get into the results of that, even though it's provided, but we do use that data in the subsequent development of the sampling bias factors.
                  DR. REBECCA KLAPER:  So does it --
                  DR. ROCHELLE BOHATY:  So the data are available in the supplemental package that was provided.
                  DR. REBECCA KLAPER:  Okay.  So it does make some difference and you've integrated that into whatever the cofactors are for the model in order to make better estimations then?
                  DR. ROCHELLE BOHATY:  So for SEAWAVE-QEX, you don't actually put in any independent environmental fate data.  It's just simply flow and the measured pesticide concentrations.  The environmental fate parameters that you're talking about, those would be in our Tier 2 model which is the pesticide water calculator so they're completely independent.
                  DR. REBECCA KLAPER:  Okay.  But no matter which of those chemicals you chose -- do all those pesticides, and I'm sorry I don't know the KOD values or whatever of all the pesticides, are they all very similar?  So they would all behave the same way in the --
                  DR. ROCHELLE BOHATY:  No, they're very different.
                  DR. REBECCA KLAPER:  -- environmentals And they still all held up equally in the same well system?
                  DR. ROCHELLE BOHATY:  I mean, you could run the tools, sure, depending on the site and the pesticide combination, we would have more confidence in the results based on the individual runs and diagnostic plots.  But we thought overall that the model performed well for what we needed it to do.
                  DR. REBECCA KLAPER:  Okay.  Then I just have one more question too.  How are you -- and maybe you're getting into this a little bit later -- but how are you dealing with the change of the flow rate over a day?  It is integrated or is it one median value of the flow for that particular day or -- because, you know, especially in some of the smaller order streams they're very flashy, et cetera.  So how do you deal with that flashiness?
                  MR. MATTHEW BISCHOF:  It's just one value for the day.
                  DR. REBECCA KLAPER:  So it's like an average or a median or something or --
                  MR. MATTHEW BISCHOF:  Yeah, I believe it's average for the day.
                  DR. REBECCA KLAPER:  Okay.  Because the other part of that question that I have then too is that, you know, when we go -- I do environmental sampling too -- when you go out, you can't go sample at the high point of a stream or when the first flush happens coming off of a field and so you're missing the highest points of the data.  What kind of confidence then do you have that you're estimating the highest concentration when most environmental samples are missing that high point of a pesticide concentration in a stream?  Do you set some kind of variability or confidence interval so that you can say, you know, 10 percent more et cetera in order to make sure that you're safe basically?  Thank you.
                  DR. ROCHELLE BOHATY:  No, we just take the data as they are.
                  DR. ANDREW MIGLINO:  Hi.  Andrew Miglino.  I just had a couple questions, hopefully easy questions, and maybe they're in Vecchia and I just didn't delve too far into it.  So sorry if it's explained somewhere else.  But these are about the fitted parameters.  So some of these are static, so like, season and rate are the same across years.  Is that a reasonable assumption, and maybe this is just ignorance on the pesticide world, but, is that a reasonable assumption to assume I apply the same timeframe at the same rate year after year for all these simulations?
                  DR. ROCHELLE BOHATY:  So Rochelle Bohaty.  It depends on the chemical so that's important to note.  One of the utilities of SEAWAVE is that you can, if you notice a change in the use profile over time, you could split up the years --
                  DR. ANDREW MIGLINO:  Okay.
                  DR. ROCHELLE BOHATY:  -- and run them, as long as it still met the data quality and quantity criteria.
                  DR. ANDREW MIGLINO:  Sure, so like a 12-year sampling split into four years each --
                  DR. ROCHELLE BOHATY:  Yeah.
                  DR. ANDREW MIGLINO:  -- or something like that.  Okay.
                  And then the other one is about censored data.  So I noted in the White Paper you talk about SEAWAVE gives a random value to censored data.  Is that a static fixed value across all hundred realizations, or does it give it a new value for each one of those?
                  DR. SARAH HAFNER:  It's a new value for each realization.
                  DR. ANDREW MIGLINO:  Okay.  And is there like a bounding on that?  I mean, obviously, the upper bound, but is there a lower bound on there?
                  DR. SARAH HAFNER:  It is somehow constrained to the model, but I couldn't tell you the specifics at this moment.
                  DR. ANDREW MIGLINO:  I'll go delving into Vecchia.  Okay.  Thank you.