c. Mosquitoes and biting flies (section (l))

Panel Recommendations:
143
Based on Tables A3.1 and A3.2 in Appendix III of the EPA supplemental document titled "Sample Size for Pet Product Studies," the required blood-fed/retention proportions (referred to as "base values" hereafter) play an important role in generating the number of pest species per vertebrate animal presented in the proposed guideline (last column of the table above); these base values are likely determined by the biology of pest-host interactions. For example, pests that are "aggressive" feeders on the vertebrate host tested may have a much higher host retention proportion than pest species with lower host feeding success. Accordingly, one way of reducing the number of pests per animal for testing is to "modify" these base values based on the "observed" (i.e., historical) information on pest-host interactions. Despite the importance of these base values in terms of sample size and power determinations, the proposed guidelines provided minimal information on their sources and biological rationales. Hence, the Panel recommended that the Agency should provide support of these base values or reexamine these base values for potential modifications; the potential sources of historical information of host-pest interaction are open literature, e.g., Taenzler et al. (2016); studies submitted by the pesticide registrants (since the inception of the 1998 testing guideline); and professional judgment from experts of these ectoparasites.

In particular, the Panel recommended that the EPA provide a clear justification for the arthropod "% retained" for inclusion of a vertebrate animal in the study. Using a higher "% retained" would decrease the number of vertebrate animals needed.  PG 33
The Agency agrees that the pest-host interactions played a role in the numbers of animals generated. The models were based on real data submitted to the Agency from studies that followed the methods chosen for the guideline and incorporated the retention rates, which were also chosen based on data typically submitted. The % retained was also generated based on historical numbers in submitted tests.
144
The Panel further recommended that the EPA should carefully consider whether there are interactions between arthropod species when making sample size recommendations for scenarios in which multiple species are permitted to be tested on the same animal. Input should be sought from subject matter experts (biologists) on these possible interactions. For mosquitoes, two experts (personal communication with Dr. Dan Kline, USDA, and Dr. Bill Reisen, UC-Davis) anticipate no problems in testing multiple species simultaneously (see response to Charge Question #18).

The statistical models in the power vs. sample size calculations did not consider interactions between arthropod species (the species were treated in the models as not interacting with each other). Significant interactions among different pest species could result in reduced retention of each pest species (from that assumed in the power analysis) and potentially could also impact the effectiveness of the treatment (e.g., effective when single species but not for the same species when with other species [i.e., the multi-species scenario]). Thus, the Panel observed that interactions could have important implications on power (e.g., reduced power with reduced retention) and could bias the estimated efficacy for a single species when assessed from a multi-species study.  PG 33-34
The suggestion to use multiple pests for testing was refined based on recommendations by the SAP, the public and peer reviewed literature to mitigate negative interactions between the pests which may affect (e.g., reduce) retention.

Based on this comment, the mosquito section was not changed and will continue to allow up to 3 mosquito species tested at a time.

The flea and tick sections were revised to indicate that a flea infestation may be combined with an infestation of 1 species of tick, however it is not encouraged since grooming or scratching due to flea bites may detach ticks.

As for mixing tick species, the recommendations will be revised to state that Ambylomma maculatum (Gulf Coast tick) should be tested alone due to the effect of their bites. Also ticks with similar appearance or markings (before or after feeding) should not be tested together, since it may lead to counting errors.
145
In addition, for the larger numbers of pests per vertebrate animal, the Panel expressed concern about the logistics of counting (and subsequent errors).  PG 34
The pest numbers presented in the guideline are not larger than what is typically used in tests currently, so the Agency does not believe counting errors are a concern.
146
4a. Below, the Panel provided Specific guidance on each arthropod.
The Panel had considerable discussion about fleas regarding the number of live fleas recovered in controls and flea retention. The literature is clear that visual thumb counting is vastly inferior to combing techniques. However, Dryden et al. (1994) provides a possible method by which the actual number of fleas could be estimated from thumb counts. This would be relevant to the earliest time studies in the guidelines. Dryden et al. (1994) also provides a study comparing counting procedures. Visual counting (six regions of a dog) was compared with a 10-minute count using a flea comb. After the counts the animals were sprayed with an alcohol based pyrethrin spray and combed for 10 minutes and the number of fleas counted. These animals were retained for 24 hours to determine if any additional fleas were counted. Area visual counting accounted for 21.5-28.0% of the fleas placed on the dog and combing counts provided 76.0-85.6 % of the fleas placed on dogs. After applying the pyrethrin sprays, 82.5-86.4% of the fleas were accounted for in the study at 24 hours. In a cross over study, Heckenberg et al. (1994) found that thumb counting found means of 8.8% and 7.7% on dogs infested with 50 and 100 fleas, respectively. Comb- counting provided 67.6% and 75.4 % of the fleas placed on the dogs. In another study, combing for 5, 10, and 15 minutes provided at least 81.5, 90.5, and 85.1% recovery of fleas placed on beagles 1 hour earlier (Zackson et al., 1995).  PG 34

For fleas, the Panel recommended:
1) Only short-haired animals like beagles should be used in the tests. Issues regarding long- haired vertebrate animals and environmental issues like sunlight or rain could be addressed in field studies.
2) Comb counting should be used based on the technique from (Dryden et al. 2016).
3) If beagles are used in the studies, retention estimates needed for the models might be obtained from the study by Dryden et al. (1994).
4) The methodology used by Dryden et al. (1994) should be repeated with short haired cats. Instead of the alcohol-based pyrethrin spray, nitenpyram could be used on the cats to remove all fleas. The use of an Elizabethan collar should also be considered as it can prevent cat grooming behaviors so its value for increasing flea retention can be determined.
5) The Charge Question regarding the issue of counting fleas for "the earliest time (e.g., 8 hours, 12 hours, etc.) needs to be revisited when EPA revises the guidelines. All on-animal counts prior to the final count may be conducted using hand counts. Values can be estimated for live fleas on beagles using the Dryden approach (Dryden et al., 1994).
6) A similar study based on the approach of Dryden et al. (1994) with short-haired cats should also be conducted.  PG 34-35
Based on SAP and public comments, hand counting options were removed from the guideline for fleas since they are considered unreliable. They will remain for ticks, which move differently than fleas and are therefore easier to hand count. The guideline was also revised to allow comb counts for fleas at those earlier time points, during which the fleas are removed then re-infested minutes later, with the stipulation that the comber avoids the treated area (so not to mechanically spread the solution) and treats the controls in the same manner with combing out and re-infesting.
147
4b. Below, the Panel provided Specific guidance on each arthropod.
The use of 50 ticks per test was viewed as reasonable since decreasing the number of ticks per test would result in requiring more than the currently recommended 14 vertebrates per group (which was a concern  -  see response to Charge #4).

For ticks, the Panel recommended 50 ticks per test.  PG 34
Thank you for the concurrence on the number of ticks used per test. No change to the guideline is necessary.
148
4c. Below, the Panel provided Specific guidance on each arthropod.
For mosquitoes, the rationale for using 50 per species when testing three species simultaneously, and 100 for individual species tests, is unclear. Unless there is a justification for the larger number for individual species tests, the Panel recommended 50 for the individual species tests, as well.

For mosquitoes, the Panel recommended 50 for the individual species tests.  PG 34
The intent of this text in the guideline was to indicate that if 100 mosquitoes or flies will be used in a study, then fewer animals (only 6) should be tested for each group.  However, if 50 mosquitoes (or specified flies) are used, then 8 cats or dogs per group should be tested. This section was revised for clarity.
149
4c. Below, the Panel provided Specific guidance on each arthropod.
For biting flies, there was considerable concern about using 50 flies per test (in terms of animal safety and welfare). Twenty-five per test was viewed as a more humane number of pests.

For biting flies, the Panel recommended 25 biting flies per test; however, the impact on power and sample size would need to be assessed.  PG 34
Based on numerous concerns from the SAP and public on using 50 painful or aggressive types of biting flies per test, the model was re-run for stable flies using 25 flies per test since they are the most painful/aggressive. Based on that model, the guideline has been revised to indicate that for studies conducted using 50 stable flies per animal8 animals should be in each group; for studies conducted using 25 stable flies per animal 12 animals should be in each group. In addition, the exposure time has been reduced from 1 hour to 30 minutes for all biting flies to reflect the study used in the model. Currently the Agency is only aware of one study conducted using stable flies on dogs and does not expect many future submissions but understands that any product that would qualify for claims against such aggressive/biting flies would be valuable.
150
The Panel also provided the following, general recommendations:
The Agency should provide further documentation of the validity of the parameter values in the simulations based on historical information regarding host-pest interaction from the literature or studies submitted to the EPA in the past.  PG 35
The models using 100 fleas were based on Beugnet et al., 2011.  The Agency relied on historical data to inform the 40% for cats and 60% for dogs control retention of fleas. The efficacy threshold of 90% by 48 hours was based on the Agency's 90% standard to support product registration.

The models using 50 ticks and 25% retention on control animals were based on Taenzler et al., 2016. The efficacy threshold of 90% by 48 hours was based on the Agency's 90% standard to support product registration.

The models using 100 mosquitoes per cage and 60% blood-feeding rates for control mosquitoes were based on Meyer et al. 2003, Hayasaki et al. 2009, and Franc et al. 2012. Although 50 mosquitoes per species (150 total mosquitoes per cage) was not a parameter found in published studies, the Agency conducted these simulations to attempt to reduce the number of animals used in a single test. The efficacy standards after 1 hour of exposure was based on the Agency's 90% standard to support product registration.

For sand flies, the models were based on Halbig et al. 2000 for 100 flies and Liénard et.al. 2013 for the 60% control blood-feeding rate. The efficacy standards after 1 hour of exposure was based on the Agency's 90% standard to support product registration.

The models using 50 stable flies were based on Fourie et al. 2006.The efficacy standards after 30 minutes of exposure was based on the Agency's 90% standard to support product registration.
151
The Panel also provided the following, general recommendations:
Simulations should be conducted using lower numbers of pests, particularly for biting flies and fleas, and improved estimates of retention rates, to assess the implication on power and sample size (i.e., the number of animals needed).  PG 35
The original simulations were run using the best estimates of retention rates based on available public literature; therefore, they were not re-run with different retention rates. The simulation for stable flies has been re-run using 25 stable flies. The guideline was revised to include options of using either 50 stable flies per animal and 8 animals per test or 25 stable flies per animal and 12 animals per test.
152
The Panel also provided the following, general recommendations:
For multiple species studies, when making an assumption of no interaction, the lack of interaction should be carefully documented from the literature. Otherwise, the power vs. sample size simulations and their validity would be suspect in the presence of interactions.  PG 35
See response to comment #144.

Public Recommendations (docket ID # in parentheses):
153
"The number of parasites proposed within the draft (i.e., 100 fleas and 50 ticks) is considered appropriate. For consideration, reduction in number of pests would lessen resources required for the study; however, reduction would also lead to smaller recoveries on control animals thereby affecting efficacy evaluation" (0008)

The Agency acknowledges that the number of parasites is appropriate.
154
Re: biting fly #: "What is the rationale for only using 50(+-5) biting flies per animal? Could a Sponsor justify using more than this number?" (0026)
See response for comment #148, however this number is the minimum recommended number; more could be used if desired.

Charge Question 6
Comment on the timing of exposing vertebrate test animals to fleas and ticks.
a. Are the number of exposures for products with different durations of efficacy adequate for determining efficacy? Why or why not?
b. If the number of exposures to pests can be decreased, please indicate specifically which exposures can be skipped. If efficacy should be evaluated at more time points, indicate when exposures should occur and for what product types (e.g., spot-ons, collars, and residual shampoos), and discuss the value provided by the additional time points.

Panel Recommendations:
155
The Panel recommended that the number of exposures should adequately reflect the actual claims of the products. For example, if a product claims efficacy for 60 days, there should be data for at least 60 days. The exact number of exposures necessary will depend on the types of products, the claims, and how the Agency will evaluate the submitted data. If it is necessary to provide >= 90% reduction of fleas and ticks at the "longest labeled duration of efficacy" claim, then there is little need for the day 1 exposures. If the claim is "provides control (assume >90%) throughout a period of 60 days," then some periodic exposures are appropriate. In addition, for products with claims of effectiveness over several months, acquired immunity to tick infestations may occur, leading to host-induced tick mortality (European Medicines Agency, 2016; Wada et al., 2010). Therefore, infestation frequencies should be reduced to the minimum necessary to test the duration of product efficacy.

However, it is unlikely that removing some exposure times would actually change the methodology or the numbers of vertebrate animals being treated because the investigators will still design the study to generate data necessary to show efficacy and residual performance. Exposure tests will be repeated until the treatment fails to reach the necessary control level. The main factor dictating the numbers of vertebrate animals to be tested are the findings from assessment provided in the EPA document entitled, "Sample Size for Pet Product Studies."  PG 36
The first exposure period for mortality claims was revised to day -1 for fleas and day -2 for ticks. The exposure before the product is applied is important to determine if the product kills current infestations. Since the power analysis includes the 4 infestations periods in the first month, and that information is useful for interpreting the data, they will all remain. After 30 days, the infestations move to monthly to limit the testing frequency.

For the mosquito and biting fly section, one of the infestations in the first month was already removed for products lasting >1 month and all weekly challenges were removed for products lasting over 1 month.
156
6a. For products with efficacy claims of > 7 days, there is no need to expose treated animals at 1 or 2 days to fleas and ticks, respectively. Again, the testing protocol should reflect the specific claims. For research purposes, testing for efficacy occurs at multiple time points up to and past the maximum claim. Laboratory bioassays without animals could be conducted to test for breakdown of the product. For tick mortality trials, the Panel suggested changing the infestation timing in Table 2 to be Day -2 for ticks (not Day -1), to give adequate time for attachment, as outlined in European Medicines Agency (2016).  PG 36
The purpose of the initial pest exposure/infestation (which is on day -1 or -2 for mortality claims) and "up to 24 hours post application" for repellency claims) is to determine if the product begins working in an adequate time that provided adequate protection.

157
6b. The Panel recommended that, depending on the specific product efficacy claims, at a minimum 14- and 21-day exposures could be removed for products with >30-day claims. The 1 or 2-day exposures could also be removed from >7-day claims depending on the EPA's goals for having 1-day data. The >4-week infestation timing is excessive; weekly repeated infestations of animals with fleas or ticks for the >4-week endpoints are unnecessary. Re-infestation of hosts every 2-4 weeks for longer duration product trials is common practice (Stanneck et al., 2012a, Wengenmayer et al., 2014; Jones et al., 2015; European Medicines Agency, 2016). For the >4- week group, infestations on Days -2, 28 and then monthly should provide the same level of information as weekly infestations. Timing of application for tick repellency testing should be similar to mortality trials at the >4-week endpoint.  PG 36
See responses to comments #s 155 & 156.

Public Recommendations (docket ID # in parentheses):

158
"Table 1 [and Table 2 and Table 3]. For Claimed Duration of Efficacy of >4 weeks, the timing of infestation is excessive and puts undo stress on the test animals. The timing should be amended to Days -1, 14 & 28 or 30, then monthly up to the longest labeled duration of efficacy" (0008)

See response to comment #155.
159
"Consideration should be given to number and timing of exposures for spot-ons and collars with potential omission of mid-month infestation count(s)" (0008)
See response to comment #155.
160
"We propose that the duration of testing to demonstrate adequate efficacy should be dependent on the proposed claims of the product. On other words, long-lasting products would be required to demonstrate its sufficient duration. This will then allow for differentiation among available products and provide users the information for better product selection. The number of exposures should be customized to the profile of the proposed product. In other words, firms should be allowed the flexibility to customize the exposure intervals to fit their proposed product. Consultation with the Agency, in advance of initiating a development program, could provide alignment between the Agency and the firms towards agreeable study designs." (0026)
The proposed schedule is currently broken down for products with varying durations. The purpose of the guideline is to standardize the data and use the power calculations, which were based on the time points given, to ensure adequate statistical data sets. For more information, see responses to comment #s 155 & 156.

Charge Question 7
In the interest of reducing the use of vertebrate animals in testing, we have suggested in the guideline to allow testing of up to two species simultaneously for fleas and ticks (e.g., fleas and 1 tick species, 2 tick species). Are there any potential known interactions between multiple tick species or ticks and fleas if on a single vertebrate animal at the same time which would require individual pest species be tested separately? If there are interactions, please describe the interactions and provide references.

Panel Recommendations:
161
The Panel observed that testing multiple species on the same host could decrease the number of vertebrate animals to be used. While flea and tick co-infestations are commonly encountered on companion animals, this would make testing 1 flea + 1 tick species simultaneously acceptable and potentially even desirable. It is, however, unknown if increased grooming by cats in response to the presence of fleas might influence normal tick attachment. While retention is not expected to
be altered by co-infestation with two different tick types, some species, such as Rhipicephalus sanguineus and Ixodes scapularis, may be very similar in body size and markings after being attached to a dog for 48-72 hours. This may make the efforts to separately identify and count these two tick species too slow and cumbersome.  Dermacentor variabilis and Ambylomma maculatum (Gulf Coast tick) also share similar coloration. Additionally, A. maculatum is an aggressive species that causes painful, inflamed attachment sites. Thus, from an animal welfare standpoint, adding a second co-infested tick species with A. maculatum exposure would probably be excessively stressful to the test animal.

Regardless of the previous comments, when combining 1 flea + 1 tick or 2 tick species, would it be practically possible to adequately "tease out" (i.e., count) the effects of each of the two species from the same animal? And is it known if one species could potentially affect the survival rate of the other, thus influencing the experimental outcomes? If one of the two species used during a trial doesn't meet the test criteria, the guidelines should address what should be done with these data for the other insect species.  PG 37
Given SAP and public comments that normal tick attachment could be affected by increased grooming caused by itching from fleas in combined flea and tick tests, the encouragement of testing with fleas and ticks combined in a single test was removed. However, a study combining the two pests may still be acceptable if it produced good results.

From an animal welfare aspect, the guideline was revised to explain that Gulf Coast ticks should be tested alone, due to their potentially painful or inflammatory bites.

Studies utilizing 2 tick species at a time are still encouraged in the guideline but will indicate that the ticks should be different in size and markings (before and after feeding) to ensure that they can be distinguished during counting.
162
When using more than one pest, the Panel recommended that the numbers of each species should be modified. Depending on the frequency of feedings and arthropod numbers used, the amount of blood consumed by a combination of ticks and fleas could be substantial, especially on control animals. Fewer ticks and fleas would need to be used on animals where multiple species are combined. There is also a potentially increased risk of inflammation at feeding sites that should be monitored. While 50 ticks total would likely be well-tolerated, doubling this to 100 ticks total to test 2 species may be excessive and infested animals should be monitored closely for any adverse reactions. In contrast, placement of 150 adult female mosquitoes (3 species x 50 mosquitoes each) will not likely lead to a risk of either undesirable mosquito-mosquito interactions or undue stress on the test animal.  PG 37
We acknowledge that this would be on the high side of a parasite burden, however animals are monitored for health during studies and it is not mandatory to run 2 tick tests at a time. Decreasing the number of pests would increase the number of animals used for testing.
163
Finally, the proposed guidelines specify using pathogen-free ticks; however, the Panel noted that a more specific definition of this is needed. There is some concern that combining 2 tick species could facilitate dermal or sub-dermal pathogen exchange that might adversely affect test results.  PG 37
Based on numerous SAP and public comments, the specification of "pathogen-free" has been revised to "free of vector-borne pathogens" to be more specific.
164
The Panel recommended allowing simultaneous testing of 1 flea + 1 tick species only, except when the tick is Amblyomma maculatum. The Panel has also provided the EPA with further information about tick species that have been used to co-infest animals without interactions that might hinder accurate data collection (see response to Charge Question # 15).  PG 37
See response to comment #161.

Public Recommendations (docket ID # in parentheses): n/a

Charge Question 8
 For pet collars, one of the typical methods of application requires cutting the collar to size and therefore the collar application rates are often dependent on neck size and not animal's weight as for other pet products like spot-ons that have dosing labeled based on weight ranges. Because neck size may not correlate with body weight or surface area, there is uncertainty that testing with the formulated collar sized for the neck may not represent the most conservative situations encountered in the field (e.g., dog breeds with small necks but relatively big bodies).
 :: Please discuss how application rate should be addressed during testing of collars. Is cutting the formulated collar to neck size sufficient for testing, even though the typical breeds used in testing may not represent the most conservative ratio of neck size to body weight/surface area? Why or why not? If not, how should this issue be practically addressed during testing?

Panel Recommendations:
165
The Panel found it unclear that the neck size to body weight/surface area or cutting off excess pieces of the collar after it is placed on the pet is having a negative impact on efficacy of the pests claimed on the label for each respective collar. There is limited peer reviewed scientific literature that discusses the efficacy of collars in controlling fleas or ticks.

The availability of collars meant for small versus large dogs (defined by weight of dog) would appear to be an effort to address this issue. Additionally, the active ingredients (a.i.) listed by % weight of a.i. on commercial collars is the same in collars for small and large dogs.

The registrant for the collar likely conducted initial efficacy testing to show that the collar delivers residual efficacy (flea and/or tick) for the desired length of time listed on the product label with any excess collar trimmed off prior to laboratory testing in both small and large dogs. Therefore, the excess collar that is cut off in pivotal studies mimics early testing and the desired length of residual flea/tick efficacy is confirmed, as well as, in clinical field studies conducted globally using dogs of different sizes and breeds (see Stanneck et al., 2012b).

Excerpt from Stanneck et al., 2012b:  Seresto(R) (Bayer Animal Health), a new collar for dogs and cats, provides long term broad spectrum parasiticidal activity by combining the insecticidal properties of imidacloprid with the acaricidal properties of flumethrin. The collar matrix system ensures that both active ingredients are slowly and continuously released from the collar towards the animal thereby avoiding peak concentrations and ensuring that acaricidal/insecticidal concentrations of both active ingredients are present in the cat's or dog's hair coat during the entire efficacy period. The active ingredients spread from the site of direct contact over the entire skin surface of the treated animal.

The Panel recommended that the EPA request that the registrant defend why they are instruct users to cut off excess collar material not needed to make contact with the neck region of the treated pet regardless of neck and body size (surface area). Additionally, the registrant should clearly indicate to the EPA reviewers how the a.i. is being released from the collar matrix, including distribution and/or release kinetics of the a.i (or multiple active ingredients).

There are two primary reasons for cutting off the excess formulated collar to the neck size. The first is that only the collar matrix that touches the skin/hair coat is going to release a.i. so any excess collar not touching the animal will not release and provide a.i to the animal so it's not needed. Removing the excess reduces the potential for exposing the pet, the pet owner, and the pet's surroundings to unnecessary amounts of a.i. The second reason is for vertebrate animal safety. Animal safety studies for the end-use collar formulation are tested with the excess collar cut off so that the animal is not exposed to excess amounts of the a.i. and to minimize potential accidental entanglement. Since some collars will break and detach as a safety feature, this may also lead to a potential health concern if the vertebrate animal ingests it.  PG 38
Since the neck size to body weight ratio are correlated for most dog breeds (except for a few outlier breeds), the current method of determining application rates of collars for efficacy testing (measuring and cutting off the excess collar) is likely the most accurate and practical method. In addition, for most active ingredients, the amount of active ingredient released from a collar is dependent on the collar material/plasticizer as well as the amount of rubbing along the animals' necks.

Public Recommendations (docket ID # in parentheses):
166
"HCPA believes that the testing currently required for registration is adequate for evaluation of product safety. Toxicity testing allows in depth consideration of potential target organs and adverse effects as related to the active(s). We acknowledge that the CAS population (e.g., Beagles, purpose-bred hounds, domestic shorthair cats) is not an ideal representation of the breeds to be exposed in the field; however, the evaluation of 5x exposures in the CAS study as designed to reveal sensitivities within the species provides a solid dataset for evaluation. Further, by their nature, collars are designed to slowly release actives to the fur and haircoat via contact with the animal over the course of the treatment period, in comparison to a spot-on product for which the entire dose is applied at one time. Finally, appropriate post-marketing surveillance ensures attention is maintained for potential safety issues" (0008)
The Agency agrees with this comment. Please see response to comment #165 for full response.
167
"The problem here is that collar application rates cannot be calculated directly from the length of the collar applied. Since the collar represents an impregnated device with slow-release characteristics, active ingredient is only slowly released to the animal, and the application rate is a function of the collar material itself and any solvents that may be incorporated as well. In addition, a correction, if it were to be applied, would depend on a study of the variance in neck size and animal weight. That data does not exist to our knowledge, so no correction can be applied." (0033)
The Agency agrees with this comment. Please see response to comment #165 for full response.

Charge Question 9
Historically the Agency has requested studies for fleas, ticks, and mosquitoes on pets be conducted with 6 to 8 animals per treatment, although, the Agency has received studies with sample sizes ranging from 2 to 12 animals per treatment. Based on the statistical simulations provided in Appendix 1, EPA is proposing a minimum number of vertebrate animal subjects for each test type (i.e., pest) in the guideline. This number varies according to test type, ranging from 6 to 14, and is intended to achieve 80% statistical power (confidence) with a precision of 4 or 5%, depending on the specific methods employed for each pest. Please discuss and provide comment on:
a. The appropriateness of statistical methods to analyze the data and calculate percent efficacy and the associated 95% confidence interval.
b. The appropriateness of statistical methods and simulations EPA has developed to estimate the power of the proposed design, and specifically to achieve an adequate estimate of precision around the estimated mortality/repellency in the treated group for each section below:
    i. Fleas (section (j))
    ii. Ticks (section (k))
    iii. Mosquitoes and biting flies (section (l))
c. The assumptions used to inform the simulations (e.g., minimum blood-feeding rate of mosquitoes on control test animals, number of pests per animal) for each section. Specifically, please discuss any information that can be used to update or clarify these assumptions, especially where doing so would increase power with a smaller sample size (i.e., using fewer animals). Where possible, indicate which assumptions can be changed for each pest species or type of test animal and provide value estimates.
    i. Fleas (section (j))
    ii. Ticks (section (k))
    iii. Mosquitoes and biting flies (section (l))

Panel Recommendations:
168
Overall the Panel considered the proposed GLMM method to be consistent with commonly accepted statistical principles for point estimation (i.e., of efficacy); however, two Panel members showed a preference for conventional data analytical methods (e.g., geometric and arithmetic means) (Pfister and Armstrong, 2016) for characterizing the product efficacy, and the public comment by Dr. Kenneth Portier (see docket # EPA-OPP-2019-0161-0034) also suggested that the implementation and interpretation of the results of the proposed GLMM method may pose challenges to practitioners without advanced statistical training. Accordingly, given the importance of using valid methods (and their complexity), the Panel recommended that the Agency create generic or example "software," a step-by-step guidance document, and fully worked-out examples for implementing the GLMM method.  PG 40
EPA will add an example using synthetic data to show step-by-step data analysis for pest product efficacy studies to the document "Sample Size for Pet Product Studies". The updated version of this document will be provided later.
169
Proposed GLMM Method for Efficacy Determination
The Panel recommended expanding the justification of why day was included as a random effect in the GLMM scenarios. Also, the day random effects were assumed to be equally correlated in the proposed GLMM framework. The Panel recommended using a temporal correlation structure to model the correlation between different days, such as a decaying correlation as the difference between days increases (e.g., an autoregressive structure).  PG 40
EPA will expand the justification of why day was included as a random effect in the GLMM scenarios in the supplemental document.
170
Proposed GLMM Method for Efficacy Determination
The use of GLMM with a random animal intercept is essential to account for the non-independence of Bernoulli trials (i.e., the binary result on each arthropod) on an animal, and standard GLMs that do not account for correlations between counts within each animal should be avoided. Also, the random animal effects account for different animal characteristics (animals have different length, weight, grooming patterns, etc.).  PG 40
GLMM with random animal effect will be used to analyze the non-independent data (each animal has multiple measurements).  In the simulations to calculate the sample sizes, GLM was used to analyze a dataset on each day separately ONLY after the GLMM with random animal effect failed to converge.  However, the non-convergence issue with GLMM has been resolved (please see the attached memo of Dr. Cohen from ICF), and GLMM with random animal effect will be used to analyze data from studies where each animal has multiple measurements.
171
Proposed GLMM Method for Efficacy Determination
Other Panel suggestions for improving the proposed method included computing a population rate (as opposed to a rate conditional on the animal random effect) after fitting the GLMMs (in particular binominal GLMM with a logit link), ......  PG 40
GLMM with logit link that computes a population rate (as opposed to a rate conditional on the animal random effect) substantially improves the convergence of the model; however, the GLMM that computes a population rate is not able to account for the overdispersion in the data.  Therefore, a GLMM model that computes a population rate will be used only after a GLMM model that computes a rate conditional on animal random effect fails to converge.
Some quick simulations showed that the power results were similar between the approaches (GLMM with conditional random animal effect model vs. GLMM for population rate); therefore, the simulations using GLMMs for population rate would not be conducted to re-calculate the sample size (please see the memo from Dr. J. Cohen from ICF).
172
Proposed GLMM Method for Efficacy Determination
Other Panel suggestions for improving the proposed method included...... the adoption of a Bayesian approach with priors to address issues such as uncertainty in "binomial" sample sizes and measurement error in counting (likely more of an issue of undercounting),....  PG 40
EPA may evaluate and consider the Bayesian approach later when there are more available data to validate the assumptions such as prior distribution proposed in the approach.
173
Proposed GLMM Method for Efficacy Determination
Other Panel suggestions for improving the proposed method included...... and the application of data submitted by the registrants to validate the proposed GLMM.  PG 40
See response of comment #172.
174
Proposed GLMM Method for Efficacy Determination
In terms of efficacy estimation from the GLMM, the Panel realized that the approach to deriving a point estimate for efficacy and effectiveness is based on a metric/statistic that has been used for many years where the difference between treatment rate and control rate is normalized by the control rate. However, the uncertainty in these estimates was computed using an asymptotic (large sample) standard error (based on the delta method) which may be problematic for small sample sizes. Accordingly, the Panel expressed concerns about the accuracy of this uncertainty calculation and the corresponding coverage of 95% interval estimates. Hence, the Panel recommended that the Agency consider alternative approaches to obtain improved characterization of the uncertainty and more accurate confidence intervals including bootstrapping or using an 'exact' fully Bayesian approach.  PG 40-41
EPA recommends using a bootstrapping approach to obtain the 95% CIs of the estimates when analyzing the data of each actual study.  However, the bootstrapping approach is not practical for simulations to calculate sample sizes of Pet Product Efficacy studies due to the large number of datasets in each of the simulations.  Furthermore, based on the results of small number of simulations (Dr. J. Cohen's ICF memo), the power resulting from using bootstrapping to obtain the 95% CIs of the estimates was slightly greater or similar to the power that resulted from using asymptotic approach to estimate 95% CIs of the estimates. Therefore, the sample sizes estimated from simulation using bootstrapping techniques would be similar to the sample sizes obtained from the simulation using asymptotic (large sample size) standard error.
175
Proposed GLMM Method for Efficacy Determination
One of the major concerns of the Panel regarding the proposed GLMM was the widespread lack of model convergence. The lack of convergence represents a major limitation for using the GLMM reliably for power versus sample size calculations. Further, an analyst outside the Agency may also have difficulty with GLMM model convergence when performing their own analysis of the data. To partially compensate for the problem of model convergence, the more proper binomial model (conditional on random effects) was replaced by a Poisson model (also conditional on random effects) in the simulations and a log pseudo-likelihood function was maximized (due to the true marginal likelihood not being able to be computed in closed form). However, the Panel noted that while the lack of convergence was not solved using this approach as the GLMM models also failed to converge in many cases, the convergence issue could be addressed by using more accurate estimation methods including the Laplace approximation and quadrature (both available in SAS) and "exact" maximization of true marginal likelihood using Monte Carlo methods (available in R). Furthermore, the choice between binomial and Poisson modeling approaches should be based on the appropriateness of the model framework for the type of data, and not on minimizing the lack of model convergence. In this case, the binomial modeling approach is more appropriate than the Poisson modeling approach. Accordingly, the Panel recommended the use of binomial modeling approach with a logit link function instead of the Poisson approach.  PG 41
EPA has revised both the guideline and the supplemental document to reflect this comment. The GLMMs for binomial distribution would be used to analyze the data of Pet Product Efficacy studies. The sample sizes in the guideline were updated with the results from simulations using GLMMs for binomial distribution.
176
Proposed GLMM Method for Efficacy Determination
Other Panel suggestions for resolving the convergence issue included analyzing the data with only the random animal effect.....  PG 41
The non-convergence issue has been resolved for most of the datasets (almost 100%), using one of the Panel's recommendations (please see response to comment #171). Removing day effect from the model to have a simpler variance-covariance matrix will be considered after the model that computes population rate fails.
177
Proposed GLMM Method for Efficacy Determination
Other Panel suggestions for resolving the convergence issue included...... and fitting the model separately by each of the "true efficacy of treatment" levels (0.95, 0.925, 0.90, and 0.85) instead of including day as an additional random effect in the model (fewer random effects should lead to fewer convergence problems).  PG 41
See response of comment #176.
178
Proposed GLMM Method for Efficacy Determination
To facilitate the implementation of the aforementioned suggestions, the Panel recommended consulting the reference by Diggle et al. (2002) and the following R packages (function performed in parentheses): MCMCglmm (fit Bayesian GLMM), glmm (fit GLMM using MC likelihood), bootstrap (bootstrapping), vglm (fit betabinominal), and Rjags (fit Bayesian GLMM with random effects distributions besides normal and complexities including measurement error etc.).  PG 41
EPA recommends using a bootstrapping approach to obtain the 95% CI of the estimates when analyzing the data of each actual study (see response of comment #174).
EPA has not been able to evaluate the recommended methods available in R packages; however, EPA has added language for submissions using any statistical methods that are appropriate for the data.
179
Also, the Panel agreed that power analysis using GLMM via simulation allows the determination of sample size for the efficacy testing of pesticide products; however, additional efforts remain to clarify the methodology and assumptions employed (please see below).  PG 40

Power Analysis Using GLMMs
The Agency performed power versus sample size analysis based on selected parameter settings and criteria for sample size determination, including (a) a true efficacy of treatment equal to 92.5%, (b) an observed efficacy of 90.0%, (c) a precision of 4%-5%, (d) 80% power, and (e) the blood-fed proportion of arthropod ectoparasites on each animal. The Panel noted that the Agency's analyses could be strengthened by including a detailed discussion of the rationale for the chosen settings. For example, animal group sizes could be reduced by requiring a greater true efficacy, less observed efficacy, or less precision. Also, increasing the number of arthropod ectoparasites per host animal will allow smaller sample sizes to be used. Guidance from subject matter experts on the proportion of arthropod ectoparasites retained would be very valuable.  PG 40-41
EPA has added more detailed discussion of the rationale for the chosen settings in the supplemental document.
180
Power Analysis Using GLMMs
One particularly challenging issue surrounds the declaration of "true efficacy" in the power calculations. In the simulations performed by the Agency, the true efficacy was set to 92.5%. The rationale for this true efficacy seems to be that setting the true efficacy to 90% with 5% precision and 80% power would result in required samples that are too large to be practical. Since setting a true efficacy equal to 92.5% in the power versus sample size calculation moves the Agency away from the previous standard of 90.0% efficacy, the Panel recommended discussing this constraint in detail.  PG 42
See response to comment #179.
181
Power Analysis Using GLMMs
Also, in standard (more common) power calculations, the statistical power might refer to an 80% chance of rejecting the null hypothesis of no treatment effect in a study when there is a true non-null effect. The Agency employed a different approach of considering accuracy in the estimation of efficacy with a pre-specified level of precision (5%); the Panel observed that this approach is a lesser known, but still an acceptable form of a power calculation.  PG 42
More discussion of the rationale for the approach of pre-specified precision power vs. typical power of testing a non-null effect will be added to the document "Sample Size for Pet Product Studies."
182
Power Analysis Using GLMMs
The Panel also realized that based on different parameter values in simulations (e.g., parameters change for different study designs in Appendix I of the "Sample Size for Pet Product Studies" document), the results of power analyses are highly variable with the power often larger than needed (which then results in a larger sample size than needed). The Panel recommended consulting subject matter experts to find other biologically plausible parameter settings that could lead to smaller sample sizes. For example, increasing the number of arthropod ectoparasites per animal would increase the study's power and potentially lead to smaller required sample sizes. However, as discussed in response to Charge Question # 5, it appears to be neither practical nor humane to further increase the number of arthropod ectoparasites per animal beyond the level already specified in the Agency's proposed guidelines (please refer to Charge Question #5 for exploring the options in reducing the number of animals used).  PG 42
The required number of animals has decreased, for example, from 14 animals down to 11 animals in studies of testing topically applied products against ticks, due to the assumption of generating the random effects from normal distributions rather than Weibull distributions. However, the option to reduce the required numbers of animals by increasing the number of pests per animals is neither practical nor feasible due to humane reasons.
183
Power Analysis Using GLMMs
Also, the Panel noted that alternative designs and (or) models could be explored to potentially decrease the number of animals needed or to address the problem of lack of GLMM model convergence. For example, the Rao-Scott approach (Rao and Scott, 1992) can be used to model correlated binary data exhibiting over dispersion and is computationally simpler than GLMMs; computer code for implementing the Rao-Scott approach are available in SAS and R (e.g., the raoscott() function in the aod package).  PG 42
Rao-Scott cannot be used to analyze the data of Pet Product Efficacy studies. There are two important reasons that Rao-Scott approach is not applicable for the data of pet product efficacy studies. The first one is that Rao-Scott approach is suitable for testing significance of the treatment effect but not for estimating the efficacy of the treatment.  Another reason is that Rao-Scott approach assumes the treatment effect does not vary from day to day, which is not true in the pet product studies where the treatment efficacy decreases over time.
184
Power Analysis Using GLMMs
As mentioned previously, the GLMM proposed by the Agency has both animal and day as random effects. GLMM modeling often assumes that the random effects are normally distributed, and SAS only allows random effects to be normally distributed. However, in the "Sample Size for Pet Product Studies" document, the power versus sample size calculations evaluated cases in which the random effects were simulated using normal distributions or Weibull distributions. The Weibull distribution is flexible and can assume many different shapes depending on the shape and scale parameters. However, with small sample sizes, or small changes to the Weibull parameters, the shape of Weibull distribution can change substantially. This difference in shape could potentially alter the sample size calculations. Tables A1.4-2 and A1.4-3 show that the GLMM based results can differ considerably when assuming the random effects follow a normal distribution or Weibull distribution. Accordingly, the Panel recommended using only the normal random effect assumption for any final sample size determination based on SAS PROC GLIMMIX and to use other software (e.g., Rjags) for non-normally distributed random effects.  PG 42
The guideline has been revised to reflect this comment. The sample sizes resulting from simulation in which random effects are generated from normal distributions will be used for the studies.
185
Appropriateness of the Assumptions Employed
Based on Tables A3.1 and A3.2 in Appendix III of the "Sample Size for Pet Product Studies" document, the blood-fed/retention proportions of 0.25-0.6 and the numbers of arthropod ectoparasites per animal of 50-100 were used by the Agency for performing the simulation exercises of sample size and power determinations. However, the Agency provided no information on their sources or biological rationales. Nevertheless, the blood-fed/retention proportions appear to be consistent with the literature values of >25% attachment rate needed for establishing an adequate tick infestation and >50% attachment rate needed for establishing an adequate flea infestation (Taenzler et al., 2016). Similarly, the numbers of arthropod ectoparasites per animal employed in the simulations are consistent with the efficacy studies involving ticks (50) (Taenzler et al., 2016), fleas (100) (Beugnet et al., 2011), and mosquitoes (100) (Meyer et al., 2003). Given the importance of the blood-fed/retention proportions and the numbers of arthropod ectoparasites per animal, the Panel recommended that the Agency verify these values based on information obtained from the open scientific literature, registrant submitted studies (since the inception of 1998 testing guideline), and experts of these ectoparasites.  PG 43
See response to comment #150.
186
Appropriateness of the Assumptions Employed
As stated above, if an ectoparasite retention proportion parameter setting that can be used is higher than the setting assumed in the current models, smaller sample sizes may be possible. Also, the "actual" retention proportion can help in a case-by-case basis for "flexibly" deciding the number of animals needed for the product efficacy testing. For example, the tick Rhipicephalus sanguineus has a high expected retention proportion on dogs, in comparison to Ixodes scapularis, with a typically low retention proportion on cats, suggesting that the number of dogs used for the product efficacy testing with R. sanguineus can be reduced. The data for ectoparasite retention proportions on individual animals may be limited in the published literature. However, the testing laboratories may already have historical ectoparasite retention data collected and available to the Agency upon request. Average retention rates can also be informative for use in reducing the number of animals used, and some sources do cite individual animal data that may be useful for the Agency's consideration (Dryden et al., 2008; Halos et al., 2014; Dumont et al., 2015a, 2015b; Baker et al., 2016; Six et al., 2016; Vatta et al., 2019).  PG 43
Companies may use their own historic data to determine a sufficient sample size of adequate power to address the study objectives for specific pests. This clarification has been added to the guideline.
187
Editorial Comments
1. The use of "true efficacy" in the EPA document titled, "Sample Size for Pet Product Studies" is difficult to follow. For example, on page 7 (line 5 under the heading) the true efficacy is for sample size calculations was listed as "...true efficacy >= 90% with a precision of >= 5%..." But it appears that the sample sizes are based on true efficacies of >= 92.5% with a required efficacy of >= 90%, and a precision of 4 or 5%.  PG 43
The revision to reflect this comment will be made to the document "Sample Size for Pet Product Studies."
188
Editorial Comments
2. Is there a discrepancy between true blood fed proportion in control group of 0.45 in the table on page 7 and the listing of 0.40 under the "input parameters for the simulations" on page 8 of the document "Sample Size for Pet Studies"?  PG 43
The value 0.40 on page 8 was a typo.  It has been corrected and updated to 0.45.
189
Editorial Comments
3. The numbers under "Details simulations" on page 37 that reads in part "P*(1-TE) and equal 0.0475, 0.045, 0.04, and 0.3." It appears that "0.3" is an error and should be "0.03."  PG 43
The value 0.3 on page 37 was a typo.  It has been corrected and updated to 0.03
190
One Panel member suggested that, as an addition to the power analysis, the Agency could address the sample size issue in terms of number of animals per test via in silico approaches such as conducting "experiments" on computers using Monte Carlo simulation or Markov Chain Monte Carlo simulation techniques integrated with biologically-based modeling (Portier and Kaplan, 1989; Thomas et al., 1996; Bernillon and Bois, 2000; EPA, 2010). Considering that the original publication of the 810.3300 Guidelines was in 1998, this Panel member considered that the wealth of information possessed by the Agency could be used to strengthen the probability distributions of the parameters needed for conducting the suggested computer simulations, and therefore, may substantially reduce the number of animals needed.  PG 40
A section about future use of NAMs has been added to the guideline. Although EPA did not pursue this for the guideline because of time constraints, using historical data submitted to the Agency in the suggested manner is an area the Agency may explore in the future.
191
The effort by the statisticians of the OPP to clarify the details of the math modeling and computer simulations in their work on SAS program of simulations (docket # EPA-HQ-OPP-2019-0161- 0003) on "Sample Size for Pet Product Studies," were very helpful and appreciated. Given the fact that the "1000 sets of data" were created by Monte Carlo Simulations, the EPA are already "doing experiments on computers." Therefore, the Panel further encouraged EPA colleagues to develop in silico approaches for the goal of reducing animal usage.  PG 43-44
EPA added language to the guideline encouraging the submission of protocols for NAMs, many computer-based models can be considered NAMs.

Public Recommendations (docket ID # in parentheses):
192
"As related to sample or group size, EPA claims to be committed to reducing animal testing in some cases while proposing more animals than currently used in others. Could the EPA please explain?" (0008)
EPA is committed to reducing animal testing while ensuring scientifically sound and efficacious products are registered for the market for public use. The numbers to be used are based on sufficient power analysis to address the study objectives which should decrease the number of studies that are currently deemed unacceptable and need to be rerun.
193
"The "Sample size for pet product studies" document was simulation based on alpha values, false positive rates, and confidence intervals, yet lacked biological evidence or data sets to justify these values. :: The power analysis presented is based on a true random sample, yet every study will have stratification of treatment groups that will not be random. :: The data generated for a typical pet product will not be normally distributed, yet this is an assumption for most statistical models. Historically, there has been no requirement for statistical modeling, because there has been no need when utilizing the cut-off values of >=90% corrected treatment mortality. The Agency reported that typical studies received were between 2 and 12 animals per treatment. The reported range seems to be in agreement with the simulated data. :: Often a product meets the cutoff criteria and "works", or it does not. Manipulating data into a statistical model is not relevant when evaluating if products satisfy the cutoff criteria (>=90% corrected treatment mortality)... We believe that for most products being developed statistical models should not be required because meeting the efficacy requirements (>=90% corrected treatment mortality) at a given time point is sufficient to support that a given product is effective. Data generated in observance of these cut-off values would lack variation that would be informative in statistical processes. Utilizing the previously established minimum of 6 animals per treatment group (6 treated, 6 untreated), and the threshold efficacy requirements should give an adequate evaluation for product efficacy while reducing the number of vertebrate animals needed for testing." (0008)
The Agency disagrees that the statistical analysis recommended in the guideline document do not provide useful information.  The purpose of the power versus sample size analysis was to determine if the historical sample sizes were correct or if the sample size was inadequate. For testing against most pests, the historical number of 6 animals per treatment group turned out to be similar to the numbers generated using the statistical models; however, when evaluating efficacy against ticks on dogs the historical number of 6 animals per group is clearly not adequate to provide confidence that a result of 90% efficacy reflects the true efficacy of the product. Basing a sample size on historical precedent is not adequate and can lead to problems with underpowered studies, as would be the case if the Agency recommended 6 animals per treatment group for tick studies and falsely concluding that the product reaches the 90% efficacy threshold when it does not reach the 90% threshold. Registering pesticide products based on underpowered study designs which result in a false conclusion of 90% efficacy when the product is not 90% effective and could have negative implications for public health. Moreover, historically registrants have calculated 90% efficacy using several different methods which were typically calculated incorrectly and often did not provide a measurement of variance.  The recommended statistical model will not only calculate 90% efficacy appropriately but will also provide 95% confidence limits to allow the agency to assess the variability in the data. Because EPA is recommending the model for calculation, using the model will improve the confidence in and consistency of the studies submitted to EPA and promote a level playing field for all registrants.

194
"The Guidelines, as currently presented, do not make clear to the reader exactly which data need to be collected, but allow the registrant choices depending on the product's control outcome. For example, in section (j)  -  specific guidance for control of flea infestations, six conditions must be specified that establish when and what data are collected.
The number of animals (6 cats, 7 dogs) to be tested per treated and control groups,
The number of fleas (100+-10) to which each animal will be exposed,
The timing and number of days (1, 2, or 6+) of exposure,
The duration of exposure (24-48h), and,
When counts are recorded (up to 72 h post treatment, but "last count <=48h post infestation).
When re-infestations are applied, and outcome counts made for studies of long-term control.
Section(j)(3)i discusses the data collected for efficacy assessment. The guidance recommends comparing counts of "live fleas on the treated animals to live fleas on the control animals" or, in the case where killing ability is being assessed, comparing counts of "dead fleas on the animal and in trays/cages between treated and control animals".  This recommendation drives the choice of statistical distribution assumed in the final data analysis. If the product under test is fully efficacious and measurement is not too long after treatment, one would expect the live count to be very low which supports the assumption of Poisson distributed counts.
The form of the Poisson distribution applicable for a count can be affected by any number of factors that can and should be controlled for in the study design. It is critical that condition 2 above - similar numbers of fleas applied per animal, holds for all animals. Too much variation in exposure counts leads to expected counts with different Poisson distributional forms.  This could lead to the need to account for extra heterogeneity in the analysis (impacting the estimates of product efficiency).  Also important is that during the selection phase of the study, animals with wildly varying retention fractions are not used.  The expected retention fraction also changes the effective number of fleas to which a animal is exposed. If animals used in the study vary wildly in number exposed or in retention fractions, blocking on retention fraction or exposure number should be used in allocating animals to treatments and a block effect accounted for in the final statistical analysis.  While this will reduce the effective sample size (number of animals by number of treatments) it should also produce more precise comparisons of average counts (Poisson or Negative Binomial assumption, or success rates for Binomial assumption) between control and treated animals. Instead of blocking, the disparity in exposure levels may be accounted for in the final statistical analysis model, but this can make the analysis potentially more difficult to accomplish successfully from unanticipated computational issues leading to lack of convergence in model fitting.
Extra variability may also be introduced to counts made following re-infestation to assess claims of sustained product efficacy.  If all fleas are not removed (or killed) at the time of re-infestation, the effective number of fleas to which each animal is exposed will also be different. This is especially important in comparisons involving negative controls where little impact (e.g. mortality) is expected. For long-duration studies without flea removal from controls before re-infestation, the effective number of fleas on control animals would be expected to go up compared to an almost constant number of fleas on treated animals. The present guidance draft is silent on this issue.
Section (j)(5) discusses data analysis. This section would benefit from much more detail. The guidance should provide clearer direction on when a Binomial, Poisson or Negative Binomial distributional assumptions should be used.
Poisson distributed counts would be most appropriate for counts of successes (live or dead) from a large number of Bernoulli trials (e.g. when 100 fleas are tested per animal as recommended). The decision to include extra variation (e.g. assume a Negative Binomial distribution) in the analysis model could be made after consideration of whether there is wide variability in the effective number of fleas applied to each test animal. In the situation where all fleas on control animals survive and all fleas on treated animals die, each animal is a Bernouilli response.  With a fully effective product and tight study design this outcome in not completely infeasible. While the data are very convincing of product efficacy, statistical analysis would be difficult. The document should provide some guidance on what should be done in this situation.
In general, any analysis using the binomial distribution requires data consisting of a count of number of pest on test per animal (e.g. total number of fleas applied) and a count of successes. This requires a clear description of what is a success (e.g. for efficacy study a success is on-animal and live, not moribund, for a killing study, moribund or dead, on or off animal).  Of most importance is the establishment of the total number tested. The uncertainty in the exact number (specified 100+-10 fleas) is less important than that animals have near similarity in the "effective number on test". In addition, if in the original acceptability assessment, a test animal is estimated to only retain 80% of fleas applied, one might assume that only 80% of fleas applied to the animal are potentially exposed to the treatment (an a priori assumption in a Bayesian formulation).  As mentioned above, complicating the analysis is the expectation that potentially 100% of fleas applied to control animal might be successes and 100% of fleas applied to treated animal might be failures.
The guidance suggests the use of a Generalized Linear Model (GLM) for comparing treatment to control (more precisely, for estimating the statistical significance of the difference between treatment and control count "averages" or success "rates") in one-infestation studies.  These models improved on the logit (or probit) "regression" models which might have been used in the past and to which many biologists and risk assessors have been exposed.  The guidance document should provide more justification for why use of a GLM is preferred over a logistic model (e.g. relies on assumptions with greater face validity and results in better overall estimates and statistical tests).
Multiple infestation studies require a more complex model depending on the inference desired. The guidance recommends use of a Generalized Linear Mixed Model (GLMM) for this kind of study. With a GLMM, both model fit and interpretation become more difficult, often requiring the help of a statistician with more advanced training. A GLMM provides potentially a better way of estimating the statistical significance of differences in treatment of estimated control average counts or success rates, both at individual sampling time points, and overall.  The guidance is not clear on which of these two types of inference are preferred, or whether both should be provided. The improvements provided by use of a GLMM (over, say use of a GLM at each test time point) comes with increased complexity of model, the need for more assumptions that may not be easily checked, and the potential that the final model will difficulty computationally "converging" (e.g. fitting).  Biologists and risk assessors will be unfamiliar with these models and have difficulty with interpretation of results. The guidance should provide more details and in particular should provide worked examples where model formulation, fitting and parameter interpretation issues are illustrated and discussed. While extensive examples are not needed, I am unable to find well worked examples in the current literature that are accessible to and understandable by the Master's level biologists most likely to be conducting these studies. Finally, statistical packages that accomplish the computations needed for fitting a GLMM often have the capability to compute and display the parameter estimates and associated (approximate or asymptotic) standard errors in the original count metric (e.g. average count or proportion and not a logit or log odds) that are needed for tables in reports submitted for product review. The document should help the analyst by pointing to these options in the functions used in the more common statistical packages (e.g. SAS, SPSS, and R). Demonstrating the same examples using SAS, SPSS and R would help as well.
Section (j)(5) suggests using Abbott's formula to compute estimates of product effectiveness at each measured time point. There are multiple issues with this formula that need to be discussed in the document but are not discussed in this draft.
First, if a Poisson-based analysis is used, the statistics produced are essentially averages, not rates. Abbott's formula is presented as a ratio of rates, so it is not clear how the output from the GLM or GLMM is to be applied in the efficacy estimate. It may be that the expectation is to use "raw" rates in Abbott's formula, but this then bypasses the benefits of using formal statistical analysis models for the estimation (and, as I discuss in the next paragraph, leads to further issues with computing a standard error estimate for the efficacy estimate) . As the sample size simulation study demonstrates there are often computational convergence issues with using a Binomial distribution-based analysis which would be expected to produce better rate estimates, and which would provide standard error estimates directly for these rates. What does the analyst do when faced with computational convergence errors?
The guidance document also recommends computation of the 95% confidence interval for the efficacy estimate computed using Abbott's formula. This is not straightforward since Abbott's formula is essentially the ratio of two estimates for which direct estimates of associated estimate uncertainty cannot be easily derived. While not mentioned in the guidance but pointed out in the second paragraph of the footnote on page 6 of the sample size simulation document, statistical software packages will not directly compute Abbott's formula nor do they provide standard error estimates and confidence intervals. As also pointed out in the footnote, estimates for standard error of the efficacy estimate can be approximated using the delta method which is based on a Taylor series expansion of Abbott's formula. But the delta method itself is not easy to apply. Typically only a first order Taylor's series approximation is used. This can be a very rough (that is, not very good) approximation and depends on estimated partial derivatives of the GLM or GLMM likelihood function in the region around the final estimates. Clearly estimated partial derivatives will not be available if the model does not converge.  If it does converge, the analyst needs to find, output and use these estimates in the formula of the delta method. Not every statistician/analyst will be familiar with the process of extracting these values or even with the delta method. This is another area where fully worked-out examples in SAS, SPSS and R should be provided in the guidance document or at least in an appendix to that document.
Section (k)(1) Test Species recommends that when testing for tick mortality and repellency a "laboratory study should be conducted in vivo with adults of three representative species of ticks." Further it specifies that more species of ticks may be used but "no more than two species should be tested on the same animal simultaneously."  Finally, section (k)(2) recommends no fewer than 14 animal subjects per treatment group be used and that animals be allocated to control or treatment groups using stratification based on attractiveness. [Note this issue does not come up with mosquito species since each animal is tested to equal number from the three major species.]
The allocation of animals based on attractiveness represents restrictions on randomization which essentially creates "blocks" in the study design.  The blocking of experimental subjects will need to be accounted for in the final analysis model, something not mentioned in the data analysis section.
The restriction that no more than two tick species be used on any one animal brings up the issue of "balance" in the design.  To illustrate, assume 14 animals are selected from the study and three tick species are used. Allocation may look like the following.

Animal
Tick Species
(number of Ticks)
Block/Strata

A
B
C

T-1
50
50

1
T-2
50

50
1
T-3

50
50
1
T-4
50
50

2
T-5
50

50
2
T-6

50
50
2
T-7
50
50

3
T-8
50

50
3
T-9

50
50
3
T-10
50
50

4
T-11
50

50
4
T-12

50
50
4
T-13
50
50

5
T-14
50

50
5
T-15

50
50
5

Note that with only 14 animals (T-1 to T-14) used, more of species A individuals (500) are tested than for species B and C (450) which can lead to a slight biasing of results to the response of tick species A to the treatment.  Without adding a 15[th] test animal the design is unbalanced. In addition, if blocks of say 6 animals per block (3 assigned to treatment and 3 assigned to control) are used, the last block would be incomplete leading to an incomplete block design and further issues with accommodating block effects.
These issues of blocking, balance and incompleteness of blocks is not addressed in the simulations used to estimate the number of animals required for even the most basic of tick study.  Note that blocking can lead to more precise estimates and hence could result in the need for fewer animals (say 12 per treatment and control versus 14). Balance ensures that no one species has advantage over another in its impact on the final estimates.
Imbalance and incompleteness of blocks are not particularly difficult conditions to accommodate into GLM and GLMM analysis models, and hence would not be expected to keep the analyst from obtaining estimates. But imbalance and incompleteness can be easily avoided by better design (and in this case, the use of one additional test animal for both treatment and control), and the resulting average and rate estimates and hence the efficacy estimate will be more precise and potentially less biased.
Note that balance and completeness become more difficult the more tick species that need to be accommodated. With 4 tick species, 6 animals are needed to handle the 6 different unique pairs of species.  In this case, sample sizes of 6,12 or 18 animals per treatment would be used to ensure balance and complete blocks.
If blocking is used, it is not clear that each block requires the same number of control animals as there are treated animals for the design to be optimal. Savings in total number of needed animals might be possible if this situation is considered. Increased precision through the use of blocking is only possible if the factor used to stratify animals is related to the response (in this case the expected count or rate). If the relationship is weak, blocking will only marginally improve estimates and reducing the number of animals in each block receiving the control might lead to reduced precision in the comparison of treatments to control. This really needs to be researched (simulated) before this option were allowed.
Nowhere in the document is the issue of who does the counts discussed, although this may be considered part of standard GLP.  In most of these studies, counting is clearly time consuming and likely to be tedious. The temptation is to assign multiple individuals to this task to ensure the task is done within a reasonable time frame and without the biases caused by fatigue. But the use of multiple individual counters creates another factor that potentially can impact the data and the precision of the final estimates. Identifying counters and taking intra-counter variability into account in the final analysis can lead to more precise estimates of averages and/or rates and better estimates of efficacy.
It is admirable that the SAS code is provided along with the study report. This allows reviewers to rerun the simulations, assuming they have access to SAS and its macro facility (which I do not have).  But each SAS code document comes with a very low number of internal comments. Some of the variables (primarily bounds on key simulation conditions) used in each macro are defined in the comment block at the beginning of the macro. The flow of the macro is not well described making it somewhat difficult to follow exactly that is happening or to determine whether there might be issues with the simulations. As far as I could tell the simulation of the data follows closely the description provided in the Sample Size for Pet Product Studies document.
From the available code I was able to better understand the simulation by rewriting one of the SAS programs (Sample size for Pet Product Studies (normal).sas) as an R program (see appendix). In the time allowed for this project I was not fully able to replicate the Proc Glimmix analysis to the respective glm() or glmer() analysis in  R. This exercise reinforced in me the need for EPA to help analysts who may not have access to SAS to perform the needed analysis in other statistical packages.
I did note that three of the four simulation rely on Proc Glimmix with the poisson distribution assumption and the log link as the analysis scenario/model used to determine the significance of treatment effects. This means that the simulations do not provide sample size estimates for situations where the binomial or negative binomial distribution would be expected to be used as the basis of the data analysis. I am uncertain whether the recommended sample sizes based as they are on the poisson are adequate for these other situations. This should be explored before the final recommendations are published." (0034)
The guideline has been updated to more clearly distinguish what data should be collected and has been revised to only collect live fleas on treated vs. control animals.

The parts of this comment about statistics differs in opinion than the SAP panel and the EPA agrees wth the SAP panel comments.

195
"Analyzing data. If corrected mortality is required to meet 90% in the treatment group, the rest of this statistical analysis is irrelevant and should be omitted." (0008)
See response to comment #193.

Charge Question 10
Please provide comments on the overall clarity, accuracy, and completeness of the draft pet product guidelines. Please provide any additional comments that highlight areas of the draft guidelines that may need to be clarified and note any critical topics that are missing. Please include references to published literature that could help improve the completeness and clarity of the draft guidelines.

Panel Recommendations:
196
Given the importance of the blood-fed/retention proportions and the numbers of arthropod ectoparasites per animal, the Panel recommended that the Agency verify these values based on information obtained from the open scientific literature, registrant submitted studies (since the inception of 1998 testing guideline), and experts of these ectoparasites.  PG 44
The references used for blood-feeding/retention and the numbers of arthropods per animal were derived from peer reviewed literature and studies submitted to the Agency. They represent what is seen consistently in submitted studies that use methods like those described in the guideline. For mosquitoes and biting flies, values were obtained from Tiawirisup et al. 2007, Franc et al. 2012, Lienard et al. 2013, and Franc et al. 2015 which show mosquito bloodfeeding in the controls to be consistently at or above 60%. For fleas on dogs, retention rates of at least 60% of fleas has been shown in Lienard et al. 2013.
197
1. There is a mix of scientific names (genus species) and common names. The Panel recommended that the scientific name be used exclusively or at least at the first use.  PG 44
The guideline has been revised accordingly.
198
2. The term "Biting fly species" is not entirely accurate, at least for the list of flies presented. For example, face flies are not biting flies. Thus, either they should be removed from the list or the term modified. If the term is modified, to include face flies, then house flies should also be included. The Panel recommended that the language be changed to "mosquitoes and stable flies" and that some language about "evaluation of other biting flies would be considered on a case-by-basis (contact EPA for guidance)."  PG 44
The guideline has been revised to delete all flies except for stable flies and sand flies and indicate that all other flies would be considered case by case.
199
3. The need for "outcrossing" is unclear and what is meant by this is not sufficiently documented.
:: Outcrossing is commonly done when the investigator wants to retain a "field relevant trait", such as a parasitoids ability to hunt for food.
:: Outcrossing is avoided for toxicology studies because responses to insecticides do not generally change and having a standard reference strain is desirable and makes reproducibility feasible.
:: Thus, for the types of assays in the proposed guidelines, The Panel suggested that outcrossing is unnecessary and even perhaps detrimental. Using a standard strain is more desirable. Outcrossing changes the strain in unpredictable ways, making reproducibility problematic.
:: The Panel suggested that outcrossing be removed from the proposed guidelines.
:: If outcrossing is to be retained in the protocol, then more details are needed as to what strain should be used for outcrossing, what methods should be employed (e.g., reciprocal crosses?), how many individuals are to be used, etc. It seems that this level of insect husbandry is not needed for these protocols.  PG 44-45
The guideline was revised to remove the outcrossing recommendation and to allow the use of wild species if lab colonies are not adequate for a pest.
200
4. The design of the repellency tests are worth reconsidering. The current method is a no-choice assay where the pests are applied directly onto the animal, whereas, the purpose of the test is to determine if a product prevents pests from associating with an animal. The Panel noted that alternative protocols could include adjoining cages where the pests can move from one animal to another (choice) or releasing the pests into the cage and determining if they are repelled (willing to get on the animal or not) (no-choice test). The Panel recommended that repellency assays be done independently of mortality assays.  PG 45
The repellency methods have been revised to include releasing the pests in a cage instead of onto the animal directly.
201
5. The procedure for the assessment of blood feeding is a bit vague. Most blood feeding arthropods can take a full blood meal or some partial bit of a blood meal. The current assay ("crushed on a light background") suggests that even a tiny blood meal is counted. Is that desired or are people being asked to score the relative size of the blood meal taken (and if so, how)? The Panel recommended that blood feeding be determined by PCR so that the blood meal can be quantitated, and the species identified.  PG 45
The guideline has been revised to include the determination of blood feeding for mosquitoes only. For topically applied products, blood feeding from mosquitoes is considered as a failure.
202
6. For shampoos, the Panel recommended that the formulation blank be used as the control because of the variability in shampoo products. Registrants will almost certainly have the formulation blank available to them.  PG 45
After consideration of SAP and public comments, the use of a non-pesticidal (vehicle) control to compare against for shampoo products has been changed to a negative control. See response to comment #237 for more information.
203
7. Pests that have fallen off the animal can be hard to find. It is unlikely that 100% of the ticks applied can be recovered. The Panel noted that this presents some statistical challenges to dealing with the data. If 50 fleas are released and only 35 fleas are recovered, how will the statistical analysis be carried out?  PG 45
The guideline has been revised to count only live fleas or ticks on treated vs. control animals, therefore, the statistics should be run using live fleas or ticks.
204
8. The Panel recommended that the title of the guideline be changed from "control" to "efficacy".  PG 45
The name of the guideline has been changed to:
Test Guidelines OCSPP 810.3300: The Efficacy of Topically Applied Pet Products Against Certain Invertebrate Pests

205
9. The Panel recommended that the suitability of a vertebrate animal to be used for the flea or tick tests be determined individually, but that the final control and treatment groups be compared as averages.  PG 45
The guideline has been revised to reflect this comment.
206
10. Although the guidelines address efficacy of new products, it appears that only pre- and posttreatment body weights are required. The same ingredients that kill ectoparasites can also cause a variety of neurological and physiological reactions in dogs/cats; scratching of bites can lead to inflammation and infections. Anemia has been reported in control dogs. The Panel recommended daily inspection of animals for health problems be conducted.  PG 45
Inspection of the health of the animals must comply with GLP regulations at 40 CFR part 160.9 and is not covered specifically in the guideline as it is not part of the efficacy methods.

207
11. Page 7 of the proposed guidelines: the last sentence of d(3) Analyzing data, second to last sentence - "Software for analysis using GLMs or GLMMs is available in many widely sold statistical analysis packages." In this sentence, the Agency appears to be endorsing the use of commercially available statistical software packages; however, there are open source software packages (e.g., R) available to perform the same task. The Panel recommended that the guidelines clarify when specific software is required and when specific software is only a suggestion.  PG 45
The reference to "widely sold statistical analysis packages" have been deleted from the guideline.
208
12. Page 9 of the proposed guidelines: first sentence of j(2) Selection and allocation of test animals  -  "The minimum number of qualified animals per group should be determined to achieve sufficient power in addressing the study objectives....." The Agency already employed the power analysis technique to determine the minimum number of animals required. Perhaps, the purpose of the aforementioned statement is to address the situation when the number of animals employed is different from (i.e., less than) the recommended value? The Panel recommended that this concept needs to be clarified. The same comment also applies to k(2) (page 12) and l(2) (page 16).  PG 45-46
To increase clarity, this statement was deleted form the individual pest sections and has been moved to the "Development of protocols for efficacy studies" section.
209
13. Page 11 of the proposed guidelines: first sentence of j(5) Data Analysis: "If the study has only one measurement (one infestation) per animal, GLMs for binomial distribution data (with log link function), Poisson distribution data, or negative binomial distribution data should be used to estimate the survival or hatching rate of each group." The Panel noted that this sentence is in contrast to the recommendation specified in section (d)(3) that techniques other than GLM or GLMM may also be used. Also, additional guidance on employing different distribution functions should be provided, e.g., betabinomial distribution for the "binomial" data exhibiting over dispersion etc. The same comment also applies to k(6) (page 15) and l(6) (page 18).  PG 46
The guideline has been revised to reflect this comment.
336  -  out of order
14. The Panel noted that the term "mode-of-action" as it appears on pages 14, 15, and 18 of the proposed guidelines needs to be defined in section (c). The "definition" of mode of action on page 15 (e.g., volatile repellent, toxicant) is not correct.  PG 46
The term "mode of action" has been changed to "effects on the arthropod pest."
210
15. Page 4 of the proposed guidelines: (c) Definitions The definitions of "Knock down" and `Moribund" are very similar and difficult to distinguish. The Panel recommended they be clarified.  PG 46
The definitions have been clarified and the recommendation to count "knocked down" pests has been deleted.
211
16. Page 6 of the proposed guidelines: d (1), v. Housing of Animals: the Panel recommended that the permissive "may" be replaced with "should" with respect to group housing of dogs and cats at times other than during infestation.  PG 46
The guidelines already indicated that prolonged isolation should be avoided and used the word "may" to allow them to be housed together at other points. The recommendation by the Panel will not be incorporated into the guideline to avoid changing the way labs currently house animals or how they may handle incompatible animals.
212
17. The Panel recommended that for analyzing and presenting data, the guidelines need to indicate that the arithmetic means must be used.  PG 46
This suggestion has been incorporated into the guideline.
213
18. The Panel noted that in the tick efficacy section, EMA tick efficacy wording should be used with in-house revisions.  PG 46
There are no specific examples given here with respect to which wording is in question, however the European Medicines Agency's guideline is cited and referenced.
214
19. The Panel recommended inserting the phrase "adequately infested" when discussing minimum percent tick/flea retention thresholds.  PG 46
The suggestion has been incorporated to clarify retention thresholds.
215
20. The Panel recommended that the guidelines define the different types of tests discussed. Specifically, the Panel suggested the term non-vertebrate animal testing (NAT, NVT, or NVAT) for tests that do not use the host. The Panel recommended that the terms in vivo, in vitro and in situ be avoided because they have different meanings in different scientific circles (e.g. some would claim that a NAT was an in vitro test, while others would claim it is not). Thus, the guidelines should avoid terms that have non-equivalent synonyms in different scientific disciplines.  PG 46
The term "in vivo" has been changed to "on animal" to provide more clarity.
216
21. The Panel recommended that the guidelines should state how many laboratory efficacy studies are required for each parasite that is included on the final approved product label. For example, FDA approved drugs require two separate pivotal dose confirmation efficacy studies for each specific parasite. Additionally, these two studies must be conducted at two different independent locations using different parasite isolates.  PG 46-47
The number of studies required depends on the specific claims desired. Adding this information would be outside the scope of the guidelines.
217
22. One Panel member indicated a desire to add a section on "The Use of Computational Technology" to encourage manufacturers to utilize such applications of computational technologies to reduce animal usage.  PG 47
See response to comment #98.
218
23. Page 5: (d) (1) i. of the proposed guidelines: The guidelines should define in practice what the term "Negative control" should encompass.  PG 47
The definition has been added to the guideline.
219
24. There are some formatting issues (for example, Tables break across pages; citations that need to be separated by a hard paragraph).  PG 47
Formatting errors have been corrected.
220
25. It is not clear if the words `must', `should' and `may' are always used appropriately in the guidelines. The Panel recommended that the use of these words be examined carefully.  PG 47
The Agency has confirmed that the proper terminology has been used throughout the guideline.
221
26. Section (k)(5)I of the proposed guidelines: The Panel suggested that the Agency consider reducing the number of categories of ticks examined from six to four. Efficacy should be based on the number of attached/free live ticks in the control group versus number of attached/free live ticks in the a.i. treated group.  PG 47
The Agency has made this revision.
222
27. Section (k)(5)i top of page 15 of the proposed guidelines, The paragraph starting with "in situations:"
:: "In situations demonstrating the product has an acaricidal effect only after the tick blood fed (engorged)", is this a realistic outcome for a topical application?
:: The statement, "Dead ticks combined with live ticks to calculate mortality?" is confusing.  PG 47
The endpoints and counting recommendations have been revised to only include live counts on the animals. See response to comment #20 for more information.
223
28. Section (k)(3)ii provides detailed information for repellency testing against ticks. The Panel recommended the equivalent information be provided for fleas if registrants also have repellency claims for fleas.  PG 47
The sentence stating that repellency should not be considered for fleas was deleted and language was added to encourage the submission of a protocol for flea repellency testing.
224
29. Table 1: The heading of column #2 should be changed to "Timing of infestations and reinfestations."  PG 47
This change is not necessary as each reinfestation is an infestation.
225
30. Section (l)(1)ii: The Panel recommended that the species that should be tested needs to be clarified. This would alleviate the need to contact EPA when registrants want to do these tests. Clarity is needed to understand if each species must be tested if it goes on the label. Also, does the Agency want only some species (or 3 of five species) tested to give a "biting flies" label?  PG 47
The guidelines indicate how to test a product's efficacy. The specific species necessary to gain a general registration will be indicated in a subsequent Product Performance Rule.
226
31. Section (l)(2): The Panel recommended that the phrase, "caged and exposed to 50-100 adults" be changed to "caged and exposed to 50-100 female adults" since blood meals are the end point for this bioassay.  PG 47
The guideline has been revised to include only female mosquitoes for testing.
227
32. Section (m)(3)ii: Delete "for testing mosquitoes" from the end of the last sentence. This is already stated.  PG 47
The guideline has been revised to incorporate the suggested revision.
228
33. Page 6: viii. c: The Panel noted that in some field studies, it might be necessary to use a positive control for animal welfare concerns; therefore, the statement that "a positive control should not be used" should be modified.  PG 48
The guideline does not provide any tests where a positive control should be used. If a study would be conducted that would need a positive control, a protocol should be submitted with the use of the positive control justified. No change to the guideline is necessary.

Public Recommendations (docket ID # in parentheses):
229
"We also note throughout the document usage of passive language in which registrants "should" do this and recommend more assertive and descriptive language to minimize ambiguity" (0008)
Since this document is a guideline and not a rule, passive language is used where appropriate, which it is most of the time. No changes to the guideline are necessary.
230
"In all sections we need better clarification on qualifying control sample numbers. So, for example in the fleas on dogs setup at least 60% retention is required, is this an average of the group total or individual animal basis. In other words, if I have 6 animals on study and 5 animals hold 90 fleas and the 6th holds 59 fleas for a count day. Average calculates to 84.8 but this study could be invalidated because of the 1 low count animal. This is an item which could be better explained and if true averages were used instead it could reduce the overall number of animals on study. Because of this possible invalidation due to random biological variability, studies are now run with 8 or 10 animals to ensure the minimum of 6 meet the requirement." (0011)
Based on SAP and public comments, these areas have been revised in the guideline to clarify the requirements.
231
"I would ask the committee to carefully consider my two major concerns. First acceptable control number percentages in all section. I highly recommend these numbers need to be based on real numbers from submitted studies and scientific literature. Have to consider biological variances. Numbers good for 1 species might not be good for all. If it is set to high and applied to all species, studies will fail due to biology not because we are doing bad science. And in My opinion it needs to be defined as average of the control group not individual animal. If it is an average it will lower the number of animals used in trials as we can use the actual minimum and not need extra buffer animals." (0032)
The references used for blood-feeding/retention and the numbers of arthropods per animal were derived from open literature and studies submitted to the Agency. Maintaining higher retention rates also allows for better statistics and in turn lower numbers of animals needed per group.

The guideline has been revised to clarify that for pre-qualification of each animal, the retention is based on each individual animal and during the study is based on the control group mean retention.

As noted in response to comment #196, mosquito and biting fly blood-feeding rates are based on studies in the peer reviewed literature.
232
"what seems to be proposed is a "Cadillac" version of test methodology versus a "Ford" version of test methodology. The EPA is not disputing that the products they have approved based on existing methodology are effective and valuable products for pest control in the marketplace. Yet it seems to be their opinion that many more details need to be specified, sample sizes need to increase, and statistical methodology should be required to determine if a product is effective. We submit that current methodology as practiced by researchers in the field is still valid, and represents a "Ford" version of studies without additional bells and whistles. This opinion (without the images) has also been expressed by a minority on the Panel, including some who have experience in the conduct of these studies. We urge the panel to resist the temptation to add requirements in order to feel like valid studies are being specified by the guideline" (0033)
The guideline has been drafted in consideration of study designs submitted to the Agency, while also aligning with the European Medicines Agency guideline where possible. The guideline is intended to describe acceptable methodologies to prevent study failures, thereby decreasing the number of studies performed. Other methodologies may also be acceptable.
233
"Generally, the draft guideline overstates pest retention averages and understates sample sizes. In addition, assumptions/requirements for pest retention do not account for the variability encountered in actual experiments. While dogs may be qualified in a study to retain more than 25% of ticks (for instance), there is no guarantee that the retention rate will be the same throughout the study. (It is often less.) It is unacceptable for a study to be rejected when control retention at a sampling point is less than 25%, even though animals were properly qualified for the study." (0033)
See response to comment #231.
234
"AHI encourages EPA to utilize a risk-based approach in determining study requirements. In this way, companies could consult with the Agency, in advance of initiating a development program, to reach alignment on agreeable study designs. Additionally, AHI proposes that the species of parasites to be testing and the duration of testing to demonstrate adequate efficacy against the selected parasites be dependent on the proposed product claims" (0035)
A protocol submission or discussion with the Agency is encouraged when novel approaches are proposed. Data requirements are based on the particular product and proposed claims.

The species of parasites required are based on representative species that were chosen based on biological factors as well as disease transmission and will be provided in the Product Performance Rule separately, or registrants may contact the Agency.
235
"Why are the WAAVP Guidelines (World Association for the Advancement of Veterinary Parasitology (W.A.A.V.P.) second edition: Guidelines for evaluating the efficacy of parasiticides for the treatment, prevention and control of flea and tick infestations on dogs and cats) not considered as a source document for the guidelines? They are used as a guidance document by FDA and other agencies and considering them would help make study designs more consistent across products and geographies, which could lead to fewer studies and less animals used." (0026)
The WAAVP Guidelines were referenced but due to a typo it was attached to another citation. It is still being referenced in the final version.
236
"FDA and EMA require GCP studies. Can the acceptance of GCP studies for EPA be considered? This would allow for a single study to be used in multiple geographies, which would lead to fewer animal studies." (0026)
Good Clinical Research Practice or GCP studies are often submitted to the Agency and are widely accepted in place of GLP.

The guideline states in (a)(4)i. "...However, with the exception that studies will comply with 40 CFR 160.90 to ensure animal welfare, studies that do not otherwise comply with GLP standards may be considered if, in the agency's judgment, the design and conduct of the study provide results that are scientifically reliable..."

METHODS FOR TESTING AGAINST FLEAS (SECTION (J))

Charge Question 11
The proposed studies currently are not blinded but untreated controls are used.
a. Should the study be blinded and/or utilize an inert control (i.e., formulation minus active ingredient)?
b. Why or why not?

Panel Recommendations:
237
The Panel concluded that the answer to this Charge Question depends on whether it is essential to identify all of the active ingredients within the product or formulation and to determine their contribution as it relates to the label claims. If it is necessary to identify the effects of formulations, carriers, diluents, ingredients, adjuvants, etc. on performance, then it will be necessary to conduct studies with the blank formulations. If only determining the performance of a formulation at some designated end point (e.g., >= 90% kill) is the primary objective, then conducting studies with "so-called" inert controls is probably not necessary, nor is conducting a blinded study.

The Panel found that this question seems most relevant to products such as powders, shampoos, and dips in which carriers, adjuvants, emulsifiers, and other ingredients may in themselves have some lethal effects against fleas. For example, the product Avon(R) Skin So Soft is widely reported to have repellent activity against cat fleas (Fehrer and Halliwell, 1987). Contact with diluted household detergents and cleaning products is lethal to German cockroaches (Szumlas, 2002; Baldwin and Koehler, 2007). Components of soaps such as fatty acids (Sims et al., 2014) and linear alcohol ethoxylates (Sims and Appel, 2007) are toxic to German cockroaches. Aliphatic alcohols and mineral oil have been shown to be toxic to German cockroaches and combinations are synergistic (Sims and O'Brien, 2011). The Panel found that it is also likely that some of these potential ingredients may synergize conventional insecticides included in the formulated products and that these combinations will also be toxic to fleas.  PG 48
The guideline has been revised to remove the recommendation to use a non-pesticidal (vehicle) control for shampoos. See responses to comment #s 310, 311 & 317 below for more information.
238
To isolate and identify each of the compounds that may contribute to the toxicity of these formulations will require a considerable amount of testing. However, the Panel noted that it should be possible to conduct these studies without the use of vertebrate animals. Topical applications of serial dilutions of individual compounds and combinations of compounds in various formulations applied to adult fleas will provide data regarding their contact toxicity allowing for the determination of synergism, additivity or antagonism. Similarly, adult fleas confined to surfaces treated with serial dilutions of individual compounds and combinations of compounds in various formulations should permit the determination of residual activity and allow for the determination of synergism, additivity or antagonism.  PG 48
See response to comment #237.

Public Recommendations (docket ID # in parentheses):
239
"The types of studies being conducted should not require blinding as there is a due process to qualify the test animals prior to the start of the study (including determination of their attractiveness to the pest of interest). To incorporate a blind study design will negate the qualification process. Additionally, including an inert control is not necessary as the outcome of the study should almost always simply answer, `does this test formulation reach 90% efficacy' as compared to an untreated control. At the stage of in vivo experimentations, the question of the efficacy of the active ingredient itself should already be answered. Further, including an inert control group together with an untreated control results in an increase in test animals." (0008)
Thank you for your comment in response to the blinding of the study; we did not add blinding to the guideline. In response to inert controls, see response to comment #237.

Charge Question 12
Repellency is typically determined by whether or not an insect takes a blood meal. Based on flea feeding behavior, repellency is not considered relevant for flea control and therefore is not included in the guideline.
a. Please discuss whether repelling fleas is a viable endpoint. Why or why not?
b. If repellency can be considered a viable endpoint, please provide detailed methods to test for repellency against fleas.
c. Can repellency be determined by assessing the number of live fleas on treated vs. untreated animals? If so, can repellency based on counts of live fleas on treated vs. untreated animals be differentiated from mortality, which is also currently measured by counts of live fleas on treated vs. untreated animals? If the two endpoints can be differentiated please describe the methods, including specifically how and why the counts of live fleas on treated vs untreated animals differ for repellency and mortality, and provide literature references.

Panel Recommendations:
240
The Panel found that the statement, "Repellency is typically determined by whether or not an insect takes a blood meal," is not supported by the scientific literature. Repellents cause oriented movements away from the source. In relation to feeding, a better term might be a feeding inhibitor (Deletre et al., 2016). Even the European guidelines define repellent effect as "a product with a repellent effect will cause the parasite to avoid contact with a treated animal completely and/or to leave a host" (EMEA/CVMP/EWP/005/2000-Rev. 3). Flea repellents have never gained wide acceptance as a strategy to control fleas or protect vertebrate animals. Repellents have been better defined for tick control. Marchiondo et al. (2013) write, "Two types of repellency are defined: sensu stricto for repellency characterized by an irritant effect, causing the tick to move away from the treated animal or leading it to fall off soon after contact with the treated hair coat within ∼6 - 8 h and sensu lato for all other tick repellency (or expellency) up to ∼24 h. The first, repellency sensu stricto, may be attributed to the vapor phase of a compound or irritant effect through direct contact (for example, oil of Citronella but also some synthetic pyrethroids), while the second repellency (or expellency) causes inhibition of attachment or detachment of already attached ticks, (for example, some synthetic pyrethroids or amitraz)."

Pfister and Armstrong (2016) write, "Repellency refers to the action of a product that causes ectoparasites to avoid or leave the dog, or to fail to feed on the dog, i.e. the ability of the compound to prevent the parasite from attaching or migrating onto the dog. A similar concept is the "anti-feeding" effect, which refers to the ability of the compound to stop the parasite from taking a meal from the dog." Herein lies the problem. The Panel recommends that the authors of the guidelines not confuse/conflate repellency or avoidance and anti-feeding properties. The Panel recommended that the Agency provide a concise definition of repellency similar to that found in the European guidelines.  PG 49
See response to comment #296.
241
a. The central premise regarding this Charge Question is based upon the following statement in the proposed guidelines section (j). "Based on flea feeding behavior, repellency should not be considered relevant for flea control (Pfister and Armstrong, 2016)." The Pfister and Armstrong (2016) paper is a review of two types of insecticides and this statement may be true for the systemic fluralaner and cutaneous delivered permethrin, but that does not mean it is true for all compounds.

Flea repellents have been reported in the literature. Chemicals were tested for repellency (prevention of feeding) with the flea Xenopsylla cheopis (Bar-Zeev and Gothilf, 1972). Unfed fleas were allowed to feed on treated skin surfaces of guinea pigs. The fleas were examined for blood feeding. Of the 538 compounds tested, four compounds were as effective as DEET. The repellency of essential oils was evaluated in a filter paper strip choice test with adult cat flea, Ctenocephalides felis. Thymol and trans-cinnamaldehyde were as repellent as 15% DEET (Su et al., 2013).  PG 50
This statement and citation have been removed from the guideline.
242
b and c.
The Panel found that if there is a claim that a compound repels fleas, then repellency must be considered as the viable endpoint. The number of live fleas off host compared with the number of live fleas off host in control should provide a relative estimate of the repellency. If the compound also exhibits some contact toxicity, then both live and dead fleas may be present off the host.

Cats or dogs could be treated with the test compound. The treatments would be permitted to dry and spread in the pelage of the animal (12-24 hours). The treated animals would be held within cages constructed with Plexiglass to ensure that fleas don't escape and are collected. Aliquots of fleas (25-50) would be placed on the fur or placed in the cage and allowed access to the treated and control animals. At 1 hour, the contents of the cage would be vacuumed, and the live and dead fleas trapped.  PG 50
See response to comment #223.
243
Repellency is not a widespread label claim of many topical products applied to cats and dogs. The Panel noted that EPA has some latitude to accept non-vertebrate studies to demonstrate repellency or request protocols for determining repellency studies on vertebrates.  PG 50
EPA has included a statement in the guideline indicating that protocols may be submitted for new approach methodologies (NAMs), and that if a protocol is proposing test methods which do not use vertebrate animals, they should also propose to run a traditional (on animal) test concurrently to provide evidence that the new methods are useful surrogates for testing that utilizes vertebrate animals.

Public Recommendations (docket ID # in parentheses):
244
"Recent literature indicates that flea repellency can be assessed by counting fleas falling off the animal in kinetic after infestation (Varloud et al., 2015; Halos et al., 2016). https://parasitesandvectors.biomedcentral.com/articles/10.1186/s13071-016-1345-4  and https://link.springer.com/article/10.1007%2Fs00436-015-4470-7  Flea repellency can be assessed by counting fleas falling off the animal in kinetic after infestation (Varloud et al., 2015; Halos et al., 2016) https://parasitesandvectors.biomedcentral.com/articles/10.1186/s13071-016-1345-4  https://link.springer.com/article/10.1007%2Fs00436-015-4470-7" (0008)

The referenced literature was considered, and a statement was added to the guideline to encourage the submission of a protocol for flea repellency tests.
245
"Yes, repelling fleas is a viable endpoint. Companies need to be given a chance to develop products and formulations that give the end user a variety of options in order to control flea populations. If the flea is never given the opportunity to take a blood meal on the host animal, there is less opportunity for transport of fleas from outdoor environments indoors. This would allow a reduction in cross contamination among multiple pet households and flea infestations on humans. b. This would have to be a defined endpoint in the protocol before trial initiation. Methods to assess repellency can be very similar to mortality assessments. Simply the presence or lack of presence of the flea is a very simple start. This method can be very similar to what is already outlined in section (5) Data collection and recording for ticks. c. Yes, it can be. The endpoint must be defined in the protocol before the initiation of the study. As the methods for evaluation of repellency versus mortality endpoints are similar, they must be tested by two separate trials (i.e., one trial to determine mortality effects and one trial to determine repellency effects). d. Flea repellency is relevant and given the oral market which does not provide repellency this is very important measure for consumers" (0008)
The guideline will be revised as indicated above in the response to comment #242.

METHODS FOR TESTING AGAINST TICKS (SECTION (K))

Charge Question 13
The guideline states that tick infestations should consist of a 50:50 ratio of male to female ticks for all species. The European guideline contains a clause that for Ixodes spp. infestations the sex ratio should consist of approximately 10% males: 90% females since males do not readily attach (Marchiondo et al., 2013). However, the scientific literature for Ixodes spp. appears to be split between using a 50:50 ratio or a 90:10 ratio. Please discuss the merits of each ratio:
a. Is one of the ratios better for testing Ixodes spp., or are they both appropriate? If one ratio is better than the other, please explain why.
b. Is one of the ratios better for testing Rhipicephalus spp., or are they both appropriate? If one ratio is better than the other, please explain why.
c. Is one of the ratios better for testing Amblyomma spp., or are they both appropriate? If one ratio is better than the other, please explain why.

Panel Recommendations:
246
a. The Panel noted that it appears that for testing Ixodes spp., both ratios 50:50 and 90:10 (females:males) would be appropriate. In general, placement of a somewhat balanced number of both females and male ticks on hosts is important because tick mating while on the host promotes feeding success by female ticks to engorgement (Akov, 1982; Wang et al., 1998; Donohue et al., 2009). The role of mating on female feeding during the very early feeding stages is unclear. Insufficient numbers of males that may survive during trials could possibly confound testing results by causing females to poorly feed due to lack of mating opportunities. So, a 50:50 ratio is probably best for all species tested to ensure normal mating behaviors.

The European guideline (European Medicines Agency, 2016) contains a clause that for Ixodes spp., there appears to be split between using a 50:50 ratio or a 90:10 ratio annotated below in the following references: Vatta et al. (2018), Becskei et al. (2017), EMEA (2016), Dumont et al. (2015), Fourie et al. (2015), Kuzner et al. (2013).

A reason some researchers prefer to use more female than male Ixodes spp. in testing is because male Ixodes spp. do not necessarily require a blood meal (attachment) to mate, which can skew the attachment rate statistics on untreated hosts, particularly when testing systemics.

Studies have shown that attachment rates of I. ricinus can be increased by using a higher number of ticks ( 60) and a lower number of males compared to females (∼25:75) as male I. ricinus attach only briefly or not at all (Kiszewski et al., 2001; Kuzner et al., 2013). However, Marchiondo et al. (2013) noted the presence of male I. ricinus is sufficient to stimulate female tick attachments and to achieve greater attachment rates.

"The metastriate ticks, including Dermacentor, Amblyomma, and Rhipicephalus, invariably attain sexual maturity and mate solely on their hosts. The more primitive prostriate Ixodes spp. ticks; however, may copulate both in the absence of hosts and while the female engorges. These expanded opportunities for insemination complicate the mating systems of the Ixodes ricinus complex of species. In these ticks, autogenous spermatogenesis must precede host contact..."
(Kiszewski et al., 2001).

In the U. S., Ixodes pacificus and I. scapularis mate on and off the host and adult males rarely attach to the host, taking little blood if they do. In general, the blood loss to a dog host is less than that taken by females and males of the other genera (Koch and Sauer, 1984). Placing 50% female and 50% male ticks (N = 50 ticks) challenges the host with 25 potentially attaching ticks, whereas 90% female to 10% male could lead up to 45 females attaching to the host. Especially if repeated infestations occur on the animal, or if more than one species of tick is used in a trial, the 50:50 ratio would cause less stress to control animals in particular. A 50:50 ratio would ensure adequate mating opportunities and normal feeding behaviors for females while being consistent with most previous studies for comparison (McCall et al., 2011).   PG 51
To allow flexibility, the tick section of the guideline has been revised to read:

Infestations should consist of a 50:50 ratio of female to male ticks (Dumont et al. 2015, Fourie et al. 2015,  Stanneck et al. 2012, Young et al. 2003, ), however other ratios may be justified (e.g., 90:10 female to male ticks ratio for Ixodes spp.)
247
b. The Panel noted that a 50:50 ratio is best for testing Rhipicephalus spp. As indicated above, with the metastriate ticks, such as Rhipicephalus spp., the males must attach and feed before they mate so there would be no statistical advantage to skewing the ratio of males to females away from 50:50.

Thus, for Dermacentor, Amblyomma, and Rhipicephalus, mating occurs only on the host, and both females and males feed. Males consume substantially less blood than females, but unlike Ixodes spp., they do take some blood. Using either ratio of ticks could lead to up to 50 ticks attaching. Placement of 50:50 would likely result in fewer adverse effects to the host during trials than 90:10, although not much difference would be seen if ticks are removed at 48 hours from the host.  PG 51-52
No changes to the guideline are necessary.
248
c. The Panel noted that a 50:50 ratio is best for testing Amblyomma spp. As indicated above, with the metastriate ticks, such as Amblyomma spp., the males must attach and feed before they mate.  PG 52
No changes to the guideline are necessary.
249
Although not included in the Charge Question, Dermacentor results should also be considered, as this is a common tick found on U.S. pets, in particular, D. variabilis on dogs.  PG 52

REPEATED FROM b:  Thus, for Dermacentor, Amblyomma, and Rhipicephalus, mating occurs only on the host, and both females and males feed. Males consume substantially less blood than females, but unlike Ixodes spp., they do take some blood. Using either ratio of ticks could lead to up to 50 ticks attaching. Placement of 50:50 would likely result in fewer adverse effects to the host during trials than 90:10, although not much difference would be seen if ticks are removed at 48 hours from the host.  PG 52
No changes to the guideline are necessary.

Public Recommendations (docket ID # in parentheses):
250
"For all three species, the ratio should be kept at 50:50. The point of efficacy trials is the eradication of the pest population in that given environment. Aside from a few parthenogenic species in the US, both males and females are needed for sexual reproduction. A decrease in both sexes could allow for an overall reduction in the population of the pest. A special caveat could be created in order to allow a full 100% ratio for species in which only females are present" (0008)
See response to comment #246.
251
"It is true that male Ixodes spp. ticks do not need to feed to reproduce, and sometimes Ixodes spp. ticks mate off host. For mortality and repellency studies, the more females used for infestations the more likely you will have adequate infestations on the control animals. That being said, you are also increasing the attachment burden on the treated animals which may result in more attached ticks in the treated group at counts." (0026)
See response to comment #246.
252
Rhipicephalus: "Males will move from host to host to find a female, feeding as they go. They also need to take a blood meal before they can mate. Therefore, a ratio of 50:50 is most appropriate." (0026)
No changes to the guideline are necessary.
253
Amblyomma: "Literature sources state, the presence of males increases the feeding behavior of the female ticks, an equal sex ratio of 50:50 is appropriate for this tick species. We recognize that the scientific literature is not fully aligned on this matter, and propose that a study should be deemed acceptable if the number of ticks in the untreated group of animals is sufficient to demonstrate that the host is able to harbor ticks, regardless of the sex ratio of the parasites used for the challenge. We believe that if a dog or cat can retain approximately 25% of the ticks used for challenge, this challenge should be considered valid for the purpose of demonstrating efficacy in a treated group" (0026)
See response to comment #246.
254
Ixodes: "This ratio is not appropriate for Ixodes scapularis ticks where ratio should be 40:10, female to male. In addition, as both sexes of all common species except Ixodes feed on the host, there is no advantage to handling the ticks to determine their sex. Infesting with 50 ticks, regardless of sex ratio, will produce a robust challenge." (0026)
See response to comment #246.

Charge Question 14
The guideline states that ticks that feed to engorgement can transmit pathogens that may cause disease; therefore, mortality after blood-feeding to the point of engorgement should not be considered a viable endpoint.
a. Is the determination of engorgement a feasible data collection point? Why or why not?
b. If so, please provide detailed methods with references for determining engorgement.
c. Can blood feeding be determined prior to engorgement and would this be a viable endpoint? If so, please describe methods for determining blood feeding prior to engorgement and provide any relevant references.

Panel Recommendations:
255
a. The Panel concluded that, no, the determination of engorgement is not a feasible data collection point. The Panel recommended that the term "engorgement" needs to be revised in the guidelines as the EMEA and WAAVP guidelines are not being used correctly. Adult tick counts are generally performed 48 hours after each infestation time point, but initiation of blood feeding during this period will not be readily apparent as very little blood is consumed within this time frame. It is impossible to accurately visualize any increase in size (plumpness) over this short period. This is even difficult if individual ticks are weighed (milligrams). All instances of "engorgement" in the guidelines should instead be changed to "partially fed." If feeding to engorgement and drop off is to be mentioned, then the term "repletion" should be used. Throughout the guidelines, for clarity, it should be specified that all ticks found on the host (attached and unattached) are removed at 48 hours post-infestation, not just counted. This is in addition to off-host ticks that are also counted and collected.

The efficacy of systemically-acting acaricides depends on tick attachment and ingestion of the compound. Transmission of bacterial/protozoan pathogens by hard (ixodid) ticks typically starts to take place between 36-72 hours after attachment, before substantial blood feeding occurs in adults. If the kill occurs prior to the 48 hours tick removal time point, this delay after attachment can allow a systemic acaricide to take effect prior to transmission of many pathogens (Uspensky and Ioffe-Uspensky, 2006; Wengengmayer et al., 2014). However, viruses such as Powassan fever virus could be transmitted within minutes to hours after attachment (Richards at al., 2017).  PG 52-53
The use of engorgement as an endpoint has been removed from the guideline and replaced with counting live ticks on the treated animal vs. the control animal for mortality testing and counting all ticks on the treated animal (live or dead) vs. the control for repellency testing, which is how these studies have been historically conducted and is consistent with the European guideline.
256
b and c.
The Panel found that engorgement status does not appear to be relevant to these revised guidelines. Researchers are assessing survival of partially fed ticks after 48 hours of exposure (contact) with a topically applied acaricide. As long as female ticks fail to survive to repletion and oviposition, the lifecycle is interrupted. As noted by the Panel in response to question 14a., all instances of "engorgement" in the guidelines should instead be changed to "partially fed."  PG 53
The guideline has been revised to incorporate this comment.
257
b and c.