APPENDIX IV: EVALUATION OF STATISTICAL MODELS FOR ANALYZING THE DATA OF PET PRODUCT STUDIES

Background

Since the data are simulated as the number of blood-fed arthropods given the total number of infesting arthropods per animal, a generalized mixed-effects model (GMM) with logit-link function for binomial distribution appears to be the appropriate statistical model to analyze the data.  Although this statistical model (i.e., binomial GMM with a logit-link function) is conceptually appropriate, it does not directly generate estimates for the calculation of efficacy and its 95% confidence interval. However, GMMs with the log-link function for either the binomial distribution or Poisson distribution does generate the statistical estimates that can be easily used to estimate efficacy and its 95% CI.  Therefore, the original power vs. sample size simulations were conducted using GMM for Poisson distributions.  However, there are theoretical issues when using GMMs with log-link function for the binomial distribution or for the Poisson distribution to analyze the data of Pet Product study.  A GMM with a log-link function (a non-canonical link function) for the binomial distribution will sometimes produce the 95% CI of estimated proportion of blood-fed arthropods (or efficacy) which is bound between 0 and 1 outside the [0,1] range (this issue is not critical since EPA would only use the estimated rate ratio and its lower 95% CI to calculate the efficacy and its lower 95%CI).  A GMM with log-link function for Poisson distribution theoretically assumes that the counts have no limit for upper bound value (which never happens in reality) while the count of blood-fed arthropods is bounded from 0 to the total number of arthropods infested on the animal in most of the times (sometimes, the total count of arthropods at the end of testing is greater than the original number arthropods infested on the animal).

After reviewing the results of simulations based on the GMM for Poisson distribution data and GMM for binomial distribution data with log-link function, Dr. Jonathon Cohen, a statistician from ICF, has provided mathematical derivation to estimate the efficacy and its 95% CI using estimates from GMM for binomial distribution with logit link function.  Dr. Cohen also provided SAS code and some quick simulations to demonstrate using the binomial GMM with logit link function.

In this appendix IV, we present a complete set of simulations for comparing 5 models: GMM for Poisson, GMM for binomial with log link function, GMM for binomial with logit link function, GMM for Poisson with "NOBOUND" option, and GMM for binomial with logit link function and "NOBOUND" option.  The last two GMM models with "NOBOUND" options, which allow the estimates of covariance matrices to be negatives, were included to address potential concerns with the zero value estimates in the covariance matrices of some datasets.  Each model was used to analyze each full dataset including the data of 2 groups on 4 testing days (called a full model).  When the full model failed to converge for a dataset, a separate model was used to analyze the data by each day. The separate models were consistent with the full model except they did not account for the repeated effects.  The separate models converged in most cases.
The simulations included 1000 random datasets for each of sample sizes (8, 9, and 10 animals/group) x each of 5 study designs (blood-fed proportions = 0.45, 0.60, 0.75, and 0.85).  The criteria to evaluate the models are as following:
 number of converged datasets  -  higher is better;
 number of datasets with positive definite G matrices- higher is better;
 the power to determine the efficacy of a product to be acceptable based on the EPA criteria to determine the acceptance of a product: an estimated efficacy >= 0.90 and a precision <= 0.05.  From a practical perspective, a model that would result in a low probability of accepting a product when the true efficacy of the product is less than 0.90 and would result in a high probability of accepting a product when the its true efficacy >= 0.90 is more desirable;
 and most the important criterion, the logic of the model that is appropriate to analyze the data.

Summarized Results of all Simulations
Tables A4.2-1 and A4.3-1 present the results of number converged datasets and number of datasets with positive definite G matrices for the simulations when the random animal effects and random day effects were generated from Weibull distributions and normal distributions, respectively.
 GMM model for Poisson distribution has highest number of converged datasets.
 All models have low proportions of datasets with positive definite G matrix.  However, the GMM model for binomial distribution with logit link function has highest number of datasets with positive definite G matrices.

Tables A4.2-2 and A4.3-2 present the power of the 5 evaluated models.
 Despite the indicated theoretical issues, all models have acceptable probabilities (< 0.05) of accepting a "fail" product, i.e. accepting a product that has a true efficacy of 0.85 < the required efficacy of 0.90 (Type I Error)
 When the random (animal and day) effects were generated from Weibull distributions
 The GMM for binomial distribution with logit link function has higher power than other models when the blood-fed arthropods in the control group = 0.45.
 The GMM for Poisson distribution has higher power than other models when the blood-fed arthropods in the control group = 0.60, 0.75, and 0.80.
 The GMM for Poisson with "NOBOUND" option, GMM model for binomial distribution with log link function, and GMM for binomial distribution with logit link function and "NOBOUND" option have lower power than the GMM for Poisson distribution and GMM for binomial distribution with logit link function.

 When the random (animal and day) effects were generated from normal distributions
 The GMM for binomial distribution with logit link function has higher power than other models.
 The GMM for Poisson with "NOBOUND" option, GMM model for binomial distribution with log link function, and GMM for binomial distribution with logit link function and "NOBOUND" option have lower power than the GMM for Poisson distribution and GMM for binomial distribution with logit link function.

Due to the mixed results in power between the GMM for Poisson distribution and GMM for binomial distribution with logit link function, the simulations for the power of GMM for binomial distributions were performed for different study designs (2000 datasets for each scenario).  The results of these power simulation are presented in Tables A4.4-1, A4.4-2, A4.5-1, A4.5-2, A4.6-1, A4.6-2, A4.7-1, A4.7-2, A4.8-1, and A4.8-2.  The results of these power simulations for GMM for binomial distributions with logit link function are summarized and presented with the power results of simulation for GMM for Poisson distributions (Appendix I) in Table A4.1.  The required sample sizes for using GMM for Poisson distribution are robust to the assumed distributions (normal or Weibull) of random animal or day effects.  However, the required sample sizes for using GMM for binomial with logit link function vary depending on the assumed distributions of random animal and day effects. Perhaps the GMM in the current SAS PROC GLIMMIX assumes the random effects are normally distributed, the power/sample size of the GMM with logit link function for binomial distributions are different depending on whether the random effects are generated from normal distributions or Weibull distributions.  When the random animal effects and random day effect were generated from normal distributions, the required sample sizes for using GMM with logit link function for binomial distributions are similar to required sample sizes for using GMM for Poisson distributions.  These are the sample size recommended for the study designs.

Table A4.1

                                 Study design
     Required lowest observed blood-fed proportion on each control animal
                  True blood fed proportion in control group
                       No. arthropods per animal per day
                                    True TE
                         Required Sample Size (power)

                         GMM for Poisson with log link
                       GMM for Binomial with logit link

                                    Normal
                                    Weibull
                                    Normal
                                    Weibull
                                       1
                                     0.25
                                     0.45
                                     50+-5
                                     0.925
                                      14
                                    (0.811)
                                      13
                                    (0.828)
                                      11
                                    (0.806)
                                      11
                                    (0.861)
                                       2
                                     0.40
                                     0.60
                                     50+-5
                                     0.925
                                      11
                                    (0.834)
                                      10
                                    (0.860)
                                       9
                                    (0.814)
                                      15
                                    (0.799)
                                       3
                                     0.40
                                     0.60
                                    100+-10
                                     0.925
                                       7
                                    (0.798)
                                       7
                                    (0.875)
                                       7
                                    (0.858)
                                      14
                                    (0.803)
                                       4
                                     0.60
                                     0.80
                                     50+-5
                                     0.925
                                       8
                                    (0.839)
                                       6
                                    (0.801)
                                       8
                                    (0.822)
                                      14
                                    (0.866)
                                       5
                                     0.60
                                     0.75
                                    100+-10
                                     0.925
                                       6
                                    (0.834)
                                       6
                                    (0.930)
                                       6
                                    (0.825)
                                      13
                                    (0.854)

Mathematical derivation of the estimated efficacy using a binomial model with logit link function (provided by Dr. Cohen)

Using the binomial model with the canonical logit link, the probability that an insect is blood fed is given by the equations:
Logit Pbloodfed=logPbloodfedPnot bloodfed=μ+groupg+dayd+group*daygd
P is Probability. In this model, μ is the intercept, groupg is the effect for group g (either the treatment or the control group), dayd is the effect for day d (1, 2, 3, or 4), and group*daygd is the group*day interaction term. The logarithms are natural logarithms. The control group is group 0 and the treatment group is group 1.
For day d, define the logits for the control group and treatment group by
α=μ+group0+dayd+group*day0d,
β=μ+group1+dayd+group*day1d.
For the SAS coding we also define the total
τ=α+β
so that the covariance between the estimates of α and β can easily be calculated.
It follows that on day d,
Pbloodfed for a control animal=11+e-α ,
Pbloodfed for a treated animal=11+e-β ,
Efficacy=1-Pbloodfed for a treated animalPbloodfed for a control animal=e-β-e-α1+e-β=f(α,β)
Let α,β, and τ be the estimated values of α,β, and τ, and let their standard errors be seα,seβ, and se τ . The estimated efficacy is f(α,β). To approximate the standard error of the efficacy we can apply the delta method, a first order Taylor series expansion using the partial derivatives of f (see Klein, 1953 or Oehlert, 1992, for example):
Estimated efficacy= fα,β≈fα,β+(α-α)dfdα(α,β) +(β-β)dfdβ(α,β).
This gives:
Estimated efficacy ≈fα,β+(α-α)e-α1+e-β +(β-β)-e-β1+e-α1+e-β2
Therefore the variance of the estimated efficacy is approximately given by:
Variance Estimated efficacy≈Varαe-α1+e-β2 +Varβ-e-β1+e-α1+e-β22+2 Cov(α,β)e-α1+e-β-e-β1+e-α1+e-β2
Taking the square root and replacing the unknown true values of the parameters by their estimates gives
Standard Error Estimated efficacy≈Varαe-α1+e-β2 +Varβ-e-β1+e-α1+e-β22+2 Cov(α,β)e-α1+e-β-e-β1+e-α1+e-β2
In the last two equations, Var denotes the variance and Cov denotes the covariance of the estimated parameters. These can be estimated as follows:
Varα=se(α)2
Varβ=se(β)2
Varτ=se(τ)2
Varα+β=Varτ=Varα+Varβ+2 Cov(α,β)
2 Covα,β=Varτ-Varα-Varβ
If the model with day effects and a group*day interaction fails to converge, then for each day, separate models are fitted of the form:
Logit Pbloodfed=logPbloodfedPnot bloodfed=μ+groupg
For each day the logits for the control group and treatment group are defined by
α=μ+group0,
β=μ+group1

Detail Results of all simulations for model comparisons

Table A4.2-1

Number of converged datasets and number of datasets with positive definite G matrices from 1000 datasets
           Random effects were generated from Weibull distributions

                           Number converged datasets
                       Number positive definite G matrix
                                                                   Study design
                                                                              N
                                                                        Poisson
                                                               Poisson nobound
                                                                  Binomial log
                                                                Binomial logit
                                                        Binomial nobound logit
                                                                       Poisson
                                                                  Binomial log
                                                                Binomial logit
                                        PC=0.45, NPDay=50, SubVar=1, DayVar=0.5
                                                                              8
                                                                            961
                                                                            755
                                                                            790
                                                                            885
                                                                            869
                                                                            476
                                                                            726
                                                                            817

                                                                              9
                                                                            965
                                                                            752
                                                                            763
                                                                            910
                                                                            895
                                                                            456
                                                                            688
                                                                            850

                                                                             10
                                                                            961
                                                                            751
                                                                            725
                                                                            918
                                                                            910
                                                                            466
                                                                            682
                                                                            880
                                        PC=0.60, NPDay=50, SubVar=1, DayVar=0.5
                                                                              8
                                                                            965
                                                                            598
                                                                            525
                                                                            556
                                                                            501
                                                                            281
                                                                            467
                                                                            512

                                                                              9
                                                                            977
                                                                            561
                                                                            433
                                                                            557
                                                                            519
                                                                            255
                                                                            386
                                                                            519

                                                                             10
                                                                            967
                                                                            557
                                                                            409
                                                                            523
                                                                            484
                                                                            258
                                                                            378
                                                                            503
                                       PC=0.60, NPDay=100, SubVar=1, DayVar=0.5
                                                                              8
                                                                            935
                                                                            832
                                                                            710
                                                                            504
                                                                            456
                                                                            603
                                                                            691
                                                                            494

                                                                              9
                                                                            939
                                                                            828
                                                                            567
                                                                            492
                                                                            468
                                                                            600
                                                                            557
                                                                            481

                                                                             10
                                                                            934
                                                                            852
                                                                            559
                                                                            437
                                                                            410
                                                                            606
                                                                            556
                                                                            433
                                        PC=0.80, NPDay=50, SubVar=1, DayVar=0.5
                                                                              8
                                                                            982
                                                                            265
                                                                             88
                                                                            273
                                                                            147
                                                                             44
                                                                             70
                                                                            208

                                                                              9
                                                                            974
                                                                            253
                                                                             49
                                                                            240
                                                                            143
                                                                             28
                                                                             33
                                                                            183

                                                                             10
                                                                            970
                                                                            238
                                                                             40
                                                                            191
                                                                            121
                                                                             15
                                                                             28
                                                                            161
                                       PC=0.75, NPDay=100, SubVar=1, DayVar=0.5
                                                                              8
                                                                            965
                                                                            702
                                                                            354
                                                                            374
                                                                            285
                                                                            290
                                                                            338
                                                                            361

                                                                              9
                                                                            965
                                                                            657
                                                                            181
                                                                            370
                                                                            296
                                                                            276
                                                                            168
                                                                            360

                                                                             10
                                                                            969
                                                                            692
                                                                            154
                                                                            308
                                                                            251
                                                                            269
                                                                            143
                                                                            302

Table A4.2-2

                                Power of models
           Random effects were generated from Weibull distributions
                                 Study design
                                       N
                                      Day
                                    true_TE
                                     ReqTE
                                     Power

                     Poisson log-link Precision <= 0.05
                 Poisson nobound log-link Precision <= 0.05
                    Binomial log-link Precision <= 0.05
                   Binomial logit-link Precision <= 0.05
               Binomial nobound logit-link Precision <= 0.05
                                        PC=0.45, NPDay=50, SubVar=1, DayVar=0.5
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.626
                                                                          0.487
                                                                          0.477
                                                                          0.880
                                                                          0.860

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.290
                                                                          0.239
                                                                          0.220
                                                                          0.638
                                                                          0.625

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.127
                                                                          0.100
                                                                          0.091
                                                                          0.328
                                                                          0.323

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.011
                                                                          0.008
                                                                          0.007
                                                                          0.030
                                                                          0.030

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.710
                                                                          0.564
                                                                          0.547
                                                                          0.919
                                                                          0.911

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.412
                                                                          0.316
                                                                          0.296
                                                                          0.750
                                                                          0.738

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.147
                                                                          0.115
                                                                          0.115
                                                                          0.385
                                                                          0.379

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.010
                                                                          0.009
                                                                          0.009
                                                                          0.030
                                                                          0.030

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.844
                                                                          0.681
                                                                          0.632
                                                                          0.941
                                                                          0.937

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.524
                                                                          0.409
                                                                          0.390
                                                                          0.827
                                                                          0.821

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.202
                                                                          0.165
                                                                          0.150
                                                                          0.479
                                                                          0.473

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.015
                                                                          0.011
                                                                          0.009
                                                                          0.046
                                                                          0.045
                                        PC=0.60, NPDay=50, SubVar=1, DayVar=0.5
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.904
                                                                          0.649
                                                                          0.474
                                                                          0.712
                                                                          0.669

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.638
                                                                          0.432
                                                                          0.323
                                                                          0.490
                                                                          0.462

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.315
                                                                          0.206
                                                                          0.156
                                                                          0.240
                                                                          0.228

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.023
                                                                          0.017
                                                                          0.011
                                                                          0.019
                                                                          0.017

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.946
                                                                          0.694
                                                                          0.435
                                                                          0.755
                                                                          0.733

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.764
                                                                          0.512
                                                                          0.315
                                                                          0.564
                                                                          0.542

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.397
                                                                          0.242
                                                                          0.149
                                                                          0.283
                                                                          0.274

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.021
                                                                          0.016
                                                                          0.007
                                                                          0.017
                                                                          0.017

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.972
                                                                          0.766
                                                                          0.420
                                                                          0.762
                                                                          0.738

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.866
                                                                          0.595
                                                                          0.374
                                                                          0.580
                                                                          0.549

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.458
                                                                          0.293
                                                                          0.182
                                                                          0.312
                                                                          0.292

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.021
                                                                          0.016
                                                                          0.003
                                                                          0.017
                                                                          0.015
                                       PC=0.60, NPDay=100, SubVar=1, DayVar=0.5
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.962
                                                                          0.894
                                                                          0.728
                                                                          0.717
                                                                          0.677

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.900
                                                                          0.816
                                                                          0.673
                                                                          0.539
                                                                          0.507

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.535
                                                                          0.494
                                                                          0.400
                                                                          0.301
                                                                          0.289

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.012
                                                                          0.013
                                                                          0.007
                                                                          0.011
                                                                          0.011

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.976
                                                                          0.923
                                                                          0.586
                                                                          0.744
                                                                          0.729

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.929
                                                                          0.850
                                                                          0.559
                                                                          0.593
                                                                          0.575

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.530
                                                                          0.485
                                                                          0.305
                                                                          0.334
                                                                          0.325

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.007
                                                                          0.006
                                                                          0.003
                                                                          0.013
                                                                          0.013

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.975
                                                                          0.948
                                                                          0.596
                                                                          0.802
                                                                          0.788

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.943
                                                                          0.895
                                                                          0.557
                                                                          0.607
                                                                          0.581

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.541
                                                                          0.499
                                                                          0.304
                                                                          0.331
                                                                          0.319

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.008
                                                                          0.006
                                                                          0.002
                                                                          0.010
                                                                          0.010
                                        PC=0.80, NPDay=50, SubVar=1, DayVar=0.5
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.997
                                                                          0.914
                                                                          0.161
                                                                          0.709
                                                                          0.662

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.944
                                                                          0.777
                                                                          0.115
                                                                          0.378
                                                                          0.331

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.533
                                                                          0.420
                                                                          0.045
                                                                          0.138
                                                                          0.122

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.011
                                                                          0.009
                                                                          0.000
                                                                          0.005
                                                                          0.004

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.998
                                                                          0.956
                                                                          0.139
                                                                          0.751
                                                                          0.725

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.946
                                                                          0.857
                                                                          0.094
                                                                          0.505
                                                                          0.461

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.526
                                                                          0.464
                                                                          0.037
                                                                          0.185
                                                                          0.170

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.012
                                                                          0.011
                                                                          0.000
                                                                          0.008
                                                                          0.007

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.999
                                                                          0.983
                                                                          0.188
                                                                          0.849
                                                                          0.842

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.964
                                                                          0.913
                                                                          0.104
                                                                          0.547
                                                                          0.519

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.537
                                                                          0.499
                                                                          0.036
                                                                          0.225
                                                                          0.201

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.010
                                                                          0.010
                                                                          0.001
                                                                          0.004
                                                                          0.003
                                       PC=0.75, NPDay=100, SubVar=1, DayVar=0.5
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.996
                                                                          0.968
                                                                          0.423
                                                                          0.749
                                                                          0.699

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.971
                                                                          0.909
                                                                          0.379
                                                                          0.488
                                                                          0.432

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.565
                                                                          0.524
                                                                          0.174
                                                                          0.227
                                                                          0.209

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.004
                                                                          0.003
                                                                          0.001
                                                                          0.000
                                                                          0.000

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.999
                                                                          0.992
                                                                          0.267
                                                                          0.827
                                                                          0.800

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.981
                                                                          0.955
                                                                          0.238
                                                                          0.601
                                                                          0.561

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.538
                                                                          0.512
                                                                          0.095
                                                                          0.312
                                                                          0.285

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.001
                                                                          0.001
                                                                          0.000
                                                                          0.001
                                                                          0.001

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          1.000
                                                                          0.997
                                                                          0.297
                                                                          0.887
                                                                          0.881

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.990
                                                                          0.980
                                                                          0.214
                                                                          0.606
                                                                          0.576

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.544
                                                                          0.535
                                                                          0.091
                                                                          0.314
                                                                          0.287

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.000
                                                                          0.000
                                                                          0.000
                                                                          0.004
                                                                          0.003

Table A4.3-1

Number of converged datasets and number of datasets with positive definite G matrices from 1000 datasets
            Random effects were generated from Normal distributions

                                                      Number converged datasets
                                              Number positive definite G matrix
                                                                   Study design
                                                                              N
                                                                        Poisson
                                                               Poisson nobound
                                                                  Binomial log
                                                                Binomial logit
                                                        Binomial nobound logit
                                                                       Poisson
                                                                  Binomial log
                                                                Binomial logit
                                   PC=0.45, NPDay=50, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                            945
                                                                            498
                                                                            743
                                                                            935
                                                                            892
                                                                            164
                                                                            442
                                                                            673

                                                                              9
                                                                            948
                                                                            445
                                                                            689
                                                                            934
                                                                            880
                                                                            126
                                                                            364
                                                                            675

                                                                             10
                                                                            938
                                                                            455
                                                                            619
                                                                            946
                                                                            906
                                                                            121
                                                                            334
                                                                            728
                                   PC=0.60, NPDay=50, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                            955
                                                                            280
                                                                            547
                                                                            901
                                                                            845
                                                                             42
                                                                            195
                                                                            626

                                                                              9
                                                                            944
                                                                            280
                                                                            527
                                                                            905
                                                                            840
                                                                             23
                                                                            153
                                                                            652

                                                                             10
                                                                            937
                                                                            268
                                                                            510
                                                                            914
                                                                            857
                                                                             19
                                                                            149
                                                                            676
                                  PC=0.60, NPDay=100, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                            965
                                                                            438
                                                                            595
                                                                            903
                                                                            876
                                                                             66
                                                                            312
                                                                            717

                                                                              9
                                                                            967
                                                                            449
                                                                            553
                                                                            906
                                                                            873
                                                                             59
                                                                            255
                                                                            752

                                                                             10
                                                                            953
                                                                            432
                                                                            495
                                                                            897
                                                                            864
                                                                             37
                                                                            205
                                                                            758
                                   PC=0.80, NPDay=50, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                            934
                                                                             41
                                                                            318
                                                                            665
                                                                            434
                                                                              .
                                                                             14
                                                                            333

                                                                              9
                                                                            917
                                                                             23
                                                                            289
                                                                            665
                                                                            435
                                                                              .
                                                                              6
                                                                            367

                                                                             10
                                                                            919
                                                                             30
                                                                            282
                                                                            654
                                                                            429
                                                                              1
                                                                              4
                                                                            336
                                  PC=0.75, NPDay=100, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                            961
                                                                            142
                                                                            452
                                                                            785
                                                                            703
                                                                              2
                                                                            104
                                                                            552

                                                                              9
                                                                            977
                                                                            112
                                                                            409
                                                                            775
                                                                            694
                                                                              4
                                                                             37
                                                                            568

                                                                             10
                                                                            952
                                                                            114
                                                                            389
                                                                            780
                                                                            686
                                                                              1
                                                                             45
                                                                            552

Table A4.3-2

                                Power of models
            Random effects were generated from Normal distributions
                                 Study design
                                       N
                                      Day
                                    true_TE
                                     ReqTE
                                     Power

                     Poisson log-link Precision <= 0.05
                 Poisson nobound log-link Precision <= 0.05
                    Binomial log-link Precision <= 0.05
                   Binomial logit-link Precision <= 0.05
               Binomial nobound logit-link Precision <= 0.05
                                   PC=0.45, NPDay=50, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.539
                                                                          0.370
                                                                          0.340
                                                                          0.903
                                                                          0.881

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.223
                                                                          0.141
                                                                          0.130
                                                                          0.639
                                                                          0.622

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.092
                                                                          0.058
                                                                          0.048
                                                                          0.336
                                                                          0.323

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.003
                                                                          0.002
                                                                          0.003
                                                                          0.034
                                                                          0.032

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.702
                                                                          0.475
                                                                          0.452
                                                                          0.944
                                                                          0.920

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.352
                                                                          0.227
                                                                          0.182
                                                                          0.699
                                                                          0.666

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.117
                                                                          0.068
                                                                          0.052
                                                                          0.373
                                                                          0.354

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.011
                                                                          0.006
                                                                          0.007
                                                                          0.036
                                                                          0.033

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.789
                                                                          0.570
                                                                          0.485
                                                                          0.962
                                                                          0.949

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.456
                                                                          0.311
                                                                          0.236
                                                                          0.788
                                                                          0.768

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.177
                                                                          0.110
                                                                          0.083
                                                                          0.416
                                                                          0.404

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.013
                                                                          0.006
                                                                          0.004
                                                                          0.036
                                                                          0.036
                                   PC=0.60, NPDay=50, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.899
                                                                          0.656
                                                                          0.513
                                                                          0.955
                                                                          0.945

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.610
                                                                          0.408
                                                                          0.264
                                                                          0.789
                                                                          0.759

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.286
                                                                          0.168
                                                                          0.114
                                                                          0.424
                                                                          0.401

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.017
                                                                          0.008
                                                                          0.005
                                                                          0.033
                                                                          0.028

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.950
                                                                          0.798
                                                                          0.608
                                                                          0.980
                                                                          0.972

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.729
                                                                          0.507
                                                                          0.360
                                                                          0.815
                                                                          0.788

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.338
                                                                          0.219
                                                                          0.138
                                                                          0.438
                                                                          0.419

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.013
                                                                          0.007
                                                                          0.003
                                                                          0.024
                                                                          0.023

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.980
                                                                          0.838
                                                                          0.655
                                                                          0.989
                                                                          0.984

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.794
                                                                          0.593
                                                                          0.435
                                                                          0.860
                                                                          0.841

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.353
                                                                          0.256
                                                                          0.179
                                                                          0.434
                                                                          0.425

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.014
                                                                          0.010
                                                                          0.007
                                                                          0.025
                                                                          0.024
                                  PC=0.60, NPDay=100, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.994
                                                                          0.912
                                                                          0.780
                                                                          0.989
                                                                          0.988

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.876
                                                                          0.749
                                                                          0.601
                                                                          0.893
                                                                          0.889

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.416
                                                                          0.331
                                                                          0.268
                                                                          0.468
                                                                          0.460

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.012
                                                                          0.008
                                                                          0.006
                                                                          0.017
                                                                          0.016

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          1.000
                                                                          0.966
                                                                          0.832
                                                                          0.996
                                                                          0.990

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.894
                                                                          0.800
                                                                          0.648
                                                                          0.900
                                                                          0.892

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.404
                                                                          0.340
                                                                          0.287
                                                                          0.449
                                                                          0.445

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.004
                                                                          0.003
                                                                          0.004
                                                                          0.006
                                                                          0.006

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.997
                                                                          0.972
                                                                          0.834
                                                                          0.996
                                                                          0.995

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.916
                                                                          0.839
                                                                          0.696
                                                                          0.921
                                                                          0.910

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.374
                                                                          0.330
                                                                          0.252
                                                                          0.433
                                                                          0.431

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.005
                                                                          0.002
                                                                          0.004
                                                                          0.007
                                                                          0.007
                                   PC=0.80, NPDay=50, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.992
                                                                          0.939
                                                                          0.714
                                                                          0.982
                                                                          0.969

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.857
                                                                          0.761
                                                                          0.465
                                                                          0.852
                                                                          0.818

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.424
                                                                          0.346
                                                                          0.201
                                                                          0.448
                                                                          0.413

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.022
                                                                          0.017
                                                                          0.014
                                                                          0.025
                                                                          0.026

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.995
                                                                          0.969
                                                                          0.805
                                                                          0.987
                                                                          0.985

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.860
                                                                          0.807
                                                                          0.551
                                                                          0.853
                                                                          0.829

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.392
                                                                          0.359
                                                                          0.229
                                                                          0.420
                                                                          0.396

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.008
                                                                          0.006
                                                                          0.003
                                                                          0.010
                                                                          0.010

                                                                             10
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          0.998
                                                                          0.991
                                                                          0.853
                                                                          0.994
                                                                          0.993

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.902
                                                                          0.853
                                                                          0.663
                                                                          0.898
                                                                          0.886

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.390
                                                                          0.354
                                                                          0.230
                                                                          0.406
                                                                          0.394

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.007
                                                                          0.006
                                                                          0.006
                                                                          0.011
                                                                          0.010
                                  PC=0.75, NPDay=100, SD_Sub=0.33, SD_Day=0.166
                                                                              8
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          1.000
                                                                          0.987
                                                                          0.886
                                                                          0.993
                                                                          0.993

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.913
                                                                          0.870
                                                                          0.721
                                                                          0.907
                                                                          0.896

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.424
                                                                          0.370
                                                                          0.300
                                                                          0.469
                                                                          0.454

                                                                              4
                                                                           0.85
                                                                            0.9
                                                                          0.005
                                                                          0.004
                                                                          0.002
                                                                          0.010
                                                                          0.010

                                                                              9
                                                                              1
                                                                           0.95
                                                                            0.9
                                                                          1.000
                                                                          0.995
                                                                          0.915
                                                                          0.998
                                                                          0.997

                                                                              2
                                                                          0.925
                                                                            0.9
                                                                          0.912
                                                                          0.889
                                                                          0.744
                                                                          0.916
                                                                          0.907

                                                                              3
                                                                            0.9
                                                                            0.9
                                                                          0.407
                                                                          0.355
                                                                          0.302
                                                                          0.424
                                                                          0.420

                                                                              4