FSIS Comparative

Risk Assessment for

Listeria monocytogenes in Ready-to-eat Meat and Poultry Deli Meats

Prepared by

Risk Assessment Division

Office of Public Health Science

Food Safety and Inspection Service

United States Department of Agriculture

March 2009

Contributors

Heather Hicks Quesenberry

Daniel Gallagher

Sarah Endrikat2

David LaBarre1

Eric Ebel1

Carl Schroeder1

Janell Kause1

Acknowledgements

Many people within and outside the Food Safety and Inspection Service, U.S. Department of Agriculture contributed to the development of this risk assessment, including Daniel Engeljohn, Carol Maczka, Regis Pouillot, James Lindsey, Flora Tsui, and researchers of the National Alliance for Food Safety and Security.

Contents

  TOC \o "1-3" \h \z \u    HYPERLINK \l "_Toc226941188"  Contributors	  PAGEREF _Toc226941188 \h  ii

  HYPERLINK \l "_Toc226941189"  Acknowledgements	  PAGEREF _Toc226941189 \h  iii

  HYPERLINK \l "_Toc226941190"  Executive Summary	  PAGEREF _Toc226941190 \h  1

  HYPERLINK \l "_Toc226941191"  Background	  PAGEREF _Toc226941191 \h  1

  HYPERLINK \l "_Toc226941192"  Results	  PAGEREF _Toc226941192 \h  2

  HYPERLINK \l "_Toc226941193"  Conclusions	  PAGEREF _Toc226941193 \h  2

  HYPERLINK \l "_Toc226941194"  Introduction	  PAGEREF _Toc226941194 \h  3

  HYPERLINK \l "_Toc226941195"  Methods	  PAGEREF _Toc226941195 \h  4

  HYPERLINK \l "_Toc226941196"  stage I: Prevalence and Level of L. monocytogenes in RTE Meat and Poultry deli meats at retail	  PAGEREF _Toc226941196 \h  5

  HYPERLINK \l "_Toc226941197"  stage II: Growth of L. monocytogenes from retail purchase to consumption	  PAGEREF _Toc226941197 \h  13

  HYPERLINK \l "_Toc226941198"  Stage III: deli meat Consumption	  PAGEREF _Toc226941198 \h  16

  HYPERLINK \l "_Toc226941199"  Stage IV: L. monocytogenes Dose-response Relationship	  PAGEREF _Toc226941199 \h  17

  HYPERLINK \l "_Toc226941200"  Results	  PAGEREF _Toc226941200 \h  17

  HYPERLINK \l "_Toc226941201"  Conclusions	  PAGEREF _Toc226941201 \h  23

  HYPERLINK \l "_Toc226941202"  Appendix I: L. monocytogenes in Ready-to-eat Meat and Poultry Deli meat	  PAGEREF _Toc226941202 \h  24

  HYPERLINK \l "_Toc226941203"  Data Collection Methods	  PAGEREF _Toc226941203 \h  24

  HYPERLINK \l "_Toc226941204"  Study Results	  PAGEREF _Toc226941204 \h  26

  HYPERLINK \l "_Toc226941205"  Prevalence and Number of Samples	  PAGEREF _Toc226941205 \h  26

  HYPERLINK \l "_Toc226941206"  Logistic Regression	  PAGEREF _Toc226941206 \h  35

  HYPERLINK \l "_Toc226941207"  Comparison of Findings of the National Alliance for Food Safety and Security with those of the Food Processors’ Association	  PAGEREF _Toc226941207 \h  37

  HYPERLINK \l "_Toc226941208"  Conclusions	  PAGEREF _Toc226941208 \h  38

  HYPERLINK \l "_Toc226941209"  Appendix II: Sensitivity Analysis	  PAGEREF _Toc226941209 \h  39

  HYPERLINK \l "_Toc226941210"  STORAGE TIME-TEMPERATURE SENSITIVITY	  PAGEREF _Toc226941210 \h  39

  HYPERLINK \l "_Toc226941211"  SHELF LIFE SENSITIVITY	  PAGEREF _Toc226941211 \h  47

  HYPERLINK \l "_Toc226941212"  RELATIVE IMPACTS OF MODEL VARIABLES	  PAGEREF _Toc226941212 \h  48

  HYPERLINK \l "_Toc226941213"  References	  PAGEREF _Toc226941213 \h  52

 Figures

  TOC \h \z \c "Figure"    HYPERLINK \l "_Toc215306753"  Figure 1. A conceptual model of the stages in this risk assessment and the critical inputs considered within each stage.	  PAGEREF _Toc215306753 \h  4

  HYPERLINK \l "_Toc215306754"  Figure 2. Cumulative density functions for the amount of L. monocytogenes in prepackaged compared to retail-sliced RTE deli meats.	  PAGEREF _Toc215306754 \h  6

  HYPERLINK \l "_Toc215306755"  Figure 3. Probability plots for fitted lognormal (base 10) distribution to observed levels for retail-sliced and prepackaged deli meat.	  PAGEREF _Toc215306755 \h  10

  HYPERLINK \l "_Toc215306756"  Figure 4. Fitted cumulative distribution plots and observed retail data for L. monocytogenes levels in retail-sliced versus prepackaged deli meats	  PAGEREF _Toc215306756 \h  11

  HYPERLINK \l "_Toc215306757"  Figure 5. Quantile-quantile plot of fitted distributions for L. monocytogenes levels from retail-sliced and prepackaged deli meat	  PAGEREF _Toc215306757 \h  12

  HYPERLINK \l "_Toc215306758"  Figure 6. Histograms of estimated number of deaths per annum for retail-sliced and prepackaged product based on the 4,000 dose-response simulations of the FDA-FSIS model.	  PAGEREF _Toc215306758 \h  21

  HYPERLINK \l "_Toc215306759"  Figure 7. Cumulative density plots estimated number of deaths per annum for retail-sliced and prepackaged product based on the 4,000 dose-response simulations of the FDA-FSIS model.	  PAGEREF _Toc215306759 \h  22

  HYPERLINK \l "_Toc215306760"  Figure 8. Number of RTE samples by location of slicing (source).	  PAGEREF _Toc215306760 \h  29

  HYPERLINK \l "_Toc215306761"  Figure 9. Prevalence of L. monocytogenes in deli meat by location of slicing.	  PAGEREF _Toc215306761 \h  30

  HYPERLINK \l "_Toc215306762"  Figure 10. Number of deli meat samples collected per store	  PAGEREF _Toc215306762 \h  31

  HYPERLINK \l "_Toc215306763"  Figure 11. Number of RTE samples by deli meat type	  PAGEREF _Toc215306763 \h  32

  HYPERLINK \l "_Toc215306764"  Figure 12. Prevalence of L. monocytogenes in RTE deli meats by deli meat type	  PAGEREF _Toc215306764 \h  33

  HYPERLINK \l "_Toc215306765"  Figure 13. Prevalence of L. monocytogenes in RTE deli meats samples sliced at retail.	  PAGEREF _Toc215306765 \h  34

  HYPERLINK \l "_Toc215306766"  Figure 14. Graphical display of logistic regression results using deli meat sample prevalence at individuals stores as the dependent variable	  PAGEREF _Toc215306766 \h  37

  HYPERLINK \l "_Toc215306767"  Figure 15. Relative ratio of deaths differing storage times between retail-sliced and prepackaged product.	  PAGEREF _Toc215306767 \h  40

  HYPERLINK \l "_Toc215306768"  Figure 16. Relative frequency of storage time for retail-sliced versus prepackaged product.	  PAGEREF _Toc215306768 \h  42

  HYPERLINK \l "_Toc215306769"  Figure 17. Storage time for retail-sliced and prepackaged product.	  PAGEREF _Toc215306769 \h  43

  HYPERLINK \l "_Toc215306770"  Figure 18. Relative frequencies for storage temperature.	  PAGEREF _Toc215306770 \h  44

  HYPERLINK \l "_Toc215306771"  Figure 19. Relative ratio of deaths between retail-sliced versus prepackaged product for differing definitions of shelf life.	  PAGEREF _Toc215306771 \h  48

  HYPERLINK \l "_Toc215306772"  Figure 20. Recursive partitioning and regression tree.	  PAGEREF _Toc215306772 \h  49

  HYPERLINK \l "_Toc215306773"  Figure 21. Box plots for each deli meat category by age group.	  PAGEREF _Toc215306773 \h  50

  HYPERLINK \l "_Toc215306774"  Figure 22. Interaction plots comparing the effect of growth inhibitor (GI) use and slicing location on the mean number of deaths from listeriosis.	  PAGEREF _Toc215306774 \h  51

 Tables

  TOC \h \z \c "Table"    HYPERLINK \l "_Toc215306775"  Table 1. Survival analysis input for statistical distribution fitting for the level of L. monocytogenes in deli meats at retail.	  PAGEREF _Toc215306775 \h  7

  HYPERLINK \l "_Toc215306776"  Table 2. Best fit maximum likelihood results and probability plot distribution parameters for retail-sliced deli meat.	  PAGEREF _Toc215306776 \h  8

  HYPERLINK \l "_Toc215306777"  Table 3. Best fit maximum likelihood results and probability plot distribution parameters for prepackaged deli meat.	  PAGEREF _Toc215306777 \h  8

  HYPERLINK \l "_Toc215306778"  Table 4. Quantiles from fitted lognormal distributions for retail-sliced and prepackaged L. monocytogenes levels.   PAGEREF _Toc215306778 \h  12

  HYPERLINK \l "_Toc215306779"  Table 5. Plant distribution and estimated fraction of production prior to the Interim Final Rule.	  PAGEREF _Toc215306779 \h  14

  HYPERLINK \l "_Toc215306780"  Table 6. Fraction of deli meat production by slicing location and antimicrobial growth inhibitor use during July 2007.	  PAGEREF _Toc215306780 \h  16

  HYPERLINK \l "_Toc215306781"  Table 7. Estimated mean number of deaths per year from L. monocytogenes in deli meat among three populations stratified by age and four deli meat categories.	  PAGEREF _Toc215306781 \h  19

  HYPERLINK \l "_Toc215306782"  Table 8. Estimated mean number of illnesses from L. monocytogenes in deli meat per year among three populations stratified by age and four deli meat categories.	  PAGEREF _Toc215306782 \h  20

  HYPERLINK \l "_Toc215306783"  Table 9. Prevalence of product samples and stores visited based on sampling locations.	  PAGEREF _Toc215306783 \h  26

  HYPERLINK \l "_Toc215306784"  Table 10. Prevalence of product samples (retail-sliced, prepackaged) and stores visited based on quarter of year.	  PAGEREF _Toc215306784 \h  27

  HYPERLINK \l "_Toc215306785"  Table 11. Prevalence of product samples (retail-sliced, prepackaged) and stores visited based on time of day (AM versus PM).	  PAGEREF _Toc215306785 \h  27

  HYPERLINK \l "_Toc215306786"  Table 12. Prevalence of only retail-sliced product and stores visited based on time of day (AM versus PM).	  PAGEREF _Toc215306786 \h  27

  HYPERLINK \l "_Toc215306787"  Table 13. Prevalence of product samples (retail-sliced, prepackaged, and chubs) and stores visited based on store type (major grocery chain versus other grocers).	  PAGEREF _Toc215306787 \h  28

  HYPERLINK \l "_Toc215306788"  Table 14. Prevalence of L. monocytogenes in retail-sliced and prepackaged deli meat by site. The number of positive samples and the total number of samples are shown in parentheses.	  PAGEREF _Toc215306788 \h  30

  HYPERLINK \l "_Toc215306789"  Table 15. Results of logistic regression for store prevalence as function of processing type, store type, and time of day indicator variables	  PAGEREF _Toc215306789 \h  35

  HYPERLINK \l "_Toc215306790"  Table 16. Prevalence of L. monocytogenes in sliced deli meat by site and processing type from the Food Products Association.	  PAGEREF _Toc215306790 \h  37

  HYPERLINK \l "_Toc215306791"  Table 17. Overall results of statistical tests for prevalence of L. monocytogenes on RTE meat and poultry deli meats by location, season, time of day for slicing at retail, and by deli meat type.	  PAGEREF _Toc215306791 \h  38

  HYPERLINK \l "_Toc215306792"  Table 18. Estimated mean number of deaths and illnesses per annum by fraction of consumer storage time.	  PAGEREF _Toc215306792 \h  39

  HYPERLINK \l "_Toc215306793"  Table 19. Fitted Weibull distributions according to the deli meat category	  PAGEREF _Toc215306793 \h  41

  HYPERLINK \l "_Toc215306794"  Table 20. Storage times for retail-sliced and prepackaged product.	  PAGEREF _Toc215306794 \h  42

  HYPERLINK \l "_Toc215306795"  Table 21. Estimated mean number of deaths per year from L. monocytogenes in deli meat among three populations stratified by age and four deli meat categories using the alternative storage time-temperature data.	  PAGEREF _Toc215306795 \h  45

  HYPERLINK \l "_Toc215306796"  Table 22. Estimated mean number of illnesses per year from L. monocytogenes in deli meat among three populations stratified by age and four deli meat categories.	  PAGEREF _Toc215306796 \h  46

  HYPERLINK \l "_Toc215306797"  Table 23. EGR for product with and without growth inhibitor by shelf life.	  PAGEREF _Toc215306797 \h  47

  HYPERLINK \l "_Toc215306798"  Table 24. Mean number of deaths and illnesses per annum by shelf life.	  PAGEREF _Toc215306798 \h  47

Executive Summary

DRAFT FSIS Comparative Risk Assessment for Listeria monocytogenes in Ready-to-eat Meat and Poultry Deli Meats

March 2009

Background

Listeria monocytogenes (L. monocytogenes) is an important foodborne pathogen, estimated to cause approximately 2,500 illnesses, 2,300 hospitalizations, and 500 deaths each year in the United States. In an effort to understand better the sources of foodborne Listeria infection, the Food and Drug Administration (FDA) and the Food Safety Inspection Service (FSIS), working collaboratively, developed a quantitative risk assessment for L. monocytogenes that compared the risk of listeriosis among twenty-three categories of ready-to-eat (RTE) foods. The results of the risk assessment, completed in 2003, indicated deli meats pose the greatest risk for listeriosis, accounting for approximately 1,600 illnesses per year.

Based on these findings, FDA and FSIS conducted a preliminary analysis using the 2003 L. monocytogenes risk assessment to evaluate the relative risk of illness from Listeria on deli meat sliced and packaged at federally-inspected processing establishments (prepackaged deli meat) compared to deli meat sliced at retail facilities. This risk assessment contained industry data for L. monocytogenes on retail deli meat from delicatessens in California and Maryland (Gombas et al. 2003). The results of this risk assessment indicated a high percentage of listeriosis cases related to deli meats were associated with those sliced at retail. Because these results, however, were based on limited retail L. monocytogenes contamination data for deli meats, FSIS sought to gather additional data specifically to examine the relative risk of illness from prepackaged deli meat compared to deli meat sliced at retail facilities more closely.

Therefore, the U.S. Department of Agriculture, Agricultural Research Service funded the National Alliance for Food Safety and Security (NAFSS) – a consortium of twenty-five research universities – to conduct a four-state study in which prepackaged deli meat and deli meat sliced and packaged at retail were analyzed for the prevalence and level of L. monocytogenes (Draughon 2006). Methods

Data from the NAFSS study, described in Appendix I of the risk assessment report, were used as inputs to the deli meat exposure pathway developed as part of the abovementioned 2003 FDA-FSIS risk assessment for Listeria in ready-to-eat foods. The pathway consists of four distinct stages. The Retail Stage determines the level of L. monocytogenes in prepackaged deli meats and in deli meats sliced at retail. The Growth Stage uses an exponential growth rate function to model growth of L. monocytogenes in deli meat between purchase at retail and consumption. The Consumption Stage uses information about deli meat serving sizes and the number of servings consumed to estimate consumer exposure to L. monocytogenes in deli meat. Lastly, by integrating the predicted exposure with a dose-response relationship, the Dose-response Stage predicts the probability of death from consuming L. monocytogenes on deli meat.

Two distinct consumer storage time-temperature distributions were used for the risk assessment. The first analysis used the same storage time-temperature distributions as the 2003 FDA-FSIS risk assessment. The storage times were taken from an American Meat Institute (AMI) 2001 survey and consumer storage temperatures were taken from an Audits International 1999 survey. For this first analysis, the storage times for both prepackaged deli meat and deli meat sliced at retail used the same values. A second analysis, described in Appendix II, was performed using the consumer survey conducted by RTI International, Tennessee State University, and Kansas State University (Cates et al. 2006). The results of the survey indicated prepackaged deli meat was stored for statistically significant longer periods than deli meat sliced at retail. The survey did not find any difference for storage temperature. This second analysis thus used different storage time distributions for prepackaged versus retail sliced product.

Results

This risk assessment, using current retail contamination data for deli meat (Draughon 2006; NAFSS) and current consumer behavior data for deli meats (Cates et al. 2006; RTI) indicates that of those listeriosis cases and deaths attributed to deli meats, approximately 83%  are associated with deli meats sliced at retail. The estimated mean number of illnesses per year from prepackaged deli meats was 188.6 with a 95% confidence interval (CI) of 184.7 – 192.4. The estimated mean number of deaths per year from L. monocytogenes prepackaged deli meats was 34.1 (95% CI: 33.4 – 34.9). In contrast, the estimated mean number of illnesses per year from retail-sliced deli meats was 919.6 (95 CI: 686.4 – 932.4). The estimated mean number of deaths per year from L. monocytogenes in retail-sliced deli meats was 166.9 (95% CI: 164.5 – 169.3).

Conclusions

Of those illnesses and deaths from L. monocytogenes from deli meat consumption, a large percentage is attributed to deli meat sliced at retail facilities. The remainder is from prepackaged deli meat. Studies are needed to determine how contamination of deli meat at retail occurs and to design effective mitigations for reducing listeriosis associated with the consumption of deli meat sliced at retail.

Introduction

In 2000, the Food and Drug Administration (FDA) and the U.S. Department of Agriculture’s Food Safety and Inspection Service (USDA/FSIS) began a risk assessment to identify which ready-to-eat (RTE) foods pose the greatest risk for listeriosis in the U.S. (FDA-FSIS 2003). Deli meat was found to pose the greatest risk of listeriosis among all RTE food categories. Based on these results and in response to public comments on the FSIS proposed rule Performance Standards for the Production of Processed Meat and Poultry Products (66 FR 12589), FSIS developed a risk assessment for L. monocytogenes in RTE meat and poultry products (FSIS 2003). The risk assessment model predicted that the use of post-lethality interventions and antimicrobial growth inhibitors significantly lowered the public health risk of listeriosis compared to either control if used independently. Post-processing lethality treatments that reduced L. monocytogenes in products formulated or processed to inhibit the growth of any remaining L. monocytogenes were predicted to be the most effective in protecting public health. Both the 2003 FDA-FSIS and 2003 FSIS Listeria risk assessments served as the scientific basis for FSIS’ interim final rule for the control of L. monocytogenes during processing (“Control of Listeria Monocytogenes in Ready-to-Eat Meat and Poultry Products,” 68 FR 34208; June 6, 2003 (revised January 1, 2006); 9 CFR 430).

Subsequently, in 2004, FDA and FSIS did a preliminary analysis using L. monocytogenes contamination data for retail deli meat from California and Maryland (Gombas et al. 2003) to estimate the relative risk of listeriosis from deli meat sliced and packaged in FSIS-inspected processing establishments (hereafter termed prepackaged) versus those sliced and packaged at retail facilities. Results suggested that deli meat sliced and packaged at retail posed the greater risk, accounting for approximately 80% of all listeriosis cases from deli meat.

In 2006, researchers with the National Alliance for Food Safety and Security (NAFSS) – a consortium of 25 research universities – completed a study of L. monocytogenes contamination in prepackaged RTE meat and poultry deli meats and those sliced and packaged at retail from California, Maryland, Georgia, and Tennessee (Draughon 2006). FSIS adapted the FDA-FSIS (2003) risk assessment model to examine data from the NAFSS study and reanalyze the comparative risk of listeriosis from prepackaged RTE deli meat versus RTE deli meat sliced and packaged at retail. This report describes the analysis and its findings.

Lastly, while doing this risk assessment, a consumer survey done by RTI International, Tennessee State University, and Kansas State University was released (Cates et al. 2006; survey available at http://www.foodrisk.org) indicating prepackaged deli meat may be stored for statistically significant longer periods than deli meat sliced at retail. Therefore, we did a sensitivity analysis by rerunning the risk assessment model using these new survey results and compared the output to that from using the AMI survey results, as described in Appendix II of this report.

Methods

The analysis uses the deli meat exposure pathway from the risk assessment model developed and utilized in a previous L. monocytogenes risk assessment (FDA-FSIS, 2003) that estimated risk of death attributable to 23 ready-to-eat (RTE) food categories. This analysis separates the deli meat category into prepackaged deli meats and those sliced at retail establishments. Because of increased use of antimicrobial growth inhibitors in deli meat, each deli meat type is divided into those with or without antimicrobial growth inhibitors. Consistent with the FDA-FSIS (2003) risk assessment model, this analysis considers four conceptual stages (  REF _Ref197160429  \* MERGEFORMAT  Figure 1 ).

Figure   SEQ Figure \* ARABIC  1 . A conceptual model of the stages in this risk assessment and the critical inputs considered within each stage.

The Retail Stage determines the presence and level of L. monocytogenes in the two deli meat types.

The Growth Stage uses an exponential growth rate modified to account for antimicrobial growth inhibitor usage to predict growth of L. monocytogenes in deli meat between retail and consumption.

The Consumption Stage predicts the L. monocytogenes exposure dose consumed in servings of deli meats, which is a consequence of serving size and the number of servings.

The Dose-response Stage predicts the probability of death from L. monocytogenes per serving by integrating the predicted exposure distribution with a dose-response relationship.

The output of these four stages in this risk assessment is the annual number of illnesses and deaths (and the corresponding risk of illness or death per serving) from Listeria deli meat. While these four stages are updated in the FDA-FSIS (2003) model for the deli meat food category, all other food categories remain as originally parameterized. This risk assessment describes the analysis used to parameterize each of the four updated stages in more detail below.

stage I: Prevalence and Level of L. monocytogenes in RTE Meat and Poultry deli meats at retail

The prevalence and level of L. monocytogenes in RTE meat and poultry deli meats at retail establishments were determined using data from a National Alliance for Food Safety and Security (NAFSS) study in which 6 of 3,522 (0.17%) samples and 49 of 3,518 (1.39%) samples tested positive for L. monocytogenes from prepackaged and retail-sliced deli meats, respectively. This difference was statistically significant (p <0.05). Analyses of these data are described in depth in Appendix I. Of the six positive samples from prepackaged deli meat, all had L. monocytogenes levels (0.3 MPN/gram. Of the 49 positive samples from deli meat sliced and packaged at retail, L. monocytogenes levels ranged from <0.3 to (110 MPN/gram. Cumulative density plots, assuming a detection limit of 0.008 MPN/gram (i.e. 1 MPN/125 g), are shown in   REF _Ref197161057  \* MERGEFORMAT  Figure 2 .

Figure   SEQ Figure \* ARABIC  2 . Cumulative density functions for the amount of L. monocytogenes in prepackaged compared to retail-sliced RTE deli meats.

Data for the prevalence and level of L. monocytogenes in deli meat sampled at retail were fitted to probability distributions as inputs to the FDA-FSIS (2003) model. Because there were few positive samples, distribution fits were considered approximate. The survival analysis module of Number Cruncher Statistical Systems (NCSS:  http://www.ncss.com/) was used to fit an appropriate statistical distribution to prepackaged and retail-sliced deli meat separately. Survival analysis allows incorporation of left and right censoring into distribution fitting. Left censoring indicates that the true level of L. monocytogenes in deli meat is less than reported. Right censoring indicates that the true level of L. monocytogenes in deli meat is higher than reported. Interval censoring indicates that the true value is between two fixed values. To be conservative, all but one of the observed L. monocytogenes positive samples of deli meat with a level of ( 0.3 MPN/gram were treated as having a level of 0.3 MPN/gram. The remaining positive sample was treated as an interval measurement between 0.008 MPN/gram and 0.3 MPN/gram. Negative samples were assumed to have L. monocytogenes levels ( 0.008 MPN/gram (i.e. ( 1 MPN/125 gram). The inputs to the survival analysis are shown in   REF _Ref197161103  \* MERGEFORMAT  Table 1 . The comparison of maximum likelihood fit to various probability distributions is provided below for retail-sliced (Table 2) and prepackaged (Table 3) deli meat. The parameters for each distribution were determined by least-squares regression fit to the corresponding probability plot.

Table   SEQ Table \* ARABIC  1 . Survival analysis input for statistical distribution fitting for the level of L. monocytogenes in deli meats at retail.

Retail (deli) sliced

Prepackaged

No. Samples1	L. monocytogenes level (MPN/gram)1	Censor Type2

No. Samples	L. monocytogenes level (MPN/gram)	Censor Type2

3,469	(0.008	L

3,516	( 0.008	L

1	Between 0.008 and 0.3	I

1	Between 0.008 and 0.3	I

29	0.3	F

5	0.3	F

3	0.92	F

1	0.93	F

1	0.94	F

3	2.3	F

1	15.3	F

1	24	F

1	46	F

3	( 110	R

1 L. monocytogenes levels were not given for five positive retail-sliced deli meat samples. These data were thus not used in the distribution fitting.

2 Censor type refers to the censoring used by the survival analysis fit. L indicates left censoring (actual value is less than observed); I indicates interval censoring (actual value is between two known values). F indicates actual value is observed level. R indicates right censoring (actual value is greater than observed).

Table   SEQ Table \* ARABIC  2 . Best fit maximum likelihood results and probability plot distribution parameters for retail-sliced deli meat.

Distribution	Log Likelihood	Shape1	Scale1

Weibull	-315.606	NA2	NA2

Lognormal	-316.634	-25.6314	9.309884

Lognormal10	-316.634	-11.1316	4.043231

Loglogistic	-318.041	-19.0915	3.277907

Logistic	-375.906	-13.795	3.168052

Extreme Value	-396.146	-22.3905	15.19124

Exponential	-13046.5	         1	0.012057

Normal	    NA2	       NA2	NA2

1 The interpretation of these parameters varies depending on the distribution. For most distributions, the shape is the mean of the distribution and the scale is the standard deviation.

2 The probability plot estimate could not be calculated for these parameters.

Table   SEQ Table \* ARABIC  3 . Best fit maximum likelihood results and probability plot distribution parameters for prepackaged deli meat.

Distribution	Log Likelihood	Shape1	Scale1

Extreme Value	-43.1107	-2.38335	1.303234

Normal	-43.2915	-1.80981	0.628146

Logistic	-43.5157	-1.14005	0.183313

Weibull	-49.6842	NA2	NA2

Lognormal10	-49.865	-27.3912	7.79663

Lognormal	-49.865	-11.8958	3.386033

Loglogistic	-50.0892	-19.078	2.275309

Exponential	-715.407	1.00E+00	1.37E-03

1 The interpretation of these parameters varies depending on the distribution. For most distributions, the shape is the mean of the distribution and the scale is the standard deviation.

2 The estimate could not be calculated for these parameters.

Though the Weibull and extreme value distributions are suggested as best fitting these data (based on the maximum likelihood criterion), the lognormal distribution was selected as the most appropriate. The lognormal fit to the distribution of the level of L. monocytogenes in retail-sliced deli meat is statistically no different from the Weibull distribution. It is preferred that both retail-sliced and prepackaged distributions are modeled as the same type. Environmental contaminants such as bacterial levels are often fit to a lognormal distribution and this distribution has theoretic justification. The probability plots and the resulting fit for both retail-sliced and prepackaged deli meat are shown on the following page in Figure 3. The fitted cumulative density plots and observed data points are shown in Figure 4a. The fit for the retail-sliced deli meat appears adequate. The distribution fit for the prepackaged deli meat is uncertain because of only two data points. Figure 4b extrapolates the cumulative density curves to lower levels. The deli meat exposure assessment model uses levels as low as 10-8 MPN/gram as inputs.

Figure   SEQ Figure \* ARABIC  3 . Probability plots for fitted lognormal (base 10) distribution to observed levels for retail-sliced (a) and prepackaged deli meat (b).

Figure   SEQ Figure \* ARABIC  4 . Fitted cumulative distribution plots and observed retail data for L. monocytogenes levels in retail-sliced versus prepackaged deli meats. Illustration (a) is over the range of observed L. monocytogenes levels. Illustration (b) is over the entire range of L. monocytogenes levels in deli meats used as an input into deli meat exposure pathway of the 2003 FDA-FSIS risk assessment model.

The quantile-quantile plot of the two fitted distributions is shown in   REF _Ref197225148  \* MERGEFORMAT  Figure 5 . Because the same distribution shape (lognormal) was selected for both retail-sliced and prepackaged, the quantile-quantile plot is a straight line. The quantile-quantile line is below the 1:1 reference line, indicating, as expected, that for a given percentile, the prepackaged L. monocytogenes level is lower than the L. monocytogenes retail-sliced level over the range of interest. The quantile-quantile line parallel to the reference line, indicating that the difference between the two distributions is greater at the extreme upper tails of the distributions.

Figure   SEQ Figure \* ARABIC  5 . Quantile-quantile plot of fitted distributions for L. monocytogenes levels from retail-sliced and prepackaged deli meat. (A 1:1 reference line is included for visual comparison).

A fixed number of quantiles from the distribution of L. monocytogenes levels in deli meats serve as inputs to the 2003 FDA-FSIS exposure assessment model. Based on the fitted parameters shown in Table 2 and Table 3, the quantiles needed for the exposure assessment model were determined using free statistical software, R (http://www.r-project.org/). These quantiles are given below in   REF _Ref197482261  \* MERGEFORMAT  Table 4 .

Table   SEQ Table \* ARABIC  4 . Quantiles from fitted lognormal distributions for retail-sliced and prepackaged L. monocytogenes levels.

Cumulative Fraction	Retail-sliced L. Monocytogenes level (MPN/gram) Prepackaged L. Monocytogenes level (MPN/gram)

0.8	1.87E-08	8.99E-10

0.85	1.15E-07	4.11E-09

0.9	1.12E-06	2.78E-08

0.95	3.30E-05	4.72E-07

0.99	1.88E-02	9.58E-05

0.995	1.92E-01	6.70E-04

0.999	2.31E+01	3.70E-02

0.9999	8.04E+03	4.98E+00

Max1	8.03E+06	6.16E+03

1Based on simulation of 1,000,000 random numbers from the appropriate fitted distribution.

The risk assessment analysis used in this report assumed independence among samples. This assumption may not be met for these data, however, because the samples collected from the same retail location are likely to be correlated. Cross-contamination or poor hygienic conditions within a retail location may result in the clustering of positive L. monocytogenes findings by store; therefore, analyzing the data by store location may provide a more accurate estimate of the relative risk ratio for retail-sliced versus prepackaged products. However, due to the blinding process used during sample collection, individual store identifiers were removed. Without these store identifiers, store visits can only be estimated based on time and date of sample collection. Also, sample collection times were not provided for samples from Minnesota, so determining individual store visits was not possible. As a result of these data limitations, all individual samples were treated as independent for this risk assessment analysis. A comparison of the results based on the assumption of independence of samples versus samples grouped by approximate store visit may be found in the Appendix.

stage II: Growth of L. monocytogenes from retail purchase to consumption

To assess consumer exposures, the growth of L. monocytogenes from retail purchase to consumption was modeled. Given regulatory changes subsequent to the development of the FDA-FSIS (2003) risk assessment, the model’s predicted growth for deli meats needed adjustment. Nevertheless, the storage time and temperature distributions were left unchanged from the FDA-FSIS (2003) risk assessment model and the same time and temperature distributions were used for both prepackaged and retail-sliced deli meat.

FSIS provides three alternatives for establishments producing certain RTE meat and poultry deli meats to control for L. monocytogenes (9 CFR 430, 2003):

Alternative 1: Employ both a post-lethality treatment and an antimicrobial growth inhibitor for Listeria monocytogenes on RTE deli meats.

Alternative 2: Employ either (a) a post-lethality treatment or (b) an antimicrobial growth inhibitor for the pathogen on RTE deli meats.

Alternative 3: Employ sanitation measures only (uses neither a post-lethality treatment nor an antimicrobial growth inhibitor).

Deli meat that uses an antimicrobial growth inhibitor is expected to have lower growth rates of L. monocytogenes than deli meat that does not use antimicrobial growth inhibitor. Data on production volumes for each category were used to estimate the use of antimicrobial growth inhibitors in RTE deli meat, and current regulations were used to estimate conservative maximum allowable growth rates.

To qualify as using an antimicrobial growth inhibitor under the Interim Final Rule 9 CFR 430, the growth of L. monocytogenes may not exceed two logs during the shelf life of the product. This information was used to modify the existing FDA-FSIS risk assessment model to account for different growth in deli meat with and without antimicrobial growth inhibitors. The comparison of retail-sliced versus prepackaged was calculated by splitting the deli meat category into four separate categories based on two factors: where the deli meat was sliced and whether antimicrobial growth inhibitor was used. Exponential growth rates for L. monocytogenes were calculated for product with and without antimicrobial growth inhibitors using data from the 2003 FDA-FSIS risk assessment and the estimated fraction of deli meat in each category prior to the implementation of the Interim Final Rule. These older data were used for calculating growth rates because it better matched the period of reported growth rates in the FDA-FSIS risk assessment model. Once the growth rates were determined, this risk assessment used more current manufacturer production volume data to calculate the fraction of deli meat in each category.

Prior to the Interim Final Rule, fewer plants employed alternatives that used antimicrobial growth inhibitors therefore less product was formulated with an antimicrobial growth inhibitor when compared to current conditions. The overall growth rate of L. monocytogenes should be lower after the implementation of the Interim Final Rule because the composition of the product is different – a greater fraction of product contains antimicrobial growth inhibitors.

To calculate the relative growth rates for deli meat with and without antimicrobial growth inhibitor, the fraction of deli meat using antimicrobial growth inhibitor prior to the Interim Final Rule was needed. The number of establishments using each L. monocytogenes control alternative (1, 2 (a or b), or 3) was estimated by FSIS economists. This is shown in Table 5. The fraction of production was estimated by assuming that each plant within a Pathogen Reduction/Hazard Analysis and Critical Control Point (PR/HACCP) size category produced the same volume, and that the total fraction of production was 48%, 48%, and 4% for large, small, and very small plants (FSIS 2003) respectively.

Table   SEQ Table \* ARABIC  5 . Plant distribution and estimated fraction of production prior to the Interim Final Rule.

Lm Control

Alternative	PR/HACCP Size Category1

	Large	Small	Very Small	Total

	No.2	Fraction3	No.2	Fraction3	No.2	Fraction3	No.2	Fraction3

(1) Both post processing lethality and antimicrobial growth inhibitor

7

0.018

20

0.007

15

0.000

42

0.026

(2a) Post processing lethality only

15

0.039

79

0.029

49

0.001

143

0.068

(2b) Antimicrobial growth inhibitor only

40

0.104

122

0.044

65

0.001

227

0.149

(2) Neither post processing lethality nor antimicrobial growth inhibitor

123

0.319

1107

0.400

2072

0.038

3302

0.757

Total	185	0.480	1328	0.480	2201	0.040	3714	1.000

1 Based on PR/HACCP classification.

2 No. is the number of plants.

3Fraction is the fraction of production = number of plants within size and alternative / total number of plants within size * total fraction by size.

Based on this analysis, it was estimated that 17.5% (2.6% + 14.9%) of deli meat used antimicrobial growth inhibitors prior to the implementation of the Interim Final Rule. The percentage of deli meat using antimicrobial growth inhibitors was assumed the same for prepackaged and retail-sliced deli meat.

The exposure assessment portion of the 2003 FDA-FSIS model was adjusted to account for possible use of antimicrobial growth inhibitor by adjusting the exponential growth rate (EGR) of L. monocytogenes among RTE meat and poultry deli meats. The FDA-FSIS model estimated that the mean EGR at 5oC was 0.282 log10 CFU/gram/day. The model treats this as a stochastic parameter and adjusts for stochastic storage time, temperature, and a correlation between the two. Appendix 8 in the 2003 FDA-FSIS risk assessment report lists the references used to calculate this value: 15 published articles with 23 reported growth rates across a range of deli meat products. Most of these reference data are from the late 1980’s to early 1990’s, which is why the use of production data from prior to the implementation of the Interim Final Rule was deemed appropriate.

FSIS L. monocytogenes Compliance Guidelines (http://www.fsis.usda.gov/OPPDE/rdad/

FRPubs/97-013F/Lm_Rule_Compliance_Guidelines_May_2006.pdf ) state that to qualify as utilizing one of two most stringent alternative L. monocytogenes control options (Alternative 1 or 2) in the Interim Final Rule, no more than 2 log10 growth is allowed over the entire shelf life of the product. No temperature is specified during this shelf life, nor is the shelf life itself specified. If this standard is interpreted to be 2 log10 growth over 14 days at 5oC, the exponential growth rate is 2 log10/gram/14 days = 0.143 log10 CFU/gram/day. Using this calculation, as the product shelf life is reduced, the calculated EGR would increase because the same 2-log10 growth would occur in a shorter time.

For comparison, consumer storage time is available based on an American Meat Institute (AMI 2001) survey. Results of the survey suggest approximately 40% of ready-to-eat product is stored for less than 3 days, and another 45% of product is stored from 4 to 7 days. A total of 96% of product is stored for less than 14 days. While consumer storage time is not the same as shelf life, the 14-day assumption appears reasonable. A sensitivity analysis of this shelf life assumption is provided in Appendix II.

If the exponential growth rate (EGR) for product with antimicrobial growth inhibitor (GI) is based on the regulation, then to calculate the EGR for product without GI:

fGI x EGRwith + (1 - fGI) x EGRwithout = EGRFDA

0.175 x 0.143 log10 CFU/gram/d + 0.825 x EGRwithout = 0.282 log10 CFU/gram/day

EGRwithout =0.311 log10 CFU/gram/day

The EGR for product both with and without antimicrobial growth inhibitor are within the observed range for the 23 literature values noted previously and within one standard deviation of the mean EGR.

The maximum L. monocytogenes level that can occur in product can also be adjusted. As there are no additional data for this parameter, it was left unchanged from the existing FDA-FSIS model.

To adjust the growth rates in the deli meat exposure pathway of the 2003 FDA-FSIS risk assessment model, an additional multiplier based on adjusting the mean EGR was added. If the product did not have GI, the stochastic EGR for each iteration was multiplied by 0.311/0.282 = 1.104. If the product did have GI, the stochastic EGR for each iteration was multiplied by 0.143/0.282 = 0.507. Note that the EGR for product with GI is calculated based on FSIS regulation, not on actual industry performance, which may be greater.

Stage III: deli meat Consumption

Serving sizes and total number of deli meat servings annually consumed that were estimated for the 2003 FDA-FSIS risk assessment were unchanged for this analysis. Nevertheless, total servings of deli meats needed to be proportioned among (i) prepackaged deli meats with antimicrobial growth inhibitors; (ii) prepackaged deli meats without antimicrobial growth inhibitors; (iii) retail-sliced deli meats with antimicrobial growth inhibitors; and (iv) retail-sliced deli meats without antimicrobial growth inhibitors.

The fraction of servings for each of the four deli meat categories was estimated from  industry survey data from USDA/FSIS Form 10,240-1, Production Information on Post-Lethality Exposed Ready-to-Eat Products, gathered in July 2007 in accordance with 9 CFR 430.4(d). For example, 32.2% of servings were calculated to be prepackaged product with antimicrobial growth inhibitor. Overall, approximately 47% of deli meat is sold prepackaged, and 53% is retail-sliced (Table 6).

Table   SEQ Table \* ARABIC  6 . Fraction of deli meat production by slicing location and antimicrobial growth inhibitor use during July 2007.

Alternative	Prepackaged (sliced at plant)	Retail-Sliced 	Total

With antimicrobial growth inhibitor	0.322	0.267	0.589

Without antimicrobial growth inhibitor	0.144	0.267	0.411

Total	0.466	0.534	1.000

Stage IV: L. monocytogenes Dose-response Relationship

In the 2003 FDA-FSIS risk assessment, there are three age-specific dose-response relationships that have been developed – one for those 60 years of age or older (referred to as “elderly” in the 2003 FDA-FSIS risk assessment), those who are more than 30 days old to 60 years of age (“intermediate” age population), and fetuses and neonates from 16 weeks after fertilization to 30 days old (perinatal subpopulation). The methods used in the 2003 FDA-FSIS risk assessment are the same as those used here.

The dose-response model was run in calibrated mode. In calibrated mode, a scaling factor was used for each of the 4,000 simulations to adjust the dose-response curve from the mouse model to meet a specified number of deaths in humans. For this analysis, the comparative risk assessment was calibrated to the number of deaths attributed to deli meats, based on data from the Centers for Disease Control and Prevention, used in the 2003 FDA-FSIS risk assessment. Given the increased implementation of Listeria control procedures at the processing plant and antimicrobial growth inhibitor use in the product, these values are likely to overstate estimated deaths under current conditions. Thus, the estimated deaths are meant for comparative purposes only, but do not affect accurate estimates of the “relative risk” of listeriosis and subsequent death associated with deli meats sliced that are retail-sliced compared to those that are prepackaged.

Results

The original deli meat category in the 2003 FDA-FSIS Listeria risk assessment was split into the four separate categories such that exposure distributions were estimated (using that model) for (i) prepackaged deli meats with antimicrobial growth inhibitors; (ii) prepackaged deli meats without antimicrobial growth inhibitors; (iii) retail-sliced deli meats with antimicrobial growth inhibitors; and (iv) retail-sliced deli meats without antimicrobial growth inhibitors. These exposure distributions were generated from the two contamination distributions at retail (i.e., one for prepackaged deli meat and another for retail-sliced deli meat). The growth predictions applied to each of these distributions predicted the effects of variable storage times and temperatures on the number of L. monocytogenes per gram of deli meat depending on whether it included antimicrobial growth inhibitors or not. The exposure distribution finally determined the variability in dose per serving by considering the (stochastic) number of grams constituting a serving of deli meat. In the FDA-FSIS (2003) risk assessment model, these exposure distributions were integrated with the FDA-FSIS L. monocytogenes dose-response models (one for each of the three age-specific subpopulations) to predict the annual number of deaths attributed to each of the four categories. The estimated mean numbers of deaths per year are given in Table 7 below. Clearly, the use of antimicrobial growth inhibitors reduces the number of estimated deaths. This is most notable for the retail-sliced product, which starts with a higher level at retail. Also notable is the impact that the lower L. monocytogenes starting distribution has on lowering the number of deaths from prepackaged products.

The estimated mean number of deaths per year associated with prepackaged product was 13.8 (4.4+9.4), and the estimated mean number of deaths per annum associated with retail-sliced product was 125.5 (23.5+102.0), with an estimated total annual number of deaths equal to 139.3. Ten percent of the estimated per annum deaths (13.8/139.3 = 9.89%) are attributable to prepackaged product, while the remaining 90% are attributable to retail-sliced product (125.6/139.3 = 90.11%). The relative risk on a per annum basis for deli meats sliced at retail versus sliced at plant is thus 125.6/13.8 = 9.1. These results are almost identical to the earlier 2003 FDA-FSIS Listeria risk assessment findings, which used NFPA retail data (Gombas et al. 2003). A similar analysis was conducted for illnesses. The 2003 FDA-FSIS model assumes a constant illness to mortality ratio by age group of 3.7, 11.3, and 12.7 for elderly, intermediate, and perinatal age groups respectively. Estimated illnesses from L. monocytogenes in deli meat are shown in   REF _Ref198950948  \* MERGEFORMAT  Table 8 . A mean of 698.0 illnesses were attributed to retail-sliced product and a mean of 76.8 illnesses were attributed to prepackaged product, for a relative risk ratio of 9.1.

Table   SEQ Table \* ARABIC  7 . Estimated mean number of deaths per year from L. monocytogenes in deli meat among three populations stratified by age and four deli meat categories.

Deli Meat Category	Elderly

(95% CI)	Intermediate Age

(95% CI)	Perinatal

(95% CI)	All Age Groups (95% CI)

Prepackaged with growth inhibitor	3.4

(3.3 - 3.5)	0.8

(0.8 - 0.8)	0.2

(0.2 - 0.2)	4.4

(4.3 - 4.5)

Prepackaged without growth inhibitor 	7.2

(7.0 - 7.3)	1.8

(1.7 - 1.8)	0.5

(0.4 - 0.5)	9.4

(9.1 - 9.6)

Retail-sliced with growth inhibitor	18.0

(17.5 - 18.4)	4.4

(4.3 - 4.6)	1.1

(1.1 - 1.2)	23.5

(23.0 - 24.1)

Retail-sliced without growth inhibitor	78.0

(76.5 - 79.6)	18.9

(18.5 - 19.3)	5.1

(5.0 - 5.2)	102.0

(100.1 - 104.0)

Subtotal: Prepackaged	10.5

(10.3 - 10.8)	2.6

(2.5 - 2.6)	0.7

(0.7 - 0.7)	13.8

(13.5 - 14.1)

Subtotal: Retail-sliced	96.0

(94.3 - 97.7)	23.3

(22.9 - 23.7)	6.2

(6.1 - 6.3)	125.6

(123.4 - 127.7)

Subtotal: With growth inhibitor	21.3

(20.8 - 21.8)	5.3

(5.1 - 5.4)	1.4

(1.3 - 1.4)	27.9

(27.3 - 28.6)

Subtotal: Without growth inhibitor	85.2

(83.6 - 86.8)	20.7

(20.3 - 21.0)	5.6

(5.5 - 5.6)	111.4

(109.4 - 113.4)

Total	106.5

(104.7 - 108.3)	25.9

(25.5 - 26.3)	6.9

(6.8 - 7.0)	139.3

(137.1 - 141.6)

Table   SEQ Table \* ARABIC  8 . Estimated mean number of illnesses from L. monocytogenes in deli meat per year among three populations stratified by age and four deli meat categories.

Deli Meat Category	Elderly

(95% CI)	Intermediate Age (95% CI)	Neonatal

(95%CI)	All Age Groups

(95% CI)

Prepackaged with growth inhibitor	12.4

(12.1 - 12.8)	9.3

(9.0 - 9.6)	2.8

(2.7 - 2.8)	24.5

(23.9 - 25.2)

Prepackaged without growth inhibitor	26.5

(25.8 - 27.2)	19.9

(19.4 - 20.5)	5.8

(5.7 - 5.9)	52.2

(50.9 - 53.5)

Retail-sliced with growth inhibitor	66.5

(64.8 - 68.1)	50.1

(48.8 - 51.4)	14.4

(14.2 - 14.7)	131.0

(127.9 - 134.2)

Retail-sliced without growth inhibitor	288.7

(283.0 - 294.4)	213.6

(209.5 - 217.7)	64.7

(63.8 - 65.6)	567.0

(556.5 - 577.6)

Subtotal: Prepackaged	38.9

(38.1 - 39.8)	29.2

(28.6 - 29.9)	8.6

(8.5 - 8.7)	76.8

(75.1 - 78.4)

Subtotal: Retail-sliced	355.2

(348.9 - 361.5)	263.7

(259.3 - 268.2)	79.1

(78.1 - 80.2)	698.0

(686.4 - 709.6)

Subtotal: With growth inhibitor	78.9

(77.1 - 80.7)	59.4

(58.0 - 60.8)	17.2

(16.9 - 17.5)	155.5

(152.1 - 159.0)

Subtotal: Without growth inhibitor	315.2

(309.3 - 321.1)	233.5

(229.3 - 237.8)	70.5

(69.5 - 71.5)	619.2

(608.2 - 630.2)

Total	394.1

(387.4 - 400.8)	292.9

(288.3 - 297.6)	87.7

(86.6 - 88.8)	774.8

(762.5 - 787.1)

The estimated number of deaths was summed across each age group for each simulation. A histogram and cumulative density plot of the estimated number of deaths between retail-sliced and prepackaged product are shown in   REF _Ref197486738  \* MERGEFORMAT  Figure 6  and   REF _Ref197486755  \* MERGEFORMAT  Figure 7 , respectively.

Figure   SEQ Figure \* ARABIC  6 . Histograms of estimated number of deaths per annum for retail-sliced and prepackaged product based on the 4,000 dose-response simulations of the FDA-FSIS model.

Figure   SEQ Figure \* ARABIC  7 . Cumulative density plots estimated number of deaths per annum for retail-sliced and prepackaged product based on the 4,000 dose-response simulations of the FDA-FSIS model.

To evaluate better if the estimated mean number of deaths among the different scenarios were statistically different, a bootstrap analysis comparing the means of the scenarios was undertaken. One hundred thousand samples (with replacement) were sampled from the 4,000 simulations of each specified scenario. The mean of each of these 100,000 samples was then calculated. This process was repeated 100,000 times to generate a distribution of means. The mean and 95% confidence interval from this distribution was then obtained. (Sensitivity analysis indicated that even the 2.5th and 97.5th % quantiles had stabilized with 100,000 runs.) Recall that these simulations were based on the starting L. monocytogenes distributions at retail for either the retail-sliced or prepackaged. Uncertainty about these distributions was not included. Thus, the resulting confidence intervals are narrower and more likely to find statistical differences than if the initial distributions included uncertainty as well. These results are presented in Table 18.

Table 18. Statistical comparison of mean number of estimated deaths by processing type.

Scenario	Mean	LCL* (2.5%)	UCL** (97.5%)

Prepackaged	13.8	13.5	14.1

Retail-sliced	125.6	123.4	127.7

Difference in means (retail-sliced – prepackaged)	111.8	109.6	114.0

*LCL=lower confidence level about the mean.

**UCL=upper confidence level about the mean.

The 95% confidence interval for the difference in means does not include 0. Thus, the difference in means is statistically significant from 0 at 95% confidence. Using the fractions of each product in Table 16 and an annual number of deli meat servings of 2.84x109, 1.78x1010, and 5.95x106 for elderly, intermediate, and perinatal, the estimated deaths per serving are shown in Table 19. The number of servings is taken from FDA-FSIS (2003). The perinatal values are based on the intermediate number of servings corrected for a pregnancy rate of 0.0174 and an exposure period of 7 days / 365 days.

Table 19. Estimated mean number of deaths per serving among the three age groups and four deli meat categories.

Food Category	Elderly	Intermediate	Perinatal

Prepackaged with antimicrobial growth inhibitor	3.67x10-9	1.43x10-10 1.15x10-7

Prepackaged without antimicrobial growth inhibitor	1.75x10-8	6.88x10-10 5.33x10-7

Retail-sliced with antimicrobial growth inhibitor	2.37x10-8	9.33x10-10 7.16x10-7

Retail-sliced without antimicrobial growth inhibitor	1.03x10-7	3.97x10-9 3.21x10-6

Conclusions

Based on this analysis, RTE meat and poultry products sliced at retail are approximately 9 times more risky on an annual basis than prepackaged product. If consumers store retail-sliced product for shorter periods than prepackaged product, this ratio might be overstated.

The analysis described in this report indicates the need for two types of data. First, environmental/ecological data are needed to indicate occurrence and origin of L. monocytogenes at retail. Second, consumer handling data are needed for how consumers treat RTE product sliced at retail versus prepackaged product sliced at the manufacturing facility.

Appendix I: L. monocytogenes in Ready-to-eat Meat and Poultry Deli meat

The presence and level of L. monocytogenes in ready-to-eat (RTE) meat and poultry products at was determined using data from a study conducted by the National Alliance for Food Safety and Security (NAFFS) (Draughon, 2006).

Data Collection Methods

The sampling group was comprised of four designated sites in the Foodborne Disease Active Surveillance Network (FoodNet). These were Northern California (CA), Georgia (GA), Minnesota (MN), and Tennessee (TN). Sampling was weighted by the populations in counties (http://www.census.gov) so that exposure could be estimated. Approximately 75% of shopping is done at major supermarket chains and 25% is done at other grocers, such as independent retailers (Gombas et al. 2003). The number of samples collected from supermarkets versus independent retailers was weighted accordingly. Also, >50% of consumers purchase RTE meat products that are sliced at delicatessens with the remainder purchasing sliced prepackaged products. USDA data suggest that approximately 47% of RTE deli meat is sliced at the processing plant and prepackaged. The relative number of samples between prepackaged and retail-sliced was therefore kept approximately equal as part of the sampling design. Sample data were encoded by the researchers to prevent identification of the store.

Approximately 2,000 samples (125 grams each) were analyzed from each of the four designated sites, with approximately equal numbers of samples sliced at retail versus sliced at a processing establishment and a smaller number of intact product samples. The sampling protocol was designed to allow for statistically valid comparisons among sites, RTE products type, and retail-sliced versus prepackaged, assuming an α = 0.05 and a 90% power of detecting a difference of 2% in the comparison of binomial proportions.

The following product types were sampled: cured poultry, uncured poultry, pork, and beef. Analysis of approximately 1,000 samples of each product type was done to support conclusions at the desired level of certainty. Use of any antimicrobial or growth inhibiting agents was noted at the time of sample collection.

Intact samples were collected by purchasing whole, intact hams, roast beefs, turkey rolls, etc. These large pieces of cooked meat are commonly referred to as “logs” or “chubs.” Each chub tends to weigh between 10 and 20 pounds. A random number table was used to choose five 35-gram core samples from each intact chub. The core samples were then tested for presence and level of L. monocytogenes.

Specific instructions were provided for sample collectors, including the product category, the number of samples of each type of product to be obtained, size of the sample to be purchased, and how to choose, collect, hold and transport the sample. Sample collection was standardized to maintain consistency.

Sampling and laboratory analyses followed standard laboratory practices. This included temperature monitoring during shipment, chain of custody documentation, aseptic transfer and handling within the laboratory, and initiating analyses within 24 hours of receipt of sample. The laboratories were instructed to discard any sample with package damage such that the microbiological integrity of the sample was not compromised. Samples not meeting quality control requirements were noted and discarded. The FSIS standard laboratory method for L. monocytogenes detection was implemented by the laboratories for use in this study. Presence/absence for L. monocytogenes was determined by inoculation in UVM broth followed by Fraser broth then modified Oxford (MOX) agar. Positive samples were quantified using a FSIS protocol 9-tube Most Probable Number (MPN) method with a reported detection limit of 0.3 MPN/gram.

NAFSS research laboratories, as approved by FSIS, were experienced in detecting L. monocytogenes in food. Samples were assigned codes and the following product information recorded: sampling location (FoodNet site along with producer information, retailer’s name, and location of purchase), date of receipt at the laboratory, whether the sample appeared to be packaged in-store or prepackaged, and the use-by or sell-by date. Any store information or identifiers were removed prior to transfer to FSIS.

Statistical Analyses

Statistical analyses were performed using Number Cruncher Statistical Systems (NCSS) 2001 (Hintz, 2001) and R version 2.6.1 (R Development Core Team, 2007). For statistical tests, p values less then 0.05 were considered statistically significant, and p values between 0.05 and 0.10 were considered marginally significant.

Data were analyzed in a variety of ways. The prevalence of L. monocytogenes among product samples sliced at retail and those that were prepackaged were analyzed by sampling site, product type, store type, time of day (morning or afternoon), and quarter of the year using tests of proportions. The null hypothesis for this test is that all the prevalences are equal. The alternative hypothesis is that at least one prevalence differs from some other. This type of statistical test assumes independence among the samples, an assumption that is not likely met for these data. Because multiple samples were collected at the same store, multiple positive L. monocytogenes findings are likely to be correlated because of cross-contamination and poor hygienic conditions at the store. Statistical tests with correlated positive samples would, on average, claim to find statistically significant results more commonly than intended.

Tests of proportions were also conducted at the retail store level. A store was considered positive for retail-sliced or prepackaged if any of the samples for that category were found positive for L. monocytogenes. Stores are much more likely to be independent, but serious problems arise from this approach as well. Store identifiers (even arbitrary labels) were removed from data provided prior to submittal to FSIS as part of the data encoding and blinding process. Store visits were therefore estimated based on date and time of sampling collection. A second problem was that sample collection times were not provided for samples from Minnesota, thus the number of stores available was much smaller than the number of samples. Statistical tests based on only a few hundred samples lack sufficient statistical power and are unlikely to detect small differences in prevalence rates at reasonable levels of confidence. Finally, this approach does not directly incorporate the number of samples collected at each store.

The final approach used was a logistic regression that predicts the store prevalence for retail-sliced and prepackaged product as a function of indicator variables: where the product was sliced, the store type, and the time of day the sample was collected. Because it is based on store prevalence, this approach is not subject to the correlation problem. The regression was weighted by the number of samples taken at the store, and evaluated more than one explanatory variable simultaneously.

Study Results

Prevalence and Number of Samples

Fifty-seven samples were found to be positive for L. monocytogenes resulting in an overall prevalence rate of 0.76%. Two of these positives were found in chub samples, six were found in prepackaged samples, and the remaining 49 positives were found in retail-sliced samples. The number of prepackaged and retail samples across the four FoodNet sites is shown in Table 9.

Table   SEQ Table \* ARABIC  9 . Prevalence of product samples1 and stores visited based on sampling locations.

Category	Sampling Locations (Site)

	CA	GA	MN	TN

Number of product samples2	0.74%

(10/1360)	0.60%

(12/2000)	0.95%

(16/1685)	0.85%

(17/1995)

Estimated number of stores sampled3	6.98%

(6/86)	4.93%

(7/142)	n/a3	10.23%

(9/88)

1  Product samples from each store include those sliced and packaged at retail and those sliced and packaged by the manufacturer.

2  Chub data are not included. Number of positive samples and total number of samples are given in parentheses.

3   Store visit estimated based on similar sampling date and time. No sample times were provided for MN, so estimate of stores sampled was not available.

Slightly fewer product samples were taken in CA than other sites. More stores were sampled in GA than other sites. In addition to prepackaged and retail-sliced product samples, 105 and 300 additional chub samples were collected in MN and TN respectively. Assuming independence, a test of proportions indicated no statistically significant difference for the prevalence within product samples among the four sites (p = 0.75). Neither was there any statistical difference for the store prevalence across the sites (p = 0.31). This allowed for pooling of the data for purposes of discussing total prevalence.

The number and prevalence for retail-sliced and prepackaged samples by quarter of the year is shown in Table 10. More product samples and more stores were visited in the 3rd quarter than in other quarters. Assuming independence, a test of proportions indicated a statistically significant difference for the prevalence within product samples (p = 0.01) but not store prevalence (p = 0.31).

Table   SEQ Table \* ARABIC  10 . Prevalence of product samples (retail-sliced, prepackaged) and stores visited based on quarter of year.

Category	Quarter of Year

	1st 	2nd 	3rd 	4th

Number of product samples	0.16%

(2/1275)	0.74%

(13/1746)	1.15%

(28/2430)	0.76%

(12/1589)

Estimated number of stores sampled1	2.63%

(2/76)	7.37%

(7/95)	5.34%

(7/131)	10.00%

(6/60)

1Store visit estimated based on similar sampling date and time. No sample times were provided for MN, so product samples include MN but stores sampled do not. Chub data are not included.

The number and prevalence for retail-sliced and prepackaged samples by time of day is shown in Table 11. Slightly more product samples and stores were sampled in the afternoon. Assuming independence, a test of proportions indicated a statistically significant difference for the prevalence within product samples (p = 0.04) but not store prevalence (p = 0.75).

Table   SEQ Table \* ARABIC  11 . Prevalence of product samples (retail-sliced, prepackaged) and stores visited based on time of day (AM versus PM).

Category	Time of Day

	AM	PM

Number of product samples1	0.51%

(13/2540)	1.04%

(32/3060)

Estimated number of stores sampled1	5.42%

(9/166)	6.81%

(13/191)

1Store visit estimated based on similar sampling date and time. No sample times were provided for MN, so neither product samples nor stores sampled include MN. Chub data are not included.

The more interesting time of day analysis looked solely at retail-sliced product as shown in Table 12. Retail-sliced product samples collected in the afternoon were more than twice as likely to test positive for L. monocytogenes – 1.92% versus 0.92%. Assuming independence, this difference was statistically significant (p = 0.04). While the store prevalences were also higher in the afternoon (7.83% versus 5.80%), the differences were not statistically significant (p = 0.64).

Table   SEQ Table \* ARABIC  12 . Prevalence of only retail-sliced product and stores visited based on time of day (AM versus PM).

Category	Time of Day

	AM	PM

Number of product samples1	0.92%

(12/1307)	1.92%

(31/1612)

Estimated number of stores sampled1	5.80%

(8/138)	7.83%

(13/166)

1Store visit estimated based on similar sampling date and time. No sample times were provided for MN, so neither product samples nor stores sampled include MN.

The number and prevalence for retail-sliced and prepackaged samples is shown in Table 13. As designed, more product samples were collected at major grocery chains. Assuming independence, a test of proportions found a marginal statistically significant difference for the prevalence within product samples (p = 0.07) but not store prevalence (p = 0.82).

Table   SEQ Table \* ARABIC  13 . Prevalence of product samples (retail-sliced, prepackaged, and chubs) and stores visited based on store type (major grocery chain versus other grocers).

Category	Store Type2

	A	B

Number of product samples1	0.64%

(31/4801)	1.10%

(24/2186)

Estimated number of stores sampled1	5.58%

(11/197)	6.71%

(11/164)

1Store visit estimated based on similar sampling date and time. No sample times were provided for MN, so product samples include MN but stores sampled do not.

2A represents major grocery chains. B represents other grocers.

Product samples were collected from prepackaged product, from product sliced at retail delis, and a limited number from intact chubs collected at retail. The number of RTE product samples by location of slicing is shown in Figure 8. A total of 3,518 retail-sliced samples, 3,522 prepackaged samples, and 405 chub samples were collected. A given chub may have been sampled multiple times making the number of unique chubs uncertain.

Figure   SEQ Figure \* ARABIC  8 . Number of RTE samples by location of slicing (source).

The data also indicate that deli meat sliced at retail is more likely to be contaminated than prepackaged deli meat (1.39% versus 0.17%). The results are shown in Figure 9. Assuming independence, a test of proportions between retail and prepackaged prevalence indicated retail-sliced deli meat had a statistically significant higher prevalence (p < 0.0001).

Figure   SEQ Figure \* ARABIC  9 . Prevalence of L. monocytogenes in deli meat by location of slicing.

The site and slicing location results for sliced deli meat only are shown in Table 14. Chub results are not included. The striking difference in prevalence between retail-sliced versus prepackaged is evident at all sites. Differences among the sites are relatively minor.

Table   SEQ Table \* ARABIC  14 . Prevalence of L. monocytogenes in retail-sliced and prepackaged deli meat by site. The number of positive samples and the total number of samples are shown in parentheses.

	Site

	CA	GA	MN	TN	Overall

         Processing

Retail-sliced	1.3%

(12/929)	1.4%

(10/731)	1.4%

(12/841)	1.5%

(15/1017)	1.4%

(49/3518)

Prepackaged	0.0%

(0/1071)	0.0%

(0/629)	0.5%

(4/844)	0.2%

(2/978)	0.2%

(6/3522)

Overall	0.6%

(12/2000)	0.7%

(10/1360)	0.9%

(16/1685)	0.9%

(17/1995)	0.8%

(55/7040)

Note:  Chub data are not included.

For the 362 stores identified across the three sites available (CA, GA, TN) retail-sliced deli meat was sampled at 308 stores and prepackaged deli meat was sampled at 313 stores. For most stores, both types of deli meat was collected – 259 of these stores had both retail-sliced and prepackaged samples collected, 49 had only retail-sliced samples collected, and 54 had only prepackaged sliced samples collected. The testing results showed that only one store had positives samples for both retail-sliced and prepackaged deli meat. An additional 20 of the stores had positive retailed-sliced samples, and one store had positive prepackaged deli meat only.

Histograms of the number of retail-sliced and prepackaged deli meat samples taken at each store are shown in Figure 10. For retail-sliced deli meat, the number of deli meat samples per store ranged from 1 to 30, with a median of 8. The 25th and 75th% quantiles were 6 and 10 respectively. For prepackaged deli meat, the number of deli meat samples ranged from 1 to 24, with a median of 9. The 25th and 75th% quantiles were 6 and 11, respectively.

Figure   SEQ Figure \* ARABIC  10 . Number of deli meat samples collected per store. MN data are not included because stores could not be identified.

Some differences existed among the different sites for labeling types of deli meats. After correcting for obvious misspellings and accounting for multiple orderings, the types of deli meats listed in the data were: beef, beef/chicken/pork, beef/chicken/turkey, beef/pork, beef/pork/turkey, bologna, chicken, chicken/pork, chicken/turkey/pork, ham, mixed,  pork, pork/turkey, poultry, poultry (chicken), poultry (chicken/pork), poultry (chicken/pork/beef), poultry (turkey), poultry (turkey/pork), and roast beef.

Many categories of deli meat types had very few samples. For purposes of this analysis, these categories were combined into 5: beef, bologna, pork, poultry, or mixed. Deli meat labeled as “bologna” was classified into different product types. If labeled by the sampler as “beef bologna,” it was categorized as beef. If labeled with mixed components, it was categorized as mixed. If labeled simply as bologna, it was categorized as bologna. Deli meat listed as poultry but containing mixed components was categorized as mixed. For example, the samples labeled “poultry (chicken/pork)” were categorized as mixed. Based on this categorization, the counts by product type are given in Figure 11.

Figure   SEQ Figure \* ARABIC  11 . Number of RTE samples by deli meat type. Chub data are not included. One sample (not shown) did not include any listing for deli meat type.

The prevalence of L. monocytogenes across the different deli meat types is shown in Figure 12. Although it appears that beef has a slightly higher prevalence, the differences were not statistically significant based on a test of proportions (p = 0.22) among the five different deli meat types (beef, bologna, mixed, pork, poultry). The corresponding L. monocytogenes prevalence for beef, bologna, mixed meat, pork, poultry deli meats were 1.28%, 0.31%, 0.44%, 0.87%, and 0.65%, respectively. There does not appear to be any difference in the prevalence of L. monocytogenes based on whether the deli meat was cured or uncured. A similar test was conducted for retail-sliced only deli meat samples with similar results. Overall, there was no statistically significant difference in the prevalence of L. monocytogenes among the different deli meat types (p = 0.43)

Figure   SEQ Figure \* ARABIC  12 . Prevalence of L. monocytogenes in RTE deli meats by deli meat type. Chubs were not included.

Samplers were asked to identify if the sample included an antimicrobial formulation. Of the 7,446 samples, 51 were identified as using an antimicrobial agent, 1,008 did not use an antimicrobial agent, and 6,387 were blank. Antimicrobial agents listed included potassium lactate, sodium diacetate, sodium erythorbate, calcium lactate, sodium phosphate, sodium benzoate, ascorbic acid, sodium citrate, and citric acid. Of the 57 samples positive for L. monocytogenes, 6 listed sodium erythorbate use, 1 listed sodium lactate/sodium diacetate use, and 50 were blank.

There is an indication that positive retail-sliced samples were clustered by store when positive L. monocytogenes results were found. Figure 13 illustrates the deli meat sample prevalence for retail-sliced deli meat among the 21 stores with at least one positive result. Three of these stores had 50% or greater prevalence, and six of these stores had greater than 30% prevalence. Of the 308 identified stores sampled for retail-sliced deli meat, 37 L. monocytogenes positive deli meat samples were found among 22 stores. The remaining positive samples were from MN, where individual stores could not be identified. Six of these stores accounted for 21 of the 37 positive samples found. Thus, it appears that a few retail stores accounted for most of the positive deli meat samples found. This finding is indicative of cross contamination at the retail establishment. It is also the reason that the independence assumption of the test of proportions for deli meat samples is likely not completely valid.

Figure   SEQ Figure \* ARABIC  13 . Prevalence of L. monocytogenes in RTE deli meats samples sliced at retail. The estimated store visit was based on similar sampling date and time. No sample times were provided for MN; thus, MN data not included. Thirty-seven total deli meat samples are shown.

Logistic Regression

To overcome the limitations with the test of proportions used above (non-independence for deli meat samples and small sample size for store samples), a logistic regression was performed. Logistic regression is appropriate when the dependent variable represents a proportion of positive results such as the deli meat prevalence for retail-sliced deli meat at an individual store. The assumptions for standard linear regression are not valid not here: the dependent variable is bounded to fall between 0 and 1, the errors are not normally distributed, and the regression must be weighted by the sample size used to calculate the prevalence. Logistic regression transforms the prevalence to a scale more suitable for regression. The analysis was performed in R using the generalized linear model (glm). In the language of R, a binomial family was specified which used the logit transformation as the link function.

The prevalence of retail-sliced and prepackaged deli meat was calculated separately for each store. This prevalence was regressed against several indicator variables: processing type (retail-sliced versus prepackaged), time of day, and store type. Retail-sliced and prepackaged prevalences from the same store were treated as independent. Given that only one store had both processing types found positive, this seemed a reasonable approach. The number of samples of each type was used to weight the regression. (Thus, store prevalences with only one sample received less weight than store samples with 30 samples.) The logistic regression approach also had the advantage that all three explanatory variables were included simultaneously.

The regression function was

where: logit() = the logit transformation function; prevalence = the deli meat sample prevalence for each store and processing type (retail-sliced versus prepackaged); processing type = 0/1 indicator variable with 0 for prepackaged and 1 for retail-sliced; store type = 0/1 indicator variable with 0 for type A stores (major grocery chains) and 1 for type B stores (other grocery stores); and time of day = 0/1 indicator variable with 0 for AM and 1 for PM.

The number of data points used in the regression was 613. This is less than twice the number of individual stores sampled (2*362=724) because not all stores had both retail-sliced and prepackaged samples collected.

The results for the parameter estimates are given in Table 15. The variables processing type and store type are statistically significant. The time of day the sample was collected is marginally significant.

Table   SEQ Table \* ARABIC  15 . Results of logistic regression for store prevalence as function of processing type, store type, and time of day indicator variables. Data for MN not included. N=613.

Parameter	Estimate	Standard Error	Z value	p

Intercept	-7.96	0.76	-10.39	<0.0001

Processing type	2.90	0.73	4.00	<0.0001

Store type	0.99	0.33	3.03	0.002

Time of day	0.59	0.35	1.68	0.093

As expected from examining the data, whether the sample was prepackaged versus retail-sliced was strongly statistically significant. This is consistent with the test of proportions for deli meat samples. The result for time of day is consistent with the deli meat sample test of proportions for time of day. Both results indicate marginal statistical significance.

Figure 14 illustrates the results using a logistic regressions based on one explanatory variable at a time as the explanatory variable. Because the vast majority of points had 0 prevalence and only two values (0/1) were used for the explanatory variables, a small random number was added to the (x,y) coordinate for each point in order to better illustrate the density of points at 0 prevalences.

Figure   SEQ Figure \* ARABIC  14 . Graphical display of logistic regression results using deli meat sample prevalence at individuals stores as the dependent variable. MN data not included.

Comparison of Findings of the National Alliance for Food Safety and Security with those of the Food Processors’ Association

A comparison of NAFSS retail contamination findings with those of the National Food Processors Association (now Food Products Association)  (Gombas et al. 2003) is enlightening, although keep in mind that sample collection methods, sample sizes and analyses methods differed and these can all affect the results. The total number of deli meat samples was roughly equivalent:  Gombas et al. sampled approximately 9,000 deli meat samples compared to about 7,000 (excluding chubs) for this research. The split between retail-sliced and prepackaged was somewhat different however. Approximately 77% of the samples from Gombas et al. were prepackaged, versus approximately 50% for this work. USDA/FSIS data suggest that approximately 47% of RTE deli meat is sliced at the processing plant and prepackaged.

Gombas et al. found retail-sliced and prepackaged prevalences of 2.7% and 0.4% respectively. This research found prevalences lower by about a factor of 2: 1.4% and 0.2% respectively. This may indicate improvements in deli meat handling, increased use of post-processing lethality and antimicrobial growth inhibitor, or other improvements at the processing plant or retail between when the studies were conducted.

The earlier research found a difference in prevalence between their two sampled sites.   REF _Ref197160220  \* MERGEFORMAT  Table 16  below shows the derived results. Compare these data to the corresponding Table 6 above for the more recent data. Whereas this work found a consistent prevalence across all sites and a significant difference between retail-sliced versus prepackaged, the earlier work found no difference in processing type at one site and a statistically significant difference at another.

Table   SEQ Table \* ARABIC  16 . Prevalence of L. monocytogenes in sliced deli meat by site and processing type from the Food Products Association (Gombas et al. 2003).

	Site

	CA	MD	Overall

         Processing1	Retail-sliced	0.70%	4.2%	2.7%

	Prepackaged	0.55%	0.19%	0.4%

	Overall	0.6%

(28/4600)	1.2%

(54/4599)	0.9%

(82/9199)

The number of positive samples and the total number of samples are shown in parentheses where available.

Gombas et al. also found that the prevalence was higher for retail-sliced deli meat, but that the levels for positives were actually higher for prepackaged deli meat. This current work found consistently that both the prevalence and levels were higher for retail-sliced deli meat compared to prepackaged.

Conclusions

  REF _Ref197160152 \h  \* MERGEFORMAT  Table 17  summarizes the results of all the statistical testing. RTE deli meat is more contaminated with L. monocytogenes, both in terms of prevalence and level, when sliced at retail than when prepackaged. The marginal statistical link between positive results and time of day as well as the clustering according to the store where the sample was collected is an indication that cross contamination within retail establishments is occurring. There was no significant difference in prevalence of L. monocytogenes among the various four FoodNet sites.

Table   SEQ Table \* ARABIC  17 . Overall results of statistical tests for prevalence of L. monocytogenes on RTE meat and poultry deli meats by location, season, time of day for slicing at retail, and by deli meat type.

Variable	Statistical Test1

	Deli meat samples2	Stores3	Logistic regression4

Geographic location	N

(p=0.75)	N

(p=0.31)

	Quarter of year	Y

(p=0.01)	N

(p=0.31)

	Time of Day	Y

(p=0.04)	N

(p=0.75)	M

(p=0.093)

Time of day (retail-sliced only)	Y

(p=0.04)	N

(p=0.64)

	Store Type	M

(p=0.07)	N

(p=0.82)	Y

(p=0.002)

Prepackaged versus retail-sliced	Y

(p<0.0001)

Y

(p<0.0001)

Deli meat Type

	N

(p=0.22)

Deli meat Type (retail-sliced only)	N

(p=0.43)

1 Chub data were not included in any of the analyses. Statistical test results were considered statistically significant if ( < 0.05 and marginal if 0.05 ( ( ( 0.10. A “Y” indicates the differences were statistically significant; an N” indicates that they were not; an “M” indicates that the differences were marginally significant. The exact p values for the test result are given in parentheses below.

2 Deli meat samples were assumed independent for the purposes of the test of proportions. In practice, because multiple samples were collected from the same store, samples were not independent. Thus, the test of proportions is more likely to erroneously claim a statistically significant result than the choice of ( would indicate.

3 A store was considered positive if at least one of the deli meat samples collected at the store was positive for L. monocytogenes.

4 All three explanatory variables were included simultaneously.

Appendix II: Sensitivity Analysis

This appendix reports the results of conducting sensitivity analyses on two major model inputs: the time-temperature storage by consumers for retail-sliced versus prepackaged deli meat and the interpretation of “shelf life” for regulatory purposes under the Interim Final Rule. The shelf life time affects the growth rates of Listeria between deli meats with and without growth inhibitors. All analyses were conducted on calibrated mode for the dose-response modeling for 390 deaths across all food groups.

STORAGE TIME-TEMPERATURE SENSITIVITY

One of the identified concerns for the comparative risk assessment was the assumption that consumers treat plant-sliced and retail sliced product similarly in the home. This assumption has been analyzed two ways. The first simply assumes that the storage time for retail-sliced product was some fraction of the storage time distribution used in the FDA 2003 model.

Consumer storage times used in the exposure assessment model were taken from a consumer survey conducted by the American Meat Institute (AMI) (2001). Results of the survey suggest that approximately 40% of ready-to-eat product is stored for less than 3 days, and another 45% of product is stored from 4 to 7 days. A total of 96% of product is stored for less than 14 days. The previous analysis used this same storage time distribution for both retail-sliced and prepackaged product. Consumers may store retail-sliced deli meats for shorter periods than prepackaged deli meats. Thus, to assess the effect of a reduced consumer storage time, the storage time distribution in the retail exposure model was adjusted by arbitrary factors of 0.25, 0.50, and 0.75. The results in terms of the number of deaths and illnesses are shown in Table 18. The ratio of deaths caused by retail-sliced versus prepackaged product is shown in Figure 15. The comparative risk ratio decreased as the consumer storage times for the retail-sliced meats decreased; however, retail-sliced product is estimated to cause 1.7 times more deaths than prepackaged product even when stored for a quarter of the time. All else being equal, if consumers store retail sliced deli meat for only 25% of the time that they store prepackaged deli meat, retail sliced product still causes a greater number of deaths than prepackaged product.

Table   SEQ Table \* ARABIC  18 . Estimated mean number of deaths and illnesses per annum by fraction of consumer storage time.

Storage Time Fraction	25%	50%	75%	100%

Deaths	70.7	105.5	127.1	139.3

Illnesses	397.8	589.9	708.0	774.7

Ratio of Deaths, Retail-sliced: Prepackaged	1.7	3.7	5.4	9.1

Figure   SEQ Figure \* ARABIC  15 . Relative ratio of deaths differing storage times between retail-sliced and prepackaged product.

The second time/temperature analysis was based on a national survey of U.S. adults using a Web-enabled panel survey approach. The survey was conducted by RTI International, Tennessee State University, and Kansas State University. The purpose of the survey was to characterize home storage and refrigeration practices for a variety of refrigerated ready-to-eat (RTE) foods and consumers' knowledge and use of open date statements among pregnant women, seniors, and the remaining population. A description of the survey and an analysis of the data are given by Cates et al. (2006). The study design, the survey questionnaire, the data dictionary and row data (Microsoft Excel format) are available at   HYPERLINK "http://www.foodrisk.org/"  http://www.foodrisk.org/ . Note that the survey asked consumers how long the packaged was stored until the product was consumed. The reported storage times represent the time for the last serving, but some product would normally be consumed prior to this. Because the same question was used for both prepackaged and retail sliced product, and because longer storage times represent the greater risk, the reported storage times for the last serving were used to compare the two product types.

For storage time, this analysis found a statistically significant difference between retail-sliced versus prepackaged product. Both storage time distributions could be fit by Weibull distributions as shown in Table 19.

Table   SEQ Table \* ARABIC  19 . Fitted Weibull distributions according to the deli meat category

Deli Meat Category	N	Shape	Scale

Retail-sliced	443	1.830	7.777

Prepackaged	387	1.137	             18.390

These different distributions are shown graphically in Figure 16. Note the long tail for the prepackaged product storage time.

Figure   SEQ Figure \* ARABIC  16 . Relative frequency of storage time for retail-sliced versus prepackaged product.

These distributions were used to calculate the storage time for the cumulative probabilities used in the FDA-FSIS model. The results, along with the existing FDA-FSIS times, are shown in Table 20 and Figure 17.

Table   SEQ Table \* ARABIC  20 . Storage times for retail-sliced and prepackaged product.

Cumulative Probability	Consumer Storage Time (d)

	FDA-FSIS	Retail-Sliced	Prepackaged

0	0.0	0.00	0.00

0.39	2.0	5.29	9.89

0.84	5.5	10.83	31.33

0.91	9.0	12.57	39.83

0.96	12.5	14.73	51.42

0.97	18.0	15.44	55.44

0.99	26.0	17.92	70.46

0.999	45.0	22.36	100.64

Figure   SEQ Figure \* ARABIC  17 . Storage time for retail-sliced and prepackaged product.

Two points stand out from this analysis. The first is that storage time for retail-sliced product is significantly shorter than for prepackaged product. The median storage time for retail-sliced product is 6.4 days, while for prepackaged product is 13.3 days. The second point is that both storage distributions are longer than the distribution used in the original FDA-FSIS model.

The same survey data were used to analyze storage temperatures. These data were fit by a logistic distribution with location parameter of 40.15 and scale parameter of 3.193. The sample size was 2,037. The FDA-FSIS model uses 939 temperature measurements as inputs, not a probability distribution. The temperature for a given run is then sampled from these values. To run the model with the new temperature distribution, R was used to generate 939 random numbers from the fitted logistic distribution. The relative frequencies are shown in Figure 18. The newer logistic distribution is slightly less peaked and has a longer tail toward higher temperatures. Storage temperatures were assumed the same regardless of whether the product was retail-sliced or prepackaged.

Figure   SEQ Figure \* ARABIC  18 . Relative frequencies for storage temperature.

The estimated mean number of deaths per year associated with prepackaged product was 34.1, and the estimated mean number of deaths per year associated with retail-sliced product was 166.9, with an estimated total annual number of deaths equal to 201.0 (Table 21). All of these values are higher than the corresponding numbers for the original analysis because the storage times were longer. There were 139.3 deaths estimated for the original analysis. Seventeen percent of the estimated per annum deaths (34.1/201.0 = 16.96%) are attributable to prepackaged product, while the remaining 83% are attributable to retail-sliced product (166.9/201.0 = 83.03%). The relative risk on a per annum basis for deli meats sliced at retail versus sliced in plants is thus 166.9/34.1 = 4.89. Corresponding results for estimated illnesses are in Table 22.

Table   SEQ Table \* ARABIC  21 . Estimated mean number of deaths per year from L. monocytogenes in deli meat among three populations stratified by age and four deli meat categories using the alternative storage time-temperature data.

Deli Meat Category	Elderly

(95% CI)	Intermediate Age

(95% CI)	Perinatal

(95% CI)	All Age Groups (95% CI)

Prepackaged with growth inhibitor	8.1

(7.9, 8.3)	1.9

(1.9, 2.0)	0.5

(0.5, 0.5)	10.5

(10.3, 10.8)

Prepackaged without growth inhibitor 	18.1

(17.7, 18.6)	4.4

(4.3, 4.5)	1.1

(1.1, 1.1)	23.6

(23.0, 24.2)

Retail-sliced with growth inhibitor	20.4

(19.9, 20.9)	4.9

(4.7, 5.0)	1.3

(1.2, 1.3)	26.5

(25.9, 27.2)

Retail-sliced without growth inhibitor	108.2

(106.4, 109.9)	25.4

(25.0, 25.8)	6.7

(6.7, 6.8)	140.3

(138.1, 142.6)

Subtotal: Prepackaged	26.2

(25.7, 26.8)	6.3

(6.2, 6.4)	1.6

(1.6, 1.6)	34.1

(33.4, 34.9)

Subtotal: Retail-sliced	128.6

(126.7, 130.5)	30.3

(29.9, 30.7)	8.0

(7.9, 8.1)	166.9

(164.5, 169.3)

Subtotal: With growth inhibitor	28.5

(27.9, 29.1)	6.8

(6.6, 6.9)	1.8

(1.7, 1.8)	37.1

(36.3, 37.8)

Subtotal: Without growth inhibitor	126.3

(124.4, 128.1)	29.8

(29.4, 30.3)	7.8

(7.7, 7.9)	163.9

(161.6, 166.3)

Total	154.8

(152.7, 156.9)	36.6

(36.2, 37.1)	9.6

(9.5, 9.7)	201.0

(198.4, 203.6)

Table   SEQ Table \* ARABIC  22 . Estimated mean number of illnesses per year from L. monocytogenes in deli meat among three populations stratified by age and four deli meat categories.

Deli Meat Category	Elderly

(95% CI)	Intermediate Age

(95% CI)	Perinatal

(95% CI)	All Age Groups (95% CI)

Prepackaged with growth inhibitor	30.0

(29.2, 30.8)	21.7

(21.1, 22.3)	6.5

(6.4, 6.6)	58.2

(56.7, 59.7)

Prepackaged without growth inhibitor 	67.1

(65.4, 68.7)	49.6

(48.4, 50.9)	13.7

(13.4, 13.9)	130.4

(127.3, 133.5)

Retail-sliced with growth inhibitor	75.5

(73.7, 77.3)	54.9

(53.6, 56.3)	16.0

(15.7, 16.2)	146.4

(143.0, 149.8)

Retail-sliced without growth inhibitor	400.2

(393.6, 406.8)	287.5

(282.8, 292.1)	85.6

(84.5, 86.6)	773.2

(761.1, 785.4)

Subtotal: Prepackaged	97.1

(95.1, 99.1)	71.3

(69.8, 72.9)	20.2

(19.8, 20.5)	188.6

(184.7, 192.4)

Subtotal: Retail-sliced	475.7

(468.7, 482.7)	342.4

(337.6, 347.2)	101.5

(100.4, 102.6)	919.6

(906.8, 932.4)

Subtotal: With growth inhibitor	105.5

(103.4, 107.7)	76.6

(75.0, 78.2)	22.5

(22.1, 22.8)	204.6

(200.5, 208.6)

Subtotal: Without growth inhibitor	467.3

(460.3, 474.2)	337.1

(332.3, 341.9)	99.2

(98.1, 100.3)	903.6

(890.9, 916.4)

Total	572.8

(565.1, 580.5)	413.7

(408.5, 418.9)	121.7

(120.5, 122.9)	1108.2

(1094.4, 1122.1)

SHELF LIFE SENSITIVITY

The second sensitivity analysis was conducted on the definition of shelf life under the Interim Final rule. This allows RTE producers to qualify for one of the categories as using growth inhibitors, and is used within the risk assessment to define the growth rates between product with and without growth inhibitor.

Maintaining the FDA-FSIS Listeria model’s assumed consumer storage distribution, the exponential growth rate (EGR) of Listeria on deli meat was then changed for both retail-sliced and prepackaged product based on a shelf life of 10, 14, and 21 days. The resulting EGRs are shown in Table 23. Increasing the shelf life decreased the number of deaths (Table 24). Recall that a maximum two-log growth is allowed during the shelf life to qualify to growth inhibitor classification. Thus, a longer shelf life requires a lower growth rate for product with growth inhibitor. The change in shelf life from 10 days to 14 days resulted in a 10% reduction in the mean number of deaths. A weeklong extension of shelf life from 14 days to 21 days resulted in a 5% reduction in the number of deaths. This suggests that the assumption of a 14-day shelf life may be adequate for predicting the number of deaths or illnesses due to Listeria.

Table   SEQ Table \* ARABIC  23 . EGR for product with and without growth inhibitor by shelf life.

Shelf Life	10 day	14 day	21 day

With growth inhibitor	0.20	0.14	0.10

Without growth inhibitor	0.30	0.31	0.32

Table   SEQ Table \* ARABIC  24 . Mean number of deaths and illnesses per annum by shelf life.

Shelf Life	10 day	14 day	21 day

Deaths	155.1	139.3	131.7

Illnesses	861.1	774.7	732.9

Ratio of Deaths, Retail-sliced: Prepackaged	8.1	9.1	7.9

The comparative risk ratios (Table 23 and Figure 19) exhibited no definitive correlation with the change in shelf life; however, the EGR for product with growth inhibitor consistently decreased, while the EGR for product without growth inhibitor increased as the shelf life increased (Table 23). The differences in the comparative risk may be a result of the iterative process used to adjust the dose-response curve and may not necessarily indicate a true difference in the relative risk.

Figure   SEQ Figure \* ARABIC  19 . Relative ratio of deaths between retail-sliced versus prepackaged product for differing definitions of shelf life.

RELATIVE IMPACTS OF MODEL VARIABLES

Using the original comparative risk assessment model results, statistical analyses were conducted to elucidate the relative importance of model inputs. A recursive partitioning and regression tree was generated in R to determine which factor (age, slicing location, or growth inhibitor use) had the greatest effect on the number of resulting deaths (Figure 20). The first division in the tree indicates that age is the most important factor and that the elderly are more likely to die from listeriosis than either the neonatal or the intermediate population. Following the tree along the elderly branch, the next division is by slicing location. The tree indicates that retail-sliced product is at greater risk for causing listeriosis than prepackaged product. Finally, the retail-sliced product is divided according the growth inhibitor use.

Figure   SEQ Figure \* ARABIC  20 . Recursive partitioning and regression tree.

Using the data from all 4,000 simulations, box plots were generated for each deli meat category by age group (Figure 21). The box plots reemphasize the effect of age on the risk of death from listeriosis, with the elderly population having the highest number of deaths for each of the deli meat categories. Within each age group, growth inhibitor reduced the number of deaths; however, the box plots show that even with the use of growth inhibitor, retail-sliced deli meats result in a greater risk of death due to listeriosis than prepackaged meats.

Figure   SEQ Figure \* ARABIC  21 . Box plots for each deli meat category by age group. Prepack = prepackaged, Retail = retail-sliced, W = with growth inhibitor, WO = without growth inhibitor.

As seen in the box plots, each of the four deli meat categories follows a similar trend, with the elderly age group at the highest risk for death. An interaction plot for the elderly age group was created to compare the effect of growth inhibitor use and product slicing location on the mean number of deaths. There is a significant difference between the mean number of deaths resulting from retail-sliced product when compared to prepackaged product (Figure 22a). While the use of growth inhibitor greatly decreased the mean number of deaths resulting from retail sliced product, prepackaged product without growth inhibitor results in fewer deaths than retail sliced product with growth inhibitor (Figure 22b).

Figure   SEQ Figure \* ARABIC  22 . Interaction plots comparing the effect of growth inhibitor (GI) use and slicing location on the mean number of deaths from listeriosis.

References

American Meat Institute. 2001. Consumer Handling of RTE meats. (Unpublished data submitted to Docket No. 99N-1168 as part of the FDA-FSIS risk assessment.)

Cates, S.C., R.A. Morales, S.A. Karns, L.A. Jaykus, K.M. Kosa, T. Teneyck, C.M. Moore, P. Cowen. 2006. Consumer knowledge, storing, and handling practices regarding Listeria in frankfurters and deli meats: results of a Web-based survey. Journal of Food Protection. 69(7):1630-39.

CDC. 2007. Preliminary FoodNet Data on the Incidence of Infection with Pathogens Transmitted Commonly Through Food --- 10 States, 2006. Morbidity and Mortality Weekly Report 56(14):336-339.

Draughon, A.F. 2006. A collaborative analysis/risk assessment of Listeria monocytogenes in ready-to-eat processed meat and poultry collected in four FoodNet states. Symposium S-16: Contamination of Ready-to-Eat Foods: Transfer and Risk: Listeria monocytogenes and Other Microorganisms. International Association for Food Protection 93rd Annual Meeting, Calgary, Alberta. August 13-16.

FDA-FSIS. 2003. Quantitative assessment of relative risk to public health from foodborne Listeria monocytogenes among selected categories of ready-to-eat foods. Washington, DC. Available at http://www.foodsafety.gov/~dms/lmr2-toc.html.

FSIS. 2003. Risk assessment for Listeria monocytogenes in deli meat. Washington, DC. Available at:  http://www.fsis.usda.gov/PDF/Lm_Deli_Risk_Assess_Final_2003.pdf.

Gombas, D.E., Y. Chen, R.S. Clavero, V.N. Scott. 2003. Survey of Listeria monocytogenes in ready-to-eat foods. Journal of Food Protection 66:559-69.

Hintz, J. 2001. NCSS and Pass. Number Cruncher Statistical Systems. Kaysville, UT.

Lianou, A., and J.N. Sofos. 2007. A review of the incidence and transmission of Listeria monocytogenes in ready-to-eat products in retail and food service environments. Journal of Food Protection 70:2172-2198.

Mead, P.S., Slutsker, L., Dietz, V., et al. 1999. Food-related illness and death in the United States. Emerging Infectious Disease 5:607–625.

R Development Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2007. ISBN 3-900051-07-0. Available at: http://www.R-project.org.

van Belle, G. 2002. Statistical rules of thumb. Wiley-Interscience, New York.

 Risk Assessment Division, Office of Public Health Science, Food Safety and Inspection Service, U.S. Department of Agriculture, Washington DC.

 Department of Civil & Environmental Engineering, Virginia Tech University, Blacksburg, VA.

 For our purpose, meat and poultry are considered together when discussing deli products (i.e., deli meat refers to any product containing beef, pork and/or poultry).

 Note that the lognormal and lognormal (base 10) are equivalent.

 See, for example, van Belle 2002.

 FSIS’ interim final rule for the control of L. monocytogenes during processing (“Control of Listeria Monocytogenes in Ready-to-Eat Meat and Poultry Products,” 68 FR 34208; June 6, 2003 (revised January 1, 2006); 9 CFR 430).

 9 CFR 430 provides requirements for the FSIS’ interim final rule, “Control of Listeria Monocytogenes in Ready-to-Eat Meat and Poultry Products,” (68 FR 34208; June 6, 2003 (revised January 1, 2006)).

 The risk assessment calibration mode used was set to 390 deaths across all food groups. This number may be lower today given the increased use of post-processing lethality and antimicrobial growth inhibitors compared to the when the original FDA-FSIS model was developed.

 Estimated based on industry survey data collected with USDA/FSIS Form 10,240-1, Production Information on Post-Lethality Exposed Ready-to-Eat Products, gathered in July 2007 in accordance with 9 CFR 430.4(d). See   REF _Ref197225400  \* MERGEFORMAT  Table 6  above for details.

 Chub data were not included in this test of proportions.

 Because of the large number of blanks, this antimicrobial formulation data was not used as part of the risk assessment described below. Instead, USDA data on current industry practices were used to estimate the fraction of product with antimicrobial usage.

 The statistical term for this is jitter.

 Estimated based on industry survey data collected with USDA/FSIS Form 10,240-1, Production Information on Post-Lethality Exposed Ready-to-Eat Products, gathered in July 2007 in accordance with 9 CFR 430.4(d). See   REF _Ref197225400  \* MERGEFORMAT  Table 6  above for details.

DRAFT FSIS Comparative Risk Assessment for Listeria monocytogenes in Ready-to-eat Meat and Poultry Deli Meats Report

 PAGE   48

 PAGE   19

This information is distributed solely for the purpose of pre-dissemination peer review under applicable information quality guidelines. It has not been formally disseminated by FSIS. It does not represent and should not be construed to represent any Agency determination or policy.

Draft – Do not Cite or Quote

DRAFT FSIS Comparative Risk Assessment for Listeria monocytogenes in Ready-to-eat Meat and Poultry Deli Meats Report

This information is distributed solely for the purpose of pre-dissemination peer review under applicable information quality guidelines. It has not been formally disseminated by FSIS. It does not represent and should not be construed to represent any Agency determination or policy.

						 PAGE   20

DRAFT FSIS Comparative Risk Assessment for Listeria monocytogenes in Ready-to-eat Meat and Poultry Deli Meats Report

 PAGE   52

„š

^„š

„š

^„š

h("

h("

h("

h("

„@

„Ð^„@

„@

^„@

m

Â

J

”

-

h”

h”

h”

h”

&

;

<

=

>

J

K

L

f

g

h

j

k

l

m

n

o

‹

Œ

Ž

ž

Ÿ

º

»

¼

¿

À

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

(

)

*

D

E

F

G

H

I

J

K

L

h

i

h”

h”

 h”

ᄀ脈栖ඔqࡕ洁H渄H甄Ĉᄔ脈栖ඔq䡭Ѐ䡮Ѐࡵᴁi

j

k

r

s

t

Ž

‘

’

“

”

•

–

²

³

´

µ

Ö

ç

h”

 h”

h”

h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

h”

j¬

 h”

h”

h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

 h”

h”

h”

h”

h”

 hqo

hÚL

hn

h:

 h:

h:

h:

h:

 h:

h:

h:

h:

h:

h…

h…

h

Ć

Ć

Ć

$

$

$

$

h

$

$

$

$

$

$

$

$

$

$

$

$

$

$

$

$

$

$

hW

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

hW

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

h

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

&4

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

&4

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

&4

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

kd

׺⎂٠׺Č

Ć

؀ᶈČ

Ć

\

$

 hn

 hn

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

hn

hn

hn

 h'

h'

h'

h'

h'

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

Ć

Ć

 hi&

" hq

& hi&

# hq

Ć

Ć

  hq

# hi&

  hi&

# hq

 hq

ô

ô

ô

ô

  hq

# hq

 hq

 hi&

 hq

" hq

}

}

阂l㐃ԁᣖČ

Č

紀섋ؑ䬘逞․紆

Ȁ沖ԀᣖČ

Č

紀섋ؑ䬘逞$紆

Ȁ沖ԀᣖČ

Č

紀섋ؑ䬘逞$紆

" hq

# hq

  hq

# hq

 hq

 hq

阂l㐃ԁᣖČ

Č

Ȁ沖ԀᣖČ

Č

$

  hq

# hq

 hq

 hq

" hq

Ȁ沖ԀᣖČ

Č

$

  hq

  hq

  hq

# hq

 hq

 hq

% hq

" hq

Ȁ沖ԀᣖČ

Č

$

u

v

x

܀

r

v

w

x

€

–

—

˜

™

š

¥

¨

Ç

Ð

Ò

Ó

,

-

8

9

<

=

x

Ò

Ó

;

<

>

¥

¦

=

>

E

F

\

]

^

_

`

o

€

ƒ

Œ

¤

¥

×

à

ù

# hq

& hq

& hq

ù

û

 hq

 hq

Ć

Ć

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

$

Ȁ沖ԀᣖČ

Č

Ć

Ć

Ȁ

Ć

Ć

h-

Ć

Ć

Ć

Ć

Ć

h-

h-

”ÿY

&Å

Ć

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

ЀL摧皔%

Ć

Ć

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

h("

 hn

 hn

h("

  not represent and should not be construed to represent any Agency determination or policy.

Outputs

■ Annual Illnesses and Deaths