Poisson regression

From Wikipedia, the free encyclopedia
Jump to: navigation, search



Regression models[edit]

If \mathbf{x} \in \mathbb{R}^n is a vector of independent variables, then the model takes the form

\log (\operatorname{E}(Y\mid\mathbf{x}))=\alpha + \mathbf{\beta}' \mathbf{x},

where \alpha \in \mathbb{R} and \mathbf{\beta} \in \mathbb{R}^n. Sometimes this is written more compactly as

\log (\operatorname{E}(Y\mid\mathbf{x}))=\boldsymbol{\theta}' \mathbf{x},\,



\operatorname{E}(Y\mid\mathbf{x})=e^{\boldsymbol{\theta}' \mathbf{x}}.\,


Maximum likelihood-based parameter estimation[edit]


\operatorname{E}(Y\mid x)=e^{\theta' x}\,,

and thus, the Poisson distribution's probability mass function is given by

p(y\mid x;\theta) = \frac{[\operatorname{E}(Y\mid x)]^y \times e^{-\operatorname{E}(Y\mid x)}}{y!} = \frac{e^{y  \theta' x} e^{-e^{\theta' x}}}{y!}

Now suppose we are given a data set consisting of m vectors x_i \in \mathbb{R}^{n+1}, \, i = 1,\ldots,m, along with a set of m values y_1,\ldots,y_m \in \mathbb{R}. Then, for a given set of parameters θ, the probability of attaining this particular set of data is given by

p(y_1,\ldots,y_m\mid x_1,\ldots,x_m;\theta) = \prod_{i=1}^m \frac{e^{y_i  \theta' x_i} e^{-e^{\theta' x_i}}}{y_i!}.


L(\theta\mid X,Y) = \prod_{i=1}^m \frac{e^{y_i \theta' x_i} e^{-e^{\theta' x_i}}}{y_i!}.


\ell(\theta\mid X,Y) = \log L(\theta\mid X,Y) = \sum_{i=1}^m \left( y_i \theta' x_i - e^{\theta' x_i} - \log(y_i!)\right).



To find a maximum, we need to solve an equation \frac{\partial \ell(\theta\mid X,Y)}{\partial \theta} = 0 which has no closed-form solution. However, the negative log-likelihood, -\ell(\theta\mid X,Y), is a convex function, and so standard convex optimization techniques such as gradient descent can be applied to find the optimal value of θ.

Poisson regression in practice[edit]


"Exposure" and offset[edit]

Poisson regression may also be appropriate for rate data, where the rate is a count of events occurring to a particular unit of observation, divided by some measure of that unit's exposure. For example, biologists may count the number of tree species in a forest, and the rate would be the number of species per square kilometre. Demographers may model death rates in geographic areas as the count of deaths divided by person−years. More generally, event rates can be calculated as events per unit time, which allows the observation window to vary for each unit. In these examples, exposure is respectively unit area, person−years and unit time. In Poisson regression this is handled as an offset, where the exposure variable enters on the right-hand side of the equation, but with a parameter estimate (for log(exposure)) constrained to 1.

\log{(\operatorname{E}(Y\mid x))} = \log{(\text{exposure})} + \theta' x

which implies

\log{(\operatorname{E}(Y\mid x))} - \log{(\text{exposure})} = 
       \log{\left(\frac{\operatorname{E}(Y\mid x)}{\text{exposure}}\right)} = \theta' x



Overdispersion and zero inflation[edit]




Use in survival analysis[edit]



Regularized Poisson regression[edit]


\sum_{i=1}^m \log(p(y_i;e^{\theta' x})),

where m is the number of examples in the data set, and p(y_i;e^{\theta' x}) is the probability mass function of the Poisson distribution with the mean set to e^{\theta' x}. Regularization can be added to this optimization problem by instead maximizing

\sum_{i=1}^m \log(p(y_i;e^{\theta' x})) - \lambda \left\|\theta\right\|_2^2,

for some positive constant \lambda. This technique, similar to ridge regression, can reduce overfitting.


Some statistics packages include implementations of Poisson regression.

  • Analytica: Provided by the built-in function PoissonRegression.
  • GenStat: Poisson regression is a standard option of the regression section, using the "MODEL", "FIT" and associated commands; it is also available in the "Stats > Regression Analysis > Generalized Linear Models" menu.
  • MATLAB Statistics Toolbox: Poisson regression can be performed using the "glmfit" and "glmval" functions.[4]
  • Microsoft Excel: Excel is not capable of doing Poisson regression by default. One of the Excel Add-ins for Poisson regression is XPost
  • mPlus: mPlus allows for Poisson regression using the command COUNT IS when specifying the data
  • R: The function for fitting a generalized linear model in R is glm(), and can be used for Poisson regression.[5][6]
  • SAS: Poisson regression in SAS is done by using GENMOD, HPGENSELECT, COUNTREG, GLIMMIX, and NLMIXED
  • SPSS: In SPSS, Poisson regression is done by using the GENLIN command
  • Stata: Stata has a procedure for Poisson regression named poisson,[7] and for panel data xtpoisson.[8]
  • CrimeStat: CrimeStat has Poisson, Poisson NB1, Poisson-Gamma(negative binomial), and Poisson-Lognormal regression models.
  • Python: Provided in the statsmodels library as statsmodels.discrete.discrete_model.Poisson.fit

See also[edit]


  2. ^ Paternoster R, Brame R (1997). "Multiple routes to delinquency? A test of developmental and general theories of crime". Criminology 35: 45–84. doi:10.1111/j.1745-9125.1997.tb00870.x. 
  3. ^ Berk R, MacDonald J (2008). "Overdispersion and Poisson regression" (PDF). Journal of Quantitative Criminology 24 (3): 269–284. doi:10.1007/s10940-008-9048-4. 
  4. ^ http://www.mathworks.com/help/toolbox/stats/glmfit.html
  5. ^ Dalgaard, Peter (2008). "Rates and Poisson Regression". Introductory Statistics with R. New York: Springer. pp. 259–274. doi:10.1007/978-0-387-79054-1_15. ISBN 978-0-387-79053-4. 
  6. ^ Faraway, Julian J. (2005). "Count Regression". Extending the Linear Model with R. CRC Press. pp. 61–75. ISBN 0-203-49228-5. 
  7. ^ Poisson regression
  8. ^ Fixed-effects, random-effects, and population-averaged Poisson models

Further reading[edit]