Inaccurate predictions with Poisson Regression in R
<p>I am trying to make a prediction on the number of visitors of a website based on historic data collected. I think this is a scenario in which I could use Poisson Regression.</p>

<p>The input consists of 6 columns:</p>

<p>id(the id of the website), day, month, year, day of week, visits.</p>

<p>So basically as input we have a CSV with columns in the format: "2","22", "7", "2015", "6","751".</p>

<p>I am trying to predict the visits based on previous number of visits. The size of the websites can vary, so I ended up dividing them in 5 categories</p>

<ul>
<li>almost zero (avg &lt; 1)</li>
<li>very small(avg &lt; 100)</li>
<li>small (avg &lt; 1000)</li>
<li>medium (avg &lt; 50.000)</li>
<li>big (avg &lt; 500.000)</li>
</ul>

<p>So I made a 7th column named type which is a int ranging from 1 to 5.</p>

<p>My code is as follows: </p>

<pre><code>train = read.csv("train.csv", header = TRUE)
model&lt;-glm(visits ~ type + day + month + year + dayofweek, train, family=poisson)
summary(model)
P = predict(model, newdata = train)
imp = round(P)
imp
</code></pre>

<p>The values predicted are not even close, I taught I could end up with something in 10-20% of the actual values, but failed to do so, most of the values predicted are 200-300% bigger than the actual values. And this is on the train data set, which should provide an optimistic view.</p>

<p>I am new to R and having some problems interpreting the data returned by the summary command. This is what it returns:</p>

<blockquote>
  <p><em>Call: glm(formula = visits ~ type + day + month + year + dayofweek, 
      family = poisson, data = train)</em></p>
  
  <p><em>Deviance Residuals: 
      Min       1Q   Median       3Q      Max<br>
  -571.05   -44.04   -11.33    -5.14   734.43</em>  </p>
  
  <p>Coefficients: </p>

<pre><code>            Estimate Std. Error  z value Pr(&gt;|z|)     

(Intercept) -9.998e+02  6.810e-01 -1468.19   &lt;2e-16 *** 

type         2.368e+00  1.280e-04 18498.53   &lt;2e-16 *** 

day         -2.473e-04  6.273e-06   -39.42   &lt;2e-16 *** 

month        1.658e-02  3.474e-05   477.31   &lt;2e-16 *** 

year         4.963e-01  3.378e-04  1469.31   &lt;2e-16 *** 

dayofweek   -3.783e-02  2.621e-05 -1443.46   &lt;2e-16 ***
</code></pre>
  
  <p>--- Signif. codes:  0 ‘<strong><em>’ 0.001 ‘</strong>’ 0.01 ‘</em>’ 0.05 ‘.’ 0.1 ‘ ’ 1</p>
  
  <p>(Dispersion parameter for poisson family taken to be 1)</p>
  
  <p>Null deviance: 1239161821  on 12370  degrees of freedom Residual
  deviance:  157095033  on 12365  degrees of freedom AIC: 157176273</p>
  
  <p>Number of Fisher Scoring iterations: 5</p>
</blockquote>

<p>Could anyone describe in more detail the values returned by the summary command and what should they look like in a Poisson Regression which would output better predictions? Are there any better approaches in R to a data which is based on a evolution over time of the value to be estimated?</p>

<p>LE. <a href="https://drive.google.com/file/d/0BxGxSvnNAF3MQ2lkRVcwb1R0NkU/view?usp=sharing" rel="nofollow">link to train.csv file</a>.</p>