Working with, preparing bag-of-word data for Regression
<p>Im trying to create a regression model that predicts an authors age. Im using (Nguyen et al,2011) as my basis.</p>

<p>Using a Bag of Words Model I count the occurences of words per Document (which are Posts from Boards) and create the vector for every Post. </p>

<p>I limit the size of each vector by using as features the top-k (k=number) most frequent used words(stopwords will not be used)</p>

<pre><code>Vectorexample_with_k_8 = [0,0,0,1,0,3,0,0]
</code></pre>

<p>My data is generally sparse like in the Example.</p>

<p>When I test the model on my test data I get a very low r² score(0.00-0.1), sometimes even a negative score. The model predicts always the same age, which happens to be the average age of my dataset, like seen in the
distribution of my data (age/amount):
<a href="https://i.stack.imgur.com/Cx1bJ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Cx1bJ.png" alt="Agedemographic"></a></p>

<p>I used diffrerent Regression Models: Linear Regression, Lasso,
SGDRegressor from scikit-learn with no improvement.</p>

<p>So the questions are:</p>

<p><strong>1.How do I improve the r² score?</strong></p>

<p><strong>2.Do I have to change the data to fit the Regression better? If yes with what method?</strong> </p>

<p><strong>3.Which Regressor/Methods should I use for text classification?</strong></p>