Python 3: How can I get news articles that contain a certain keyword
<p>I'm trying to write a little web app that returns the sentiment of a news article involving a keyword.</p>

<p>I used the TextBlob and Newspaper3K python 3 packages. I tried to make the url string for Newspaper3K the result of a search query on Google News but the newspaper package just seems to redirect to the "main page" of Google News.</p>

<p>Is there any way to get a list of newspaper articles that contain a certain keyword? In addition, is it possible for newspaper to iterate through pages?</p>

<p>The following is my code:</p>

<pre><code>from textblob import TextBlob
import newspaper

#keyword = input("Please enter the keyword: ")
keyword = "Apple" #for testing only
keyword_lowercase = keyword.lower()

search_string = "" # only for google news
split_keyword = keyword.split()
for i in range(len(split_keyword)):
    search_string += split_keyword[i]
    if i != len(split_keyword)-1:
        search_string += '+'

def google_news_site(search_query):
    prefix = 'http://news.google.com/news?q='
    return prefix+search_string

#Currently for news.google.com only
url_string = google_news_site(search_string)
paper = newspaper.build(url_string, memoize_articles=False)

def sentiment(text):
    return TextBlob(text).sentiment.polarity

current_sum = 0.0
relevant_article_count = 0
for article in paper.articles:
    print(article.url)
    article_text = article.text
    article_text_lowercase = article_text.lower()
    if keyword_lowercase in article_text_lowercase:
        current_sum += sentiment(article_text)

print("Article count is", str(relevant_article_count)+".")

rating = current_sum/max(relevant_article_count, 1)
print("The rating for", keyword, "is", str(rating)+".")
</code></pre>