parsing google news using beautiful soup python
<p>I have python code as below. It searches a google news page and prints hyperlinks and titles for each news. My problem is that googlenews groups news that are similar in one bucket and below script prints only 1st news in each bucket. How can I print all new from all buckets?</p>

<pre><code>from bs4 import BeautifulSoup
import requests

headers = {'user-agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36'}

#r = requests.get('http://www.aflcio.org/Legislation-and-Politics/Legislative-Alerts', headers=headers)
r = requests.get('https://www.google.com/search?q=%22eric+bledsoe%22&amp;tbm=nws&amp;tbs=qdr:d', headers=headers)
r = requests.get('https://www.google.com/search?q=%22lebron+james%22&amp;tbm=nws&amp;tbs=qdr:y', headers=headers)

soup = BeautifulSoup(r.text, "html.parser")

letters = soup.find_all("div", class_="_cnc")
#print soup.prettify() 
#print letters
print type(letters)
print len(letters)
print("\n")

for x in range(0, len(letters)):
    print x
    print letters[x].a["href"]


print("\n")

letters2 = soup.find_all("a", class_="l _HId")
for x in range(0, len(letters2)):
    print x
    print letters2[x].get_text()

print ("\n----------content")
#print letters[0]
</code></pre>

<p>By bucketing news I mean that in the below image, first few news are grouped together. The news "LeBron James compares one of his teammates to Denn" is part of another group.</p>

<p><a href="https://i.stack.imgur.com/vTbwG.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/vTbwG.png" alt="enter image description here"></a></p>