Getting "None" and 'NoneType object...' error when using BeautifulSoup4 to get Text from a webpage
<p>I'm trying to pull the main headline (currently: "Wenger predicts 'active' January") from BBC Sport page. The ID is 'lead-caption' and it's in a <code>&lt;h2&gt;</code> and an <code>&lt;a&gt;</code> tag. I'm using Python.</p>

<pre><code>from bs4 import BeautifulSoup
import urllib2
url = urllib2.urlopen("http://www.bbc.co.uk/sport/football/teams/arsenal")
soup=BeautifulSoup(url.read())
#Things I've tried
headline=soup.find('a', attrs={'id': 'lead-caption'})
print headline
#The above prints 'None'
headline1=soup.find('lead-caption').getText()
print headline1
#The above print "'NoneTpye' Object has no attirbute 'getText'
tag = soup.a
tag ['id'] = 'lead-caption'
type(tag)
print tag.string
#Error: NoneType object does not support item assignment
</code></pre>

<p>Any help would be much appreciated. Thanks :)</p>