Set cookie in BeautifulSoup Python web scraper
<p>I'm trying to create a python script that goes to a web page and check if there is a div with a specified id in it and if there isn't, it tries again deleting a given cookie.</p>

<p>So far, this is my code:</p>

<pre><code>import urllib2
from BeautifulSoup import BeautifulSoup
import time

url = 'http://google.com'
cookie = 'hello'

while True:
    page = urllib2.urlopen(url).read()
    soup = BeautifulSoup(page)
    soup.prettify()
    if soup.find(id='hello'):
        print "Found!"
        break
    else:
        #DELETE THE GIVEN COOKIE AND TRY AGAIN
    time.sleep(1)
</code></pre>

<p>What I'm asking for is: <strong>How do I delete the cookie? Or I don't need to delete it because BeautifulSoup will retry the request using a different instance?</strong> </p>

<p>Also, is it possible to set things like headers, user agent etc with this method? If so, how?</p>