Bypassing intrusive cookie statement with requests library
<p>I'm trying to crawl a website using the <code>requests</code> library. However, the particular website I am trying to access (<a href="http://www.vi.nl/matchcenter/vandaag.shtml" rel="nofollow">http://www.vi.nl/matchcenter/vandaag.shtml</a>) has a very intrusive cookie statement.</p>

<p>I am trying to access the website as follows:</p>

<pre><code>from bs4 import BeautifulSoup as soup
import requests
website = r"http://www.vi.nl/matchcenter/vandaag.shtml"
html = requests.get(website, headers={"User-Agent": "Mozilla/5.0"})
htmlsoup = soup(html.text, "html.parser")
</code></pre>

<p>This returns a web page that consists of just the cookie statement with a big button to accept. If you try accessing this page in a browser, you find that pressing the button redirects you to the requested page. How can I do this using <code>requests</code>?</p>

<p>I considered using <code>mechanize.Browser</code> but that seems a pretty roundabout way of doing it.</p>