Web scraping - how to access content rendered in JavaScript via Angular.js?
<p>I'm trying to scrape data from the public site <a href="http://www.asx.com.au" rel="noreferrer">asx.com.au</a></p>

<p>The page <a href="http://www.asx.com.au/asx/research/company.do#!/ACB/details" rel="noreferrer">http://www.asx.com.au/asx/research/company.do#!/ACB/details</a> contains a <code>div</code> with class 'view-content', which has the information I need:</p>

<p><a href="https://i.stack.imgur.com/Siny9.png" rel="noreferrer"><img src="https://i.stack.imgur.com/Siny9.png" alt="enter image description here"></a></p>

<p>But when I try to view this page via Python's <code>urllib2.urlopen</code> that div is empty:</p>

<pre><code>import urllib2
from bs4 import BeautifulSoup

url = 'http://www.asx.com.au/asx/research/company.do#!/ACB/details'
page = urllib2.urlopen(url).read()
soup = BeautifulSoup(page, "html.parser")
contentDiv = soup.find("div", {"class": "view-content"})
print(contentDiv)

# the results is an empty div:
# &lt;div class="view-content" ui-view=""&gt;&lt;/div&gt;
</code></pre>

<p>Is it possible to access the contents of that div programmatically?</p>

<p>Edit: as per the comment it appears that the content is rendered via <code>Angular.js</code>. Is it possible to trigger the rendering of that content via Python?</p>