<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 bf_qt_scraping
This notebook describes how hotel data can be scraped using PyQT.
The items we want to extract are
Step1:  Now the Hotels
Step2:  Now Get the Links
Step3:  Loading Reviews
Next, we want to step through each page, and scrape the reviews for each hotel.
<ASSISTANT_TASK:>
Python Code:
import sys  
from PyQt4.QtGui import *  
from PyQt4.QtCore import *  
from PyQt4.QtWebKit import *  
from lxml import html 
class Render(QWebPage):  
    def __init__(self, url):  
        self.app = QApplication(sys.argv)  
        QWebPage.__init__(self)  
        self.loadFinished.connect(self._loadFinished)  
        self.mainFrame().load(QUrl(url))  
        self.app.exec_()  
    def _loadFinished(self, result):  
        self.frame = self.mainFrame()  
        self.app.quit() 
    
    def update_url(self, url):
        self.mainFrame().load(QUrl(url))  
        self.app.exec_()  
        
url = 'http://www.bringfido.com/lodging/city/new_haven_ct_us'  
#This does the magic.Loads everything
r = Render(url)  
#result is a QString.
result = r.frame.toHtml()
# result
#QString should be converted to string before processed by lxml
formatted_result = str(result.toAscii())
#Next build lxml tree from formatted_result
tree = html.fromstring(formatted_result)
tree.text_content
#Now using correct Xpath we are fetching URL of archives
archive_links = tree.xpath('//*[@id="results_list"]/div')
print archive_links
url = 'http://pycoders.com/archive/'  
r = Render(url)  
result = r.frame.toHtml()
#QString should be converted to string before processed by lxml
formatted_result = str(result.toAscii())
tree = html.fromstring(formatted_result)
#Now using correct Xpath we are fetching URL of archives
archive_links = tree.xpath('//*[@class="campaign"]/a/@href')
# for lnk in archive_links:
#     print(lnk)
Explanation: bf_qt_scraping
This notebook describes how hotel data can be scraped using PyQT.
The items we want to extract are:
- the hotels for a given city
- links to each hotel page
- text hotel summary
- text hotel description
Once the links for each hotel are determined, I then want to extract the following items pertaining to each review:
- title
- author
- text
- rating
End of explanation
url = 'http://www.bringfido.com/lodging/city/new_haven_ct_us'  
r = Render(url)  
result = r.frame.toHtml()
#QString should be converted to string before processed by lxml
formatted_result = str(result.toAscii())
tree = html.fromstring(formatted_result)
#Now using correct Xpath we are fetching URL of archives
archive_links = tree.xpath('//*[@id="results_list"]/div')
print(archive_links)
print('')
for lnk in archive_links:
    print(lnk.xpath('div[2]/h1/a/text()')[0])
    print(lnk.text_content())
    print('*'*25)
Explanation: Now the Hotels
End of explanation
links = []
for lnk in archive_links:
    print(lnk.xpath('div/h1/a/@href')[0])
    links.append(lnk.xpath('div/h1/a/@href')[0])
    print('*'*25)
lnk.xpath('//*/div/h1/a/@href')[0]
links
Explanation: Now Get the Links
End of explanation
url_base = 'http://www.bringfido.com'  
r.update_url(url_base+links[0])  
result = r.frame.toHtml()
#QString should be converted to string before processed by lxml
formatted_result = str(result.toAscii())
tree = html.fromstring(formatted_result)
hotel_description = tree.xpath('//*[@class="body"]/text()')
details = tree.xpath('//*[@class="address"]/text()')
address = details[0]
csczip = details[1]
phone = details[2]
#Now using correct Xpath we are fetching URL of archives
reviews = tree.xpath('//*[@class="review_container"]')
texts = []
titles = []
authors = []
ratings = []
print(reviews)
print('')
for rev in reviews:
    titles.append(rev.xpath('div/div[1]/text()')[0])
    authors.append(rev.xpath('div/div[2]/text()')[0])
    texts.append(rev.xpath('div/div[3]/text()')[0])
    ratings.append(rev.xpath('div[2]/img/@src')[0].split('/')[-1][0:1])
    print(rev.xpath('div[2]/img/@src')[0].split('/')[-1][0:1])
titles
authors
texts
ratings
Explanation: Loading Reviews
Next, we want to step through each page, and scrape the reviews for each hotel.
End of explanation
<END_TASK>