<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 数据抓取：
Beautifulsoup简介
王成军
wangchengjun@nju.edu.cn
计算传播网 http
Step1:  一般的数据抓取，使用urllib2和beautifulsoup配合就可以了。
尤其是对于翻页时url出现规则变化的网页，只需要处理规则化的url就可以了。
以简单的例子是抓取天涯论坛上关于某一个关键词的帖子。
在天涯论坛，关于雾霾的帖子的第一页是：
http
Step2:  html.parser
Beautiful Soup supports the html.parser included in Python’s standard library
lxml
but it also supports a number of third-party Python parsers. One is the lxml parser lxml. Depending on your setup, you might install lxml with one of these commands
Step3:  html
head
title
body
p (class = 'title', 'story' )
a (class = 'sister')
href/id
Select 方法
Step4:  find_all方法
Step5:  数据抓取：
根据URL抓取微信公众号文章内容
王成军
wangchengjun@nju.edu.cn
计算传播网 http
Step6:  查看源代码
Inspect
Step7:  尝试抓取阅读数和点赞数
<ASSISTANT_TASK:>
Python Code:
import urllib2
from bs4 import BeautifulSoup
Explanation: 数据抓取：
Beautifulsoup简介
王成军
wangchengjun@nju.edu.cn
计算传播网 http://computational-communication.com
需要解决的问题
页面解析
获取Javascript隐藏源数据
自动翻页
自动登录
连接API接口
End of explanation
url = 'file:///Users/chengjun/GitHub/cjc2016/data/test.html'
# http://computational-class.github.io/cjc2016/data/test.html
content = urllib2.urlopen(url).read() 
soup = BeautifulSoup(content, 'html.parser') 
soup
Explanation: 一般的数据抓取，使用urllib2和beautifulsoup配合就可以了。
尤其是对于翻页时url出现规则变化的网页，只需要处理规则化的url就可以了。
以简单的例子是抓取天涯论坛上关于某一个关键词的帖子。
在天涯论坛，关于雾霾的帖子的第一页是：
http://bbs.tianya.cn/list.jsp?item=free&nextid=0&order=8&k=雾霾
第二页是：
http://bbs.tianya.cn/list.jsp?item=free&nextid=1&order=8&k=雾霾
Beautiful Soup
Beautiful Soup is a Python library designed for quick turnaround projects like screen-scraping. Three features make it powerful:
Beautiful Soup provides a few simple methods. It doesn't take much code to write an application
Beautiful Soup automatically converts incoming documents to Unicode and outgoing documents to UTF-8. Then you just have to specify the original encoding.
Beautiful Soup sits on top of popular Python parsers like lxml and html5lib.
Install beautifulsoup4
open your terminal/cmd
$ pip install beautifulsoup4
第一个爬虫
Beautifulsoup Quick Start 
http://www.crummy.com/software/BeautifulSoup/bs4/doc/
End of explanation
print(soup.prettify())
Explanation: html.parser
Beautiful Soup supports the html.parser included in Python’s standard library
lxml
but it also supports a number of third-party Python parsers. One is the lxml parser lxml. Depending on your setup, you might install lxml with one of these commands:
$ apt-get install python-lxml
$ easy_install lxml
$ pip install lxml
html5lib
Another alternative is the pure-Python html5lib parser html5lib, which parses HTML the way a web browser does. Depending on your setup, you might install html5lib with one of these commands:
$ apt-get install python-html5lib
$ easy_install html5lib
$ pip install html5lib
End of explanation
soup.select('.sister')
soup.select('.story')
soup.select('#link2')
soup.select('#link2')[0]['href']
Explanation: html
head
title
body
p (class = 'title', 'story' )
a (class = 'sister')
href/id
Select 方法
End of explanation
soup('p')
soup.find_all('p')
[i.text for i in soup('p')]
for i in soup('p'):
    print i.text
for tag in soup.find_all(True):
    print(tag.name)
soup('head') # or soup.head
soup('body') # or soup.body
soup('title')  # or  soup.title
soup('p')
soup.p
soup.title.name
soup.title.string
soup.title.text
# 推荐使用text方法
soup.title.parent.name
soup.p
soup.p['class']
soup.find_all('p', {'class', 'title'})
soup.find_all('p', class_= 'title')
soup.find_all('p', {'class', 'story'})
soup.find_all('p', {'class', 'story'})[0].find_all('a')
soup.a
soup('a')
soup.find(id="link3")
soup.find_all('a')
soup.find_all('a', {'class', 'sister'}) # compare with soup.find_all('a')
soup.find_all('a', {'class', 'sister'})[0]
soup.find_all('a', {'class', 'sister'})[0].text
soup.find_all('a', {'class', 'sister'})[0]['href']
soup.find_all('a', {'class', 'sister'})[0]['id']
soup.find_all(["a", "b"])
print(soup.get_text())
Explanation: find_all方法
End of explanation
from IPython.display import display_html, HTML
HTML('<iframe src=http://mp.weixin.qq.com/s?__biz=MzA3MjQ5MTE3OA==&\
mid=206241627&idx=1&sn=471e59c6cf7c8dae452245dbea22c8f3&3rd=MzA3MDU4NTYzMw==&scene=6#rd\
width=500 height=500></iframe>')
# the webpage we would like to crawl
Explanation: 数据抓取：
根据URL抓取微信公众号文章内容
王成军
wangchengjun@nju.edu.cn
计算传播网 http://computational-communication.com
End of explanation
url = "http://mp.weixin.qq.com/s?__biz=MzA3MjQ5MTE3OA==&\
mid=206241627&idx=1&sn=471e59c6cf7c8dae452245dbea22c8f3&3rd=MzA3MDU4NTYzMw==&scene=6#rd"
content = urllib2.urlopen(url).read() #获取网页的html文本
soup = BeautifulSoup(content, 'html.parser') 
title = soup.select("div .title #id1 img ")
for i in title:
    print i.text
print soup.find('h2', {'class', 'rich_media_title'}).text
print soup.find('div', {'class', 'rich_media_meta_list'})
print soup.find('em').text
article = soup.find('div', {'class' , 'rich_media_content'}).text
print article
rmml = soup.find('div', {'class', 'rich_media_meta_list'})
date = rmml.find(id = 'post-date').text
rmc = soup.find('div', {'class', 'rich_media_content'})
content = rmc.get_text()
print title
print date
print content
Explanation: 查看源代码
Inspect
End of explanation
url = 'http://mp.weixin.qq.com/s?src=3&timestamp=1463191394&ver=1&signature=cV3qMIBeBTS6i8cJJOPu98j-H9veEPx0Y0BekUE7F6*sal9nkYG*w*FwDiaySIfR4XZL-XFbo2TFzrMxEniDETDYIMRuKmisV8xjfOcCjEWmPkYfK57G*cffYv4JxuM*RUtN8LUIg*n6Kd0AKB8--w=='
content = urllib2.urlopen(url).read() #获取网页的html文本
soup = BeautifulSoup(content, 'html.parser') 
print soup
soup.find(id='sg_likeNum3')
Explanation: 尝试抓取阅读数和点赞数
End of explanation
<END_TASK>