<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Practical use of Jupyter notebook
Second motivation 
Step1:  Expected results
Step2:  Techniques used
Regular expressions
Pythonic / Functional programming 
Step3:  Data wrangling in action
Step4:  The extracted tree still contains much noise
Step5:  Use of lambda functions and piping
Step6:  Further clean-up 
<ASSISTANT_TASK:>
Python Code:
Image("img/init.png")
Explanation: Practical use of Jupyter notebook
Second motivation : learning Python by web scraping
Scraping data from the WHO
End of explanation
Image("img/target_result.png")
Explanation: Expected results
End of explanation
# FOR WEB SCRAPING
from lxml import html
import requests
# FOR FUNCTIONAL PROGRAMMING
import cytoolz  # pipe
# FOR DATA WRANGLING
import pandas as pd # use of R like dataframes
import re #re for regular expressions
# TO INSERT IMAGES
from IPython.display import Image
Explanation: Techniques used
Regular expressions
Pythonic / Functional programming :
use lists (iterable) : 
avoid looping on indices whenever possible
list / for comprehensions
lambda expressions
pipe and map essentially : (based on cytoolz)
Python web scraping : 
lxml (Python library)
XPath (web page content)
End of explanation
### Target URL
outbreakNewsURL = "http://www.who.int/csr/don/archive/disease/zika-virus-infection/en/"
page = requests.get(outbreakNewsURL)
tree = html.fromstring(page.content)
newsXPath = '//li'
zikaNews = tree.xpath(newsXPath)
### Store the relevant news in a list
zikaNews_dirty = [p.text_content() for p in zikaNews]
# Printing the first 20 elements
zikaNews_dirty[1:20] # omitting first element
Explanation: Data wrangling in action
End of explanation
Image("img/flatten_tree_data.png")
# Extract only the items containing the pattern "Zika virus infection "
#sample= '\n22 April 2016\n\t\t\tZika virus infection – Papua New Guinea - USA\n'
keywdEN ="Zika virus infection "
zikaNews_content = [s for s in zikaNews_dirty if re.search(keywdEN, s)] 
zikaNews_content[0:10] # first 11 elements
Explanation: The extracted tree still contains much noise
End of explanation
#### Use of lambdas (avoid creating verbose Python functions with def f():{})
substitudeUnicodeDash = lambda s : re.sub(u'–',"@", s)
substituteNonUnicode = lambda s : re.sub(r"\s"," ",s)
removeSpace = lambda s: s.strip()
# Use of pipe to chain lambda functions within a list comprehension
### Should be familiar to those using R dplyr %>%
zikaNews_dirty = [cytoolz.pipe(s, 
                               removeSpace, 
                               substituteNonUnicode) 
                  for s in zikaNews_content] 
# List comprehension
zikaNews_dirty = [s.split("Zika virus infection") for s in zikaNews_dirty ]
zikaNews_dirty[0:10]
Explanation: Use of lambda functions and piping
End of explanation
# Structure data into a Pandas dataframe
zika = pd.DataFrame(zikaNews_dirty, columns = ["Date","Locations"])
zika.head(n=20)
### Removing the first dash sign / for zika["Locations"]
# Step 1 : transform in a list of strings, via str.split()
# Step 2 : copy the list, except the first element list[1:]
# Step 3 : reconstitute the entire string using ' '.join(list[1:])
# Step 1 : transform in a list of strings, via str.split()
zika["Split_Locations"] = pd.Series(zika["Locations"].iloc[i].split()  for i in range(len(zika)))
# Step 2 : copy the list, except the first element list[1:]
zika["Split_Locations"] = pd.Series([s[1:] for s in zika["Split_Locations"]])
# Step 3 : reconstitute the entire string using ' '.join(list[1:])
zika["Split_Locations"] = pd.Series([" ".join(s) for s in zika["Split_Locations"]])
zika["Split_Locations"] = pd.Series([s.split("-") for s in zika["Split_Locations"]])
zika["Split_Date"] = pd.Series([s.split() for s in zika["Date"]])
# Show the first 10 rows using HEAD
zika.head(n=10)
### Extract Day / Month / Year in the Split_Date column, 1 row is of the form [21, January, 2016]
zika["Day"]= pd.Series(zika["Split_Date"].iloc[i][0] for i in range(len(zika)))
zika["Month"]= pd.Series(zika["Split_Date"].iloc[i][1] for i in range(len(zika)))
zika["Year"]= pd.Series(zika["Split_Date"].iloc[i][2] for i in range(len(zika)))
# Show the first 10 rows using HEAD
zika.head(n=10)
# Extract Country and Territory
zika["Country"] = pd.Series(zika["Split_Locations"].iloc[i][0] for i in range(len(zika)))
zika["Territory"] = pd.Series(zika["Split_Locations"].iloc[i][len(zika["Split_Locations"].iloc[i])-1] for i in range(len(zika)))
# Show the first 20 rows using HEAD
zika[['Split_Locations','Country','Territory']].head(20)
zika["Territory"] =pd.Series(zika["Territory"][i] 
                             if zika["Territory"][i] != zika["Country"][i]  
                             else " " for i in range(len(zika))
                            )
# Show the first 20 rows using HEAD
zika[['Split_Locations','Country','Territory']].head(20)
Explanation: Further clean-up : use of the Pandas library
Use extensiveley the Pandas library
End of explanation
<END_TASK>