<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 English Wikipedia View analysis for 2008-2017
The data is obtained through an API created by the wikimedia foundation enabling us to access the traffic data for mobile-web, mobile-app and desktop respectively
Step 1
Step1:  API for pagecounts initiated
Step2:  Step 2
Step3:  Giving names based on acronyms of the type of data.
Naming convention
Step4:  Filling the dataframe with values of page views and counts
To separate year and month for each csv file, created a function to do that.
Step5:  Changed all dataframes to dfs with year and month separated
Step6:  Dropping any unnecessary columns to avoid too many columns in the intermediate dataframes while merging
Making final dataframe
Step7:  Analysis
In this part, I have plotted the timeseries graph using matplotlib
<ASSISTANT_TASK:>
Python Code:
import requests
import json
import csv
import numpy as np
import pandas as pd
endpoint = 'https://wikimedia.org/api/rest_v1/metrics/pageviews/aggregate/{project}/{access}/{agent}/{granularity}/{start}/{end}'
headers={'User-Agent' : 'https://github.com/your_github_username', 'From' : 'your_uw_email@uw.edu'}
params = {'project' : 'en.wikipedia.org',
            'access' : 'mobile-web',
            'agent' : 'user',
            'granularity' : 'monthly',
            'start' : '2015070100',
            'end' : '2017101000'#use the first day of the following month to ensure a full month of data is collected
            }
api_call = requests.get(endpoint.format(**params))
response_pageviews_mobile_web = api_call.json()
with open('pageviews_mobile_web_201507_201709.json', 'w') as outfile:
    json.dump(response_pageviews_mobile_web, outfile)
    
import requests
import json
import csv
endpoint = 'https://wikimedia.org/api/rest_v1/metrics/pageviews/aggregate/{project}/{access}/{agent}/{granularity}/{start}/{end}'
headers={'User-Agent' : 'https://github.com/your_github_username', 'From' : 'your_uw_email@uw.edu'}
params = {'project' : 'en.wikipedia.org',
            'access' : 'mobile-app',
            'agent' : 'user',
            'granularity' : 'monthly',
            'start' : '2015070100',
            'end' : '2017101000'#use the first day of the following month to ensure a full month of data is collected
            }
api_call = requests.get(endpoint.format(**params))
response_pageviews_mobile_app = api_call.json()
with open('pageviews_mobile_app_201507_201709.json', 'w') as outfile:
    json.dump(response_pageviews_mobile_app, outfile)
    
import requests
import json
import csv
endpoint = 'https://wikimedia.org/api/rest_v1/metrics/pageviews/aggregate/{project}/{access}/{agent}/{granularity}/{start}/{end}'
headers={'User-Agent' : 'https://github.com/your_github_username', 'From' : 'your_uw_email@uw.edu'}
params = {'project' : 'en.wikipedia.org',
            'access' : 'desktop',
            'agent' : 'user',
            'granularity' : 'monthly',
            'start' : '2015070100',
            'end' : '2017101000'#use the first day of the following month to ensure a full month of data is collected
            }
api_call = requests.get(endpoint.format(**params))
response_pageviews_desktop = api_call.json()
with open('pageviews_desktop_201507_201709.json', 'w') as outfile:
    json.dump(response_pageviews_desktop, outfile)
    
Explanation: English Wikipedia View analysis for 2008-2017
The data is obtained through an API created by the wikimedia foundation enabling us to access the traffic data for mobile-web, mobile-app and desktop respectively
Step 1: Data acquisition
We will collect monthly data on web traffic to English Wikipedia from two Wikipedia APIs and save the data in JSON files. We will make five API calls and then save the JSON results into five separate JSON data files. We will make the API requests in python code below, but these are the complete URLs for all five API requests:
End of explanation
import requests
import json
import csv
endpoint = 'https://wikimedia.org/api/rest_v1/metrics/legacy/pagecounts/aggregate/{project}/{access}/{granularity}/{start}/{end}'
headers={'User-Agent' : 'https://github.com/your_github_username', 'From' : 'your_uw_email@uw.edu'}
params = {'project' : 'de.wikipedia.org',
            'access' : 'desktop-site',
            'granularity' : 'monthly',
            'start' : '2008060100',
            'end' : '2016060100'#use the first day of the following month to ensure a full month of data is collected
            }
api_call = requests.get(endpoint.format(**params))
response_pagecounts_desktop = api_call.json()
with open('pagecounts_desktop_200806_201606.json', 'w') as outfile:
    json.dump(response_pagecounts_desktop, outfile)
    
import requests
import json
import csv
endpoint = 'https://wikimedia.org/api/rest_v1/metrics/legacy/pagecounts/aggregate/{project}/{access}/{granularity}/{start}/{end}'
headers={'User-Agent' : 'https://github.com/your_github_username', 'From' : 'your_uw_email@uw.edu'}
params = {'project' : 'de.wikipedia.org',
            'access' : 'mobile-site',
            'granularity' : 'monthly',
            'start' : '2008010100',
            'end' : '2016060100'#use the first day of the following month to ensure a full month of data is collected
            }
api_call = requests.get(endpoint.format(**params))
response_pagecounts_mobile = api_call.json()
with open('pagecounts_mobile_200801_201606.json', 'w') as outfile:
    json.dump(response_pagecounts_mobile, outfile)
    
Explanation: API for pagecounts initiated
End of explanation
(pd.DataFrame.from_dict(data=response_pageviews_mobile_web['items'], orient='columns').
to_csv('pageviews_mobile_web_201507_201709.csv', header=True))
(pd.DataFrame.from_dict(data=response_pageviews_desktop['items'], orient='columns').
to_csv('pageviews_desktop_201507_201709.csv', header=True))
(pd.DataFrame.from_dict(data=response_pageviews_mobile_app['items'], orient='columns').
to_csv('pageviews_mobile_app_201507_201709.csv', header=True))
(pd.DataFrame.from_dict(data=response_pagecounts_desktop['items'], orient='columns').
to_csv('pagecounts_desktop_200801_201606.csv', header=True))
(pd.DataFrame.from_dict(data=response_pagecounts_mobile['items'], orient='columns').
to_csv('pagecounts_mobile_200801_201606.csv', header=True))
Explanation: Step 2: Data processing
In this step, we perform a series of processing steps on the saved JSON data files generated by the API requests. This processing is being done to prepare the data for analysis and combine the relevant features into a single CSV-formatted data file.
In the following cells, I have converted the JSON files to csv files
End of explanation
pvmw = pd.read_csv('pageviews_mobile_web_201507_201709.csv')
# To remove extra column since on doing read_csv, the column 'Unnamed: 0' is automatically added to the dataframe
del pvmw['Unnamed: 0']
pvma = pd.read_csv('pageviews_mobile_app_201507_201709.csv')
del pvma['Unnamed: 0']
pcd = pd.read_csv('pagecounts_desktop_200801_201606.csv')
del pcd['Unnamed: 0']
pcm = pd.read_csv('pagecounts_mobile_200801_201606.csv')
del pcm['Unnamed: 0']
pvd = pd.read_csv('pageviews_desktop_201507_201709.csv')
del pvd['Unnamed: 0']
Explanation: Giving names based on acronyms of the type of data.
Naming convention:  
pvmw = pageview mobile-web
  pvma = pageview mobile app
pcd = pagecount desktop
pcm = pagecount mobile
End of explanation
def change_csv(df):
    df_time = df['timestamp'].values.tolist()
    df_time = [str(x) for x in df_time]
    df_year=[]
    for i in range(0,len(df_time)):
        df_year.append(df_time[i][0:4])
    df['year'] = df_year
    df_month = []
    for i in range(0,len(df_time)):
        df_month.append(df_time[i][4:6])
    df['month'] = df_month
    del df['timestamp']
    return df
Explanation: Filling the dataframe with values of page views and counts
To separate year and month for each csv file, created a function to do that.
End of explanation
pcm = change_csv(pcm)
pcd = change_csv(pcd)
pvmw = change_csv(pvmw)
pvma = change_csv(pvma)
pvd = change_csv(pvd)
Explanation: Changed all dataframes to dfs with year and month separated
End of explanation
#Merging pagecount mobile and pagecount desktop and replacing Nan values with 0
pc = pcm.merge(pcd, on=['year','month'],how='outer')
pc= pc.fillna(0)
# To calculate total views
pc["pagecount_all_views"] = pc['count_y'] + pc['count_x']
#Merging pageview mobile and pageview desktop and replacing Nan values with 0
mo = pvmw.merge(pvma,on=['year','month'],how='outer')
mo["pageview_mobile_views"] = mo["views_x"] + mo["views_y"]  
pv = mo.merge(pvd, how='outer',on=["year", "month"])
pv = pv.fillna(0)
#To get total views
pv["pageview_all_views"] =  pv["pageview_mobile_views"] + pv["views_x"]
mo.drop(["views_x", "views_y"],axis=1,inplace=True)
#Combining both pagecount and pageview dataframes to make the final dataframe
final = pc.merge(pv, how='outer',on=["year", "month"])
final = final[["year", "month", "pagecount_all_views","count_y", "count_x","pageview_all_views", "views",
"pageview_mobile_views"]]
final = final.fillna(0)
final.to_csv('en_wikipedia_traffic_200801_201709 .csv')
Explanation: Dropping any unnecessary columns to avoid too many columns in the intermediate dataframes while merging
Making final dataframe
End of explanation
import matplotlib.pyplot as plt
import pandas as pd
import datetime as dt
#Creating a date range for the x-axis
daterange = pd.date_range('2008-01', '2017-05',freq='M')
x = daterange
pc1 = final["pagecount_all_views"]
pc2 = final["count_y"]
pc3 = final["count_x"]
pv1 = final["pageview_all_views"]
pv2 = final["views"]
pv3 = final["pageview_mobile_views"]
%matplotlib inline
plt.plot(x, pc1, label = "PageCount All", color = "black")
plt.plot(x, pc2, label = "PageCount Desktop", color = "blue")
plt.plot(x, pc3, label = "PageCount Mobile", color = "yellow")
plt.plot(x, pv1, label = "PageView All", color = "green")
plt.plot(x, pv2, label = "PageView  Desktop", color = "red")
plt.plot(x, pv3, label = "PageView Mobile", color = "orange")
plt.legend(loc=2)
plt.xlabel("Time")
plt.ylabel("Page Views (Raised to the power 9)")
plt.show()
fig.savefig("Plot.png")
Explanation: Analysis
In this part, I have plotted the timeseries graph using matplotlib
End of explanation
<END_TASK>