<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Data Exploration
This notebook will perform exploratory analysis on the european soccer dataset before new feature creation.
Additional exploration of new features is located within the feature creation notebook.
Step1:  First step is to read in the csv files created by the extraction notebook
Step2:  Lets remove any variables from matches df that we won't need for this analysis
Step3:  'Stage' variable must mean 'week' of the season. Each 'stage' consists of 10 matches. This is a way to group matches by date.
Step4:  Now let's check for missing values
Step5:  Many of the betting odds have null values. Let's remove the columns that have excessive nulls.
Step6:  From 'null_dict' object, only the attribute 'buildUpPlayDribbling' numeric attribute has null values.
Step7:  Because players will not be used in this version of the model, we will not explore these attributes.
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
Explanation: Data Exploration
This notebook will perform exploratory analysis on the european soccer dataset before new feature creation.
Additional exploration of new features is located within the feature creation notebook.
End of explanation
matches = pd.read_csv('/Users/mtetkosk/Google Drive/Data Science Projects/data/processed/EPL_matches.csv')
print len(matches)
print matches.head()
matches.columns[:11]  # Columns 1 - 10 identify the match and the number of goals scored by each team
matches.columns[85:]   # Columns 85 - 115 are betting odds from different websites
matches.columns[11:55]  # Columns 11-55 are (X,Y) coordinates for players on the pitch - Describing formation
matches.columns[55:85] # Columns 55 - 77 give the player names. Columns 77-84 give some statistics based on the match.
Explanation: First step is to read in the csv files created by the extraction notebook
End of explanation
matches_reduced = matches.copy()
removecols = matches.columns[11:85]
removecols_other = ['country_id','league_id']
for col in matches_reduced.columns:
    if col in removecols or col in removecols_other:
        del matches_reduced[col]  
print matches_reduced.shape   #Reduced from 115 columns to 106 columns
matches_reduced.season.value_counts()   #Equal numer of matches per-season
# What does the 'stage' variable mean?
matches_reduced[matches_reduced.season=='2008/2009'].stage.value_counts()
Explanation: Lets remove any variables from matches df that we won't need for this analysis
End of explanation
matches_reduced.head()
Explanation: 'Stage' variable must mean 'week' of the season. Each 'stage' consists of 10 matches. This is a way to group matches by date.
End of explanation
null_dict = {}
for col in matches_reduced.columns[4:]:
    nulls = matches_reduced[col].isnull().sum()
    if nulls > 0:
        null_dict[col] = nulls
null_dict
Explanation: Now let's check for missing values
End of explanation
for key in null_dict.keys():
    if null_dict[key] > 10:
        del matches_reduced[key]
        
matches_reduced.shape
matches_reduced.to_csv('/Users/mtetkosk/Google Drive/Data Science Projects/data/processed/EPL_Matches_Reduced.csv',index= False)
team_attributes = pd.read_csv('/Users/mtetkosk/Google Drive/Data Science Projects/data/processed/EPL_team_attributes.csv')
print len(team_attributes)
print team_attributes.head()
team_attributes['date']
team_attributes.columns
null_dict = {}
for col in team_attributes.columns[4:]:
    nulls = team_attributes[col].isnull().sum()
    if nulls > 0:
        null_dict[col] = nulls
    
    if team_attributes[col].dtype == 'int64' or team_attributes[col].dtype == 'float64':
        team_attributes[col].plot(kind = 'hist')
        plt.xlabel(col)
        plt.title(col + 'Histogram')
        plt.show()
    elif team_attributes[col].dtype == 'object':
        team_attributes[col].value_counts().plot(kind ='bar')  #Build up play passing class value counts totals to 204, no nulls
        plt.title(col + 'Bar Chart')
        plt.show()
        
    
    
null_dict
Explanation: Many of the betting odds have null values. Let's remove the columns that have excessive nulls.
End of explanation
teams = pd.read_csv('/Users/mtetkosk/Google Drive/Data Science Projects/data/processed/EPL_teams.csv')
print len(teams)
print teams.head()
teams.head()
Explanation: From 'null_dict' object, only the attribute 'buildUpPlayDribbling' numeric attribute has null values.
End of explanation
player_attributes = pd.read_csv('/Users/mtetkosk/Google Drive/Data Science Projects/data/processed/Player_Attributes.csv')
print len(player_attributes)
print player_attributes.head()
players = pd.read_csv('/Users/mtetkosk/Google Drive/Data Science Projects/data/processed/Players.csv')
print len(players)
print players.head()
Explanation: Because players will not be used in this version of the model, we will not explore these attributes.
End of explanation
<END_TASK>