<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Indexing and selecting data
Step1:  Some notes on selecting data
One of pandas' basic features is the labeling of rows and columns, but this makes indexing also a bit more complex compared to numpy. We now have to distuinguish between
Step2:  or multiple columns
Step3:  But, slicing accesses the rows
Step4:  So as a summary, [] provides the following convenience shortcuts
Step5:  But the row or column indexer can also be a list, slice, boolean array, ..
Step6:  Selecting by position with iloc works similar as indexing numpy arrays
Step7:  The different indexing methods can also be used to assign data
Step8:  Boolean indexing (filtering)
Like a where clause in SQL. The indexer (or boolean mask) should be 1-dimensional and the same length as the thing being indexed.
Step9:  <div class="alert alert-success">
    <b>EXERCISE</b>
Step10:  This can then be used to filter the dataframe with boolean indexing
Step11:  Let's say we want to select all data for which the capital starts with a 'B'. In Python, when having a string, we could use the startswith method
Step12:  In pandas, these are available on a Series through the str namespace
Step13:  For an overview of all string methods, see
Step14:  How to avoid this?
Use loc instead of chained indexing if possible!
Or copy explicitly if you don't want to change the original data.
More exercises!
For the quick ones among you, here are some more exercises with some larger dataframe with film data. These exercises are based on the PyCon tutorial of Brandon Rhodes (so all credit to him!) and the datasets he prepared for that. You can download these data from here
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
try:
    import seaborn
except ImportError:
    pass
data = {'country': ['Belgium', 'France', 'Germany', 'Netherlands', 'United Kingdom'],
        'population': [11.3, 64.3, 81.3, 16.9, 64.9],
        'area': [30510, 671308, 357050, 41526, 244820],
        'capital': ['Brussels', 'Paris', 'Berlin', 'Amsterdam', 'London']}
countries = pd.DataFrame(data)
countries
countries = countries.set_index('country')
countries
Explanation: Indexing and selecting data
End of explanation
countries['area']
Explanation: Some notes on selecting data
One of pandas' basic features is the labeling of rows and columns, but this makes indexing also a bit more complex compared to numpy. We now have to distuinguish between:
selection by label
selection by position.
data[] provides some convenience shortcuts
For a DataFrame, basic indexing selects the columns.
Selecting a single column:
End of explanation
countries[['area', 'population']]
Explanation: or multiple columns:
End of explanation
countries['France':'Netherlands']
Explanation: But, slicing accesses the rows:
End of explanation
countries.loc['Germany', 'area']
Explanation: So as a summary, [] provides the following convenience shortcuts:
Series: selecting a label: s[label]
DataFrame: selecting a single or multiple columns: df['col'] or df[['col1', 'col2']]
DataFrame: slicing the rows: df['row_label1':'row_label2'] or df[mask]
Systematic indexing with loc and iloc
When using [] like above, you can only select from one axis at once (rows or columns, not both). For more advanced indexing, you have some extra attributes:
loc: selection by label
iloc: selection by position
These methods index the different dimensions of the frame:
df.loc[row_indexer, column_indexer]
df.iloc[row_indexer, column_indexer]
Selecting a single element:
End of explanation
countries.loc['France':'Germany', ['area', 'population']]
Explanation: But the row or column indexer can also be a list, slice, boolean array, ..
End of explanation
countries.iloc[0:2,1:3]
Explanation: Selecting by position with iloc works similar as indexing numpy arrays:
End of explanation
countries2 = countries.copy()
countries2.loc['Belgium':'Germany', 'population'] = 10
countries2
Explanation: The different indexing methods can also be used to assign data:
End of explanation
countries['area'] > 100000
Explanation: Boolean indexing (filtering)
Like a where clause in SQL. The indexer (or boolean mask) should be 1-dimensional and the same length as the thing being indexed.
End of explanation
s = countries['capital']
s.isin?
s.isin(['Berlin', 'London'])
Explanation: <div class="alert alert-success">
    <b>EXERCISE</b>: Add a column `density` with the population density (note: population column is expressed in millions)
</div>
<div class="alert alert-success">
    <b>EXERCISE</b>: Select the capital and the population column of those countries where the density is larger than 300
</div>
<div class="alert alert-success">
    <b>EXERCISE</b>: Add a column 'density_ratio' with the ratio of the density to the mean density
</div>
<div class="alert alert-success">
    <b>EXERCISE</b>: Change the capital of the UK to Cambridge
</div>
<div class="alert alert-success">
    <b>EXERCISE</b>: Select all countries whose population density is between 100 and 300 people/km²
</div>
Some other useful methods: isin and string methods
The isin method of Series is very useful to select rows that may contain certain values:
End of explanation
countries[countries['capital'].isin(['Berlin', 'London'])]
Explanation: This can then be used to filter the dataframe with boolean indexing:
End of explanation
'Berlin'.startswith('B')
Explanation: Let's say we want to select all data for which the capital starts with a 'B'. In Python, when having a string, we could use the startswith method:
End of explanation
countries['capital'].str.startswith('B')
Explanation: In pandas, these are available on a Series through the str namespace:
End of explanation
countries.loc['Belgium', 'capital'] = 'Ghent' 
countries
countries['capital']['Belgium'] = 'Antwerp' 
countries
countries[countries['capital'] == 'Antwerp']['capital'] = 'Brussels' 
countries
Explanation: For an overview of all string methods, see: http://pandas.pydata.org/pandas-docs/stable/api.html#string-handling
<div class="alert alert-success">
    <b>EXERCISE</b>: Select all countries that have capital names with more than 7 characters
</div>
<div class="alert alert-success">
    <b>EXERCISE</b>: Select all countries that have capital names that contain the character sequence 'am'
</div>
Pitfall: chained indexing (and the 'SettingWithCopyWarning')
End of explanation
cast = pd.read_csv('data/cast.csv')
cast.head()
titles = pd.read_csv('data/titles.csv')
titles.head()
Explanation: How to avoid this?
Use loc instead of chained indexing if possible!
Or copy explicitly if you don't want to change the original data.
More exercises!
For the quick ones among you, here are some more exercises with some larger dataframe with film data. These exercises are based on the PyCon tutorial of Brandon Rhodes (so all credit to him!) and the datasets he prepared for that. You can download these data from here: titles.csv and cast.csv and put them in the /data folder.
End of explanation
<END_TASK>