<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Tools for CSV
Step1:  Exploring
csvkit's csvlook tool gives us a way to inspect the contents of CSV files without opening them up in a program like Excel (especially when the file is so large it might crash our program)
Step2:  What we're seeing here is effectively a plain text display of our CSV data … not super pretty but faster than opening Excel. Let's make this more useful
Selecting and Slicing
We can combine this facility with the csvcut tool to see only subsets of our data. First, let's use the -n flag to print out the column headings
Step3:  We can look at just a subset of columns …
Step4:  Once you know the column names you can also use those to subset or slice
Step5:  Now combine these tools together using unix pipes
Step6:  In our case, this still doesn't look that helpful because some of these columns (like name) are really wide
Summary Statistics
The csvkit tools do offer other ways of peeking at out data that can prove useful to us &mdash; for instance through providing summary statistics.
Step7:  Super useful!
Searching
We can know use tools for searching (or "grepping") to find specific bits of information in our file &mdash; csvkit provides a csvgrep tool for doing just this
Step8:  Let's sort by the year of birth to see which of our 2005 decedents were the oldest
Step9:  Power Usage
One of the really cool features of csvkit is that it allows us to move back and forth between CSVs and relational database structures quickly and easily &mdash; and even to treat our CSVs (for the purpose of rapid exploration) as though they were databases we could run SQL queries against
Step10:  This facility is best for one-off ad hoc queries and, since csvkit is building a little in-memory database behind the scenes to make it possible, large datasets can be very slow to work with this way.
However, csvkit SQL tools also make it really easy to turn CSVs into a proper database &mdash; by creating the necessary SQL statements for you and trying to guess the correct data types for columns. csvsql will even load the data for you if you have an SQL database installed.
Step11:  Imported successfully, now we can run any SQL query we want
<ASSISTANT_TASK:>
Python Code:
import tarfile
import re
import os
from itertools import count
# You have a copy of this file in your `data` directory. Tate provides the data in a single TAR (tape archive) file
DATA_PATH = '../data/tate-collection-1.2.tar.gz'
DATA_FOBJ = tarfile.open(DATA_PATH)
# We can use Python's tools for working with tar files to inspect the data package
# For instance by listing the files it contains without unpacking it
FILES = DATA_FOBJ.getmembers()
for i, f in enumerate(FILES[:10]):
    print('{0} \t {1}'.format(i, f))
# This time we're only going to extract the CSV file
DATA_FOBJ.extractall(path='../data/tate-collection', members=FILES[0:4])
CSV_FILE_PATH = '../data/tate-collection/collection-1.2/artist_data.csv'
Explanation: Tools for CSV: csvkit
Like jq for JSON, csvkit provides a suite of tools for working with data in comma separated value (CSV) formats. Again, one of the chief virtues of the csvkit tools is the way they can be combined with other common unix tools. Also the documentation is quite good.
End of explanation
%%bash
csvlook ../data/tate-collection/collection-1.2/artist_data.csv | head -n 5
Explanation: Exploring
csvkit's csvlook tool gives us a way to inspect the contents of CSV files without opening them up in a program like Excel (especially when the file is so large it might crash our program)
End of explanation
!csvcut -n ../data/tate-collection/collection-1.2/artist_data.csv
Explanation: What we're seeing here is effectively a plain text display of our CSV data … not super pretty but faster than opening Excel. Let's make this more useful
Selecting and Slicing
We can combine this facility with the csvcut tool to see only subsets of our data. First, let's use the -n flag to print out the column headings
End of explanation
!csvcut -c 1,2,5,6,9 ../data/tate-collection/collection-1.2/artist_data.csv
Explanation: We can look at just a subset of columns …
End of explanation
!csvcut -c name,url ../data/tate-collection/collection-1.2/artist_data.csv
Explanation: Once you know the column names you can also use those to subset or slice:
End of explanation
!csvcut -c name,dates ../data/tate-collection/collection-1.2/artist_data.csv | csvlook | head -n 10
Explanation: Now combine these tools together using unix pipes
End of explanation
!csvcut -c name,yearOfBirth,yearOfDeath,url ../data/tate-collection/collection-1.2/artist_data.csv | \
csvstat
Explanation: In our case, this still doesn't look that helpful because some of these columns (like name) are really wide
Summary Statistics
The csvkit tools do offer other ways of peeking at out data that can prove useful to us &mdash; for instance through providing summary statistics.
End of explanation
!csvcut -c name,yearOfBirth,yearOfDeath ../data/tate-collection/collection-1.2/artist_data.csv | \
csvgrep -c yearOfBirth -m 1497 | csvlook
!csvcut -c name,yearOfBirth,yearOfDeath ../data/tate-collection/collection-1.2/artist_data.csv | \
csvgrep -c yearOfDeath -m 2005 | csvlook
Explanation: Super useful!
Searching
We can know use tools for searching (or "grepping") to find specific bits of information in our file &mdash; csvkit provides a csvgrep tool for doing just this
End of explanation
!csvcut -c name,yearOfBirth,yearOfDeath ../data/tate-collection/collection-1.2/artist_data.csv | \
csvgrep -c yearOfDeath -m 2005 | csvsort -c yearOfBirth | csvlook
Explanation: Let's sort by the year of birth to see which of our 2005 decedents were the oldest
End of explanation
!csvsql --query "select name from artist_data where yearOfBirth > 1700;" \
../data/tate-collection/collection-1.2/artist_data.csv | csvlook
Explanation: Power Usage
One of the really cool features of csvkit is that it allows us to move back and forth between CSVs and relational database structures quickly and easily &mdash; and even to treat our CSVs (for the purpose of rapid exploration) as though they were databases we could run SQL queries against
End of explanation
!csvsql -i sqlite ../data/tate-collection/collection-1.2/artist_data.csv
!csvsql --db sqlite:///tate_artists.db --insert ../data/tate-collection/collection-1.2/artist_data.csv
!sql2csv --db sqlite:///tate_artists.db --query "select * from artist_data"
Explanation: This facility is best for one-off ad hoc queries and, since csvkit is building a little in-memory database behind the scenes to make it possible, large datasets can be very slow to work with this way.
However, csvkit SQL tools also make it really easy to turn CSVs into a proper database &mdash; by creating the necessary SQL statements for you and trying to guess the correct data types for columns. csvsql will even load the data for you if you have an SQL database installed.
End of explanation
!sql2csv --db sqlite:///tate_artists.db --query "select * from artist_data where gender='Female'"
!sql2csv --db sqlite:///tate_artists.db --query "select name,dates from artist_data where gender='Female'" | \
    csvlook
Explanation: Imported successfully, now we can run any SQL query we want
End of explanation
<END_TASK>