<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Data Import - Testing
Class definitions
module DataImport
We want to import data directly from the ECB data warehouse, so for example rather than going to the series we want to download the csv data. In fact, the ECB provides three different download format (two csv's, one generic and one for Excel) and one XML download.
There is is also an sdmx query facility that allows more granula control over what data will be downloaded.
The URI's are as follows (most also allow https)
Step1:  module PDataFrame
that's a little side project that creates persistent data frames
Step2:  Testing
Bookmarks
that's a little side project, which is to create a file containing bookmarks for interesting series in the ECB database; it uses the PDataFrame class defined above.
Note that the following lines can generally be commented out
Step3:  just to check what bookmarks we have defined...
Step4:  ...and how to get the values back
Step5:  Data
we fetch three data series, the ECB deposit facility, the ECB lending facility, and the ECB marginal lending facility using the fetch method that takes as parameter the series key (see below and explanation for the skip_end parameter)
Step6:  the dataset returned contains a number of additonal info items, for example a description
Step7:  The time information is in a funny format eg (eg, "2008w21"). So we then reformat the datatables into something that can be plotted, ie a float. For this we have the static method data_table that takes the data and a reformatting function for the time. Normally it returns a 2-tuple, the first component being the time-tuple, the second component being the value-tuple
If desired, additionally an interpolation function can be return as the third component. This is necessary if we want to do operations on series that are not based on the same time values. We see this in the last line below
Step8:  The functions for converting time are implemented as static methods on the object. For the time being there are two of them
Step9:  We now can plot the data series. Note that that would not have been that trivial to do in Excel because one of the data series is monthly, the other one is weekly
<ASSISTANT_TASK:>
Python Code:
#!wget https://www.dropbox.com/s//DataImport.py -O DataImport.py
import DataImport as di
#help('DataImport')
Explanation: Data Import - Testing
Class definitions
module DataImport
We want to import data directly from the ECB data warehouse, so for example rather than going to the series we want to download the csv data. In fact, the ECB provides three different download format (two csv's, one generic and one for Excel) and one XML download.
There is is also an sdmx query facility that allows more granula control over what data will be downloaded.
The URI's are as follows (most also allow https):
human readable series
~~~
http://sdw.ecb.europa.eu/quickview.do?SERIES_KEY=-key-
~~~
csv file (generic and Excel format)
~~~
http://sdw.ecb.europa.eu/quickviewexport.do?SERIES_KEY=-key-&type=csv
http://sdw.ecb.europa.eu/quickviewexport.do?SERIES_KEY=-key-&type=xls
~~~
sdmx file
~~~
http://sdw.ecb.europa.eu/quickviewexport.do?SERIES_KEY=-key-&type=sdmx
~~~
sdmx query and endpoint
~~~
http://sdw.ecb.europa.eu/quickviewexport.do?SERIES_KEY=-key-&type=sdmxQuery
http://sdw-ws.ecb.europa.eu/
~~~
End of explanation
#!wget https://www.dropbox.com/s//PDataFrame.py -O PDataFrame.py
import PDataFrame as pdf
#help('PDataFrame')
Explanation: module PDataFrame
that's a little side project that creates persistent data frames
End of explanation
#pdf.PDataFrame.create('DataImport.csv', ('key', 'description'))
#bm = pdf.PDataFrame('DataImport.csv')
#bm.set('deposit', ('ILM.W.U2.C.L022.U2.EUR', 'current usage of the deposit facility'))
#bm.set('lending', ('ILM.M.U2.C.A05B.U2.EUR', 'current aggregate usage of major lending facilities'))
#bm.set('lending_marg', ('ILM.W.U2.C.A055.U2.EUR', 'current usage of the marginal lending facility')
Explanation: Testing
Bookmarks
that's a little side project, which is to create a file containing bookmarks for interesting series in the ECB database; it uses the PDataFrame class defined above.
Note that the following lines can generally be commented out: the whole idea here is that the bookmarks are kept in persistant storage (here the file ECB_DataSeries.csv) so one only has to execute bm.set() once to add a new bookmark (provided the csv file is being moved around with this note book)
End of explanation
bm = pdf.PDataFrame('DataImport.csv')
bm._df
Explanation: just to check what bookmarks we have defined...
End of explanation
bm.get('deposit', 'key')
Explanation: ...and how to get the values back
End of explanation
ei = di.ECBDataImport()
deposit = ei.fetch(bm.get('deposit', 'key'), skip_end=10)
lending = ei.fetch(bm.get('lending', 'key'))
lending_marg = ei.fetch(bm.get('lending_marg', 'key'))
Explanation: Data
we fetch three data series, the ECB deposit facility, the ECB lending facility, and the ECB marginal lending facility using the fetch method that takes as parameter the series key (see below and explanation for the skip_end parameter)
End of explanation
deposit.keys()
deposit['descr']
Explanation: the dataset returned contains a number of additonal info items, for example a description
End of explanation
unit = 1000000
dp = ei.data_table(deposit, ei.time_reformat1, unit)
le = ei.data_table(lending, ei.time_reformat2, unit, True)
lm = ei.data_table(lending_marg, ei.time_reformat1, unit)
diff = le[2](dp[0]) - dp[1]
Explanation: The time information is in a funny format eg (eg, "2008w21"). So we then reformat the datatables into something that can be plotted, ie a float. For this we have the static method data_table that takes the data and a reformatting function for the time. Normally it returns a 2-tuple, the first component being the time-tuple, the second component being the value-tuple
If desired, additionally an interpolation function can be return as the third component. This is necessary if we want to do operations on series that are not based on the same time values. We see this in the last line below: le[2] is the interpolation function for the lending, and it is applied to dp[0] which are the time values for the deposit function. Now the two series are on the same basis and can hence be substracted (note that in fetch() we needed the skip_end parameter, because the available deposit data series goes further than the available lending series, which makes the interpolation fail).
End of explanation
ei.time_reformat1("2010w2")
ei.time_reformat2("2010mar")
Explanation: The functions for converting time are implemented as static methods on the object. For the time being there are two of them
End of explanation
plot(le[0], le[1])
plot(lm[0], lm[1])
plot(dp[0], dp[1])
plot(dp[0], diff)
Explanation: We now can plot the data series. Note that that would not have been that trivial to do in Excel because one of the data series is monthly, the other one is weekly
End of explanation
<END_TASK>