<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 In this Tutorial we will explore how to work with columnar data in HoloViews. Columnar data has a fixed list of column headings, with values stored in an arbitrarily long list of rows.  Spreadsheets, relational databases, CSV files, and many other typical data sources fit naturally into this format.  HoloViews defines an extensible system of interfaces to load, manipulate, and visualize this kind of data, as well as allowing conversion of any of the non-columnar data types into columnar data for analysis or data interchange.
By default HoloViews will use one of three storage formats for columnar data
Step1:  Simple Dataset
Usually when working with data we have one or more independent variables, taking the form of categories, labels, discrete sample coordinates, or bins.  These variables are what we refer to as key dimensions (or kdims for short) in HoloViews. The observer or dependent variables, on the other hand, are referred to as value dimensions (vdims), and are ordinarily measured or calculated given the independent variables. The simplest useful form of a Dataset object is therefore a column 'x' and a column 'y' corresponding to the key dimensions and value dimensions respectively. An obvious visual representation of this data is a Table
Step2:  However, this data has many more meaningful visual representations, and therefore the first important concept is that Dataset objects are interchangeable as long as their dimensionality allows it, meaning that you can easily create the different objects from the same data (and cast between the objects once created)
Step3:  Each of these three plots uses the same data, but represents a different assumption about the semantic meaning of that data -- the Scatter plot is appropriate if that data consists of independent samples, the Curve plot is appropriate for samples chosen from an underlying smooth function, and the Bars plot is appropriate for independent categories of data.  Since all these plots have the same dimensionality, they can easily be converted to each other, but there is normally only one of these representations that is semantically appropriate for the underlying data.  For this particular data, the semantically appropriate choice is Curve, since the y values are samples from the continuous function exp.
As a guide to which Elements can be converted to each other, those of the same dimensionality here should be interchangeable, because of the underlying similarity of their columnar representation
Step4:  Literals
In addition to the main storage formats, Dataset Elements support construction from three Python literal formats
Step5:  For these inputs, the data will need to be copied to a new data structure, having one of the three storage formats above.  By default Dataset will try to construct a simple array, falling back to either pandas dataframes (if available) or the dictionary-based format if the data is not purely numeric. Additionally, the interfaces will try to maintain the provided data's type, so numpy arrays and pandas DataFrames will therefore always be parsed by the array and dataframe interfaces first respectively.
Step6:  Dataset will attempt to parse the supplied data, falling back to each consecutive interface if the previous could not interpret the data. The default list of fallbacks and simultaneously the list of allowed datatypes is
Step7:  To select a particular storage format explicitly, supply one or more allowed datatypes
Step8:  Sharing Data
Since the formats with labelled columns do not require any specific order, each Element can effectively become a view into a single set of data. By specifying different key and value dimensions, many Elements can show different values, while sharing the same underlying data source.
Step9:  We can quickly confirm that the data is actually shared
Step10:  For columnar data, this approach is much more efficient than creating copies of the data for each Element, and allows for some advanced features like linked brushing in the Bokeh backend.
Converting to raw data
Column types make it easy to export the data to the three basic formats
Step11:  Pandas DataFrame
Step12:  Dataset dictionary
Step13:  Creating tabular data from Elements using the .table and .dframe methods
If you have data in some other HoloViews element and would like to use the columnar data features, you can easily tabularize any of the core Element types into a Table Element, using the .table() method.  Similarly, the .dframe() method will convert an Element into a pandas DataFrame. These methods are very useful if you want to then transform the data into a different Element type, or to perform different types of analysis.
Tabularizing simple Elements
For a simple example, we can create a Curve of an exponential function and convert it to a Table with the .table method, with the same result as creating the Table directly from the data as done earlier on this Tutorial
Step14:  Similarly, we can get a pandas dataframe of the Curve using curve.dframe().  Here we wrap that call as raw HTML to allow automated testing of this notebook, but just calling curve.dframe() would give the same result visually
Step15:  Although 2D image-like objects are not inherently well suited to a flat columnar representation, serializing them by converting to tabular data is a good way to reveal the differences between Image and Raster elements.  Rasters are a very simple type of element, using array-like integer indexing of rows and columns from their top-left corner as in computer graphics applications.  Conversely, Image elements are a higher-level abstraction that provides a general-purpose continuous Cartesian coordinate system, with x and y increasing to the right and upwards as in mathematical applications, and each point interpreted as a sample representing the pixel in which it is located (and thus centered within that pixel).  Given the same data, the .table() representation will show how the data is being interpreted (and accessed) differently in the two cases (as explained in detail in the Continuous Coordinates Tutorial)
Step16:  Tabularizing space containers
Even deeply nested objects can be deconstructed in this way, serializing them to make it easier to get your raw data out of a collection of specialized Element types. Let's say we want to make multiple observations of a noisy signal. We can collect the data into a HoloMap to visualize it and then call .table() to get a columnar object where we can perform operations or transform it to other Element types. Deconstructing nested data in this way only works if the data is homogenous. In practical terms, the requirement is that your data structure contains Elements (of any types) in these Container types
Step17:  Now we can serialize this data just as before, where this time we get a four-column (4D) table. The key dimensions of both the HoloMap and the Images, as well as the z-values of each Image, are all merged into a single table. We can visualize the samples we have collected by converting it to a Scatter3D object.
Step18:  Here the z dimension is shown by color, as in the original images, and the other three dimensions determine where the datapoint is shown in 3D. This way of deconstructing will work for any data structure that satisfies the conditions described above, no matter how nested. If we vary the amount of noise while continuing to performing multiple observations, we can create an NdLayout of HoloMaps, one for each level of noise, and animated by the observation number.
Step19:  And again, we can easily convert the object to a Table
Step20:  Applying operations to the data
Sorting by columns
Once data is in columnar form, it is simple to apply a variety of operations.  For instance, Dataset can be sorted by their dimensions using the .sort() method.  By default, this method will sort by the key dimensions, but any other dimension(s) can be supplied to specify sorting along any other dimensions
Step21:  Working with categorical or grouped data
Data is often grouped in various ways, and the Dataset interface provides various means to easily compare between groups and apply statistical aggregates. We'll start by generating some synthetic data with two groups along the x-axis and 4 groups along the y axis.
Step22:  Since there are repeat observations of the same x- and y-values, we have to reduce the data before we display it or else use a datatype that supports plotting distributions in this way. The BoxWhisker type allows doing exactly that
Step23:  Aggregating/Reducing dimensions
Most types require the data to be non-duplicated before being displayed.  For this purpose, HoloViews makes it easy to aggregate and reduce the data. These two operations are simple inverses of each other--aggregate computes a statistic for each group in the supplied dimensions, while reduce combines all the groups except the supplied dimensions. Supplying only a function and no dimensions will simply aggregate or reduce all available key dimensions.
Step24:  (A) aggregates over both the x and y dimension, computing the mean for each x/y group, while (B) reduces the x dimension leaving just the mean for each group along y.
Collapsing multiple Dataset Elements
When multiple observations are broken out into a HoloMap they can easily be combined using the collapse method. Here we create a number of Curves with increasingly larger y-values. By collapsing them with a function and a spreadfn we can compute the mean curve with a confidence interval. We then simply cast the collapsed Curve to a Spread and Curve Element to visualize them.
Step25:  Working with complex data
In the last section we only scratched the surface of what the Dataset interface can do.  When it really comes into its own is when working with high-dimensional datasets. As an illustration, we'll load a dataset of some macro-economic indicators for OECD countries from 1964-1990, cached on the HoloViews website.
Step26:  We'll also take this opportunity to set default options for all the following plots.
Step27:  Loading the data
As we saw above, we can supply a dataframe to any Dataset type. When dealing with so many dimensions it would be cumbersome to supply all the dimensions explicitly, but luckily Dataset can easily infer the dimensions from the dataframe itself. We simply supply the kdims, and it will infer that all other numeric dimensions should be treated as value dimensions (vdims).
Step28:  To get an overview of the data we'll quickly sort it and then view the data for one year.
Step29:  Most of the examples above focus on converting a Table to simple Element types, but HoloViews also provides powerful container objects to explore high-dimensional data, such as HoloMap, NdOverlay, NdLayout, and GridSpace. HoloMaps work as a useful interchange format from which you can conveniently convert to the other container types using its .overlay(), .layout(), and .grid() methods. This way we can easily create an overlay of GDP Growth curves by year for each country. Here Year is a key dimension and GDP Growth a value dimension. We are then left with the Country dimension, which we can overlay using the .overlay() method.
Step30:  Now that we've extracted the gdp_curves, we can apply some operations to them. As in the simpler example above we will collapse the HoloMap of Curves using a number of functions to visualize the distribution of GDP Growth rates over time. First we find the mean curve with np.std as the spreadfn and cast the result to a Spread type, then we compute the min, mean and max curve in the same way and put them all inside an Overlay.
Step31:  Many HoloViews Element types support multiple kdims, including HeatMap, Points, Scatter, Scatter3D, and Bars. Bars in particular allows you to lay out your data in groups, categories and stacks. By supplying the index of that dimension as a plotting option you can choose to lay out your data as groups of bars, categories in each group, and stacks. Here we choose to lay out the trade surplus of each country with groups for each year, no categories, and stacked by country. Finally, we choose to color the Bars for each item in the stack.
Step32:  This plot contains a lot of data, and so it's probably a good idea to focus on specific aspects of it, telling a simpler story about them.  For instance, using the .select method we can then customize the palettes (e.g. to use consistent colors per country across multiple analyses).
Palettes can customized by selecting only a subrange of the underlying cmap to draw the colors from. The Palette draws samples from the colormap using the supplied sample_fn, which by default just draws linear samples but may be overriden with any function that draws samples in the supplied ranges. By slicing the Set1 colormap we draw colors only from the upper half of the palette and then reverse it.
Step33:  Many HoloViews Elements support multiple key and value dimensions. A HeatMap is indexed by two kdims, so we can visualize each of the economic indicators by year and country in a Layout. Layouts are useful for heterogeneous data you want to lay out next to each other.
Before we display the Layout let's apply some styling; we'll suppress the value labels applied to a HeatMap by default and substitute it for a colorbar. Additionally we up the number of xticks that are drawn and rotate them by 90 degrees to avoid overlapping. Flipping the y-axis ensures that the countries appear in alphabetical order. Finally we reduce some of the margins of the Layout and increase the size.
Step34:  Another way of combining heterogeneous data dimensions is to map them to a multi-dimensional plot type. Scatter Elements, for example, support multiple vdims, which may be mapped onto the color and size of the drawn points in addition to the y-axis position. 
As for the Curves above we supply 'Year' as the sole key dimension and rely on the Table to automatically convert the Country to a map dimension, which we'll overlay. However this time we select both GDP Growth and Unemployment, to be plotted as points. To get a sensible chart, we adjust the scaling_factor for the points to get a reasonable distribution in sizes and apply a categorical Palette so we can distinguish each country.
Step35:  In this way we can plot any dimension against any other dimension, very easily allowing us to iterate through different ways of revealing relationships in the dataset.
Step36:  This view, for example, immediately highlights the high unemployment rates of the 1980s.
Since all HoloViews Elements are composable, we can generate complex figures just by applying the * operator. We'll simply reuse the GDP curves we generated earlier, combine them with the scatter points (which indicate the unemployment rate by size) and annotate the data with some descriptions of what happened economically in these years.
Step37:  Since we didn't map the country to some other container type, we get a widget allowing us to view the plot separately for each country, reducing the forest of curves we encountered before to manageable chunks. 
While looking at the plots individually like this allows us to study trends for each country, we may want to lay out a subset of the countries side by side, e.g. for non-interactive publications. We can easily achieve this by selecting the countries we want to view and and then applying the .layout method. We'll also want to restore the square aspect ratio so the plots compose nicely.
Step38:  Finally, let's combine some plots for each country into a Layout, giving us a quick overview of each economic indicator for each country
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
import holoviews as hv
from IPython.display import HTML
hv.notebook_extension()
Explanation: In this Tutorial we will explore how to work with columnar data in HoloViews. Columnar data has a fixed list of column headings, with values stored in an arbitrarily long list of rows.  Spreadsheets, relational databases, CSV files, and many other typical data sources fit naturally into this format.  HoloViews defines an extensible system of interfaces to load, manipulate, and visualize this kind of data, as well as allowing conversion of any of the non-columnar data types into columnar data for analysis or data interchange.
By default HoloViews will use one of three storage formats for columnar data:
A pure Python dictionary containing each column.
A purely NumPy-based format for numeric data.
Pandas DataFrames
End of explanation
xs = range(10)
ys = np.exp(xs)
table = hv.Table((xs, ys), kdims=['x'], vdims=['y'])
table
Explanation: Simple Dataset
Usually when working with data we have one or more independent variables, taking the form of categories, labels, discrete sample coordinates, or bins.  These variables are what we refer to as key dimensions (or kdims for short) in HoloViews. The observer or dependent variables, on the other hand, are referred to as value dimensions (vdims), and are ordinarily measured or calculated given the independent variables. The simplest useful form of a Dataset object is therefore a column 'x' and a column 'y' corresponding to the key dimensions and value dimensions respectively. An obvious visual representation of this data is a Table:
End of explanation
hv.Scatter(table) + hv.Curve(table) + hv.Bars(table)
Explanation: However, this data has many more meaningful visual representations, and therefore the first important concept is that Dataset objects are interchangeable as long as their dimensionality allows it, meaning that you can easily create the different objects from the same data (and cast between the objects once created):
End of explanation
print(repr(hv.Scatter({'x': xs, 'y': ys}) +
           hv.Scatter(np.column_stack([xs, ys])) +
           hv.Scatter(pd.DataFrame({'x': xs, 'y': ys}))))
Explanation: Each of these three plots uses the same data, but represents a different assumption about the semantic meaning of that data -- the Scatter plot is appropriate if that data consists of independent samples, the Curve plot is appropriate for samples chosen from an underlying smooth function, and the Bars plot is appropriate for independent categories of data.  Since all these plots have the same dimensionality, they can easily be converted to each other, but there is normally only one of these representations that is semantically appropriate for the underlying data.  For this particular data, the semantically appropriate choice is Curve, since the y values are samples from the continuous function exp.
As a guide to which Elements can be converted to each other, those of the same dimensionality here should be interchangeable, because of the underlying similarity of their columnar representation:
0D: BoxWhisker, Spikes, Distribution*, 
1D: Scatter, Curve, ErrorBars, Spread, Bars, BoxWhisker, Regression*
2D: Points, HeatMap, Bars, BoxWhisker, Bivariate*
3D: Scatter3D, Trisurface, VectorField, BoxWhisker, Bars
* - requires Seaborn
This categorization is based only on the kdims, which define the space in which the data has been sampled or defined. An Element can also have any number of value dimensions (vdims), which may be mapped onto various attributes of a plot such as the color, size, and orientation of the plotted items.  For a reference of how to use these various Element types, see the Elements Tutorial.
Data types and Constructors
As discussed above, Dataset provide an extensible interface to store and operate on data in different formats. All interfaces support a number of standard constructors.
Storage formats
Dataset types can be constructed using one of three supported formats, (a) a dictionary of columns, (b) an NxD array with N rows and D columns, or (c) pandas dataframes:
End of explanation
print(repr(hv.Scatter(ys) + hv.Scatter((xs, ys)) + hv.Scatter(zip(xs, ys))))
Explanation: Literals
In addition to the main storage formats, Dataset Elements support construction from three Python literal formats: (a) An iterator of y-values, (b) a tuple of columns, and (c) an iterator of row tuples.
End of explanation
df = pd.DataFrame({'x': xs, 'y': ys, 'z': ys*2})
print(type(hv.Scatter(df).data))
Explanation: For these inputs, the data will need to be copied to a new data structure, having one of the three storage formats above.  By default Dataset will try to construct a simple array, falling back to either pandas dataframes (if available) or the dictionary-based format if the data is not purely numeric. Additionally, the interfaces will try to maintain the provided data's type, so numpy arrays and pandas DataFrames will therefore always be parsed by the array and dataframe interfaces first respectively.
End of explanation
hv.Dataset.datatype
Explanation: Dataset will attempt to parse the supplied data, falling back to each consecutive interface if the previous could not interpret the data. The default list of fallbacks and simultaneously the list of allowed datatypes is:
End of explanation
print(type(hv.Scatter((xs, ys), datatype=['array']).data))
print(type(hv.Scatter((xs, ys), datatype=['dictionary']).data))
print(type(hv.Scatter((xs, ys), datatype=['dataframe']).data))
Explanation: To select a particular storage format explicitly, supply one or more allowed datatypes:
End of explanation
overlay = hv.Scatter(df, kdims='x', vdims='y') * hv.Scatter(df, kdims='x', vdims='z')
overlay
Explanation: Sharing Data
Since the formats with labelled columns do not require any specific order, each Element can effectively become a view into a single set of data. By specifying different key and value dimensions, many Elements can show different values, while sharing the same underlying data source.
End of explanation
overlay.Scatter.I.data is overlay.Scatter.II.data
Explanation: We can quickly confirm that the data is actually shared:
End of explanation
table.array()
Explanation: For columnar data, this approach is much more efficient than creating copies of the data for each Element, and allows for some advanced features like linked brushing in the Bokeh backend.
Converting to raw data
Column types make it easy to export the data to the three basic formats: arrays, dataframes, and a dictionary of columns.
Array
End of explanation
HTML(table.dframe().head().to_html())
Explanation: Pandas DataFrame
End of explanation
table.columns()
Explanation: Dataset dictionary
End of explanation
xs = np.arange(10)
curve = hv.Curve(zip(xs, np.exp(xs)))
curve * hv.Scatter(zip(xs, curve)) + curve.table()
Explanation: Creating tabular data from Elements using the .table and .dframe methods
If you have data in some other HoloViews element and would like to use the columnar data features, you can easily tabularize any of the core Element types into a Table Element, using the .table() method.  Similarly, the .dframe() method will convert an Element into a pandas DataFrame. These methods are very useful if you want to then transform the data into a different Element type, or to perform different types of analysis.
Tabularizing simple Elements
For a simple example, we can create a Curve of an exponential function and convert it to a Table with the .table method, with the same result as creating the Table directly from the data as done earlier on this Tutorial:
End of explanation
HTML(curve.dframe().to_html())
Explanation: Similarly, we can get a pandas dataframe of the Curve using curve.dframe().  Here we wrap that call as raw HTML to allow automated testing of this notebook, but just calling curve.dframe() would give the same result visually:
End of explanation
%%opts Points (s=200) [size_index=None]
extents = (-1.6,-2.7,2.0,3)
np.random.seed(42)
mat = np.random.rand(3, 3)
img = hv.Image(mat, bounds=extents)
raster = hv.Raster(mat)
img    * hv.Points(img)    + img.table() + \
raster * hv.Points(raster) + raster.table()
Explanation: Although 2D image-like objects are not inherently well suited to a flat columnar representation, serializing them by converting to tabular data is a good way to reveal the differences between Image and Raster elements.  Rasters are a very simple type of element, using array-like integer indexing of rows and columns from their top-left corner as in computer graphics applications.  Conversely, Image elements are a higher-level abstraction that provides a general-purpose continuous Cartesian coordinate system, with x and y increasing to the right and upwards as in mathematical applications, and each point interpreted as a sample representing the pixel in which it is located (and thus centered within that pixel).  Given the same data, the .table() representation will show how the data is being interpreted (and accessed) differently in the two cases (as explained in detail in the Continuous Coordinates Tutorial):
End of explanation
obs_hmap = hv.HoloMap({i: hv.Image(np.random.randn(10, 10), bounds=(0,0,3,3))
                   for i in range(3)}, key_dimensions=['Observation'])
obs_hmap
Explanation: Tabularizing space containers
Even deeply nested objects can be deconstructed in this way, serializing them to make it easier to get your raw data out of a collection of specialized Element types. Let's say we want to make multiple observations of a noisy signal. We can collect the data into a HoloMap to visualize it and then call .table() to get a columnar object where we can perform operations or transform it to other Element types. Deconstructing nested data in this way only works if the data is homogenous. In practical terms, the requirement is that your data structure contains Elements (of any types) in these Container types: NdLayout, GridSpace, HoloMap, and NdOverlay, with all dimensions consistent throughout (so that they can all fit into the same set of columns).
Let's now go back to the Image example. We will now collect a number of observations of some noisy data into a HoloMap and display it:
End of explanation
%%opts Layout [fig_size=150] Scatter3D [color_index=3 size_index=None] (cmap='hot' edgecolor='k' s=50)
obs_hmap.table().to.scatter3d() + obs_hmap.table()
Explanation: Now we can serialize this data just as before, where this time we get a four-column (4D) table. The key dimensions of both the HoloMap and the Images, as well as the z-values of each Image, are all merged into a single table. We can visualize the samples we have collected by converting it to a Scatter3D object.
End of explanation
from itertools import product
extents = (0,0,3,3)
error_hmap = hv.HoloMap({(i, j): hv.Image(j*np.random.randn(3, 3), bounds=extents)
                         for i, j in product(range(3), np.linspace(0, 1, 3))},
                        key_dimensions=['Observation', 'noise'])
noise_layout = error_hmap.layout('noise')
noise_layout
Explanation: Here the z dimension is shown by color, as in the original images, and the other three dimensions determine where the datapoint is shown in 3D. This way of deconstructing will work for any data structure that satisfies the conditions described above, no matter how nested. If we vary the amount of noise while continuing to performing multiple observations, we can create an NdLayout of HoloMaps, one for each level of noise, and animated by the observation number.
End of explanation
%%opts Table [fig_size=150]
noise_layout.table()
Explanation: And again, we can easily convert the object to a Table:
End of explanation
bars = hv.Bars((['C', 'A', 'B', 'D'], [2, 7, 3, 4]))
bars + bars.sort() + bars.sort(['y'])
Explanation: Applying operations to the data
Sorting by columns
Once data is in columnar form, it is simple to apply a variety of operations.  For instance, Dataset can be sorted by their dimensions using the .sort() method.  By default, this method will sort by the key dimensions, but any other dimension(s) can be supplied to specify sorting along any other dimensions:
End of explanation
n = np.arange(1000)
xs = np.repeat(range(2), 500)
ys = n%4
zs = np.random.randn(1000)
table = hv.Table((xs, ys, zs), kdims=['x', 'y'], vdims=['z'])
table
Explanation: Working with categorical or grouped data
Data is often grouped in various ways, and the Dataset interface provides various means to easily compare between groups and apply statistical aggregates. We'll start by generating some synthetic data with two groups along the x-axis and 4 groups along the y axis.
End of explanation
%%opts BoxWhisker [aspect=2 fig_size=200 bgcolor='w']
hv.BoxWhisker(table)
Explanation: Since there are repeat observations of the same x- and y-values, we have to reduce the data before we display it or else use a datatype that supports plotting distributions in this way. The BoxWhisker type allows doing exactly that:
End of explanation
%%opts Bars [show_legend=False] {+axiswise}
hv.Bars(table).aggregate(function=np.mean) + hv.Bars(table).reduce(x=np.mean)
Explanation: Aggregating/Reducing dimensions
Most types require the data to be non-duplicated before being displayed.  For this purpose, HoloViews makes it easy to aggregate and reduce the data. These two operations are simple inverses of each other--aggregate computes a statistic for each group in the supplied dimensions, while reduce combines all the groups except the supplied dimensions. Supplying only a function and no dimensions will simply aggregate or reduce all available key dimensions.
End of explanation
hmap = hv.HoloMap({i: hv.Curve(np.arange(10)*i) for i in range(10)})
collapsed = hmap.collapse(function=np.mean, spreadfn=np.std)
hv.Spread(collapsed) * hv.Curve(collapsed) + collapsed.table()
Explanation: (A) aggregates over both the x and y dimension, computing the mean for each x/y group, while (B) reduces the x dimension leaving just the mean for each group along y.
Collapsing multiple Dataset Elements
When multiple observations are broken out into a HoloMap they can easily be combined using the collapse method. Here we create a number of Curves with increasingly larger y-values. By collapsing them with a function and a spreadfn we can compute the mean curve with a confidence interval. We then simply cast the collapsed Curve to a Spread and Curve Element to visualize them.
End of explanation
macro_df = pd.read_csv('http://assets.holoviews.org/macro.csv', '\t')
dimensions = {'unem':    'Unemployment',
              'capmob':  'Capital Mobility',
              'gdp':     'GDP Growth', 
              'trade':   'Trade',
              'year':    'Year', 
              'country': 'Country'}
macro_df = macro_df.rename(columns=dimensions)
Explanation: Working with complex data
In the last section we only scratched the surface of what the Dataset interface can do.  When it really comes into its own is when working with high-dimensional datasets. As an illustration, we'll load a dataset of some macro-economic indicators for OECD countries from 1964-1990, cached on the HoloViews website.
End of explanation
%output dpi=100
options = hv.Store.options()
opts = hv.Options('plot', aspect=2, fig_size=250, show_frame=False, show_grid=True, legend_position='right')
options.NdOverlay = opts
options.Overlay = opts
Explanation: We'll also take this opportunity to set default options for all the following plots.
End of explanation
macro = hv.Table(macro_df, kdims=['Year', 'Country'])
Explanation: Loading the data
As we saw above, we can supply a dataframe to any Dataset type. When dealing with so many dimensions it would be cumbersome to supply all the dimensions explicitly, but luckily Dataset can easily infer the dimensions from the dataframe itself. We simply supply the kdims, and it will infer that all other numeric dimensions should be treated as value dimensions (vdims).
End of explanation
%%opts Table [aspect=1.5 fig_size=300]
macro = macro.sort()
macro[1988]
Explanation: To get an overview of the data we'll quickly sort it and then view the data for one year.
End of explanation
%%opts Curve (color=Palette('Set3'))
gdp_curves = macro.to.curve('Year', 'GDP Growth')
gdp_curves.overlay('Country')
Explanation: Most of the examples above focus on converting a Table to simple Element types, but HoloViews also provides powerful container objects to explore high-dimensional data, such as HoloMap, NdOverlay, NdLayout, and GridSpace. HoloMaps work as a useful interchange format from which you can conveniently convert to the other container types using its .overlay(), .layout(), and .grid() methods. This way we can easily create an overlay of GDP Growth curves by year for each country. Here Year is a key dimension and GDP Growth a value dimension. We are then left with the Country dimension, which we can overlay using the .overlay() method.
End of explanation
%%opts Overlay [bgcolor='w' legend_position='top_right'] Curve (color='k' linewidth=1) Spread (facecolor='gray' alpha=0.2)
hv.Spread(gdp_curves.collapse('Country', np.mean, np.std), label='std') *\
hv.Overlay([gdp_curves.collapse('Country', fn).relabel(name)(style=dict(linestyle=ls))
            for name, fn, ls in [('max', np.max, '--'), ('mean', np.mean, '-'), ('min', np.min, '--')]])
Explanation: Now that we've extracted the gdp_curves, we can apply some operations to them. As in the simpler example above we will collapse the HoloMap of Curves using a number of functions to visualize the distribution of GDP Growth rates over time. First we find the mean curve with np.std as the spreadfn and cast the result to a Spread type, then we compute the min, mean and max curve in the same way and put them all inside an Overlay.
End of explanation
%opts Bars [bgcolor='w' aspect=3 figure_size=450 show_frame=False]
%%opts Bars [category_index=2 stack_index=0 group_index=1 legend_position='top' legend_cols=7 color_by=['stack']] (color=Palette('Dark2'))
macro.to.bars(['Country', 'Year'], 'Trade', [])
Explanation: Many HoloViews Element types support multiple kdims, including HeatMap, Points, Scatter, Scatter3D, and Bars. Bars in particular allows you to lay out your data in groups, categories and stacks. By supplying the index of that dimension as a plotting option you can choose to lay out your data as groups of bars, categories in each group, and stacks. Here we choose to lay out the trade surplus of each country with groups for each year, no categories, and stacked by country. Finally, we choose to color the Bars for each item in the stack.
End of explanation
%%opts Bars [padding=0.02 color_by=['group']] (alpha=0.6, color=Palette('Set1', reverse=True)[0.:.2])
countries = {'Belgium', 'Netherlands', 'Sweden', 'Norway'}
macro.to.bars(['Country', 'Year'], 'Unemployment').select(Year=(1978, 1985), Country=countries)
Explanation: This plot contains a lot of data, and so it's probably a good idea to focus on specific aspects of it, telling a simpler story about them.  For instance, using the .select method we can then customize the palettes (e.g. to use consistent colors per country across multiple analyses).
Palettes can customized by selecting only a subrange of the underlying cmap to draw the colors from. The Palette draws samples from the colormap using the supplied sample_fn, which by default just draws linear samples but may be overriden with any function that draws samples in the supplied ranges. By slicing the Set1 colormap we draw colors only from the upper half of the palette and then reverse it.
End of explanation
%opts HeatMap [show_values=False xticks=40 xrotation=90 aspect=1.2 invert_yaxis=True colorbar=True]
%opts Layout [figure_size=120 aspect_weight=0.5 hspace=0.8 vspace=0]
hv.Layout([macro.to.heatmap(['Year', 'Country'], value)
           for value in macro.data.columns[2:]]).cols(2)
Explanation: Many HoloViews Elements support multiple key and value dimensions. A HeatMap is indexed by two kdims, so we can visualize each of the economic indicators by year and country in a Layout. Layouts are useful for heterogeneous data you want to lay out next to each other.
Before we display the Layout let's apply some styling; we'll suppress the value labels applied to a HeatMap by default and substitute it for a colorbar. Additionally we up the number of xticks that are drawn and rotate them by 90 degrees to avoid overlapping. Flipping the y-axis ensures that the countries appear in alphabetical order. Finally we reduce some of the margins of the Layout and increase the size.
End of explanation
%%opts Scatter [scaling_method='width' scaling_factor=2] (color=Palette('Set3') edgecolors='k')
gdp_unem_scatter = macro.to.scatter('Year', ['GDP Growth', 'Unemployment'])
gdp_unem_scatter.overlay('Country')
Explanation: Another way of combining heterogeneous data dimensions is to map them to a multi-dimensional plot type. Scatter Elements, for example, support multiple vdims, which may be mapped onto the color and size of the drawn points in addition to the y-axis position. 
As for the Curves above we supply 'Year' as the sole key dimension and rely on the Table to automatically convert the Country to a map dimension, which we'll overlay. However this time we select both GDP Growth and Unemployment, to be plotted as points. To get a sensible chart, we adjust the scaling_factor for the points to get a reasonable distribution in sizes and apply a categorical Palette so we can distinguish each country.
End of explanation
%%opts NdOverlay [legend_cols=2] Scatter [size_index=1] (color=Palette('Blues'))
macro.to.scatter('GDP Growth', 'Unemployment', ['Year']).overlay()
Explanation: In this way we can plot any dimension against any other dimension, very easily allowing us to iterate through different ways of revealing relationships in the dataset.
End of explanation
%%opts Curve (color='k') Scatter [color_index=2 size_index=2 scaling_factor=1.4] (cmap='Blues' edgecolors='k')
macro_overlay = gdp_curves * gdp_unem_scatter
annotations = hv.Arrow(1973, 8, 'Oil Crisis', 'v') * hv.Arrow(1975, 6, 'Stagflation', 'v') *\
hv.Arrow(1979, 8, 'Energy Crisis', 'v') * hv.Arrow(1981.9, 5, 'Early Eighties\n Recession', 'v')
macro_overlay * annotations
Explanation: This view, for example, immediately highlights the high unemployment rates of the 1980s.
Since all HoloViews Elements are composable, we can generate complex figures just by applying the * operator. We'll simply reuse the GDP curves we generated earlier, combine them with the scatter points (which indicate the unemployment rate by size) and annotate the data with some descriptions of what happened economically in these years.
End of explanation
%opts Overlay [aspect=1]
%%opts NdLayout [figure_size=100] Scatter [color_index=2] (cmap='Reds')
countries = {'United States', 'Canada', 'United Kingdom'}
(gdp_curves * gdp_unem_scatter).select(Country=countries).layout('Country')
Explanation: Since we didn't map the country to some other container type, we get a widget allowing us to view the plot separately for each country, reducing the forest of curves we encountered before to manageable chunks. 
While looking at the plots individually like this allows us to study trends for each country, we may want to lay out a subset of the countries side by side, e.g. for non-interactive publications. We can easily achieve this by selecting the countries we want to view and and then applying the .layout method. We'll also want to restore the square aspect ratio so the plots compose nicely.
End of explanation
%%opts Layout [fig_size=100] Scatter [color_index=2] (cmap='Reds')
(macro_overlay.relabel('GDP Growth', depth=1) +\
macro.to.curve('Year', 'Unemployment', ['Country'], group='Unemployment',) +\
macro.to.curve('Year', 'Trade', ['Country'], group='Trade') +\
macro.to.scatter('GDP Growth', 'Unemployment', ['Country'])).cols(2)
Explanation: Finally, let's combine some plots for each country into a Layout, giving us a quick overview of each economic indicator for each country:
End of explanation
<END_TASK>