<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Series
A Series object is a distributed collection of 1D arrays, all of which share a common index. It supports several manipulations and computations on 1D data, some of which take the index into account.
The most common examples of Series objects are likely time series data, in which each array represents a quantity that has been temporally sampled at regular intervals. For example, a movie can be represented as Series data, where a single record might consist of a pixel time series, as in the fish-series and mouse-series example data from the basic usage tutorial.
The keys associated with Series records can be any data used to identify the series, for example, a single sensor channel identifier, or tuples representing pixel coordiantes (if derived from a movie). For example, a Series record representing a time series from an outer corner of a movie might have key (0, 0, 0), with the value being an array representing the measurements at that location over time
Step1:  Loading series
Series can be loaded from a ThunderContext using the loadSeries method. (See the Input formats tutorial for more detail on loading methods and formats.) Here, we'll again load the "fish-series" example data set
Step2:  Inspection
Let's plot a random subset of the data using the subset method, which thresholds based on the standard deviation
Step3:  Note the variation in raw intensity levels.
Processing
A Series object has various methods for processing and transforming the data. For example, center subtracts the mean, squelch sets to 0 any records not exceeding a threshold, and zscore subtracts the mean and divides by the standard deviation. Note the changes in the y-axis.
Step4:  Related methods include standardize, detrend, and normalize (the latter two are specified to TimeSeries, see below)
Selections
A Series has a 1D index, which can be used to subselect values.
Step5:  For example, to select a range
Step6:  Note that the index changes to reflect the subselected range
Step7:  We can also select based on an arbitrary criterion function
Step8:  The default index generated for Series objects will be the range of integers starting at zero and ending one before the length of the series data, as shown in these examples. However, other data types can also be used as the index for a series object, such as a sequence of strings, providing text labels for each element in the series array, or a tuple with indices at different levels. See the tutorial on Multi-indexing tutorial for this usage.
Statistics
A Series can be summarized with statistics both within and across records. To summarize across records
Step9:  To summarize within records
Step10:  We can also correlate each record with a signal of interest. As expected, for a random signal, the correlation should be near 0.
Step11:  Keys
We often use integer keys to index the records of a Series, and sometimes these keys represent indices into some original array (for example, indices into a multi-dimensional tensor, or pixel coordinates if the records are time series from a movie).
A few convenience methods are provided for working with keys. First, the attribute dims contains information about the range of the keys. Computing this requires a pass through the data, but if computed on a Series it will be inherited by all derived objects, to avoid recomputation.
Step12:  For keys that correspond to subscripts (e.g. indices of the rows and columns of a matrix, coordinates in space), we can convert between subscript and linear indexing. The default for these conversions is currently onebased subscript indexing, so we need to set onebased to False (this will likely change in a future release).
Step13:  The query method can be used to average subselected records based on their (linearized) keys. It returns the mean value and  key for each of the provided index lists.
Step14:  The pack method collects a series into a local array, reshaped based on the keys. If there are multiple values per record, all will be collected into the local array, so typically we select a subset of values before packing to avoid overwhelming the local returning a very large amount of data.
Step15:  Conversions
Subclasses of Series provide additional functionality for more specialized data types.
A TimeSeries provides methods specific to time-varying data, like fourier, for computing the statistics of a Fourier transform
Step16:  Or detrend for detrending data over time
Step17:  A RowMatrix provides a variety of methods for working with distributed matrices and matrix operations
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_context('notebook')
Explanation: Series
A Series object is a distributed collection of 1D arrays, all of which share a common index. It supports several manipulations and computations on 1D data, some of which take the index into account.
The most common examples of Series objects are likely time series data, in which each array represents a quantity that has been temporally sampled at regular intervals. For example, a movie can be represented as Series data, where a single record might consist of a pixel time series, as in the fish-series and mouse-series example data from the basic usage tutorial.
The keys associated with Series records can be any data used to identify the series, for example, a single sensor channel identifier, or tuples representing pixel coordiantes (if derived from a movie). For example, a Series record representing a time series from an outer corner of a movie might have key (0, 0, 0), with the value being an array representing the measurements at that location over time: [112, 110, 118, ..., 124].
Here, we show examples of loading and manipulating Series data.
Setup plotting
End of explanation
data = tsc.loadExample('fish-series')
Explanation: Loading series
Series can be loaded from a ThunderContext using the loadSeries method. (See the Input formats tutorial for more detail on loading methods and formats.) Here, we'll again load the "fish-series" example data set:
End of explanation
examples = data.subset(nsamples=50, thresh=1)
plt.plot(examples.T[0:20,:]);
Explanation: Inspection
Let's plot a random subset of the data using the subset method, which thresholds based on the standard deviation:
End of explanation
examples = data.center().subset(nsamples=50, thresh=10)
plt.plot(examples.T[0:20,:]);
examples = data.squelch(150).zscore().subset(nsamples=50, thresh=0.1)
plt.plot(examples.T[0:20,:]);
Explanation: Note the variation in raw intensity levels.
Processing
A Series object has various methods for processing and transforming the data. For example, center subtracts the mean, squelch sets to 0 any records not exceeding a threshold, and zscore subtracts the mean and divides by the standard deviation. Note the changes in the y-axis.
End of explanation
data.index.shape
Explanation: Related methods include standardize, detrend, and normalize (the latter two are specified to TimeSeries, see below)
Selections
A Series has a 1D index, which can be used to subselect values.
End of explanation
data.between(0,8).first()
Explanation: For example, to select a range:
End of explanation
data.between(0,8).index
Explanation: Note that the index changes to reflect the subselected range:
End of explanation
data.select(lambda x: x < 5).index
Explanation: We can also select based on an arbitrary criterion function:
End of explanation
plt.plot(data.toTimeSeries().normalize().max());
plt.plot(data.toTimeSeries().normalize().mean());
plt.plot(data.toTimeSeries().normalize().min());
Explanation: The default index generated for Series objects will be the range of integers starting at zero and ending one before the length of the series data, as shown in these examples. However, other data types can also be used as the index for a series object, such as a sequence of strings, providing text labels for each element in the series array, or a tuple with indices at different levels. See the tutorial on Multi-indexing tutorial for this usage.
Statistics
A Series can be summarized with statistics both within and across records. To summarize across records:
End of explanation
data.seriesMean().first()
data.seriesStdev().first()
Explanation: To summarize within records:
End of explanation
from numpy import random
signal = random.randn(240)
data.correlate(signal).first()
Explanation: We can also correlate each record with a signal of interest. As expected, for a random signal, the correlation should be near 0.
End of explanation
data.dims.max
data.dims.min
Explanation: Keys
We often use integer keys to index the records of a Series, and sometimes these keys represent indices into some original array (for example, indices into a multi-dimensional tensor, or pixel coordinates if the records are time series from a movie).
A few convenience methods are provided for working with keys. First, the attribute dims contains information about the range of the keys. Computing this requires a pass through the data, but if computed on a Series it will be inherited by all derived objects, to avoid recomputation.
End of explanation
data.keys().take(10)
data.subToInd(isOneBased=False).keys().take(10)
data.subToInd(isOneBased=False).indToSub(isOneBased=False).keys().take(10)
Explanation: For keys that correspond to subscripts (e.g. indices of the rows and columns of a matrix, coordinates in space), we can convert between subscript and linear indexing. The default for these conversions is currently onebased subscript indexing, so we need to set onebased to False (this will likely change in a future release).
End of explanation
keys, values = data.query(inds=[[100,101],[200]], isOneBased=False)
keys
values.shape
Explanation: The query method can be used to average subselected records based on their (linearized) keys. It returns the mean value and  key for each of the provided index lists.
End of explanation
out = data.select(0).pack()
out.shape
out = data.between(0,2).pack()
out.shape
Explanation: The pack method collects a series into a local array, reshaped based on the keys. If there are multiple values per record, all will be collected into the local array, so typically we select a subset of values before packing to avoid overwhelming the local returning a very large amount of data.
End of explanation
ts = data.toTimeSeries()
fr = ts.fourier(freq=5)
fr.index
fr.select('coherence').first()
Explanation: Conversions
Subclasses of Series provide additional functionality for more specialized data types.
A TimeSeries provides methods specific to time-varying data, like fourier, for computing the statistics of a Fourier transform:
End of explanation
plt.plot(ts.mean())
plt.plot(ts.detrend('nonlinear', order=5).mean());
Explanation: Or detrend for detrending data over time
End of explanation
mat = data.toRowMatrix()
from thunder import Colorize
Colorize.image(mat.cov())
Explanation: A RowMatrix provides a variety of methods for working with distributed matrices and matrix operations:
End of explanation
<END_TASK>