<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <a href="https
Step1:  Install the right software
(telluride-decoding plus prereqs)
Step2:  Decode the Telluride4 EEG Data
Step3:  Run complete jackknife test
Run a complete jackknife (leave-one-out) test using the Telluride4 data. This takes a while to run, but gives us error bounds (really standard deviations).  Compute the test-set correlation when using CCA.
<ASSISTANT_TASK:>
Python Code:
#@title Default title text
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Explanation: <a href="https://colab.research.google.com/github/google/telluride_decoding/blob/master/Telluride_Decoding_Toolbox_TF2_Demo.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
Telluride Decoding Toolbox, TF2 Version
This colab demonstrates the basic operation of the Telluride Decoding Toolbox. The toolbox is designed for large-scale experiments, where the raw data is stored on disk because is too large to fit into memory, or you want to run multiple decoding jobs in parallel for hyperparameter searches.
In general, there are three stages to our decoding pipeline: ingest, decode, decide. This colab demonstrates the first two.
Ingest the raw data, and store it on disk in TFRecord format. This is the native format for Tensorflow, and each file consists of the raw audio and brain data, one (vector) sample per time step for one continuous trial.  Multiple forms of the data can be stored in each record.  Thus you might have 64 channels of EEG data, the audio intensity, as well as perhaps the phone class and the corresponding spectrogram.  All data is sampled as the same frame rate (i.e. 100Hz).
Build a decoding model. The decoding model can take several forms, such as linear regression, CCA or several forms of DNN. When building regression models you specify which input fields (from the TFRecord data) are used as input, along with any desired temporal context, and which field is the output and to be predicted. For a backward model you might use EEG with 30 frames of post_context to predict the current (audio) intensity.  For CCA you specify two sets of input fields, which are rotated to provide the highest correlation.
Decide to which signal the subject is attending. This code implements a State Space Model (from Maryland) and a simple Markov model suggested by KUL.
It uses EEG data, first published as part of version 1 of the Telluride Decoding Toolbox. This data consists of 32 trials from four simultaneous subjects listening to 4 different audio tracks. There are 64 electrodes in each trial, split across the 4 subjects. The original data is provided in Matlab format and is downloaded by the colab.
This colab shows how to download and import the toolbox, download and import the original Matlab data, and then measure linear and CCA models predicting the intensity that generated the EEG data.
This is a work in progress.  Comments and/or questions are welcome as we finalize the documention and the code. The source code (a lousy way to document the toolkit) is available on GitHub.
Copyright 2018 Google LLC.
Licensed under the Apache License, Version 2.0 (the "License");
End of explanation
!pip uninstall -y telluride-decoding
# To get test versions and install them on this machine.
# !pip install mock pyedflib  # Need to list these since they are not on test.pypi
# !pip install --index-url https://test.pypi.org/project telluride-decoding
# To install the latest released version:
!pip install telluride-decoding
from matplotlib import pyplot as plt
import numpy as np
import tensorflow as tf
tf.compat.v1.enable_v2_behavior()
from telluride_decoding import brain_data
from telluride_decoding import brain_model
from telluride_decoding import decoding
from telluride_decoding import regression
from telluride_decoding import regression_data
# Reset the plot library backend to the default. 
import matplotlib
matplotlib.use('module://ipykernel.pylab.backend_inline')
# Force flags to be parsed (none here) to set flag defaults. There are some parts
# of the toolbox that still use FLAGS, but shouldn't :-(
from absl import flags
flags.FLAGS(['colab'])  # Cause flags library to set the default.
telluride_data = regression_data.RegressionDataTelluride4()
cache_dir = '/tmp'
telluride4_url = 'https://drive.google.com/uc?id=0ByZjGXodIlspWmpBcUhvenVQa1k'
if not telluride_data.is_data_local(cache_dir):
  print('Downloading the Telluride4 data...')
  telluride_data.download_data(telluride4_url, cache_dir)
tf_dir = '/tmp/telluride_tf'
!mkdir {tf_dir}
if not telluride_data.is_data_ingested(tf_dir):
  print('Ingesting the Telluride4 data...')
  telluride_data.ingest_data(cache_dir, tf_dir, 100)
!ls {tf_dir}
!cat {tf_dir}/README.txt
Explanation: Install the right software
(telluride-decoding plus prereqs)
End of explanation
# If you have problems or get unexplained errors, you might want to turn on
# log messages with these calls.  Change the False to True to enable the logging.
if False:
  from absl import logging
  logging.set_verbosity(logging.INFO)
  logging.set_stderrthreshold('info')
  logging.info('Testing')
telluride4_options = decoding.DecodingOptions()
telluride4_options.input_field = 'eeg'
telluride4_options.output_field = 'intensity'
telluride4_options.input2_field = 'intensity'
telluride4_options.tfexample_dir = tf_dir
telluride4_options.dnn_regressor = 'cca'
telluride4_options.post_context = 21
telluride4_options.input2_pre_context = 15
telluride4_options.input2_post_context = 15
telluride4_options.test_metric = 'cca_pearson_correlation_first'
telluride4_options.shuffle_buffer_size = 0  # No need when training a CCA model
telluride4_options.cca_dimensions = 5
print(telluride4_options.experiment_parameters('\n'))
telluride4_data = regression.get_brain_data_object(telluride4_options)
# Get the actual TF dataset, as an example of the full TDT data object.
telluride4_dataset = telluride4_data.create_dataset('train')
# Create the brain model (CCA as specified above.)
brain_model = decoding.create_brain_model(telluride4_options, telluride4_dataset)
# Now train the regressor. Since this regressor is built with CCA, only one
# pass through the data is needed to collect the statistics and create the model.
train_results, test_results = decoding.train_and_test(telluride4_options, telluride4_data, brain_model)
test_results
Explanation: Decode the Telluride4 EEG Data
End of explanation
telluride4_regression = regression.Telluride4CCA(telluride4_options)
reg_values = np.power(10.0, np.arange(-3, 2, 1))
results = telluride4_regression.jackknife_over_regularizations(telluride4_options, 
                                                               reg_values)
results
reg_values = list(results.keys())
result_means = [results[k][0] for k in reg_values] 
result_stddev = [results[k][1] for k in reg_values] 
plt.errorbar(reg_values, result_means, result_stddev)
matplotlib.pyplot.xscale('log')
plt.xlabel('Regularization Value')
plt.ylabel('Jackknifed Correlation');
Explanation: Run complete jackknife test
Run a complete jackknife (leave-one-out) test using the Telluride4 data. This takes a while to run, but gives us error bounds (really standard deviations).  Compute the test-set correlation when using CCA.
End of explanation
<END_TASK>