<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Reading SETI Hackathon Data
This tutorial will show you how to programmatically download the SETI code challenge data to your local file space and 
start to analyze it.
Please see the Step_1_Get_Data.ipynb notebook on information about all of the data available for this code challenge. 
This tutorial will use the basic data set, but will work, of course, with any of the data sets.
Step1:  No Spark Here
You'll notice that this tutorial doesn't use parallelization with Spark. This is to keep this simple and make this code generalizable to folks that are running this analysis on their local machines.
Step2:  Assume you have the data in a local folder
Step3:  Use ibmseti for convenience
While it's somewhat trivial to read these data, the ibmseti.compamp.SimCompamp class will extract the JSON header and the complex-value time-series data for you.
Step4:  The Goal
The goal is to take each simulation data file and 
1. convert the time-series data into a 2D spectrogram
2. Use the 2D spectrogram as an image to train an image classification model
There are multiple ways to improve your model's ability to classify signals. You can
* Modify the time-series data with some signals processing to make a better 2D spectrogram
* Build a really good image classification system
* Try something entirely different, such as 
    * transforming the time-series data in different ways (KTL transform)?
    * use the time-series data directly in model
Here we just show how to view the data as a spectrogram
1. Converting the time-series data into a spectrogram with ibmseti
Step5:  2. Build the spectogram yourself
You don't need to use ibmseti python package to calculate the spectrogram for you. 
This is especially important if you want to apply some signals processing to the time-series data before you create your spectrogram
<ASSISTANT_TASK:>
Python Code:
#The ibmseti package contains some useful tools to faciliate reading the data.
#The `ibmseti` package version 1.0.5 works on Python 2.7. 
#  !pip install --user ibmseti
#A development version runs on Python 3.5. 
#  !pip install --user ibmseti==2.0.0.dev5
# If running on DSX, YOU WILL NEED TO RESTART YOUR SPARK KERNEL to use a newly installed Python Package. 
# Click Kernel -> Restart above!
Explanation: Reading SETI Hackathon Data
This tutorial will show you how to programmatically download the SETI code challenge data to your local file space and 
start to analyze it.
Please see the Step_1_Get_Data.ipynb notebook on information about all of the data available for this code challenge. 
This tutorial will use the basic data set, but will work, of course, with any of the data sets.
End of explanation
import ibmseti
import os
import zipfile
Explanation: No Spark Here
You'll notice that this tutorial doesn't use parallelization with Spark. This is to keep this simple and make this code generalizable to folks that are running this analysis on their local machines.
End of explanation
!ls my_data_folder/basic4
zz = zipfile.ZipFile(mydatafolder + '/' + 'basic4.zip')
basic4list = zz.namelist()
firstfile = basic4list[0]
print firstfile
Explanation: Assume you have the data in a local folder
End of explanation
import ibmseti
aca = ibmseti.compamp.SimCompamp(zz.open(firstfile, 'rb').read())
# This data file is classified as a 'squiggle'
aca.header()
Explanation: Use ibmseti for convenience
While it's somewhat trivial to read these data, the ibmseti.compamp.SimCompamp class will extract the JSON header and the complex-value time-series data for you.
End of explanation
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
## ibmseti.compamp.SimCompamp has a method to calculate the spectrogram for you (without any signal processing applied to the time-series data)
spectrogram = aca.get_spectrogram()
fig, ax = plt.subplots(figsize=(10, 5))
ax.imshow(np.log(spectrogram),  aspect = 0.5*float(spectrogram.shape[1]) / spectrogram.shape[0])
Explanation: The Goal
The goal is to take each simulation data file and 
1. convert the time-series data into a 2D spectrogram
2. Use the 2D spectrogram as an image to train an image classification model
There are multiple ways to improve your model's ability to classify signals. You can
* Modify the time-series data with some signals processing to make a better 2D spectrogram
* Build a really good image classification system
* Try something entirely different, such as 
    * transforming the time-series data in different ways (KTL transform)?
    * use the time-series data directly in model
Here we just show how to view the data as a spectrogram
1. Converting the time-series data into a spectrogram with ibmseti
End of explanation
complex_data = aca.complex_data()
#complex valued time-series
complex_data
complex_data = complex_data.reshape(32, 6144)
complex_data
#Apply a Hanning Window
complex_data = complex_data * np.hanning(complex_data.shape[1])
complex_data
# Build Spectogram & Plot
cpfft = np.fft.fftshift( np.fft.fft(complex_data), 1)
spectrogram = np.abs(cpfft)**2
fig, ax = plt.subplots(figsize=(10, 5))
ax.imshow(np.log(spectrogram), aspect = 0.5*float(spectrogram.shape[1]) / spectrogram.shape[0])
Explanation: 2. Build the spectogram yourself
You don't need to use ibmseti python package to calculate the spectrogram for you. 
This is especially important if you want to apply some signals processing to the time-series data before you create your spectrogram
End of explanation
<END_TASK>