<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Basic Python Packages for Science
The Aeropython’s guide to the Python Galaxy!
Siro Moreno Martín
Alejandro Sáez Mollejo
0. Introduction
Python in the Scientific environment
Principal Python Packages for scientific purposes
Anaconda & conda
http
Step1:  Main objectives of this workshop
Provide you with a first insight into the principal Python tools & libraries used in Science
Step2:  Array creation
Step3:  Basic slicing
Step4:  [start
Step5:  2. Drawing
Step6:  Operations & linalg
Step7:  Air quality data
Step8:  Loading the data
Step9:  Dealing with missing values
Step10:  Plotting the data
 Maximum values  from
Step11:  CO 
Máxima diaria de las medias móviles octohorarias
Step12:  O3
Máxima diaria de las medias móviles octohorarias
Step13:  The file contains data from 2004 to 2015 (included). Each row corresponds to a day of the year, so evey 365 lines contain data from a whole year*
Note1
Step14:  We can also get information about percentiles!
Rearranging the data
Step15:  Let's visualize data!
Using matplotlib styles http
Step16:  Let's see if 2015 was a normal year...
Step17:  But the power of Matplotlib does not end here!
For example, lets represent a function over a 2D domain!
For this we will use the contour function, which requires some special inputs...
Step18:  In oder to plot the 2D function, we will need a grid.
For 1D domain, we just needed one 1D array containning the X position and another 1D array containing the value.
Now, we will create a grid, a distribution of points covering a surface. For the 2D domain, we will need
Step19:  Note that with the meshgrid function we can only create rectangular grids
Step20:  We can try a little more resolution...
Step21:  The countourf function is simmilar, but it colours also between the lines. In both functions, we can manually adjust the number of lines/zones we want to differentiate on the plot.
Step22:  These functions can be enormously useful when you want to visualize something.
And remember!
Always visualize data!
Let's try it with Real data!
Step23:  The time and frequency vectors contain the values at which the instrument was reading, and the intensity matrix, the postprocessed strength measured for each frequency at each time.
We need again to create the 2D arrays of coordinates.
Step24:  Wow! What is that? Let's zoom into it!
Step25:  IPython Widgets
The IPython Widgets are interactive tools to use in the notebook. They are fun and very useful to quickly understand how different parameters affect a certain function.
This is based on a section of the PyConEs 14 talk by Kiko Correoso "Hacking the notebook"
Step26:  If you want a dropdown menu that passes non-string values to the Python function, you can pass a dictionary. The keys in the dictionary are used for the names in the dropdown menu UI and the values are the arguments that are passed to the underlying Python function.
Let's have some fun! We talked before about frequencys and waves. Have you ever learn about AM and FM modulation? It's the process used to send radio communications!
Step27:  In order to interact with it, we will need to transform it into a function
Step28:  Other options...
5. Other packages
Symbolic calculations with SymPy
SymPy is a Python package for symbolic math. We will not cover it in depth, but let's take a picure of the basics!
Step29:  The basic unit of this package is the symbol. A simbol object has name and graphic representation, which can be different
Step30:  By default, SymPy takes symbols as complex numbers. That can lead to unexpected results in front of certain operations, like logarithms. We can explicitly signal that a symbol is real when we create it. We can also create several symbols at a time.
Step31:  Expressions can be created from symbols
Step32:  We can manipulate the expression in several ways. For example
Step33:  We can derivate and integrate
Step34:  We also have ecuations and differential ecuations
Step35:  Data Analysis with pandas
Pandas is a package that focus on data structures and data analysis tools. We will not cover it because the next workshop, by Kiko Correoso, will develop it in depth.
Machine Learning with scikit-learn
Scikit-learn is a very complete Python package focusing on machin learning, and data mining and analysis. We will not cover it in depth because it will be the focus of many more talks at the PyData.
A world of possibilities...
Thanks for yor attention!
Any Questions?
<ASSISTANT_TASK:>
Python Code:
from IPython.display import HTML
HTML('<iframe src="http://conda.pydata.org/docs/_downloads/conda-cheatsheet.pdf" width="700" height="400"></iframe>')
Explanation: Basic Python Packages for Science
The Aeropython’s guide to the Python Galaxy!
Siro Moreno Martín
Alejandro Sáez Mollejo
0. Introduction
Python in the Scientific environment
Principal Python Packages for scientific purposes
Anaconda & conda
http://conda.pydata.org/docs/intro.html
Conda is a package manager application that quickly installs, runs, and updates packages and their dependencies. The conda command is the primary interface for managing installations of various packages. It can query and search the package index and current installation, create new environments, and install and update packages into existing conda environments.
End of explanation
# importing numpy
# performance list sum
# performance array sum
%timeit np.sum(array)
Explanation: Main objectives of this workshop
Provide you with a first insight into the principal Python tools & libraries used in Science:
conda.
Jupyter Notebook.
NumPy, matplotlib, SciPy
Provide you with the basic skills to face basic tasks such as:
Show other common libraries:
Pandas, scikit-learn (some talks & workshops will focus on these packages)
SymPy
Numba ¿?
1. Jupyter Notebook
The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more.
It has been widely recognised as a great way to distribute scientific papers, because of the capability to have an integrated format with text and executable code, highly reproducible. Top level investigators around the world are already using it, like the team behind the Gravitational Waves discovery (LIGO), whose analysis was translated to an interactive dowloadable Jupyter notebook. You can see it here: https://github.com/minrk/ligo-binder/blob/master/GW150914_tutorial.ipynb
2. Using arrays: NumPy
ndarray object
| index     | 0     | 1     | 2     | 3     | ...   | n-1   | n  |
| ---------- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| value      | 2.1   | 3.6   | 7.8   | 1.5   | ...   | 5.4   | 6.3 |
N-dimensional data structure.
Homogeneously typed.
Efficient!
A universal function (or ufunc for short) is a function that operates on ndarrays. It is a “vectorized function".
End of explanation
one_dim_array = 
two_dim_array = 
# size & shape
# data type
# usual arrays
# changing the shape
# linspace
Explanation: Array creation
End of explanation
one_dim_array
two_dim_array
Explanation: Basic slicing
End of explanation
# Chess board
chess_board = np.zeros([8, 8], dtype=int)
# your code
chess_board
Explanation: [start:stop:step]
End of explanation
# drawing the chessboard
Explanation: 2. Drawing: Matplotlib
End of explanation
# numpy functions
x = 
y = 
# plotting
# another function
# transpose
two_dim_array = 
# matrix multiplication
# matrix vector
# inv
# eigenvectors & eigenvalues
Explanation: Operations & linalg
End of explanation
from IPython.display import HTML
HTML('<iframe src="http://www.mambiente.munimadrid.es/sica/scripts/index.php" \
            width="700" height="400"></iframe>')
Explanation: Air quality data
End of explanation
# Linux command 
!head ./data/barrio_del_pilar-20160322.csv
# Windows
# !gc log.txt | select -first 10 # head
# loading the data
# ./data/barrio_del_pilar-20160322.csv
data2016 = 
Explanation: Loading the data
End of explanation
# mean
# masking invalid data
data2015 = 
Explanation: Dealing with missing values
End of explanation
from IPython.display import HTML
HTML('<iframe src="http://ccaa.elpais.com/ccaa/2015/12/24/madrid/1450960217_181674.html" width="700" height="400"></iframe>')
Explanation: Plotting the data
 Maximum values  from: http://www.mambiente.munimadrid.es/opencms/export/sites/default/calaire/Anexos/valores_limite_1.pdf
NO2
Media anual: 40 µg/m3
Media horaria: 200 µg/m3
End of explanation
# http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.convolve.html
def moving_average(x, N=8):
    return np.convolve(x, np.ones(N)/N, mode='same')
Explanation: CO 
Máxima diaria de las medias móviles octohorarias: 10 mg/m³
End of explanation
HTML('<iframe src="http://eportal.magrama.gob.es/websiar/Ficha.aspx?IdProvincia=28&IdEstacion=1" width="700" height="400"></iframe>')
Explanation: O3
Máxima diaria de las medias móviles octohorarias: 120 µg/m3
Umbral de información. 180 µg/m3
Media horaria. Umbral de alerta. 240 µg/m3
4. Scientific functions: SciPy
```
scipy.linalg: ATLAS LAPACK and BLAS libraries
scipy.stats: distributions, statistical functions...
scipy.integrate: integration of functions and ODEs
scipy.optimization: local and global optimization, fitting, root finding...
scipy.interpolate: interpolation, splines...
scipy.fftpack: Fourier trasnforms
scipy.signal, scipy.special, scipy.io
```
Temperature data
Now, we will use some temperature data from the Spanish Ministry of Agriculture.
End of explanation
!head data/M01_Center_Finca_temperature_data_2004_2015.csv
# Loading the data
temp_data = 
# Importing SciPy stats
# Applying some functions: describe, mode, mean...
Explanation: The file contains data from 2004 to 2015 (included). Each row corresponds to a day of the year, so evey 365 lines contain data from a whole year*
Note1: 29th February has been removed for leap-years.
Note2: Missing values have been replaced with the immediately prior valid data.
These kind of events are better handled with Pandas!
End of explanation
temp_data2 = np.zeros([365, 3, 12])
# Calculating mean of mean temp
# max of max
# min of min
Explanation: We can also get information about percentiles!
Rearranging the data
End of explanation
plt.style.available
# plotting max_max, min_min, mean_mean
Explanation: Let's visualize data!
Using matplotlib styles http://matplotlib.org/users/whats_new.html#styles
End of explanation
# mean vs mean_mean
# and max, min 2015
Explanation: Let's see if 2015 was a normal year...
End of explanation
#we will use numpy functions in order to work with numpy arrays
def funcion(x,y):
    return 
# 0D: works!
funcion(3,5)
# 1D: works!
x = np.
plt.plot( , )
Explanation: But the power of Matplotlib does not end here!
For example, lets represent a function over a 2D domain!
For this we will use the contour function, which requires some special inputs...
End of explanation
#We can create the X and Y matrices by hand, or use a function designed to make ir easy:
#we create two 1D arrays of the desired lengths:
x_1d = np.linspace(0, 5, 5)
y_1d = np.linspace(-2, 4, 7)
#And we use the meshgrid function to create the X and Y matrices!
X, Y = 
X
Y
Explanation: In oder to plot the 2D function, we will need a grid.
For 1D domain, we just needed one 1D array containning the X position and another 1D array containing the value.
Now, we will create a grid, a distribution of points covering a surface. For the 2D domain, we will need:
- One 2D array containing the X coordinate of the points.
- One 2D array containing the Y coordinate of the points.
- One 2D array containing the function value at the points.
The three matrices must have the exact same dimensions, because each cell of them represents a particular point.
End of explanation
#Using Numpy arrays, calculating the function value at the points is easy!
Z 
#Let's plot it!
Explanation: Note that with the meshgrid function we can only create rectangular grids
End of explanation
x_1d = np.???(0, 5, 100)
y_1d = np.???(-2, 4, 100)
X, Y = np.???( , )
Z = funcion(X,Y)
plt.contour(X, Y, Z)
plt.colorbar()
Explanation: We can try a little more resolution...
End of explanation
plt.contourf( , , , ,cmap=plt.cm.Spectral) #With cmap, a color map is specified
plt.colorbar()
plt.contourf( , , , ,cmap=plt.cm.Spectral)
plt.colorbar()
#We can even combine them!
plt.contourf(X, Y, Z, np.linspace(-2, 2, 100),cmap=plt.cm.Spectral)
plt.colorbar()
cs = plt.???(X, Y, Z, np.linspace(-2, 2, 9), colors='k')
plt.clabel(cs)
Explanation: The countourf function is simmilar, but it colours also between the lines. In both functions, we can manually adjust the number of lines/zones we want to differentiate on the plot.
End of explanation
time_vector = np. ('data/ligo_tiempos.txt')
frequency_vector =  np. ('data/ligo_frecuencias.txt')
intensity_matrix =  np. ('data/ligo_datos.txt')
Explanation: These functions can be enormously useful when you want to visualize something.
And remember!
Always visualize data!
Let's try it with Real data!
End of explanation
time_2D, freq_2D = np.
plt. ( ) #We can manually adjust the sice of the picture
plt. ( , , ,np.linspace(0, 0.02313, 200),cmap='bone')
plt.xlabel('time (s)')
plt.ylabel('Frequency (Hz)')
plt.colorbar()
Explanation: The time and frequency vectors contain the values at which the instrument was reading, and the intensity matrix, the postprocessed strength measured for each frequency at each time.
We need again to create the 2D arrays of coordinates.
End of explanation
plt.figure(figsize=(10,6))
plt.contourf(time_2D, freq_2D,intensity_matrix,np.linspace(0, 0.02313, 200),cmap = plt.cm.Spectral)
plt.colorbar()
plt.contour(time_2D, freq_2D,intensity_matrix,np.linspace(0, 0.02313, 9), colors='k')
plt.xlabel('time (s)')
plt.ylabel('Frequency (Hz)')
plt.axis([9.9, 10.05, 0, 300])
Explanation: Wow! What is that? Let's zoom into it!
End of explanation
from ipywidgets import interact
#Lets define a extremely simple function:
def ejemplo(x):
    print(x)
 #Try changing the value of x to True, 'Hello' or ['hello', 'world']
#We can control the slider values with more precission:
Explanation: IPython Widgets
The IPython Widgets are interactive tools to use in the notebook. They are fun and very useful to quickly understand how different parameters affect a certain function.
This is based on a section of the PyConEs 14 talk by Kiko Correoso "Hacking the notebook": http://nbviewer.jupyter.org/github/kikocorreoso/PyConES14_talk-Hacking_the_Notebook/blob/master/notebooks/Using%20Interact.ipynb
End of explanation
x = np.linspace(-1, 7, 1000)
fig = plt.figure()
fig.tight_layout()
plt.subplot(211)#This allows us to display multiple sub-plots, and where to put them
plt.plot(x, np.sin(x))
plt.grid(False)
plt.title("Audio signal: modulator")
plt.subplot(212)
plt.plot(x, np.sin(50 * x))
plt.grid(False)
plt.title("Radio signal: carrier")
#Am modulation simply works like this:
am_wave = np.sin(50 * x) * (0.5 + 0.5 * np.sin(x))
plt.plot(x, am_wave)
Explanation: If you want a dropdown menu that passes non-string values to the Python function, you can pass a dictionary. The keys in the dictionary are used for the names in the dropdown menu UI and the values are the arguments that are passed to the underlying Python function.
Let's have some fun! We talked before about frequencys and waves. Have you ever learn about AM and FM modulation? It's the process used to send radio communications!
End of explanation
def am_mod (f_carr=50, f_mod=1, depth=0.5): #The default values will be the starting points of the sliders
    
    
interact(am_mod,
        f_carr = (1,100,2),
        f_mod = (0.2, 2, 0.1),
        depth = (0, 1, 0.1))
Explanation: In order to interact with it, we will need to transform it into a function
End of explanation
# Importación
from sympy import init_session
init_session(use_latex='matplotlib') #We must start calling this function
Explanation: Other options...
5. Other packages
Symbolic calculations with SymPy
SymPy is a Python package for symbolic math. We will not cover it in depth, but let's take a picure of the basics!
End of explanation
coef_traccion = 
w = 
W = 
w, W
Explanation: The basic unit of this package is the symbol. A simbol object has name and graphic representation, which can be different:
End of explanation
x, y, z, t = symbols('x y z t', real=True)
x.assumptions0
Explanation: By default, SymPy takes symbols as complex numbers. That can lead to unexpected results in front of certain operations, like logarithms. We can explicitly signal that a symbol is real when we create it. We can also create several symbols at a time.
End of explanation
expr = 
expr
#We can substitute pieces of the expression:
expr.
#We can particularize on a certain value:
(sin(x) + 3 * x).
#We can evaluate the numerical value with a certain precission:
(sin(x) + 3 * x).
Explanation: Expressions can be created from symbols:
End of explanation
expr1 = (x ** 3 + 3 * y + 2) ** 2
expr1
expr1.
Explanation: We can manipulate the expression in several ways. For example:
End of explanation
expr = cos(2*x)
expr.
expr_xy = y ** 3 * sin(x) ** 2 + x ** 2 * cos(y)
expr_xy
int2 =  1 / sin(x)
x, a = symbols('x a', real=True)
int3 = 1 / (x**2 + a**2)**2
Explanation: We can derivate and integrate:
End of explanation
a, x, t, C = symbols('a, x, t, C', real=True)
ecuacion = 
ecuacion
x = symbols('x')
f = Function('y')
ecuacion_dif = 
ecuacion_dif
Explanation: We also have ecuations and differential ecuations:
End of explanation
# Notebook style
from IPython.core.display import HTML
css_file = './static/style.css'
HTML(open(css_file, "r").read())
Explanation: Data Analysis with pandas
Pandas is a package that focus on data structures and data analysis tools. We will not cover it because the next workshop, by Kiko Correoso, will develop it in depth.
Machine Learning with scikit-learn
Scikit-learn is a very complete Python package focusing on machin learning, and data mining and analysis. We will not cover it in depth because it will be the focus of many more talks at the PyData.
A world of possibilities...
Thanks for yor attention!
Any Questions?
End of explanation
<END_TASK>