<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 How to use notebook
Notebook is a wonderful environment to write your research notes.
You can merge comments and code in a single document, pass this to
your colleagues and let them check what you did.
Starting is super easy. It comes with the anaconda distribution.
So just start with
Step1:  See the source of python functions/classes with question marks (? or ??)
The information appears in a pop-up window
Step2:  %load 
Use %load http
Step3:  %run  
to run an external code or another notebook
Step4:  %load code 
to load code directly into a cell
Notebook magics
By running %lsmagic in a cell we get a list of all magic commands available.
Step5:  You absolutely need to know how to make appear plots in a page using
Step6:  %bash to run cell with bash
Step7:  %%latex to render cell content in LaTeX
Step8:  Collapsing windows
After embedding big figures, it is possible to collapse or expand them by clicking on the left side of them.
By clicking two times, the window collapses. By clicking one time, the window opens.
Using R from Python ...
To use R from notebook, install rpy2
Step9:  Mixing Python and R
Step10:  %R push X   is  equivalent to %R -i X
%R pull X   is  equivalent to %R -o X
Step11:  We can also directly write in R
Step12:  Timing and profiling
Notebook has several functions that allow one to time their
scripts and profile them.
%time  times a script
Step13:  %%timeit does several times the same operation and computes the average time
Step14:  To limit the number of repetitions use "-n number"
Step15:  To run profilers (processing and memory) you have to install further
packages
Step16:  %memit Profiling the memory usage
Step17:  %mprun  for profiling a code (testmem in the code mymem.py)
Step18:  This works only on an external file.
Results appear in another window.
So, first let's write a code externally and then profile it.
Step19:  The line profiler doesn't currently work.
We use the following work-around.
Step20:  Speed
Check how many cores there are in your machine
Step21:  Widgets
First, we have to install the package ipywidgets
Step22:  Text widgets
Step23:  Buttons
Step24:  Progress bar
Add a bar to show the progress of a loop
Step25:  Interaction
To produce a slider
Step26:  To create a checkbox
Step27:  To pass a string
Step28:  As a decorator of a function
Step29:  Finally, we can bind the input of one widget to the output of another one.
Step30:  Interactive visualization
<ASSISTANT_TASK:>
Python Code:
! pwd
names = !ls *.py
names[:3]
Explanation: How to use notebook
Notebook is a wonderful environment to write your research notes.
You can merge comments and code in a single document, pass this to
your colleagues and let them check what you did.
Starting is super easy. It comes with the anaconda distribution.
So just start with:
jupiter notebook
It will open the browser you chose as default and you will be able to
open an existent notebook (files ending in *.ipynb  as ipython notebook)
or create a new one by opening the menu File -> New Notebook.
Writing in a cell
A cell can be either text or code.
As a default it starts as code. 
You can change it into text (markdown) with ESC-m and back into code with Esc-c.
Markdown cells
You can define a header by adding a pound sign (#) or two (##) to obtain different
sizes in boldface.
Headers
# H1
## H2
### H3
#### H4
##### H5
###### H6
H1
H2
H3
H4
H5
H6
Emphasis
Italic:  * asterisks *   or  _underscores_
asterisks * or _ underscores _
Bold:  ** asterisks **  or __ underscores __
 asterisks  or  underscores 
Combined:  ** asterisks and __ underscores __ **
 asterisks and  underscores 
Strikethrough:  \~\~ scratch this \~\~
~~ scratch this ~~
Each of these symbols can be escaped using a backslash before it,
such a \#  or \*.
Lists
Let's learn with an example:
1. First ordered list item
2. Another item
⋅⋅* Unordered sub-list. 
1. Actual numbers don't matter, just that it's a number
⋅⋅1. Ordered sub-list
4. And another item.
⋅⋅⋅You can have properly indented paragraphs within list items. Notice the blank line above, and the leading spaces (at least one, but we'll use three here to also align the raw Markdown).
⋅⋅⋅To have a line break without a paragraph, you will need to use two trailing spaces.⋅⋅
⋅⋅⋅Note that this line is separate, but within the same paragraph.⋅⋅
⋅⋅⋅(This is contrary to the typical GFM line break behaviour, where trailing spaces are not required.)
* Unordered list can use asterisks
- Or minuses
+ Or pluses
This is the result:
First ordered list item
Another item
⋅⋅* Unordered sub-list. 
Actual numbers don't matter, just that it's a number
⋅⋅1. Ordered sub-list
And another item.
⋅⋅⋅You can have properly indented paragraphs within list items. Notice the blank line above, and the leading spaces (at least one, but we'll use three here to also align the raw Markdown).
⋅⋅⋅To have a line break without a paragraph, you will need to use two trailing spaces.⋅⋅
⋅⋅⋅Note that this line is separate, but within the same paragraph.⋅⋅
⋅⋅⋅(This is contrary to the typical GFM line break behaviour, where trailing spaces are not required.)
Unordered list can use asterisks
Or minuses
Or pluses
Mathematical formulae
Mathematical formulae can be used by surrounding a typical Latex expression
with \$ ... \$:
$ \int x^2 dx = x^3/3 $
URL
You can add a link with [ linked name ] (http://...)
ipython
or simply by writing the link inside angle brackets:
http://ipython.org
or simply with http at the beginning:
http://ipython.org
Code highlighting
You can write highlighted code surrounding it with  ```python  ....  ```
python
print('hello world')
or the same with bash:
bash
for i in (a b c);
do
echo $i
done
Inserting images
You can insert images with:  ![alt](...)
To add an hovering title: ![alt text](... "some text"):
Tables
Tables are created using pipes "|"
Alignment inside the cells can be written using:
- |:-   to align on the left
- |-:   to align on the right
- |:-:  to center
For instance:
\|  Column 1 \| Column 2 \| Column 3 \|
\|  :-   \| :-:  \|  :-  \|
\| column \| center \| left \|
will give:
|Column 1|Column 2|Column 3|
|:-|:-:|-:|
|column |center| left|
Typical shortcuts
| Command | Definition |
| :- | :-:|
| Esc m      | change to markdown cell|
| Esc c      |change to code cell|
| Shift Enter| run cell|
| Ctrl Enter | run cell in place|
| Alt Enter  | run cell and open new cell|
| Esc Enter   | go back to edit mode|
| Ctrl Shift - | split a cell|
| Shift + | merge the cell below|
| Esc dd  |    delete the current cell|
| Esc a   |  insert new cell above|
| Esc b   |  insert new cell below|
| Enter   | switch to edit mode|
Writing in a code cell
First of all, we can use the esclamation mark to use shell commands.
End of explanation
%pycat?
Explanation: See the source of python functions/classes with question marks (? or ??)
The information appears in a pop-up window
End of explanation
%%writefile pythoncode.py
import numpy
def append_if_not_exists(arr, x):
    if x not in arr:
        arr.append(x)
        
def some_useless_slow_function():
    arr = list()
    for i in range(10000):
        x = numpy.random.randint(0, 10000)
        append_if_not_exists(arr, x)
Explanation: %load 
Use %load http://matplotlib.org/mpl_examples/pylab_examples/contour_demo.py
to load the content of a python script.
 %%writefile 
Overwrite pythoncode.py
End of explanation
%run ./foo.py
Explanation: %run  
to run an external code or another notebook
End of explanation
%lsmagic
Explanation: %load code 
to load code directly into a cell
Notebook magics
By running %lsmagic in a cell we get a list of all magic commands available.
End of explanation
%matplotlib inline
import  matplotlib.pyplot as plt
import numpy as np
# The semicolon at the end avoid to print the function
plt.hist(np.linspace(0, 1, 1000)**1.5);
Explanation: You absolutely need to know how to make appear plots in a page using:
% matplolib inline
End of explanation
%%bash
for i in a b c;
do
echo $i
done
Explanation: %bash to run cell with bash
End of explanation
%%latex
\begin{align}
a = \frac{1}{2} && b = \frac{1}{3} && c = \frac{1}{4}\\
a && b && c \\
1 && 2 && 3 
\end{align}
Explanation: %%latex to render cell content in LaTeX
End of explanation
%load_ext rpy2.ipython
# example of R ...
%R X=c(1,4,5,7); sd(X); mean(X)
Explanation: Collapsing windows
After embedding big figures, it is possible to collapse or expand them by clicking on the left side of them.
By clicking two times, the window collapses. By clicking one time, the window opens.
Using R from Python ...
To use R from notebook, install rpy2:
conda install -c r rpy2
End of explanation
import numpy as np
import pylab
X = np.array([0,1,2,3,4])
Y = np.array([3,5,4,6,7])
# Push variables in rpy2
%Rpush X Y
v1 = %R plot(X,Y); print(summary(lm(Y~X))); vv=mean(X)*mean(Y)
# Compute the fit with R and save coefficients in Python
b = %R lm(Y~X)$coef
# Or compute variables in rpy2 and then pull them from the rpy2 space
#%R a=resid(lm(Y-X))
#%Rpull a
#print a, b
Explanation: Mixing Python and R
End of explanation
b = %R a=resid(lm(Y~X))
%Rpull a
print "a", a
print "b", b
Explanation: %R push X   is  equivalent to %R -i X
%R pull X   is  equivalent to %R -o X
End of explanation
%%R -i X,Y -o XYcoef
XYlm = lm(Y~X)
XYcoef = coef(XYlm)
print(summary(XYlm))
par(mfrow=c(2,2))
plot(XYlm)
Explanation: We can also directly write in R
End of explanation
%time {1 for i in xrange(10*1000000)}
Explanation: Timing and profiling
Notebook has several functions that allow one to time their
scripts and profile them.
%time  times a script
End of explanation
%%timeit
x = range(10000)
max(x)
Explanation: %%timeit does several times the same operation and computes the average time
End of explanation
%%timeit -n 100
x = range(10000)
max(x)
Explanation: To limit the number of repetitions use "-n number"
End of explanation
from numpy.random import randn
def add_and_sum(x, y):
    added = x + y
    summed = added.sum(axis=1)
    return summed
x = randn(3000, 3000)
y = randn(3000, 3000)
%prun add_and_sum(x, y)
!python -m cProfile foo.py
!python -m cProfile -o foo.out foo.py
import pstats
stats = pstats.Stats('foo.out')
stats.print_stats()
Explanation: To run profilers (processing and memory) you have to install further
packages:
conda install line_profiler
conda install memory_profiler
%prun  profiles a script by checking how much time each functions takes to run
End of explanation
%load_ext memory_profiler
%memit range(1000000)
%memit list(range(1000000))
Explanation: %memit Profiling the memory usage
End of explanation
import numpy as np
def testmem():
    a = np.arange(1000000)
    b = list(range(1000000))
    del(a)
    del(b)
    
%mprun -f testmem testmem()
Explanation: %mprun  for profiling a code (testmem in the code mymem.py)
End of explanation
%%writefile mymem.py
#mymem.py
import numpy as np
def testmem():
    a = np.arange(1000000)
    b = list(range(1000000))
    del(a)
    del(b)
testmem()
from mymem import testmem
%mprun -f testmem testmem()
from pythoncode import some_useless_slow_function, append_if_not_exists
%prun some_useless_slow_function()
%load_ext memory_profiler
%mprun -f append_if_not_exists some_useless_slow_function()
Explanation: This works only on an external file.
Results appear in another window.
So, first let's write a code externally and then profile it.
End of explanation
import line_profiler
lp = line_profiler.LineProfiler()
lp.add_function(some_useless_slow_function)
lp.runctx('some_useless_slow_function()', locals=locals(), globals=globals())
lp.print_stats()
Explanation: The line profiler doesn't currently work.
We use the following work-around.
End of explanation
import multiprocessing
print multiprocessing.cpu_count(), 'xeon cores'
Explanation: Speed
Check how many cores there are in your machine
End of explanation
from ipywidgets import widgets
Explanation: Widgets
First, we have to install the package ipywidgets:
conda install ipywidgets
then, import it.
End of explanation
from IPython.display import display
text = widgets.Text()
display(text)
def handle_submit(sender):
    print(text.value)
    
text.on_submit(handle_submit)
Explanation: Text widgets
End of explanation
button = widgets.Button(description="Click me !")
display(button)
def on_button_clicked(b):
    print ("Yes, you clicked me !")
    
button.on_click(on_button_clicked)
Explanation: Buttons
End of explanation
from IPython.display import display
from ipywidgets import FloatProgress
import numpy as np
f = FloatProgress(min=0,max=1000)
f.value=0
display(f)
for i in range(1000):
    a = np.exp(np.arange(100.))
    f.value += 1
    
Explanation: Progress bar
Add a bar to show the progress of a loop
End of explanation
from ipywidgets import interact
def f(x):
    print(x)
interact(f,x=10)
Explanation: Interaction
To produce a slider:
End of explanation
interact(f, x=True);
Explanation: To create a checkbox:
End of explanation
interact(f, x='Hi there!');
Explanation: To pass a string:
End of explanation
@interact(x=True, y=1.0)
def g(x, y):
    return (x, y)
Explanation: As a decorator of a function:
End of explanation
outputText = widgets.Text()
outputText
inputText = widgets.Text()
def makeUpperCase(sender):
    outputText.value = inputText.value.upper()
    
inputText.on_submit(makeUpperCase)
inputText
Explanation: Finally, we can bind the input of one widget to the output of another one.
End of explanation
#from IPython.html.widgets import *
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
t = np.arange(0.0,1.0,0.01)
def pltsin(f):
    plt.plot(t,np.sin(2*np.pi*t*f))
    plt.show()
    
interact(pltsin, f=(1,10,0.1))
Explanation: Interactive visualization:
End of explanation
<END_TASK>