<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Table of Contents
Tasks for those who "feel like a pro"
Step1:  data containers
list
tuple
set
dictionary
for more details see docs
Step2:  Indexing starts with zero.
General indexing rule (mind the brackets)
Step3:  tuples
immutable type!
Step4:  sets
Immutable type. Stores only unique elements.
Step5:  dictionaries
Step7:  Functions
general patterns
Step8:  function is just another object (like almost everything in python)
Step9:  Guidence on how to create meaningful docstring
Step10:  lambda evaluation
Step11:  Don't assign lambda expressions to variables. If you need named instance - create standard function with def
Step12:  vs
Step13:  Numpy - scientific computing
Building matrices and vectors
Step14:  Basic manipulations
matvec
Step15:  broadcasting
Step16:  forcing dtype
Step17:  converting dtypes
Step18:  shapes (singletons)
mind dimensionality!
Step19:  adding new dimension
Step20:  Indexing, slicing
Step21:  Guess what is the output
Step22:  Guess what is the output
Step23:  Reshaping
Step24:  reshape always returns view!
Step25:  Boolean indexing
Step26:  Useful numpy functions
eye, ones, zeros, diag
Example
Step27:  reducers
Step28:  numpy math functions
Step29:  managing output
Step30:  Meshes
linspace, meshgrid
Let's produce a function
$$
f(x, y) = sin(x+y)
$$
on some mesh.
Step31:  Scipy - scientific computing 2
Building sparse matrix
Step32:  How does scipy represent sparse matrix?
Step33:  Sparse matrix stores only non-zero elements (and their indices)
Step34:  Restoring full matrix
Step35:  Popular  (not sparse) matrices
Step36:  Timing - measuring performance
Simplest way to measure time
Step37:  You can also use %%timeit magic to measure run time of the whole cell
Step38:  Storing timings in a separate variable
Avoid using time.time() or time.clock() directly as their behaviour's different depending on platform; default_timer makes the best choice for you. It measures wall time though, e.g. not very precise.
Step39:  Let's make the code less redundant
Step40:  timeit with -o parameter
more details on different parameters
Step41:  Our new benchmark procedure
Step42:  Matplotlib - plotting in python
don't forget to check
* http
Step43:  %matplotlib inline ensures all graphs are plotted inside your notebook
Global controls
(more at http
Step44:  Combined plot
Step45:  Think, why
Step46:  Even simpler way - also gives you granular control on plot objects
Step47:  Plot formatting
matplotlib has a number of different options for styling your plot
Step48:  Subplots
for advanced usage of subplots start here
* http
Step49:  Manual control of subplots
Step50:  Task
Step51:  method 1
Step52:  method 2
Step53:  method 3
Step54:  method 4
Step55:  Task 2
Step56:  Hankel matrix
<ASSISTANT_TASK:>
Python Code:
greeting = 'Hello'
guest = "John"
my_string = 'Hello "John"'
named_greeting = 'Hello, {name}'.format(name=guest)
named_greeting2 = '{}, {}'.format(greeting, guest)
print named_greeting
print named_greeting2
Explanation: Table of Contents
Tasks for those who "feel like a pro":
Learning Resources
Online
Reading (in the future)
Programming in python
Writing code
Some anti-patterns
Python basics
Basic types
variables
strings
data containers
lists
tuples
sets
dictionaries
Functions
general patterns
functions as arguments
lambda evaluation
Numpy - scientific computing
Building matrices and vectors
Basic manipulations
matvec
broadcasting
forcing dtype
converting dtypes
shapes (singletons)
adding new dimension
Indexing, slicing
View vs Copy
Reshaping
Boolean indexing
Useful numpy functions
reducers: sum, mean, max, min, all, any
numpy math functions
managing output
Meshes
Scipy - scientific computing 2
Building sparse matrix
How does scipy represent sparse matrix?
Restoring full matrix
Popular  (not sparse) matrices:
Timing - measuring performance
Simplest way to measure time
Storing timings in a separate variable
timeit with -o parameter
Matplotlib - plotting in python
Configuring matplotlib
Global controls
Combined plot
Combined plot "one-liner"
Plot formatting
Subplots
Iterating over subplots
Manual control of subplots
Other topics
Solutions
Tasks for those who "feel like a pro":
TASK 1
Write the code to enumerate items in the list:
* items are not ordered
* items are not unique
* don't use loops
* try to be as short as possible (not considering import statements)
Example:
Input
```
items = ['foo', 'bar', 'baz', 'foo', 'baz', 'bar']
```
Output
```
something like:
[0, 1, 2, 0, 2, 1]
```
TASK 2
For each element in a list [0, 1, 2, ..., N] build all possible pairs with other elements of that list.
exclude "self-pairing" (e.g. 0-0, 1-1, 2-2)
don't use loops
try to be as short as possible   (not considering import statements)
Example:
Input:
[0, 1, 2, 3] or just 4
Output:
```
0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3
1, 2, 3, 0, 2, 3, 0, 1, 3, 0, 1, 2
```
Learning Resources
Online
The Hitchhiker’s Guide to Python
http://docs.python-guide.org/en/latest/
Hard way is easier http://learnpythonthehardway.org
Google python class
https://developers.google.com/edu/python/
Python tutorial
https://docs.python.org/2/tutorial/
Python Tutor - code visualizing (developed by MIT)
http://pythontutor.com/
If you feel lost: CodeAcademy https://www.codecademy.com/en/tracks/python
Learning by doing!
Reading (in the future)
Al Sweigart, "Automate the Boring Stuff with Python", https://automatetheboringstuff.com
Mark Lutz, "Python Pocket Reference" (250 pages)
Mark Lutz, "Learning Python" (1600 pages!)
Programming in python
Writing code
code should be readable first!
style guides
PEP8 (PEP = Python Enhancement Proposal) http://legacy.python.org/dev/peps/pep-0008/
writing idiomatic code http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html
Some anti-patterns
looping through dictionaries
http://docs.quantifiedcode.com/python-anti-patterns/performance/index.html
using wildcard imports (from ... import *)
http://docs.quantifiedcode.com/python-anti-patterns/maintainability/from_module_import_all_used.html
Using single letter to name your variables
http://docs.quantifiedcode.com/python-anti-patterns/maintainability/using_single_letter_as_variable_name.html
Comparing things to None the wrong way
http://docs.quantifiedcode.com/python-anti-patterns/readability/comparison_to_none.html
Comparing things to True the wrong way
http://docs.quantifiedcode.com/python-anti-patterns/readability/comparison_to_true.html
Using type() to compare types
http://docs.quantifiedcode.com/python-anti-patterns/readability/do_not_compare_types_use_isinstance.html
Using an unpythonic loop
http://docs.quantifiedcode.com/python-anti-patterns/readability/using_an_unpythonic_loop.html
Using CamelCase in function names
http://docs.quantifiedcode.com/python-anti-patterns/readability/using_camelcase_in_function_names.html
Python basics
Verify your python version by running
python
python --version
This notebook is written in pyhton 2.
Basic types
variables
```python
a = b = 3
c, d = 4, 5
c, d = d, c
```
strings
End of explanation
fruit_list = ['apple', 'orange', 'peach', 'mango', 'bananas', 'pineapple']
name_length = [len(fruit) for fruit in fruit_list]
print name_length
name_with_p = [fruit for fruit in fruit_list if fruit[0]=='p']  #even better: fruit.startswith('p')
numbered_fruits = []
for i, fruit in enumerate(fruit_list):
    numbered_fruits.append('{}.{}'.format(i, fruit))
    
numbered_fruits
Explanation: data containers
list
tuple
set
dictionary
for more details see docs:  https://docs.python.org/2/tutorial/datastructures.html
lists
End of explanation
numbered_fruits[0] = None
numbered_fruits[1:4]
numbered_fruits[1:-1:2]
numbered_fruits[::-1]
Explanation: Indexing starts with zero.
General indexing rule (mind the brackets): [start:stop:step]
End of explanation
p_fruits = (name_with_p[1], name_with_p[0])
p_fruits[1] = 'mango'
single_number_tuple = 3,
single_number_tuple
single_number_tuple + (2,) + (1, 0)
Explanation: tuples
immutable type!
End of explanation
set([0, 1, 2, 1, 1, 1, 3])
Explanation: sets
Immutable type. Stores only unique elements.
End of explanation
fruit_list = ['apple', 'orange', 'mango', 'banana', 'pineapple']
quantities = [3, 5, 2, 3, 4]
order_fruits = {fruit: num \
                for fruit, num in zip(fruit_list, quantities)}
order_fruits
order_fruits['pineapple'] = 2
order_fruits
print order_fruits.keys()
print order_fruits.values()
for fruit, amount in order_fruits.iteritems():
    print 'Buy {num} {entity}s'.format(num=amount, entity=fruit)
Explanation: dictionaries
End of explanation
def my_func(var1, var2, default_var1=0, default_var2 = False):
    
    This is a generic example of python a function.
    You can see this string when do call: my_func?
    
    #do something with vars
    if not default_var2:
        result = var1
    elif default_var1 == 0:
        result = var1
    else:
        result = var1 + var2
    return result
Explanation: Functions
general patterns
End of explanation
print 'Function {} has the following docstring:\n{}'\
        .format(my_func.func_name, my_func.func_doc)
Explanation: function is just another object (like almost everything in python)
End of explanation
def function_over_function(func, *args, **kwargs):
    function_result = func(*args, **kwargs)
    return function_result
function_over_function(my_func, 3, 5, default_var1=1, default_var2=True)
Explanation: Guidence on how to create meaningful docstring:
https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt#docstring-standard
functions as arguments
End of explanation
function_over_function(lambda x, y, factor=10: (x+y)*factor, 1, 2, 5)
Explanation: lambda evaluation
End of explanation
my_simple_func = lambda x: x+1
Explanation: Don't assign lambda expressions to variables. If you need named instance - create standard function with def
End of explanation
def my_simple_func(x):
    return x + 1
Explanation: vs
End of explanation
import numpy as np
matrix_from_list = np.array([[1, 3, 4],
                             [2, 0, 5],
                             [4, 4, 1],
                             [0, 1, 0]])
vector_from_list = np.array([2, 1, 3])
print 'The matrix is\n{matrix}\n\nthe vector is\n{vector}'\
        .format(vector=vector_from_list, matrix=matrix_from_list)
Explanation: Numpy - scientific computing
Building matrices and vectors
End of explanation
matrix_from_list.dot(vector_from_list)
Explanation: Basic manipulations
matvec
End of explanation
matrix_from_list + vector_from_list
Explanation: broadcasting
End of explanation
single_precision_vector = np.array([1, 3, 5, 2], dtype=np.float32)
single_precision_vector.dtype
Explanation: forcing dtype
End of explanation
vector_from_list.dtype
vector_from_list.astype(np.int16)
Explanation: converting dtypes
End of explanation
row_vector = np.array([[1,2,3]])
print 'New vector {} has dimensionality {}'\
        .format(row_vector, row_vector.shape)
print 'The dot-product is: ', matrix_from_list.dot(row_vector)
singleton_vector = row_vector.squeeze()
print 'Squeezed vector {} has shape {}'.format(singleton_vector, singleton_vector.shape)
matrix_from_list.dot(singleton_vector)
Explanation: shapes (singletons)
mind dimensionality!
End of explanation
print singleton_vector[:, np.newaxis]
mat = np.arange(12)
mat.reshape(-1, 4)
mat
print singleton_vector[:, None]
Explanation: adding new dimension
End of explanation
vector12 = np.arange(12)
vector12
Explanation: Indexing, slicing
End of explanation
matrix43 = vector12.reshape(4, 3)
matrix43
Explanation: Guess what is the output:
python
vector12[:3]
vector12[-1]
vector12[:-2]
vector12[3:7]
vector12[::2]
vector12[::-1]
End of explanation
matrix43_copy = matrix43[:]
Explanation: Guess what is the output:
python
matrix43[:, 0]
matrix43[-1, :]
matrix43[::2, :]
matrix43[:3, :-1]
matrix43[3:, 1]
Unlike Matlab, numpy arrays are column-major (or C-major) by default, not row-major (or F-major).
View vs Copy
Working with views is more efficient and is a preferred way.
view is returned whenever basic slicing is used
more details at http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html
making copy is simple:
End of explanation
matrix_to_reshape = np.random.randint(10, 99, size=(6, 4))
matrix_to_reshape
reshaped_matrix = matrix_to_reshape.reshape(8, 3)
reshaped_matrix
Explanation: Reshaping
End of explanation
reshaped_matrix[-1, 0] = 1
np.set_printoptions(formatter={'all':lambda x: '_{}_'.format(x) if x < 10 else str(x)})
matrix_to_reshape[:]
np.set_printoptions()
Explanation: reshape always returns view!
End of explanation
idx = matrix43 > 4
matrix43[idx]
Explanation: Boolean indexing
End of explanation
def three_diagonal(N):
    A = np.zeros((N, N), dtype=np.int)
    for i in range(N):
        A[i, i] = -2
        if i > 0:
            A[i, i-1] = 1
        if i < N-1:
            A[i, i+1] = 1
    return A
print three_diagonal(5)
def numpy_three_diagonal(N):
    main_diagonal = -2 * np.eye(N)
    
    suddiag_value = np.ones(N-1,)
    lower_subdiag = np.diag(suddiag_value, k=-1)
    upper_subdiag = np.diag(suddiag_value, k=1)
    
    result = main_diagonal + lower_subdiag + upper_subdiag
    return result.astype(np.int)
numpy_three_diagonal(5)
Explanation: Useful numpy functions
eye, ones, zeros, diag
Example:
Build three-diagonal matrix with -2's on main diagonal and 1's and subdiagonals
Is this code valid?
End of explanation
A = numpy_three_diagonal(5)
A[0, -1] = 5
A[-1, 0] = 3
print A
print A.sum()
print A.min()
print A.max(axis=0)
print A.sum(axis=0)
print A.mean(axis=1)
print (A > 4).any(axis=1)
Explanation: reducers: sum, mean, max, min, all, any
End of explanation
print np.pi
args = np.arange(0, 2.5*np.pi, 0.5*np.pi)
print np.sin(args)
print np.round(np.sin(args), decimals=2)
Explanation: numpy math functions
End of explanation
'{}, {:.1%}, {:e}, {:.2f}, {:.0f}'.format(*np.sin(args))
np.set_printoptions(formatter={'all':lambda x: '{:.2f}'.format(x)})
print np.sin(args)
np.set_printoptions()
Explanation: managing output
End of explanation
linear_index = np.linspace(0, np.pi, 10, endpoint=True)
mesh_x, mesh_y = np.meshgrid(linear_index, linear_index)
values_3D = np.sin(mesh_x + mesh_y)
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
%matplotlib inline
fig = plt.figure(figsize=(10,6))
ax = fig.gca(projection='3d')
ax.plot_wireframe(mesh_x, mesh_y, values_3D)
ax.view_init(azim=-45, elev=30)
plt.title('The plot of $f(x, y) = sin(x+y)$')
Explanation: Meshes
linspace, meshgrid
Let's produce a function
$$
f(x, y) = sin(x+y)
$$
on some mesh.
End of explanation
import scipy.sparse as sp
def scipy_three_diagonal(N):
    main_diagonal = -2 * np.ones(N, )
    suddiag_values = np.ones(N-1,)
    
    diagonals = [main_diagonal, suddiag_values, suddiag_values]
    # Another option: use sp.eye(N) and add subdiagonals
    offsets = [0, 1, -1]
    
    result = sp.diags(diagonals, offsets, shape=(N, N), format='coo')
    return result
my_sparse_matrix = scipy_three_diagonal(5)
Explanation: Scipy - scientific computing 2
Building sparse matrix
End of explanation
my_sparse_matrix
Explanation: How does scipy represent sparse matrix?
End of explanation
print my_sparse_matrix
Explanation: Sparse matrix stores only non-zero elements (and their indices)
End of explanation
my_sparse_matrix.toarray()
my_sparse_matrix.A
Explanation: Restoring full matrix
End of explanation
from scipy.linalg import toeplitz, hankel
hankel(xrange(4), [-1, -2, -3, -4])
toeplitz(xrange(4))
Explanation: Popular  (not sparse) matrices:
End of explanation
N = 1000
%timeit three_diagonal(N)
%timeit numpy_three_diagonal(N)
%timeit scipy_three_diagonal(N)
Explanation: Timing - measuring performance
Simplest way to measure time
End of explanation
%%timeit
N = 1000
calc = three_diagonal(N)
calc = scipy_three_diagonal(N)
del calc
Explanation: You can also use %%timeit magic to measure run time of the whole cell
End of explanation
from timeit import default_timer as timer
dims = [300, 1000, 3000, 10000]
bench_names = ['loop', 'numpy', 'scipy']
timings = {bench:[] for bench in bench_names}
for n in dims:
    start_time = timer()
    calc = three_diagonal(n)
    time_delta = timer() - start_time
    timings['loop'].append(time_delta)
    
    start_time = timer()
    calc = numpy_three_diagonal(n)
    time_delta = timer() - start_time
    timings['numpy'].append(time_delta)
    
    start_time = timer()
    calc = scipy_three_diagonal(n)
    time_delta = timer() - start_time
    timings['scipy'].append(time_delta)
Explanation: Storing timings in a separate variable
Avoid using time.time() or time.clock() directly as their behaviour's different depending on platform; default_timer makes the best choice for you. It measures wall time though, e.g. not very precise.
End of explanation
dims = [300, 1000, 3000, 10000]
bench_names = ['loop', 'numpy', 'scipy']
timings = {bench_name: [] for bench_name in bench_names}
def timing_machine(func, *args, **kwargs):
    start_time = timer()
    result = func(*args, **kwargs)
    time_delta = timer() - start_time
    return time_delta
for n in dims:
    timings['loop'].append(timing_machine(three_diagonal, n))
    timings['numpy'].append(timing_machine(numpy_three_diagonal, n))
    timings['scipy'].append(timing_machine(scipy_three_diagonal, n))
Explanation: Let's make the code less redundant
End of explanation
timeit_result = %timeit -q -r 5 -o three_diagonal(10)
print 'Best of {} runs: {:.8f}s'.format(timeit_result.repeat,
                                        timeit_result.best)
Explanation: timeit with -o parameter
more details on different parameters:
https://ipython.org/ipython-doc/dev/interactive/magics.html#magic-timeit
End of explanation
dims = [300, 1000, 3000, 10000]
bench_names = ['loop', 'numpy', 'scipy']
bench_funcs = [three_diagonal, numpy_three_diagonal, scipy_three_diagonal]
timings_best = {bench_name: [] for bench_name in bench_names}
for bench_name, bench_func in zip(bench_names, bench_funcs):
    print '\nMeasuring {}'.format(bench_func.func_name)
    for n in dims:
        print n,
        time_result = %timeit -q -o bench_func(n)
        timings_best[bench_name].append(time_result.best)
Explanation: Our new benchmark procedure
End of explanation
import matplotlib.pyplot as plt
%matplotlib inline 
Explanation: Matplotlib - plotting in python
don't forget to check
* http://matplotlib.org/users/pyplot_tutorial.html
* http://matplotlib.org/gallery.html
* http://matplotlib.org/examples/index.html
Configuring matplotlib
End of explanation
# plt.rcParams.update({'axes.labelsize': 'large'})
plt.rcParams.update({'font.size': 14})
Explanation: %matplotlib inline ensures all graphs are plotted inside your notebook
Global controls
(more at http://matplotlib.org/users/customizing.html)
End of explanation
plt.figure(figsize=(10,8))
for bench_name, values in timings_best.iteritems():
    plt.semilogy(dims, values, label=bench_name)
    
plt.legend(loc='best')
plt.title('Benchmarking results with best of timeit', y=1.03)
plt.xlabel('Matrix dimension size')
plt.ylabel('Time, s')
plt.figure(figsize=(10,8))
for bench_name, values in timings.iteritems():
    plt.semilogy(dims, values, label=bench_name)
    
plt.legend(loc='best')
plt.title('Benchmarking results with default_timer', y=1.03)
plt.xlabel('Matrix dimension size')
plt.ylabel('Time, s')
Explanation: Combined plot
End of explanation
k = len(timings_best)
iter_xyf = [item for sublist in zip([dims]*k,
                                    timings_best.values(),
                                    list('rgb'))\
                                for item in sublist]
plt.figure(figsize=(10, 8))
plt.semilogy(*iter_xyf)
plt.legend(timings_best.keys(), loc=2, frameon=False)
plt.title('Benchmarking results - "one-liner"', y=1.03)
plt.xlabel('Matrix dimension size')
plt.ylabel('Time, s')
Explanation: Think, why:
* "loop" was faster then "numpy"
* "scipy" is almost constant
* results for default_timer and "best of timeit" are different
You might want to read the docs:
* https://docs.python.org/2/library/timeit.html#timeit.default_timer
* https://docs.python.org/2/library/time.html#time.clock and https://docs.python.org/2/library/time.html#time.time
Remark: starting from python 3.3 it's recommended to use time.perf_counter() and time.process_time()
https://docs.python.org/3/library/time.html#time.perf_counter
Also note, that for advanced benchmarking it's better to use profiling tools.
Combined plot "one-liner"
Use plt.plot? to get detailed info on function usage.
Task: given lists of x-values, y-falues and plot format strings, plot all three graphs in one line.
Hint: use list comprehensions
End of explanation
plt.figure(figsize=(10, 8))
figs = [plt.semilogy(dims, values, label=bench_name)\
        for bench_name, values in timings.iteritems()];
ax0, = figs[0]
ax0.set_dashes([5, 10, 20, 10, 5, 10])
ax1, = figs[1]
ax1.set_marker('s')
ax1.set_markerfacecolor('r')
ax2, = figs[2]
ax2.set_linewidth(6)
ax2.set_alpha(0.3)
ax2.set_color('m')
Explanation: Even simpler way - also gives you granular control on plot objects
End of explanation
all_markers = [
'.', # point
',', # pixel
'o', # circle
'v', # triangle down
'^', # triangle up
'<', # triangle_left
'>', # triangle_right
'1', # tri_down
'2', # tri_up
'3', # tri_left
'4', # tri_right
'8', # octagon
's', # square
'p', # pentagon
'*', # star
'h', # hexagon1
'H', # hexagon2
'+', # plus
'x', # x
'D', # diamond
'd', # thin_diamond
'|', # vline
]
all_linestyles = [
'-',  # solid line style
'--', # dashed line style
'-.', # dash-dot line style
':',  # dotted line style
'None'# no line
]
all_colors = [
'b', # blue
'g', # green
'r', # red
'c', # cyan
'm', # magenta
'y', # yellow
'k', # black
'w', # white
]
Explanation: Plot formatting
matplotlib has a number of different options for styling your plot
End of explanation
n = len(timings)
experiment_names = timings.keys()
fig, axes = plt.subplots(1, n, sharey=True, figsize=(16,4))
colors = np.random.choice(list('rgbcmyk'), n, replace=False)
markers = np.random.choice(all_markers, n, replace=False)
lines = np.random.choice(all_linestyles, n, replace=False)
for ax_num, ax in enumerate(axes):
    key = experiment_names[ax_num]
    ax.semilogy(dims, timings[key], label=key,
            color=colors[ax_num],
            marker=markers[ax_num],
            markersize=8,
            linestyle=lines[ax_num],
            lw=3)
    ax.set_xlabel('matrix dimension')
    ax.set_title(key)
axes[0].set_ylabel('Time, s')
plt.suptitle('Benchmarking results', fontsize=16,  y=1.03)
Explanation: Subplots
for advanced usage of subplots start here
* http://matplotlib.org/examples/pylab_examples/subplots_demo.html
* http://matplotlib.org/users/tight_layout_guide.html
* http://matplotlib.org/users/gridspec.html
Iterating over subplots
End of explanation
plt.figure()
plt.subplot(211)
plt.plot([1,2,3])
plt.subplot(212)
plt.plot([2,5,4])
Explanation: Manual control of subplots
End of explanation
items = ['foo', 'bar', 'baz', 'foo', 'baz', 'bar']
Explanation: Task: create subplot with 2 columns and 2 rows. Leave bottom left quarter empty. Scipy and numpy benchmarks should go into top row.
Other topics
function wrappers and decorators
installing packages
importing modules
ipyton magic
qtconsole
environment
extensions
profiles (deprecated in jupyter)
profiling
debugging
cython, numba
openmp
OOP
python 2 vs python 3
plotting in python - palletes and colormaps, styles
pandas (presenting results)
numpy strides, contiguousness, vectorize function, broadcasting, saving output
magic functions (applied to line and to code cell)
jupyter configuration
Solutions
Task 1
End of explanation
from collections import defaultdict
item_ids = defaultdict(lambda: len(item_ids))
map(item_ids.__getitem__, items)
Explanation: method 1
End of explanation
import pandas as pd
pd.DataFrame({'items': items}).groupby('items', sort=False).grouper.group_info[0]
Explanation: method 2
End of explanation
import numpy as np
np.unique(items, return_inverse=True)[1]
Explanation: method 3
End of explanation
last = 0
counts = {}
result = []
for item in items:
    try:
        count = counts[item]
    except KeyError:
        counts[item] = count = last
        last += 1
    result.append(count)
result
Explanation: method 4
End of explanation
N = 1000
from itertools import permutations
%timeit list(permutations(xrange(N), 2))
Explanation: Task 2
End of explanation
import numpy as np
from scipy.linalg import hankel
def pairs_idx(n):
    return np.vstack((np.repeat(xrange(n), n-1), hankel(xrange(1, n), xrange(-1, n-1)).ravel()))
%timeit pairs_idx(N)
Explanation: Hankel matrix: $a_{ij} = a_{i-1, j+1}$
End of explanation
<END_TASK>