<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
Step3:  Theano exercises
This notebook contains Theano exercises not related to machine learning.
The exercises work in the following way
Step4:  Solution
Step9:  Exercise 2
This exercise requires you to create Theano variable, and apply elementwise multiplication and matrix/vector dot product.
Step10:  Solution
Step14:  Exercise 3
This exercise requires you to create Theano tensor variables, do broadcastable addition, and to compute the maximum over part of a tensor.
Step15:  Solution
Step17:  Exercise 4
This exercise requires you to compile a Theano function and call it to execute "x + y".
Step18:  Solution
Step22:  Exercise 5
This exercise makes you use shared variables. You must create them and update them by swapping 2 shared variables values.
Step23:  Solution
Step25:  Exercise 6
This exercise makes you use Theano symbolic differentiation features, grad.
Step26:  Solution
<ASSISTANT_TASK:>
Python Code:
import numpy as np
from theano import function
raise NotImplementedError("TODO: add any other imports you need")
def make_scalar():
    
    Returns a new Theano scalar.
    
    raise NotImplementedError("TODO: implement this function.")
def log(x):
    
    Returns the logarithm of a Theano scalar x.
    
    raise NotImplementedError("TODO: implement this function.")
def add(x, y):
    
    Adds two theano scalars together and returns the result.
    
    raise NotImplementedError("TODO: implement this function.")
    
# The following code will use your code and test it.
a = make_scalar()
b = make_scalar()
c = log(b)
d = add(a, c)
f = function([a, b], d)
a = np.cast[a.dtype](1.)
b = np.cast[b.dtype](2.)
actual = f(a, b)
expected = 1. + np.log(2.)
assert np.allclose(actual, expected)
print("SUCCESS!")
Explanation: Theano exercises
This notebook contains Theano exercises not related to machine learning.
The exercises work in the following way:
You have a cell with TODOs that raise errors with a description of what is needed.
The cell contains a description at the top.
Run the cell (Ctrl-Enter) to execute it. At first, it raises an error.
Modify the cell to implement what is asked in the error.
If you implement correctly all the todos and run the cell, it should print "Success" at the end (there is validation code in the cell). If not, try again.
If you want to check the solution, execute the cell that starts with "%load" after the exercise.
Exercise 1
This exercise requires you to create Theano variables and perform some computation on them.
End of explanation
%load solutions/01_scalar_soln.py
Explanation: Solution
End of explanation
import numpy as np
from theano import function
raise NotImplementedError("TODO: add any other imports you need")
def make_vector():
    
    Returns a new Theano vector.
    
    raise NotImplementedError("TODO: implement this function.")
def make_matrix():
    
    Returns a new Theano matrix.
    
    raise NotImplementedError("TODO: implement this function.")
def elemwise_mul(a, b):
    
    a: A theano matrix
    b: A theano matrix
    Returns the elementwise product of a and b
    
    raise NotImplementedError("TODO: implement this function.")
def matrix_vector_mul(a, b):
    
    a: A theano matrix
    b: A theano vector
    Returns the matrix-vector product of a and b
    
    raise NotImplementedError("TODO: implement this function.")
# The following code will use your code and test it.
a = make_vector()
b = make_vector()
c = elemwise_mul(a, b)
d = make_matrix()
e = matrix_vector_mul(d, c)
f = function([a, b, d], e)
rng = np.random.RandomState([1, 2, 3])
a_value = rng.randn(5).astype(a.dtype)
b_value = rng.rand(5).astype(b.dtype)
c_value = a_value * b_value
d_value = rng.randn(5, 5).astype(d.dtype)
expected = np.dot(d_value, c_value)
actual = f(a_value, b_value, d_value)
assert np.allclose(actual, expected)
print("SUCCESS!")
Explanation: Exercise 2
This exercise requires you to create Theano variable, and apply elementwise multiplication and matrix/vector dot product.
End of explanation
%load solutions/02_vector_mat_soln.py
Explanation: Solution
End of explanation
import numpy as np
from theano import function
raise NotImplementedError("TODO: add any other imports you need")
def make_tensor(dim):
    
    Returns a new Theano tensor with no broadcastable dimensions.
    dim: the total number of dimensions of the tensor.
    (You can use any dtype you like)
    
    raise NotImplementedError("TODO: implement this function.")
def broadcasted_add(a, b):
    
    a: a 3D theano tensor
    b: a 4D theano tensor
    Returns c, a 4D theano tensor, where
    c[i, j, k, l] = a[l, k, i] + b[i, j, k, l]
    for all i, j, k, l
    
    raise NotImplementedError("TODO: implement this function.")
def partial_max(a):
    
    a: a 4D theano tensor
    Returns b, a theano matrix, where
    b[i, j] = max_{k,l} a[i, k, l, j]
    for all i, j
    
    raise NotImplementedError("TODO: implement this function.")
# The following code use your code and test it.
a = make_tensor(3)
b = make_tensor(4)
c = broadcasted_add(a, b)
d = partial_max(c)
f = function([a, b], d)
rng = np.random.RandomState([1, 2, 3])
a_value = rng.randn(2, 2, 2).astype(a.dtype)
b_value = rng.rand(2, 2, 2, 2).astype(b.dtype)
c_value = np.transpose(a_value, (2, 1, 0))[:, None, :, :] + b_value
expected = c_value.max(axis=1).max(axis=1)
actual = f(a_value, b_value)
assert np.allclose(actual, expected), (actual, expected)
print("SUCCESS!")
Explanation: Exercise 3
This exercise requires you to create Theano tensor variables, do broadcastable addition, and to compute the maximum over part of a tensor.
End of explanation
%load solutions/03_tensor_soln.py
Explanation: Solution
End of explanation
from theano import tensor as T
raise NotImplementedError("TODO: add any other imports you need")
def evaluate(x, y, expr, x_value, y_value):
    
    x: A theano variable
    y: A theano variable
    expr: A theano expression involving x and y
    x_value: A numpy value
    y_value: A numpy value
    Returns the value of expr when x_value is substituted for x
    and y_value is substituted for y
    
    raise NotImplementedError("TODO: implement this function.")
# The following code use your code and test it.
x = T.iscalar()
y = T.iscalar()
z = x + y
assert evaluate(x, y, z, 1, 2) == 3
print("SUCCESS!")
Explanation: Exercise 4
This exercise requires you to compile a Theano function and call it to execute "x + y".
End of explanation
%load solutions/04_function_soln.py
Explanation: Solution
End of explanation
import numpy as np
raise NotImplementedError("TODO: add any other imports you need")
def make_shared(shape):
    
    Returns a theano shared variable containing a tensor of the specified
    shape.
    You can use any value you want.
    
    raise NotImplementedError("TODO: implement the function")
def exchange_shared(a, b):
    
    a: a theano shared variable
    b: a theano shared variable
    Uses get_value and set_value to swap the values stored in a and b
    
    raise NotImplementedError("TODO: implement the function")
def make_exchange_func(a, b):
    
    a: a theano shared variable
    b: a theano shared variable
    Returns f
    where f is a theano function, that, when called, swaps the
    values in a and b
    f should not return anything
    
    raise NotImplementedError("TODO: implement the function")
# The following code will use your code and test it.
a = make_shared((5, 4, 3))
assert a.get_value().shape == (5, 4, 3)
b = make_shared((5, 4, 3))
assert a.get_value().shape == (5, 4, 3)
a.set_value(np.zeros((5, 4, 3), dtype=a.dtype))
b.set_value(np.ones((5, 4, 3), dtype=b.dtype))
exchange_shared(a, b)
assert np.all(a.get_value() == 1.)
assert np.all(b.get_value() == 0.)
f = make_exchange_func(a, b)
rval = f()
assert isinstance(rval, list)
assert len(rval) == 0
assert np.all(a.get_value() == 0.)
assert np.all(b.get_value() == 1.)
print("SUCCESS!")
Explanation: Exercise 5
This exercise makes you use shared variables. You must create them and update them by swapping 2 shared variables values.
End of explanation
%load solutions/05_shared_soln.py
Explanation: Solution
End of explanation
from theano import tensor as T
def grad_sum(x, y, z):
    
    x: A theano variable
    y: A theano variable
    z: A theano expression involving x and y
    Returns dz / dx + dz / dy
    
    raise NotImplementedError("TODO: implement this function.")
# The following code will use your code and test it.
x = T.scalar()
y = T.scalar()
z = x + y
s = grad_sum(x, y, z)
assert s.eval({x: 0, y: 0}) == 2
print("SUCCESS!")
Explanation: Exercise 6
This exercise makes you use Theano symbolic differentiation features, grad.
End of explanation
%load solutions/06_grad_soln.py
Explanation: Solution
End of explanation
<END_TASK>