<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Error messages
Very important
Have lots of information in them
Take the time to read them.
If you get multiple error messages, the first one is the most important, then the last one, then the others in the middle.
An error message has 4 parts
Step1:  Where in your code does this error come from?
The HINT tells you how to get a better error message.
In Python, you can pass the parameter mode=theano.Mode(optimizer=TODO) to theano.function, instead of changing the Theano flag.
This allows you to see in your code, which line caused the problem!
Step2:  Stack trace
The stack trace can be very useful. You don't need to understand the part in Theano (where the file is inside theano/), just check the part coming from your code files.
DebugMode
Checks and double-checks everything, extremely slow.
Compares Python, C and GPU implementations.
Compares values before and after each optimization.
By default, raises an error on NaN.
Sensitive
Step3:  Printing during execution
Step4:  Printing attributes of a variable
Step5:  Most Frequent NaN Causes
Hyperparameters (ex
Step6:  Test Value
Give sample values to symbolic variables and have the graph execute
as it is being built. This allows to get some type of error like shape
errors when you build the graph instead of during the execution.
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import theano
import theano.tensor as T
x = T.vector()
y = T.vector()
z = x + x
z = z * y
f = theano.function([x, y], z)
f(np.ones((2,)), np.ones((3,)))
Explanation: Error messages
Very important
Have lots of information in them
Take the time to read them.
If you get multiple error messages, the first one is the most important, then the last one, then the others in the middle.
An error message has 4 parts:
The stack trace
The error itself and a small description (the most important) (only 1 line)
Sometimes: extra details
Sometimes: hint to help you get more information
Here is an example of code with a user error. Don't try to find the error by code inspection, check only the error:
End of explanation
# TODO: finish to define the mode below
mode=...
import numpy as np
import theano
import theano.tensor as T
x = T.vector()
y = T.vector()
z = x + x
z.name = "z1"
z = z * y
z.name = "z2"
f = theano.function([x, y], z, mode=mode)
f(np.ones((2,)), np.ones((3,)))
Explanation: Where in your code does this error come from?
The HINT tells you how to get a better error message.
In Python, you can pass the parameter mode=theano.Mode(optimizer=TODO) to theano.function, instead of changing the Theano flag.
This allows you to see in your code, which line caused the problem!
End of explanation
import theano
import theano.tensor as T
from theano.tests.breakpoint import PdbBreakpoint
input, target = T.fvectors(['x', 'y'])
mse = (input - target) ** 2
# Conditional breakpoint to be activated if the total
# MSE > 100. The breakpoint will monitor the inputs,
# targets as well as the individual error values
breakpointOp = PdbBreakpoint("MSE too high")
condition = T.gt(mse.sum(), 100)
mse, monitored_input, monitored_target = breakpointOp(
    condition, mse, input, target)
# Compile the theano function
fct = theano.function([input, target], mse)
# Use the function
print fct([10, 0], [10, 5]) # Will NOT activate the breakpoint
print fct([0, 0], [10, 5]) # Will activate the breakpoint
Explanation: Stack trace
The stack trace can be very useful. You don't need to understand the part in Theano (where the file is inside theano/), just check the part coming from your code files.
DebugMode
Checks and double-checks everything, extremely slow.
Compares Python, C and GPU implementations.
Compares values before and after each optimization.
By default, raises an error on NaN.
Sensitive: so frequently reported errors are OK.
Use the Theano flag mode=DebugMode or the the parameter mode=theano.compile.DebugMode() to theano.function().
Python debugger (PDB)
PDB is similar to GDB
n: next
c: continue
l: list
p var: print
s: step,
bt: print the stack trace
...
To get into PDB:
import pdb; pdb.set_trace()
python -m pdb script.py
To get into PDB with Theano:
Some Theano flags:
mode=FAST_COMPILE disables most of optimizations, and C code generation.
linker=py disables C code generation.
warn_float64=pdb
...
PdbBreakpoint: breakpoint during execution.
Breakpoint during execution
End of explanation
import theano
x = theano.tensor.vector()
o = theano.printing.Print("a message")(x)
f = theano.function([x], o)
d = f([3, 4])
Explanation: Printing during execution
End of explanation
o = theano.printing.Print("Attributes of x:", attrs=('min', 'mean', 'max'))(x)
f = theano.function([x], o)
d = f([3, 1, 4, 9])
Explanation: Printing attributes of a variable
End of explanation
import numpy
import theano
import theano.compile.nanguardmode
from theano import tensor as T
x = T.matrix()
w = theano.shared(numpy.random.randn(5, 7).astype(theano.config.floatX))
y = T.dot(x, w)
mode=theano.compile.nanguardmode.NanGuardMode(nan_is_error=True,
                  inf_is_error=True,
                  big_is_error=True)
fun = theano.function(
    [x], y, mode=mode)
infa = numpy.tile(
    (numpy.asarray(100.) ** 1000000), (3, 5))
fun(infa)
Explanation: Most Frequent NaN Causes
Hyperparameters (ex: learning rate)
Initialization of parameters
Numerical Stability
Algorithm Related
Run in NanGuardMode, DebugMode, or MonitorMode.
NanGuardMode
Can check for:
* Nan
* Inf
* Big values (greater than 1e10)
End of explanation
# Can also be 'off', 'ignore', 'raise', 'pdb'
theano.config.compute_test_value = 'warn'
# input which will be of shape (5, 10)
x, y  = T.matrices('xy')
# provide Theano with a default test-value
x.tag.test_value = numpy.random.rand(5, 10)
y.tag.test_value = numpy.random.rand(4, 10)
x + y # warn about the shape error
Explanation: Test Value
Give sample values to symbolic variables and have the graph execute
as it is being built. This allows to get some type of error like shape
errors when you build the graph instead of during the execution.
End of explanation
<END_TASK>