<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 tangent
Source-to-Source Debuggable Derivatives in Pure Python
As a result, you can finally read your automatic derivative code just like the rest of your program. Tangent is useful to researchers and students who not only want to write their models in Python, but also read and debug automatically-generated derivative code without sacrificing speed and flexibility.
<img src="https
Step1:  Forward mode
Reverse-mode autodiff, or backpropagation, generates efficient derivatives for the types of functions we use in machine learning, where there are usually many (perhaps millions) of input variables and only a single output (our loss). When the inverse is true, where there are many more outputs than inputs, reverse mode is not an efficient algorithm, as it has to be run as many times as there are output variables. However, a less famous algorithm, forward-mode autodiff, only has to be run as many times as there are input variables.). Tangent supports forward-mode autodiff.
Step2:  Hessian-vector products
To take higher-order derivatives, you can use any combination of forward- and reverse-mode autodiff in Tangent. This works because the code Tangent produces can also be fed back in as input. The autodiff literature recommends calculating HVPs in a “Forward-over-Reverse” style. This means first apply reverse mode autodiff to the function, and then apply forward mode to that.
<ASSISTANT_TASK:>
Python Code:
import tangent
import tensorflow as tf
def f(x):
    a = x * x
    b = x * a
    c = a + b
    return c
df = tangent.grad(f)
df
df(33)
Explanation: tangent
Source-to-Source Debuggable Derivatives in Pure Python
As a result, you can finally read your automatic derivative code just like the rest of your program. Tangent is useful to researchers and students who not only want to write their models in Python, but also read and debug automatically-generated derivative code without sacrificing speed and flexibility.
<img src="https://raw.githubusercontent.com/google/tangent/master/docs/toolspace.png">
How
Under the hood, tangent.grad grabs the source code of the Python function you pass it (using inspect.getsource, which is available in the Python standard library), converts the source code into an abstract syntax tree (AST) using ast.parse (also built into the Python standard library), and walks the syntax tree in reverse order.
End of explanation
forward_df = tangent.grad(f, mode='forward')
forward_df(33, dx=1)
forward_df(33, dx=2)
forward_df(33, dx=-1)
Explanation: Forward mode
Reverse-mode autodiff, or backpropagation, generates efficient derivatives for the types of functions we use in machine learning, where there are usually many (perhaps millions) of input variables and only a single output (our loss). When the inverse is true, where there are many more outputs than inputs, reverse mode is not an efficient algorithm, as it has to be run as many times as there are output variables. However, a less famous algorithm, forward-mode autodiff, only has to be run as many times as there are input variables.). Tangent supports forward-mode autodiff.
End of explanation
def f(x):
    a = x * x * x
    b = a * x ** 2.0
    return tf.reduce_sum(b)
hvp = tangent.grad(tangent.grad(f, mode='reverse'), mode='forward')
def f(W,x):
    h1 = tf.matmul(x,W)
    h2 = tf.tanh(h1)
    out = tf.reduce_sum(h2)
    return out
dfdW = tangent.grad(f)
dfdW
W = tf.Variable(tf.zeros([100, 10]))
x = tf.Variable(tf.zeros([10, 100]))
dfdW(W, x)
Explanation: Hessian-vector products
To take higher-order derivatives, you can use any combination of forward- and reverse-mode autodiff in Tangent. This works because the code Tangent produces can also be fed back in as input. The autodiff literature recommends calculating HVPs in a “Forward-over-Reverse” style. This means first apply reverse mode autodiff to the function, and then apply forward mode to that.
End of explanation
<END_TASK>