<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Caffe2 Basic Concepts - Operators & Nets
In this tutorial we will go through a set of Caffe2 basics
Step1:  You might see a warning saying that caffe2 does not have GPU support. That means you are running a CPU-only build. Don't be alarmed - anything CPU is still runnable without problem.
Workspaces
Let's cover workspaces first, where all the data reside.
If you are familiar with Matlab, workspace consists of blobs you create and store in memory. For now, consider a blob to be a N-dimensional Tensor similar to numpy's ndarray, but is contiguous. Down the road, we will show you that a blob is actually a typed pointer that can store any type of C++ objects, but Tensor is the most common type stored in a blob. Let's show what the interface looks like.
Blobs() prints out all existing blobs in the workspace. 
HasBlob() queries if a blob exists in the workspace. For now, we don't have anything yet.
Step2:  We can feed blobs into the workspace using FeedBlob().
Step3:  Now, let's take a look what blobs there are in the workspace.
Step4:  Let's verify that the arrays are equal.
Step5:  Also, if you are trying to access a blob that does not exist, an error will be thrown
Step6:  One thing that you might not use immediately
Step7:  Let's switch back to the default workspace.
Step8:  Finally, ResetWorkspace() clears anything that is in the current workspace.
Step9:  Operators
Operators in Caffe2 are kind of like functions. From the C++ side, they all derive from a common interface, and are registered by type, so that we can call different operators during runtime. The interface of operators is defined in caffe2/proto/caffe2.proto. Basically, it takes in a bunch of inputs, and produces a bunch of outputs.
Remember, when we say "create an operator" in Caffe2 Python, nothing gets run yet. All it does is to create the protocol buffere that specifies what the operator should be. At a later time it will be sent to the C++ backend for execution. If you are not familiar with protobuf, it is a json-like serialization tool for structured data. Find more about protocol buffers here.
Let's see an actual example.
Step10:  As we mentioned, the created op is actually a protobuf object. Let's show the content.
Step11:  OK, let's run the operator. We first feed in the input X to the workspace. 
Then the simplest way to run an operator is to do workspace.RunOperatorOnce(operator)
Step12:  After execution, let's see if the operator is doing the right thing, which is our neural network's activation function (Relu) in this case.
Step13:  This is working if your Expected output matches your Y output in this example.
Operators also take optional arguments if needed. They are specified as key-value pairs. Let's take a look at one simple example, which takes a tensor and fills it with Gaussian random variables.
Step14:  Let's run it and see if things are as intended.
Step15:  If you see a bell shaped curve then it worked!
Nets
Nets are essentially computation graphs. We keep the name Net for backward consistency (and also to pay tribute to neural nets). A Net is composed of multiple operators just like a program written as a sequence of commands. Let's take a look.
When we talk about nets, we will also talk about BlobReference, which is an object that wraps around a string so we can do easy chaining of operators.
Let's create a network that is essentially the equivalent of the following python math
Step16:  Let's create a blob called X, and use GaussianFill to fill it with some random data.
Step17:  You might have observed a few differences from the earlier core.CreateOperator call. Basically, when we have a net, you can direct create an operator and add it to the net at the same time using Python tricks
Step18:  Let's continue to create W and b.
Step19:  Now, one simple code sugar
Step20:  Under the hood, X.FC(...) simply delegates to net.FC by inserting X as the first input of the corresponding operator, so what we did above is equivalent to
Y = net.FC([X, W, b], ["Y"])
Let's take a look at the current network.
Step21:  Too verbose huh? Let's try to visualize it as a graph. Caffe2 ships with a very minimal graph visualization tool for this purpose. Let's show that in ipython.
Step22:  So we have defined a Net, but nothing gets executed yet. Remember that the net above is essentially a protobuf that holds the definition of the network. When we actually want to run the network, what happens under the hood is
Step23:  Now let's try the second way to create the net, and run it. First clear the variables with ResetWorkspace(), create the net with the workspace's net object you created earlier CreateNet(net_object), and then run the net by name with RunNet(net_name).
Step24:  There are a few differences between RunNetOnce and RunNet, but probably the main difference is the computation time overhead. Since RunNetOnce involves serializing the protobuf to pass between Python and C and instantiating the network, it may take longer to run. Let's see in this case what the overhead is.
<ASSISTANT_TASK:>
Python Code:
# We'll also import a few standard python libraries
from matplotlib import pyplot
import numpy as np
import time
# These are the droids you are looking for.
from caffe2.python import core, workspace
from caffe2.proto import caffe2_pb2
# Let's show all plots inline.
%matplotlib inline
Explanation: Caffe2 Basic Concepts - Operators & Nets
In this tutorial we will go through a set of Caffe2 basics: the basic concepts including how operators and nets are being written.
First, let's import caffe2. core and workspace are usually the two that you need most. If you want to manipulate protocol buffers generated by caffe2, you probably also want to import caffe2_pb2 from caffe2.proto.
End of explanation
print("Current blobs in the workspace: {}".format(workspace.Blobs()))
print("Workspace has blob 'X'? {}".format(workspace.HasBlob("X")))
Explanation: You might see a warning saying that caffe2 does not have GPU support. That means you are running a CPU-only build. Don't be alarmed - anything CPU is still runnable without problem.
Workspaces
Let's cover workspaces first, where all the data reside.
If you are familiar with Matlab, workspace consists of blobs you create and store in memory. For now, consider a blob to be a N-dimensional Tensor similar to numpy's ndarray, but is contiguous. Down the road, we will show you that a blob is actually a typed pointer that can store any type of C++ objects, but Tensor is the most common type stored in a blob. Let's show what the interface looks like.
Blobs() prints out all existing blobs in the workspace. 
HasBlob() queries if a blob exists in the workspace. For now, we don't have anything yet.
End of explanation
X = np.random.randn(2, 3).astype(np.float32)
print("Generated X from numpy:\n{}".format(X))
workspace.FeedBlob("X", X)
Explanation: We can feed blobs into the workspace using FeedBlob().
End of explanation
print("Current blobs in the workspace: {}".format(workspace.Blobs()))
print("Workspace has blob 'X'? {}".format(workspace.HasBlob("X")))
print("Fetched X:\n{}".format(workspace.FetchBlob("X")))
Explanation: Now, let's take a look what blobs there are in the workspace.
End of explanation
np.testing.assert_array_equal(X, workspace.FetchBlob("X"))
Explanation: Let's verify that the arrays are equal.
End of explanation
try:
    workspace.FetchBlob("invincible_pink_unicorn")
except RuntimeError as err:
    print(err)
Explanation: Also, if you are trying to access a blob that does not exist, an error will be thrown:
End of explanation
print("Current workspace: {}".format(workspace.CurrentWorkspace()))
print("Current blobs in the workspace: {}".format(workspace.Blobs()))
# Switch the workspace. The second argument "True" means creating 
# the workspace if it is missing.
workspace.SwitchWorkspace("gutentag", True)
# Let's print the current workspace. Note that there is nothing in the
# workspace yet.
print("Current workspace: {}".format(workspace.CurrentWorkspace()))
print("Current blobs in the workspace: {}".format(workspace.Blobs()))
Explanation: One thing that you might not use immediately: you can have multiple workspaces in Python using different names, and switch between them. Blobs in different workspaces are separate from each other. You can query the current workspace using CurrentWorkspace. Let's try switching the workspace by name (gutentag) and creating a new one if it doesn't exist.
End of explanation
workspace.SwitchWorkspace("default")
print("Current workspace: {}".format(workspace.CurrentWorkspace()))
print("Current blobs in the workspace: {}".format(workspace.Blobs()))
Explanation: Let's switch back to the default workspace.
End of explanation
workspace.ResetWorkspace()
Explanation: Finally, ResetWorkspace() clears anything that is in the current workspace.
End of explanation
# Create an operator.
op = core.CreateOperator(
    "Relu", # The type of operator that we want to run
    ["X"], # A list of input blobs by their names
    ["Y"], # A list of output blobs by their names
)
# and we are done!
Explanation: Operators
Operators in Caffe2 are kind of like functions. From the C++ side, they all derive from a common interface, and are registered by type, so that we can call different operators during runtime. The interface of operators is defined in caffe2/proto/caffe2.proto. Basically, it takes in a bunch of inputs, and produces a bunch of outputs.
Remember, when we say "create an operator" in Caffe2 Python, nothing gets run yet. All it does is to create the protocol buffere that specifies what the operator should be. At a later time it will be sent to the C++ backend for execution. If you are not familiar with protobuf, it is a json-like serialization tool for structured data. Find more about protocol buffers here.
Let's see an actual example.
End of explanation
print("Type of the created op is: {}".format(type(op)))
print("Content:\n")
print(str(op))
Explanation: As we mentioned, the created op is actually a protobuf object. Let's show the content.
End of explanation
workspace.FeedBlob("X", np.random.randn(2, 3).astype(np.float32))
workspace.RunOperatorOnce(op)
Explanation: OK, let's run the operator. We first feed in the input X to the workspace. 
Then the simplest way to run an operator is to do workspace.RunOperatorOnce(operator)
End of explanation
print("Current blobs in the workspace: {}\n".format(workspace.Blobs()))
print("X:\n{}\n".format(workspace.FetchBlob("X")))
print("Y:\n{}\n".format(workspace.FetchBlob("Y")))
print("Expected:\n{}\n".format(np.maximum(workspace.FetchBlob("X"), 0)))
Explanation: After execution, let's see if the operator is doing the right thing, which is our neural network's activation function (Relu) in this case.
End of explanation
op = core.CreateOperator(
    "GaussianFill",
    [], # GaussianFill does not need any parameters.
    ["Z"],
    shape=[100, 100], # shape argument as a list of ints.
    mean=1.0,  # mean as a single float
    std=1.0, # std as a single float
)
print("Content of op:\n")
print(str(op))
Explanation: This is working if your Expected output matches your Y output in this example.
Operators also take optional arguments if needed. They are specified as key-value pairs. Let's take a look at one simple example, which takes a tensor and fills it with Gaussian random variables.
End of explanation
workspace.RunOperatorOnce(op)
temp = workspace.FetchBlob("Z")
pyplot.hist(temp.flatten(), bins=50)
pyplot.title("Distribution of Z")
Explanation: Let's run it and see if things are as intended.
End of explanation
net = core.Net("my_first_net")
print("Current network proto:\n\n{}".format(net.Proto()))
Explanation: If you see a bell shaped curve then it worked!
Nets
Nets are essentially computation graphs. We keep the name Net for backward consistency (and also to pay tribute to neural nets). A Net is composed of multiple operators just like a program written as a sequence of commands. Let's take a look.
When we talk about nets, we will also talk about BlobReference, which is an object that wraps around a string so we can do easy chaining of operators.
Let's create a network that is essentially the equivalent of the following python math:
X = np.random.randn(2, 3)
W = np.random.randn(5, 3)
b = np.ones(5)
Y = X * W^T + b
We'll show the progress step by step. Caffe2's core.Net is a wrapper class around a NetDef protocol buffer.
When creating a network, its underlying protocol buffer is essentially empty other than the network name. Let's create the net and then show the proto content.
End of explanation
X = net.GaussianFill([], ["X"], mean=0.0, std=1.0, shape=[2, 3], run_once=0)
print("New network proto:\n\n{}".format(net.Proto()))
Explanation: Let's create a blob called X, and use GaussianFill to fill it with some random data.
End of explanation
print("Type of X is: {}".format(type(X)))
print("The blob name is: {}".format(str(X)))
Explanation: You might have observed a few differences from the earlier core.CreateOperator call. Basically, when we have a net, you can direct create an operator and add it to the net at the same time using Python tricks: essentially, if you call net.SomeOp where SomeOp is a registered type string of an operator, this essentially gets translated to
op = core.CreateOperator("SomeOp", ...)
net.Proto().op.append(op)
Also, you might be wondering what X is. X is a BlobReference which basically records two things:
- what its name is. You can access the name by str(X)
- which net it gets created from. It is recorded by an internal variable _from_net, but most likely
you won't need that.
Let's verify it. Also, remember, we are not actually running anything yet, so X contains nothing but a symbol. Don't expect to get any numerical values out of it right now :)
End of explanation
W = net.GaussianFill([], ["W"], mean=0.0, std=1.0, shape=[5, 3], run_once=0)
b = net.ConstantFill([], ["b"], shape=[5,], value=1.0, run_once=0)
Explanation: Let's continue to create W and b.
End of explanation
Y = X.FC([W, b], ["Y"])
Explanation: Now, one simple code sugar: since the BlobReference objects know what net it is generated from, in addition to creating operators from net, you can also create operators from BlobReferences. Let's create the FC operator in this way.
End of explanation
print("Current network proto:\n\n{}".format(net.Proto()))
Explanation: Under the hood, X.FC(...) simply delegates to net.FC by inserting X as the first input of the corresponding operator, so what we did above is equivalent to
Y = net.FC([X, W, b], ["Y"])
Let's take a look at the current network.
End of explanation
from caffe2.python import net_drawer
from IPython import display
graph = net_drawer.GetPydotGraph(net, rankdir="LR")
display.Image(graph.create_png(), width=800)
Explanation: Too verbose huh? Let's try to visualize it as a graph. Caffe2 ships with a very minimal graph visualization tool for this purpose. Let's show that in ipython.
End of explanation
workspace.ResetWorkspace()
print("Current blobs in the workspace: {}".format(workspace.Blobs()))
workspace.RunNetOnce(net)
print("Blobs in the workspace after execution: {}".format(workspace.Blobs()))
# Let's dump the contents of the blobs
for name in workspace.Blobs():
    print("{}:\n{}".format(name, workspace.FetchBlob(name)))
Explanation: So we have defined a Net, but nothing gets executed yet. Remember that the net above is essentially a protobuf that holds the definition of the network. When we actually want to run the network, what happens under the hood is:
- Instantiate a C++ net object from the protobuf;
- Call the instantiated net's Run() function.
Before we do anything, we should clear any earlier workspace variables with ResetWorkspace().
Then there are two ways to run a net from Python. We will do the first option in the example below.
Using workspace.RunNetOnce(), which instantiates, runs and immediately destructs the network. 
A little bit more complex and involves two steps: 
    (a) call workspace.CreateNet() to create the C++ net object owned by the workspace, and
    (b) use workspace.RunNet() by passing the name of the network to it.
End of explanation
workspace.ResetWorkspace()
print("Current blobs in the workspace: {}".format(workspace.Blobs()))
workspace.CreateNet(net)
workspace.RunNet(net.Proto().name)
print("Blobs in the workspace after execution: {}".format(workspace.Blobs()))
for name in workspace.Blobs():
    print("{}:\n{}".format(name, workspace.FetchBlob(name)))
Explanation: Now let's try the second way to create the net, and run it. First clear the variables with ResetWorkspace(), create the net with the workspace's net object you created earlier CreateNet(net_object), and then run the net by name with RunNet(net_name).
End of explanation
# It seems that %timeit magic does not work well with
# C++ extensions so we'll basically do for loops
start = time.time()
for i in range(1000):
    workspace.RunNetOnce(net)
end = time.time()
print('Run time per RunNetOnce: {}'.format((end - start) / 1000))
start = time.time()
for i in range(1000):
    workspace.RunNet(net.Proto().name)
end = time.time()
print('Run time per RunNet: {}'.format((end - start) / 1000))
Explanation: There are a few differences between RunNetOnce and RunNet, but probably the main difference is the computation time overhead. Since RunNetOnce involves serializing the protobuf to pass between Python and C and instantiating the network, it may take longer to run. Let's see in this case what the overhead is.
End of explanation
<END_TASK>