<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Visualizing Thinc models (with shape inference)
This is a simple notebook showing how you can easily visualize your Thinc models and their inputs and outputs using Graphviz and pydot. If you're installing pydot via the notebook, make sure to restart your kernel (or Google Colab VM – see here for details) after installation.
Step1:  Let's start by defining a model with a number of layers chained together using the chain combinator
Step2:  Here's the visualization we want to achieve for this model
Step3:  We can now use pydot to create a visualization for a given model. You can customize the direction of the notes by setting "rankdir" (e.g. "TB" for "top to bottom") and adjust the font and arrow styling. To make the visualization render nicely in a notebook, we can call into IPython's utilities.
Step4:  Calling visualize_model on the model defined above will render the visualization. However, most dimensions will now show up as (?, ?), instead of the actual dimensions as shown in the graph above. That's because Thinc allows defining models with missing shapes and is able to infer the missing shapes from the data when you call model.initialize. The model visualized here doesn't define all its shapes, so the labels are incomplete.
Step5:  To fill in the missing shapes, we can call model.initialize with examples of the expected input X and output Y. Running visualize_model again now shows the complete shapes.
<ASSISTANT_TASK:>
Python Code:
!pip install "thinc>=8.0.0" pydot graphviz svgwrite
Explanation: Visualizing Thinc models (with shape inference)
This is a simple notebook showing how you can easily visualize your Thinc models and their inputs and outputs using Graphviz and pydot. If you're installing pydot via the notebook, make sure to restart your kernel (or Google Colab VM – see here for details) after installation.
End of explanation
from thinc.api import chain, expand_window, Relu, Maxout, Linear, Softmax
n_hidden = 32
dropout = 0.2
model= chain(
    expand_window(3),
    Relu(nO=n_hidden, dropout=dropout, normalize=True),
    Maxout(nO=n_hidden * 4),
    Linear(nO=n_hidden * 2),
    Relu(nO=n_hidden, dropout=dropout, normalize=True),
    Linear(nO=n_hidden),
    Relu(nO=n_hidden, dropout=dropout),
    Softmax(),
)
Explanation: Let's start by defining a model with a number of layers chained together using the chain combinator:
End of explanation
def get_label(layer):
    layer_name = layer.name
    nO = layer.get_dim("nO") if layer.has_dim("nO") else "?"
    nI = layer.get_dim("nI") if layer.has_dim("nI") else "?"
    return f"{layer.name}|({nO}, {nI})".replace(">", "&gt;")
Explanation: Here's the visualization we want to achieve for this model: the name of the layer or combination of layers and the input and output dimensions. Note that &gt;&gt; refers to a chaining of layers.
This means we need to add a node for each layer, edges connecting the nodes to the previous node (except for the first/last), and labels like "name|(nO,nI)" – for instance, "maxout|(128,32)". Here's a simple function that takes a Thinc layer (i.e. a Model instance) and returns a label with the layer name and its dimensions, if available:
End of explanation
import pydot
from IPython.display import SVG, display
def visualize_model(model):
    dot = pydot.Dot()
    dot.set("rankdir", "LR")
    dot.set_node_defaults(shape="record", fontname="arial", fontsize="10")
    dot.set_edge_defaults(arrowsize="0.7")
    nodes = {}
    for i, layer in enumerate(model.layers):
        label = get_label(layer)
        node = pydot.Node(layer.id, label=label)
        dot.add_node(node)
        nodes[layer.id] = node
        if i == 0:
            continue
        from_node = nodes[model.layers[i - 1].id]
        to_node = nodes[layer.id]
        if not dot.get_edge(from_node, to_node):
            dot.add_edge(pydot.Edge(from_node, to_node))
    display(SVG(dot.create_svg()))
Explanation: We can now use pydot to create a visualization for a given model. You can customize the direction of the notes by setting "rankdir" (e.g. "TB" for "top to bottom") and adjust the font and arrow styling. To make the visualization render nicely in a notebook, we can call into IPython's utilities.
End of explanation
visualize_model(model)
Explanation: Calling visualize_model on the model defined above will render the visualization. However, most dimensions will now show up as (?, ?), instead of the actual dimensions as shown in the graph above. That's because Thinc allows defining models with missing shapes and is able to infer the missing shapes from the data when you call model.initialize. The model visualized here doesn't define all its shapes, so the labels are incomplete.
End of explanation
import numpy
X = numpy.zeros((5, 784), dtype="f")
Y = numpy.zeros((54000, 10), dtype="f")
model.initialize(X=X, Y=Y)
visualize_model(model)
Explanation: To fill in the missing shapes, we can call model.initialize with examples of the expected input X and output Y. Running visualize_model again now shows the complete shapes.
End of explanation
<END_TASK>