<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Pragmatic color describers
Step1:  Contents
Overview
Set-up
The corpus
Corpus reader
ColorsCorpusExample instances
Far, Split, and Close conditions
Toy problems for development work
Core model
Toy dataset illustration
Predicting sequences
Listener-based evaluation
BLEU scores
Other prediction and evaluation methods
Cross-validation
Baseline SCC model
Modifying the core model
Illustration
Step2:  The Stanford English Colors in Context corpus (SCC) is included in the data distribution for this course. If you store the data in a non-standard place, you'll need to update the following
Step3:  The corpus
The SCC corpus is based in a two-player interactive game. The two players share a context consisting of three color patches, with the display order randomized between them so that they can't use positional information when communicating.
The speaker is privately assigned a target color and asked to produce a description of it that will enable the listener to identify the speaker's target. The listener makes a choice based on the speaker's message, and the two  succeed if and only if the listener identifies the target correctly.
In the game, the two players played repeated reference games and could communicate with each other in a free-form way. This opens up the possibility of modeling these repeated interactions as task-oriented dialogues. However, for this unit, we'll ignore most of this structure. We'll treat the corpus as a bunch of independent reference games played by anonymous players, and we will ignore the listener and their choices entirely.
For the bake-off, we will be distributing a separate test set. Thus, all of the data in the SCC can be used for exploration and development.
Corpus reader
The corpus reader class is ColorsCorpusReader in colors.py. The reader's primary function is to let you iterate over corpus examples
Step4:  The two keyword arguments have their default values here. 
If you supply word_count with an interger value, it will restrict to just examples where the utterance has that number of words (using a whitespace heuristic). This creates smaller corpora that are useful for development.
The colors in the corpus are in HLS format. With normalize_colors=False, the first (hue) value is an integer between 1 and 360 inclusive, and the L (lightness) and S (saturation) values are between 1 and 100 inclusive. With normalize_colors=True, these values are all scaled to between 0 and 1 inclusive. The default is normalize_colors=True because this is a better choice for all the machine learning models we'll consider.
Step5:  We can verify that we read in the same number of examples as reported in Monroe et al. 2017
Step6:  ColorsCorpusExample instances
The examples are ColorsCorpusExample instances
Step7:  These objects have a lot of attributes and methods designed to help you study the corpus and use it for our machine learning tasks. Let's review some highlights.
Displaying examples
You can see what the speaker saw, with the utterance they chose printed above the patches
Step8:  This is the original order of patches for the speaker. The target happens to be the leftmost patch, as indicated by the black box around it.
Here's what the listener saw, with the speaker's message printed above the patches
Step9:  The listener isn't shown the target, of course, so no patches are highlighted.
If display is called with no arguments, then the target is placed in the final position and the other two are given in an order determined by the corpus metadata
Step10:  This is the representation order we use for our machine learning models.
Color representations
For machine learning, we'll often need to access the color representations directly. The primary attribute for this is colors
Step11:  In this display order, the third element is the target color and the first two are the distractors. The attributes speaker_context and listener_context return the same colors but in the order that those players saw them. For example
Step12:  Utterance texts
Utterances are just strings
Step13:  There are cases where the speaker made a sequences of utterances for the same trial. We follow Monroe et al. 2017 in concatenating these into a single utterance. To preserve the original information, the individual turns are separated by " ### ". Example 3 is the first with this property – let's check it out
Step14:  The method parse_turns will parse this into individual turns
Step15:  For examples consisting of a single turn, parse_turns returns a list of length 1
Step16:  Far, Split, and Close conditions
The SCC contains three conditions
Step17:  Split condition
Step18:  Close condition
Step19:  These conditions go from easiest to hardest when it comes to reliable communication. In the Far condition, the context is hardly relevant, whereas the nature of the distractors reliably shapes the speaker's choices in the other two conditions. 
You can begin to see how this affects speaker choices in the above examples
Step20:  The following verifies that we have the same number of examples per condition as reported in Monroe et al. 2017
Step21:  Toy problems for development work
The SCC corpus is fairly large and quite challenging as an NLU task. This means it isn't ideal when it comes to testing hypotheses and debugging code. Poor performance could trace to a mistake, but it could just as easily trace to the fact that the problem is very challenging from the point of view of optimization.
To address this, the module torch_color_describer.py includes a function create_example_dataset for creating small, easy datasets with the same basic properties as the SCC corpus.
Here's a toy problem containing just six examples
Step22:  Each member of tiny_contexts contains three vectors. This is meant to be an easy problem, so the final (target) vector always has values that unambiguously determine which utterance is produced. Thus, the model basically just needs to learn to ignore the distractors and find the association between the target vector and the corresponding sequence. 
All the models we study have a capacity to solve this task with very little data, so you should see perfect or near perfect performance on reasonably-sized versions of this task.
Core model
Our core model for this problem is implemented in torch_color_describer.py as ContextualColorDescriber. At its heart, this is a pretty standard encoder–decoder model
Step23:  ContextualColorDescriber is a subclass of TorchModelBase, so all of the optimization parameters from that model are available here; see torch_model_base.py for full details.
Here is a simple use of ContextualColorDescriber
Step24:  Predicting sequences
The predict method takes a list of color contexts as input and returns model descriptions
Step25:  We can then check that we predicted all correct sequences
Step26:  For real problems, this is too stringent a requirement, since there are generally many equally good descriptions. This insight gives rise to metrics like BLEU, METEOR, ROUGE, CIDEr, and others, which seek to relax the requirement of an exact match with the test sequence. These are reasonable options to explore, but we will instead adopt a communcation-based evaluation, as discussed in the next section.
Listener-based evaluation
ContextualColorDescriber implements a method listener_accuracy that we will use for our primary evaluations in the assignment and bake-off. The essence of the method is that we can calculate
$$
c^{*} = \text{argmax}_{c \in C} P_S(\text{utterance} \mid c)
$$
where $P_S$ is our describer model and $C$ is the set of all permutations of all three colors in the color context. We take $c^{*}$ to be a correct prediction if it is one where the target is in the privileged final position. (There are two such contexts; we try both in case the order of the distractors influences the predictions, and the model is correct if one of them has the highest probability.)
Here's the listener accuracy of our toy model
Step27:  BLEU scores
The listener-based evaluation scheme has the unusual property that, in some sense, it assesses the model's ability to communicate with itself. This creates a danger that it will drift far from English as we know it but still succeed in signaling the target color. Ideally, we would train a separate listener model to help prevent this, but doing so would be cumbersome and could limit creative system development. Thus, as a quick check that our systems are still going to be able to communicate with us, we can calculate a BLEU score
Step28:  For discussion of BLEU scores, see the evaluation metrics notebook.
Other prediction and evaluation methods
You can get the perplexities for test examples with perplexities
Step29:  You can use predict_proba to see the full probability distributions assigned to test examples
Step30:  Cross-validation
You can use utils.fit_classifier_with_hyperparameter_search to cross-validate these models. Just be sure to set scoring=None so that the sklearn model selection methods use the score method of ContextualColorDescriber, which is an alias for listener_accuracy
Step31:  Baseline SCC model
Just to show how all the pieces come together, here's a very basic SCC experiment using the core code and very simplistic assumptions (which you will revisit in the assignment) about how to represent the examples
Step32:  Here we extract the raw colors and texts (as strings)
Step33:  To tokenize the examples, we'll just split on whitespace, taking care to add the required boundary symbols
Step34:  We'll use a random train–test split
Step35:  Our vocab is determined by the train set, and we take care to include the $UNK token
Step36:  And now we're ready to train a model
Step37:  And finally an evaluation in terms of listener accuracy and BLEU scores. The evaluate method combines these
Step38:  Modifying the core model
The first few assignment problems concern how you preprocess the data for your model. After that, the goal is to subclass model components in torch_color_describer.py. For the bake-off submission, you can do whatever you like in terms of modeling, but my hope is that you'll be able to continue subclassing based on torch_color_describer.py.
This section provides some illustrative examples designed to give you a feel for how the code is structured and what your options are in terms of creating subclasses. The principles are the same as those reviewed for a wider range of models in tutorial_pytorch_models.ipynb.
Illustration
Step39:  Step 2
Step40:  Step 3
Step41:  Here's an example run
Step42:  Illustration
Step43:  Step 2
Step44:  An example/test run
<ASSISTANT_TASK:>
Python Code:
__author__ = "Christopher Potts"
__version__ = "CS224u, Stanford, Spring 2022"
Explanation: Pragmatic color describers
End of explanation
from colors import ColorsCorpusReader
import os
import pandas as pd
from sklearn.model_selection import train_test_split
import torch
from torch_color_describer import (
    ContextualColorDescriber, create_example_dataset)
import utils
from utils import START_SYMBOL, END_SYMBOL, UNK_SYMBOL
utils.fix_random_seeds()
Explanation: Contents
Overview
Set-up
The corpus
Corpus reader
ColorsCorpusExample instances
Far, Split, and Close conditions
Toy problems for development work
Core model
Toy dataset illustration
Predicting sequences
Listener-based evaluation
BLEU scores
Other prediction and evaluation methods
Cross-validation
Baseline SCC model
Modifying the core model
Illustration: LSTM Cells
Illustration: Deeper models
Overview
This notebook is part of our unit on grounding. It illustrates core concepts from the unit, and it provides useful background material for the associated homework and bake-off.
Set-up
End of explanation
COLORS_SRC_FILENAME = os.path.join(
    "data", "colors", "filteredCorpus.csv")
Explanation: The Stanford English Colors in Context corpus (SCC) is included in the data distribution for this course. If you store the data in a non-standard place, you'll need to update the following:
End of explanation
corpus = ColorsCorpusReader(
    COLORS_SRC_FILENAME,
    word_count=None,
    normalize_colors=True)
Explanation: The corpus
The SCC corpus is based in a two-player interactive game. The two players share a context consisting of three color patches, with the display order randomized between them so that they can't use positional information when communicating.
The speaker is privately assigned a target color and asked to produce a description of it that will enable the listener to identify the speaker's target. The listener makes a choice based on the speaker's message, and the two  succeed if and only if the listener identifies the target correctly.
In the game, the two players played repeated reference games and could communicate with each other in a free-form way. This opens up the possibility of modeling these repeated interactions as task-oriented dialogues. However, for this unit, we'll ignore most of this structure. We'll treat the corpus as a bunch of independent reference games played by anonymous players, and we will ignore the listener and their choices entirely.
For the bake-off, we will be distributing a separate test set. Thus, all of the data in the SCC can be used for exploration and development.
Corpus reader
The corpus reader class is ColorsCorpusReader in colors.py. The reader's primary function is to let you iterate over corpus examples:
End of explanation
examples = list(corpus.read())
Explanation: The two keyword arguments have their default values here. 
If you supply word_count with an interger value, it will restrict to just examples where the utterance has that number of words (using a whitespace heuristic). This creates smaller corpora that are useful for development.
The colors in the corpus are in HLS format. With normalize_colors=False, the first (hue) value is an integer between 1 and 360 inclusive, and the L (lightness) and S (saturation) values are between 1 and 100 inclusive. With normalize_colors=True, these values are all scaled to between 0 and 1 inclusive. The default is normalize_colors=True because this is a better choice for all the machine learning models we'll consider.
End of explanation
# Should be 46994:
len(examples)
Explanation: We can verify that we read in the same number of examples as reported in Monroe et al. 2017:
End of explanation
ex1 = next(corpus.read())
Explanation: ColorsCorpusExample instances
The examples are ColorsCorpusExample instances:
End of explanation
ex1.display(typ='speaker')
Explanation: These objects have a lot of attributes and methods designed to help you study the corpus and use it for our machine learning tasks. Let's review some highlights.
Displaying examples
You can see what the speaker saw, with the utterance they chose printed above the patches:
End of explanation
ex1.display(typ='listener')
Explanation: This is the original order of patches for the speaker. The target happens to be the leftmost patch, as indicated by the black box around it.
Here's what the listener saw, with the speaker's message printed above the patches:
End of explanation
ex1.display()
Explanation: The listener isn't shown the target, of course, so no patches are highlighted.
If display is called with no arguments, then the target is placed in the final position and the other two are given in an order determined by the corpus metadata:
End of explanation
ex1.colors
Explanation: This is the representation order we use for our machine learning models.
Color representations
For machine learning, we'll often need to access the color representations directly. The primary attribute for this is colors:
End of explanation
ex1.speaker_context
Explanation: In this display order, the third element is the target color and the first two are the distractors. The attributes speaker_context and listener_context return the same colors but in the order that those players saw them. For example:
End of explanation
ex1.contents
Explanation: Utterance texts
Utterances are just strings:
End of explanation
ex3 = examples[2]
ex3.contents
Explanation: There are cases where the speaker made a sequences of utterances for the same trial. We follow Monroe et al. 2017 in concatenating these into a single utterance. To preserve the original information, the individual turns are separated by " ### ". Example 3 is the first with this property – let's check it out:
End of explanation
ex3.parse_turns()
Explanation: The method parse_turns will parse this into individual turns:
End of explanation
ex1.parse_turns()
Explanation: For examples consisting of a single turn, parse_turns returns a list of length 1:
End of explanation
print("Condition type:", examples[1].condition)
examples[1].display()
Explanation: Far, Split, and Close conditions
The SCC contains three conditions:
Far condition: All three colors are far apart in color space.  Example:
End of explanation
print("Condition type:", examples[3].condition)
examples[3].display()
Explanation: Split condition: The target is close to one of the distractors, and the other is far away from both of them. Example:
End of explanation
print("Condition type:", examples[2].condition)
examples[2].display()
Explanation: Close condition: The target is similar to both distractors. Example:
End of explanation
ex1.condition
Explanation: These conditions go from easiest to hardest when it comes to reliable communication. In the Far condition, the context is hardly relevant, whereas the nature of the distractors reliably shapes the speaker's choices in the other two conditions. 
You can begin to see how this affects speaker choices in the above examples: "purple" suffices for the Far condition, a more marked single word ("lime") is used in the Split condition, and the Close condition triggers a pretty long, complex description.
The condition attribute provides access to this value:
End of explanation
pd.Series([ex.condition for ex in examples]).value_counts()
Explanation: The following verifies that we have the same number of examples per condition as reported in Monroe et al. 2017:
End of explanation
tiny_contexts, tiny_words, tiny_vocab = create_example_dataset(
    group_size=2, vec_dim=2)
tiny_vocab
tiny_words
tiny_contexts
Explanation: Toy problems for development work
The SCC corpus is fairly large and quite challenging as an NLU task. This means it isn't ideal when it comes to testing hypotheses and debugging code. Poor performance could trace to a mistake, but it could just as easily trace to the fact that the problem is very challenging from the point of view of optimization.
To address this, the module torch_color_describer.py includes a function create_example_dataset for creating small, easy datasets with the same basic properties as the SCC corpus.
Here's a toy problem containing just six examples:
End of explanation
toy_color_seqs, toy_word_seqs, toy_vocab = create_example_dataset(
    group_size=50, vec_dim=2)
toy_color_seqs_train, toy_color_seqs_test, toy_word_seqs_train, toy_word_seqs_test = \
    train_test_split(toy_color_seqs, toy_word_seqs)
Explanation: Each member of tiny_contexts contains three vectors. This is meant to be an easy problem, so the final (target) vector always has values that unambiguously determine which utterance is produced. Thus, the model basically just needs to learn to ignore the distractors and find the association between the target vector and the corresponding sequence. 
All the models we study have a capacity to solve this task with very little data, so you should see perfect or near perfect performance on reasonably-sized versions of this task.
Core model
Our core model for this problem is implemented in torch_color_describer.py as ContextualColorDescriber. At its heart, this is a pretty standard encoder–decoder model:
Encoder: Processes the color contexts as a sequence. We always place the target in final position so that it is closest to the supervision signals that we get when decoding.
Decoder: A neural language model whose initial hidden representation is the final hidden representation of the Encoder.
EncoderDecoder: Coordinates the operations of the Encoder and Decoder.
Finally, ContextualColorDescriber is a wrapper around these model components. It handles the details of training and implements the prediction and evaluation functions that we will use.
Many additional details about this model are included in the slides for this unit.
Toy dataset illustration
To highlight the core functionality of ContextualColorDescriber, let's create a small toy dataset and use it to train and evaluate a model:
End of explanation
toy_mod = ContextualColorDescriber(toy_vocab, max_iter=200)
_ = toy_mod.fit(toy_color_seqs_train, toy_word_seqs_train)
Explanation: ContextualColorDescriber is a subclass of TorchModelBase, so all of the optimization parameters from that model are available here; see torch_model_base.py for full details.
Here is a simple use of ContextualColorDescriber:
End of explanation
toy_preds = toy_mod.predict(toy_color_seqs_test)
toy_preds[0]
Explanation: Predicting sequences
The predict method takes a list of color contexts as input and returns model descriptions:
End of explanation
toy_correct = sum(1 for x, p in zip(toy_word_seqs_test, toy_preds) if x == p)
toy_correct / len(toy_word_seqs_test)
Explanation: We can then check that we predicted all correct sequences:
End of explanation
toy_mod.listener_accuracy(toy_color_seqs_test, toy_word_seqs_test)
Explanation: For real problems, this is too stringent a requirement, since there are generally many equally good descriptions. This insight gives rise to metrics like BLEU, METEOR, ROUGE, CIDEr, and others, which seek to relax the requirement of an exact match with the test sequence. These are reasonable options to explore, but we will instead adopt a communcation-based evaluation, as discussed in the next section.
Listener-based evaluation
ContextualColorDescriber implements a method listener_accuracy that we will use for our primary evaluations in the assignment and bake-off. The essence of the method is that we can calculate
$$
c^{*} = \text{argmax}_{c \in C} P_S(\text{utterance} \mid c)
$$
where $P_S$ is our describer model and $C$ is the set of all permutations of all three colors in the color context. We take $c^{*}$ to be a correct prediction if it is one where the target is in the privileged final position. (There are two such contexts; we try both in case the order of the distractors influences the predictions, and the model is correct if one of them has the highest probability.)
Here's the listener accuracy of our toy model:
End of explanation
bleu_score, predicted_texts = toy_mod.corpus_bleu(toy_color_seqs_test, toy_word_seqs_test)
bleu_score
Explanation: BLEU scores
The listener-based evaluation scheme has the unusual property that, in some sense, it assesses the model's ability to communicate with itself. This creates a danger that it will drift far from English as we know it but still succeed in signaling the target color. Ideally, we would train a separate listener model to help prevent this, but doing so would be cumbersome and could limit creative system development. Thus, as a quick check that our systems are still going to be able to communicate with us, we can calculate a BLEU score:
End of explanation
toy_perp = toy_mod.perplexities(toy_color_seqs_test, toy_word_seqs_test)
toy_perp[0]
Explanation: For discussion of BLEU scores, see the evaluation metrics notebook.
Other prediction and evaluation methods
You can get the perplexities for test examples with perplexities:
End of explanation
toy_proba = toy_mod.predict_proba(toy_color_seqs_test, toy_word_seqs_test)
# 4 tokens, each assigned a distribution over 5 vocab items:
print(toy_word_seqs_test[0])
toy_proba[0].shape
for timestep in toy_proba[0]:
    print(dict(zip(toy_vocab, timestep)))
Explanation: You can use predict_proba to see the full probability distributions assigned to test examples:
End of explanation
best_mod = utils.fit_classifier_with_hyperparameter_search(
    toy_color_seqs_train,
    toy_word_seqs_train,
    toy_mod,
    cv=2,
    scoring=None,
    param_grid={'hidden_dim': [10, 20]})
Explanation: Cross-validation
You can use utils.fit_classifier_with_hyperparameter_search to cross-validate these models. Just be sure to set scoring=None so that the sklearn model selection methods use the score method of ContextualColorDescriber, which is an alias for listener_accuracy:
End of explanation
dev_corpus = ColorsCorpusReader(COLORS_SRC_FILENAME, word_count=2)
dev_examples = list(dev_corpus.read())
len(dev_examples)
Explanation: Baseline SCC model
Just to show how all the pieces come together, here's a very basic SCC experiment using the core code and very simplistic assumptions (which you will revisit in the assignment) about how to represent the examples:
To facilitate quick development, we'll restrict attention to the two-word examples:
End of explanation
dev_cols, dev_texts = zip(*[[ex.colors, ex.contents] for ex in dev_examples])
Explanation: Here we extract the raw colors and texts (as strings):
End of explanation
dev_word_seqs = [[START_SYMBOL] + text.split() + [END_SYMBOL] for text in dev_texts]
Explanation: To tokenize the examples, we'll just split on whitespace, taking care to add the required boundary symbols:
End of explanation
dev_cols_train, dev_cols_test, dev_word_seqs_train, dev_word_seqs_test = \
    train_test_split(dev_cols, dev_word_seqs)
Explanation: We'll use a random train–test split:
End of explanation
dev_vocab = sorted({w for toks in dev_word_seqs_train for w in toks})
dev_vocab += [UNK_SYMBOL]
Explanation: Our vocab is determined by the train set, and we take care to include the $UNK token:
End of explanation
dev_mod = ContextualColorDescriber(
    dev_vocab,
    embed_dim=10,
    hidden_dim=10,
    early_stopping=True)
%time _ = dev_mod.fit(dev_cols_train, dev_word_seqs_train)
Explanation: And now we're ready to train a model:
End of explanation
dev_mod_eval = dev_mod.evaluate(dev_cols_test, dev_word_seqs_test)
dev_mod_eval['listener_accuracy']
dev_mod_eval['corpus_bleu']
Explanation: And finally an evaluation in terms of listener accuracy and BLEU scores. The evaluate method combines these:
End of explanation
import torch.nn as nn
from torch_color_describer import Encoder
class LSTMEncoder(Encoder):
    def __init__(self, color_dim, hidden_dim):
        super().__init__(color_dim, hidden_dim)
        self.rnn = nn.LSTM(
            input_size=self.color_dim,
            hidden_size=self.hidden_dim,
            batch_first=True)
Explanation: Modifying the core model
The first few assignment problems concern how you preprocess the data for your model. After that, the goal is to subclass model components in torch_color_describer.py. For the bake-off submission, you can do whatever you like in terms of modeling, but my hope is that you'll be able to continue subclassing based on torch_color_describer.py.
This section provides some illustrative examples designed to give you a feel for how the code is structured and what your options are in terms of creating subclasses. The principles are the same as those reviewed for a wider range of models in tutorial_pytorch_models.ipynb.
Illustration: LSTM Cells
Both the Encoder and the Decoder of torch_color_describer are currently GRU cells. Switching to another cell type is easy:
Step 1: Subclass the Encoder; all we have to do here is change GRU from the original to LSTM:
End of explanation
import torch.nn as nn
from torch_color_describer import Encoder, Decoder
class LSTMDecoder(Decoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.rnn = nn.LSTM(
            input_size=self.embed_dim,
            hidden_size=self.hidden_dim,
            batch_first=True)
Explanation: Step 2: Subclass the Decoder, making the same simple change as above:
End of explanation
from torch_color_describer import EncoderDecoder
class LSTMContextualColorDescriber(ContextualColorDescriber):
    def build_graph(self):
        # Use the new Encoder:
        encoder = LSTMEncoder(
            color_dim=self.color_dim,
            hidden_dim=self.hidden_dim)
        # Use the new Decoder:
        decoder = LSTMDecoder(
            vocab_size=self.vocab_size,
            embed_dim=self.embed_dim,
            embedding=self.embedding,
            hidden_dim=self.hidden_dim)
        return EncoderDecoder(encoder, decoder)
Explanation: Step 3:ContextualColorDescriber has a method called build_graph that sets up the Encoder and Decoder. The needed revision just uses LSTMEncoder:
End of explanation
lstm_mod = LSTMContextualColorDescriber(
    toy_vocab,
    embed_dim=10,
    hidden_dim=10)
_ = lstm_mod.fit(toy_color_seqs_train, toy_word_seqs_train)
lstm_mod.listener_accuracy(toy_color_seqs_test, toy_word_seqs_test)
Explanation: Here's an example run:
End of explanation
import torch.nn as nn
from torch_color_describer import Encoder, Decoder
class DeepEncoder(Encoder):
    def __init__(self, *args, num_layers=2, **kwargs):
        super().__init__(*args, **kwargs)
        self.num_layers = num_layers
        self.rnn = nn.GRU(
            input_size=self.color_dim,
            hidden_size=self.hidden_dim,
            num_layers=self.num_layers,
            batch_first=True)
class DeepDecoder(Decoder):
    def __init__(self, *args, num_layers=2, **kwargs):
        super().__init__(*args, **kwargs)
        self.num_layers = num_layers
        self.rnn = nn.GRU(
            input_size=self.embed_dim,
            hidden_size=self.hidden_dim,
            num_layers=self.num_layers,
            batch_first=True)
Explanation: Illustration: Deeper models
The Encoder and Decoder are both currently hard-coded to have just one hidden layer. It is straightforward to make them deeper as long as we ensure that both the Encoder and Decoder have the same depth; since the Encoder final states are the initial hidden states for the Decoder, we need this alignment. 
(Strictly speaking, we could have different numbers of Encoder and Decoder layers, as long as we did some kind of averaging or copying to achieve the hand-off from Encoder to Decoder. I'll set this possibility aside.)
Step 1: We need to subclass the Encoder and Decoder so that they have num_layers argument that is fed into the RNN cell:
End of explanation
from torch_color_describer import EncoderDecoder
class DeepContextualColorDescriber(ContextualColorDescriber):
    def __init__(self, *args, num_layers=2, **kwargs):
        self.num_layers = num_layers
        super().__init__(*args, **kwargs)
    def build_graph(self):
        encoder = DeepEncoder(
            color_dim=self.color_dim,
            hidden_dim=self.hidden_dim,
            num_layers=self.num_layers)  # The new piece is this argument.
        decoder = DeepDecoder(
            vocab_size=self.vocab_size,
            embed_dim=self.embed_dim,
            embedding=self.embedding,
            hidden_dim=self.hidden_dim,
            num_layers=self.num_layers)  # The new piece is this argument.
        return EncoderDecoder(encoder, decoder)
Explanation: Step 2: As before, we need to update the build_graph method of ContextualColorDescriber. The needed revision just uses DeepEncoder and DeepDecoder. To expose this new argument to the user, we also add a new keyword argument to ContextualColorDescriber:
End of explanation
mod_deep = DeepContextualColorDescriber(
    toy_vocab,
    embed_dim=10,
    hidden_dim=10)
_ = mod_deep.fit(toy_color_seqs_train, toy_word_seqs_train)
mod_deep.listener_accuracy(toy_color_seqs_test, toy_word_seqs_test)
Explanation: An example/test run:
End of explanation
<END_TASK>