---
bibliography:
- 'supplement.bib'
title: |
    Supplementary Material:\
    Enhancing Generalized Zero-Shot Learning via Adversarial Visual-Semantic Interaction
---

In this supplementary section, we discuss the following details, which could not be included in the main paper owing to space constraints:

-   Implementation details of our experiments (in continuation to Sec 4)

-   Details describing the Alignment loss (in continuation to Sec 3.2)

-   Related work on Generative Models (in continuation to Sec 2)

We have also added a complete figure, Figure \[fig:A\], with both training and testing phases of the recognition module for clarity of understanding.

Implementation Details {#intro}
======================

In this section, we describe the implementation details for our methodology, which we could not include in our experiments section. The generator ($G$), inference network ($I$) and discriminators ($D_{1},D_{2},D_{3}$) are all implemented using fully connected neural networks. In order to ensure fair comparison, we follow the architecture used in [@eighteen] for all our components. Formally, the generator and inference network both consist of 2 dense layers of size 4096 with leaky ReLU activation except at the output layer which has ReLU activation. These layers in the inference network form the latent features $\textbf{f}_{1},\textbf{f}_{2}$ in our recognition module as shown in Figure \[fig:A\]. The dimension of output layer is 2048 in case of the generator and $d_{h}$ in case of the inference network where $d_{h}$ denotes the dimension of semantic attributes. The three discriminators consist of 2 fully connected hidden layers of size 4096 with leaky ReLU activation. The noise vector $z$ is sampled from a random unit Gaussian. We find that taking the dimension of noise vector same as that of semantic embeddings works well as in [@eighteen]. The generator is updated every 5 discriminator iterations as suggested in WGAN paper  [@thirtyone]. As for the optimization, we use an Adam optimizer. We use a single layered softmax classifier in our recognition module for fair comparison (most earlier work use this) and simplicity. Table \[tab:my\_label\] presents the details of hyperparameters used for each of the considered benchmark datasets.

   Dataset   $\beta$   $\lambda$   $\gamma$   $\alpha_{1}$   $\alpha_{2}$
  --------- --------- ----------- ---------- -------------- --------------
     CUB      0.01        10          3            1              2
     FLO      0.01        10         0.01          1              1
    AWA1      0.01        10        0.001          10             2
    AWA2      0.01        10         0.01          5              4

  : Hyperparameters used for different datasets[]{data-label="tab:my_label"}

Alignment Loss
==============

In this section, we explain the details regarding of the $L_{wasserstein}$ term in Eqn 11 of the main paper. We use the sinkhorn distance based lightspeed computation proposed by Cuturi in 2013 [@sinkhorn] for computing our alignment loss. The sinkhorn distance metric has also been used for approximating the Wasserstein distance in [@tdsl] for a very different projection-based (non-generative) ZSL method for alignment in visual space. We instead use the metric to provide distributional alignment in the semantic space which helps us to preserve high-level semantics better and reduce semantic loss. To the best of our knowledge, this has not been done before in ZSL literature. We use the Wasserstein distance to model the joint probability of visual-semantic features better by combining it with adversarial loss in a generative GZSL setting. Formally, the sinkhorn distance can be written as: $$\mathcal{L}_{Wasserstein} = \underset{X}{\min}\sum _ { i , j } \text{dis}_{i j} x_{i j} - \epsilon H(X)$$ where $H(X)$ is an entropy-based regularization term and $\text{dis}_ {i j} (\cdot)$ is as defined in [@tdsl]. We compute $\text{dis}$ however on an assignment matrix $X$ with entries given by $x_{i j}$, which defines the matching relationship between the class centers of output semantic attributes, $\hat{\textbf{h}}$, and ground truth semantic centers $\textbf{h}$. Here, $i \in A$ and $j \in B$ where $A$ and $B$ are sets of class centres of output semantic attributes and ground truth semantic attributes respectively. This helps compute the term $P(\textbf{h}|\hat{ \textbf{h}})$ in our methodology.

Related Work: Generative Models
===============================

Since ours is a generative approach to GZSL, we briefly present the earlier work in generative models underlying our methodology, for completeness of our discussion on related work. Generative modeling aims to learn the probability distribution of data points such that we can randomly sample data from it. The idea behind Generative Adversarial Networks (GANs) is to learn a generative model to capture an arbitrary data distribution via a min-max training procedure which consists of a generator that synthesizes fake data and a discriminator that distinguishes fake and real data. These models have been used in many interesting computer vision applications especially for image generation [@twentyseven; @twentyeight; @twentynine] and have achieved compelling results. However, GANs are also known for their instability in training, and are known to suffer from the mode collapse problem. In order to mitigate these problems and improve the quality of synthetic samples, methods like WGAN [@thirty] and WGAN-GP [@thirtyone] have been proposed, which we leverage in this work. GANs have also been used for tasks like multi-view generation and learning cross-modal representations for downstream tasks like retrieval or semi-supervised classification [@twentyeight] [@ali]. Such generative models can be trained explicitly to model conditional/joint distributions of random variables, which we leverage in this work. For example, [@twentythree] shows how such generative models can be used to generate data for a specific class by conditioning them on the label. [@ali; @twenty; @twentyone; @twentytwo] show how adversarial training can be used to model joint and utilize the trained model for semi-supervised learning.
