Graph convolutional neural networks have recently shown great potential for the task of zero-shot learning. These models are highly sample efficient as related concepts in the graph structure share statistical strength allowing generalization to new classes when faced with a lack of data. However, we find that the extensive use of Laplacian smoothing at each layer in current approaches can easily dilute the knowledge from distant nodes and consequently decrease the performance in zero-shot learning. In order to still enjoy the benefit brought by the graph structure while preventing the dilution of knowledge from distant nodes, we propose a Dense Graph Propagation (DGP) module with carefully designed direct links among distant nodes. DGP allows us to exploit the hierarchical graph structure of the knowledge graph through additional connections. These connections are added based on a node's relationship to its ancestors and descendants. A weighting scheme is further used to weigh their contribution depending on the distance to the node. Combined with finetuning of the representations in a two-stage training approach our method outperforms state-of-the-art zero-shot learning approaches. With the ever-growing supply of image data, from an ever-expanding number of classes, there is an increasing need to use prior knowledge to classify images from unseen classes into correct categories based on semantic relationships between seen and unseen classes. This task is called zero-shot image classification. To obtain satisfactory performance on this task, it is crucial to model precise class relationships based on prior class knowledge. Previously prior knowledge has been incorporated in form of semantic descriptions of classes, such as attributes BID0 BID27 BID18 or word embeddings BID29 BID10 , or by using semantic relations such as knowledge graphs BID23 BID26 BID28 BID19 . Approaches that use knowledge graphs are less-explored and generally are based on the assumption that unknown classes can exploit similarity to known classes. Recently the benefit of hybrid approaches that combine knowledge graph and semantic class descriptions has been illustrated BID31 .The current state-of-the-art approach BID31 processes knowledge graphs by making use of recent developments in applying neural network techniques to non-euclidean spaces, such as graph and manifold spaces BID1 . A deep graph convolutional neural network (GCN) BID13 ) is used and the problem is phrased as weight regression, where the GCN is trained to regress classifier weights for each class. GCNs balance model complexity and expressiveness with a simple scalable model relying on the idea of message passing, i.e. nodes pass knowledge to their neighbors. However, these models were originally designed for classification tasks, albeit semi-supervised, an arguably simpler task than regression. In recent work , it has been shown that GCNs perform a form of Laplacian smoothing, where feature representations will become more similar as depth increases leading to easier classification BID16 . In the regression setting, instead, the aim is to exchange information between nodes in the graph and extensive smoothing is not desired as it dilutes information and does not allow for accurate regression. For instance, in a connected graph all features in a GCN with n layers will converge to the same representation as n → ∞ under some conditions, hence washing out all information BID16 . Here, graph propagation represents the knowledge that a node receives in a single layer for previous approaches. b) Proposed dense graph propagation for node 'Cat'. The node receives knowledge from all its descendants during the descendant phase (blue arrows) and its ancestors during the ancestor phase (red arrows). This leads to a densely connected graph where knowledge can directly propagate between related nodes. Weights α k are used to weigh nodes that are k-hops away from a given node.We, therefore, argue that this approach is not ideal for the task of zero-shot learning and that the number of layers in the graph should be small in order to avoid smoothing. We illustrate this phenomenon in practice , by showing that a shallow GCN consistently outperforms previously reported results. We employ a model-of-models framework by training the method to predict a set of logistic regression classifier for each class on top of a set of extracted features produced by a CNN. Choosing a small number of layers, however , has the effect that knowledge will not propagate well through the graph. A 1-layer GCN for instance only considers neighbors that are two hops away in the graph such that only immediate neighbors influence a given node. Thus, we propose a dense connectivity scheme , where nodes are connected directly to descendants/ancestors in order to include distant information. These connections allow us to propagate information without many smoothing operations but leads to the problem that all descendants/ancestors are weighed equally when computing the regression weight vector for a given class. However, intuitively, nodes closer to a given node should have higher importance. To remedy this, we extend this framework by adding a weighting scheme that considers the distance between nodes in order to weigh the contribution of different nodes. Making use of shared weights based on the distance also has the advantage that it only adds a minimal amount of additional parameters, is computationally efficient, and provides a balance between increasing flexibility of the model and keeping it restrictive enough to allow good predictions for the nodes of the unseen classes. FIG0 illustrates the difference in the way knowledge is propagated in this proposed Dense Graph Propagation (DGP) module compared to a GCN layer.To allow the feature extraction stage of the pre-trained CNN to adjust to the newly learned classifiers we propose a two-phase training scheme. In the first step, the DGP is trained to predict the last layer CNN weights. In the second phase, we replace the last layer weights of the CNN with the weights predicted by the DGP, freeze the weights and finetune the remaining weights of the CNN by optimizing the cross entropy classification loss on the seen classes.Our contributions can be summarized as follows. We present• an analysis of our intuitions for zero-shot learning and illustrate how these intuitions can be combined to design a DGP that outperforms previous zero-shot learning results. In contrast to previous approaches using graph convolutional neural networks for zero-shot learning, we illustrate that the task of zero-shot learning benefits from shallow networks. Further, to avoid the lack of information propagation between distant nodes in shallow models, we propose DGP, which exploits the hierarchical structure of the knowledge graph by adding a dense connection scheme. Experiments illustrate the ability of the proposed methods, outperforming previous state-of-the-art methods for zero-shot learning. In future work, we aim to investigate the potential of more advanced weighting mechanisms to further improve the performance of DGP compared to the SGCN. The inclusion of additional semantic information for settings where these are available for a subset of nodes is another future direction.A QUALITATIVE RESULTS Figure 4 and 5 provide further qualitative results of our finetuned Graph Propagation Module GPM and Dense Graph Propagation Module DGP compared to a standard ResNet and GCNZ, our reimplementation of BID31 .upright , grand piano, organ, accordion, barbershop piano, spinet, keyboard instrument, concert grand, baby grand piano, spinet, concert grand, baby grand, keyboard instrument piano, baby grand, concert grand, spinet, keyboard instrument B TWO-PHASE PROPAGATION TAB4 illustrates the benefit of a two-phase directed propagation rule where ancestors and descendants are considered individually compared to two consecutive updates using the full adjacency matrix in the dense method. C ANALYSIS OF NUMBER OF LAYERS TAB5 illustrates the drop in performance that is caused by using additional hidden layers in the GCN for the 2-hops experiment. All hidden layers have dimensionality of 2048 with 0.5 dropout. TAB6 explains the performance difference between our SGCN, our reimplementation of GCNZ and the reported results in BID31 . Note, unless otherwise stated training is performed for 3000 epochs. Non-symmetric normalization (D −1 A) is denoted as non-sym in the normalization column, while a symmetric normalization (D −1/2 AD −1/2 ) is denoted as sym. No finetuning has been performed for SGCN in these results. TAB7 shows the mean and std for 3 runs for the 2-hops and All dataset. It can clearly be observed that as the number of classes increases (2-hops to all), results become more stable.
We rethink the way information can be exploited more efficiently in the knowledge graph in order to improve performance on the Zero-Shot Learning task and propose a dense graph propagation (DGP) module for this purpose. This authors propose a solution to the problem of over-smoothing in Graph conv networks by allowing dense propagation between all related nodes, weighted by the mutual distance. Proposes a novel graph convolutional neural network to tackle the problem of zero-shot classification by using relational structures between classes as input of graph convolutional networks to learn classifiers of unseen classes