Identifying the hypernym relations that hold between words is a fundamental task in NLP. Word embedding methods have recently shown some capability to encode hypernymy. However, such methods tend not to explicitly encode the hypernym hierarchy that exists between words. In this paper, we propose a method to learn a hierarchical word embedding in a speciﬁc order to capture the hypernymy. To learn the word embeddings, the proposed method considers not only the hypernym relations that exists between words on a taxonomy, but also their contextual information in a large text corpus. The experimental results on a supervised hypernymy detection and a newly-proposed hierarchical path completion tasks show the ability of the proposed method to encode the hierarchy. Moreover, the proposed method outperforms previously proposed methods for learning word and hypernym-speciﬁc word embeddings on multiple benchmarks. Hypernymy relation is an essential component for many Natural Language Processing (NLP) tasks. It represents an asymmetric relation between name of a class (hypernym) and a particular instance of it (hyponym). For example, given the hypernymy pair (bird, vertebrate), the hyponym word bird is a particular instance of the hypernym word vertebrate, and one can simply say "a bird is a vertebrate". Hypernymy identification plays a major role in various NLP tasks such as question answering BID14 , taxonomy construction BID25 , textual entailment BID8 and text generation BID5 , to name a few.The task of identifying hypernymy has been long addressed by relying on either the lexicalsyntactic patterns where a particular textual pattern suggests the existence of a hypernymy relation, or the distributional representation of a given pair of words. In a pattern-based approach, from a sentence like "a bird such as a falcon" we can identify a hypernymy relation between bird and fal-con. Despite its simplicity and efficiency, pattern-based approaches suffer from a low precision and coverage issue BID44 . For example, let us consider the sentence "some birds recorded in Africa such as Gadwall", a typical lexical-pattern based approach may incorrectly detect (Gadwall, Africa) as hypernymy.In contrast to the requirement of the pattern-based approaches to have the hypernym and hyponym words occurring in the same sentence, the distributional family of approaches rely on the words co-occurrence statistics in a large text corpus to represent the words and then deduce the hold of hypernymy. It works on the assumption that taxonomically related words tend to occur in a similar context. In particular, a hypernym word has a broader context than its hypernym, and therefore the contextual features of the hyponym word are usually a subset of that of its hypernym. However, such approaches commonly struggle from discriminating the hypernym relations from other lexico-semantic relations BID34 .Lately , distributed word representations (a.k.a word embeddings) BID22 BID29 have shown some capability to encode hypernymy. Such methods typically embed the words into dense, low-dimensional, real-valued vectors, using the co-occurrence statistics obtained from a text corpora, and then used in a supervised settings to detect hypernymy. However , typical word embeddings models rely only on co-occurrence based similarity, and therefore are insufficient to encode taxonomic relations in the learnt word embeddings BID2 .Several recent studies have proposed methods for learning hypernymy-specific word embeddings BID44 BID2 BID26 . BID44 design a neural network to learn word embeddings that perceive hypernymy, purely relying on extracted pairwise training data from a web corpus. Moreover, Anh et al. [2016] proposed a similar approach, but further utilise the contextual information of the pre-extracted pairwise hypernymy. Similarly, Nguyen et al. [2017] introduce the HyperVec model that use the Skip-gram with Negative Sampling (SGNS) BID22 objective to learn the word embeddings from a corpus subject to pairwise hypernymy constraints extracted from a taxonomy. Moreover, BID27 proposed a new model that embed symbolic data into hyperbolic space, particularly into Poincaré ball, to learn hierarchical embeddings. Another model that is recently proposed is the LEAR model of BID39 . LEAR is a post-processing model that takes any word vectors and then adjust the input vectors to emphasis the hypernym relations. The aforementioned methods merely consider pairwise hypernymy relations rather than the hierarchical hypernymy path connecting a word to the root in a taxonomy.In this paper, we propose a method that jointly learns hierarchical word embeddings (HWE) from a corpus and a taxonomy. The proposed method begins by embedding the words into random low-dimensional real-valued vectors, and subsequently updates the embeddings to encode the hierarchical structure available in the taxonomy. To train the proposed method, we use a taxonomy to extract the hierarchical hypernymy paths for the hyponym words, and use the global vector BID29 as a context-aware objective between the hypernym and hyponym words. As such, the proposed model benefits from both the contextual information as well as the taxonomic relations to learn the embeddings.In our experiments, we evaluate our hierarchical word embeddings on the standard supervised hypernymy task on various benchmarks, graded lexical entailment prediction and a newly-proposed hierarchical path completion task. On the supervised hypernymy identification , the proposed method reports an improvement over several previously proposed methods, showing the benefit of considering the full hierarchical hypernymy paths instead of only pairwise relations. Moreover, a quantitative and qualitative analysis on the newly-proposed hierarchical path completion task further illustrate the ability of the proposed method to encode hypernymy in the learn embeddings. We presented a method to learn a Hierarchical Word Embedding (HWE) for identifying the hypernymy relations between words. For this purpose, we introduced a joint objective that make use of both a taxonomy and a large text corpus to learn hierarchical word embeddings. We evaluated HWE on the standard supervised hypernymy identification and a newly-proposed hierarchical hypernymy path completion tasks. The experiments conducted in this paper on the above mentioned two tasks demonstrate that HWE was able to encode the hypernymy relations between words into the learnt embeddings, and reports an improvement over several previously proposed methods that learn either general word embeddings or hypernymy-specific word embeddings.
We presented a method to jointly learn a Hierarchical Word Embedding (HWE) using a corpus and a taxonomy for identifying the hypernymy relations between words. The paper presents a method to jointly learn word embeddings using co-occurrence statistics as well as incorporating hierarchal information from semantic networks. This paper proposed a joint learning method of hypernym from both raw text and supervised taxonomy data.  This paper proposes adding a measure of "distributional inclusion" difference to the GloVE objective for the purpose of representing hypernym relations.