We present a novel method to precisely impose tree-structured category information onto word-embeddings, resulting in ball embeddings in higher dimensional spaces (N-balls for short). Inclusion relations among N-balls implicitly encode subordinate relations among categories. The similarity measurement in terms of the cosine function is enriched by category information. Using a geometric construction method instead of back-propagation, we create large N-ball embeddings that satisfy two conditions: (1) category trees are precisely imposed onto word embeddings at zero energy cost; (2) pre-trained word embeddings are well preserved. A new benchmark data set is created for validating the category of unknown words. Experiments show that N-ball embeddings, carrying category information, significantly outperform word embeddings in the test of nearest neighborhoods, and demonstrate surprisingly good performance in validating categories of unknown words. Source codes and data-sets are free for public access \url{https://github.com/gnodisnait/nball4tree.git} and \url{https://github.com/gnodisnait/bp94nball.git}. Words in similar contexts have similar semantic and syntactic information. Word embeddings are vector representations of words that reflect this characteristic BID16 BID19 and have been widely used in AI applications such as question-answering BID24 , text classification BID22 , information retrieval BID14 , or even as a building-block for a unified NLP system to process common NLP tasks BID2 . To enhance semantic reasoning, researchers proposed to represent words in terms of regions instead of vectors. For example, BID5 extended a word vector into a region by estimating the log-linear probability of weighted feature distances and found that hyponym regions often do not fall inside of their hypernym regions. By using external hyponym relations, she obtained 95.2% precision and 43.4% recall in hypernym prediction for a small scale data set. Her experiments suggest that regions structured by hyponym relations may not be located within the same dimension as the space of word embeddings. Yet, how to construct strict inclusion relations among regions is still an open problem when representing hypernym relations.In this paper, we restrict regions to be n dimensional balls (N -ball for short) and propose a novel geometrical construction approach to impose tree-structured category information onto word embeddings. This is guided by two criteria: (1) Subordinate relations among categories shall be implicitly and precisely represented by inclusion relations among corresponding N -balls. This way, the energy costs of imposing structure will be zero; (2) Pre-trained word embeddings shall be well-preserved. Our particular contributions are as follows: (1) The proposed novel geometric approach achieves zero energy costs of imposing tree structures onto word-embeddings. (2) By considering category information in terms of the boundary of an N -ball, we propose a new similarity measurement that is We create a large data set of N -ball embeddings using the pre-trained GloVe embeddings and a large category tree of word senses extracted from Word-Net 3.0.The remainder of our presentation is structured as follows: Section 2 presents the structure of N -ball embeddings; Section 3 describes the geometric approach to construct N -ball embeddings; Section 4 presents experiment results; Section 5 briefly reviews related work; Section 6 concludes the presented work, and lists on-going research. FIG0 (a), so they are not RCC regions that can be either open or closed, or even a mixture, thus avoiding a number of problems BID4 BID3 . We proposed a novel geometric method to precisely impose external tree-structured category information onto word embeddings, resulting in region-based (N -ball embeddings) word sense embeddings. They can be viewed as Venn diagrams (Venn, 1880) of the tree structure, if zero energy cost is achieved. Our N -ball method has demonstrated great performance in validating the category of unknown words, the reason for this being under further investigation. Our on-going work also includes multi-lingual region-based knowledge graph embedding where multiple relations on directed acyclic graphs need to be considered. N -balls carry both vector information from deep learning and region information from symbolic structures. Therefore, N -balls establish a harmony between Connectionism and Symbolicism, as discussed by BID15 , and thus may serve as a novel building block for the commonsense representation and reasoning in Artificial Intelligence. N -balls, in particular, contribute a new topic to Qualitative Spatial Reasoning (QSR) that dates back to BID26 .
we show a geometric method to perfectly encode categroy tree information into pre-trained word-embeddings. The paper proposes N-ball embedding for taxonomic data where an N-ball is a pair of a centroid vector and the radius from the center. The paper presents a method for tweaking existing vector embeddings of categorical objects (such as words), to convert them to ball embeddings that follow hierarchies. Focuses on adjusting the pretrained word embeddings so that they respect the hypernymy/hyponymy relationship by appropriate n-ball encapsulation.