Distributional Semantics Models(DSM) derive word space from linguistic items
 in context. Meaning is obtained by defining a distance measure between vectors
 corresponding to lexical entities. Such vectors present several problems. This
 work concentrates on quality of word embeddings, improvement of word embedding
 vectors, applicability of a novel similarity metric used ‘on top’ of the
 word embeddings. In this paper we provide comparison between two methods
 for post process improvements to the baseline DSM vectors. The counter-fitting
 method which enforces antonymy and synonymy constraints into the Paragram
 vector space representations recently showed improvement in the vectors’ capability
 for judging semantic similarity. The second method is our novel RESM
 method applied to GloVe baseline vectors. By applying the hubness reduction
 method, implementing relational knowledge into the model by retrofitting synonyms
 and providing a new ranking similarity definition RESM that gives maximum
 weight to the top vector component values we equal the results for the ESL
 and TOEFL sets in comparison with our calculations using the Paragram and Paragram
 + Counter-fitting methods. For SIMLEX-999 gold standard since we cannot
 use the RESM the results using GloVe and PPDB are significantly worse compared
 to Paragram. Apparently, counter-fitting corrects hubness. The Paragram
 or our cosine retrofitting method are state-of-the-art results for the SIMLEX-999
 gold standard. They are 0.2 better for SIMLEX-999 than word2vec with sense
 de-conflation (that was announced to be state-of the-art method for less reliable
 gold standards). Apparently relational knowledge and counter-fitting is more important
 for judging semantic similarity than sense determination for words. It is to
 be mentioned, though that Paragram hyperparameters are fitted to SIMLEX-999
 results. The lesson is that many corrections to word embeddings are necessary
 and methods with more parameters and hyperparameters perform better.
 Distributional language models are frequently used to measure word similarity in natural language. This is a basis for many semantic tasks. The DSM often consists of a set of vectors; each vector corresponds to a character string, which represents a word. BID14 and BID19 implemented the most commonly used word embedding (WE) algorithms. Vector components in language models created by these algorithms are latent. Similarity between words is defined as a function of vectors corresponding to given words. The cosine measure is the most frequently used similarity function, although many other functional forms were attempted. BID25 highlights the fact that the cosine can be outperformed by ranking based functions.As pointed out by many works, e.g. , evidence suggests that distributional models are far from perfect.Vector space word representations obtained from purely distributional information of words in large unlabeled corpora are not enough to best the state-of-the-art results in query answering benchmarks, because they suffer from several types of weaknesses: 3. Appearance of hubness that distorts distances between vectors, 4. Inability of distinguishing from antonyms. 5. In case of retrofitting distortion vector space -loss of information contained in the original vectorsIn this paper we use the existing word embedding model but with several post process enhancement techniques. We address three of these problems. In particular, we define a novel similarity measure, dedicated for language models.Similarity is a function, which is monotonically opposite to distance. As the distance between two given entities gets shorter, entities are more similar. This holds for language models. Similarity between words is equal to similarity between their corresponding vectors. There are various definitions of distance. The most common Euclidean distance is defined as follows: DISPLAYFORM0 Similarity based on the Euclidean definition is inverse to the distance: DISPLAYFORM1 Angular definition of distance is defined with cosine function: DISPLAYFORM2 We define angular similarity as: DISPLAYFORM3 Both Euclidean and Cosine definitions of distance could be looked at as the analysis of vector components. Simple operations, like addition and multiplication work really well in low dimensional spaces. We believe, that applying those metrics in spaces of higher order is not ideal, hence we compare cosine similarity to a measure of distance dedicated for high dimensional spaces.In this paper we restrict ourselves to three gold standards: TOEFL, ESL and SIMLEX-999. The first two are small but reliably annotated (and therefore confidence in their performance can be assumed 100%). Other used benchmarks suffer from several drawbacks. Both WS- 353 Finkelstein et al. (2001) and MEN Bruni et al. (2012) do not measure the ability of models to reflect similarity. Moreover, as pointed out by , for WS-353, RG Rubenstein & Goodenough (1965) and MEN, state-of-the-art models have reached the average performance of a human annotator on these evaluations. This work compares the state-of-the-art word embedding methods for three most reliable gold standards: TOEFL, ESL and SIMLEX-999. For TOEFL and ESL the GloVe, PPDB baseline with retrofitting, our novel RESM similarity measure and hubness reduction we are able to equal the Paragram results. For SIMLEX-999 Paragram with Counter-fitting results are clearly better than the Glove based methods using the PPDB 1.0. However, we propose the cosine retrofitting that basically achieves the Paragram with Counter-fitting results. The Paragram with Counter-fitting method contains several hyperparameters which is one source of its success. Its effects can be seen in TAB0 at https://arxiv.org/pdf/1506.03487.pdf. The Spearman ρ values for SIMLEX-999 are 0.667 for Paragram300 fitted to WS353, and 0.685 for Paragram300 fitted to SIMLEX-999. The difference is even larger for WS353. Then the Spearman ρ values for WS-353 are 0. 769 for Paragram300 fitted toWS353, and 0.720 for Paragram300 fitted to SIMLEX-999. Still the best word embedding based methods are not able to achieve the performance of other dedicated methods for TOEFL and ESL. The work of BID13 employed 2 fitting constants (and it is not clear that they were the same for all questions) for answering the TOEFL test where only 50 questions are used. Techniques introduced in the paper are lightweight and easy to implement, yet they provide a significant performance boost to the language model. Since the single word embedding is a basic element of any semantic task one can expect a significant improvement of results for these tasks. In particular, SemEval-2017 International Workshop on Semantic Evaluation run (among others) the following tasks(se2):1. Task 1: Semantic Textual Similarity 2. Task 2: Multilingual and Cross-lingual Semantic Word Similarity 3. Task 3: Community Question Answering in the category Semantic comparison for words and texts. Another immediate application would be information retrieval (IR). Expanding queries by adding potentially relevant terms is a common practice in improving relevance in IR systems. There are many methods of query expansion. Relevance feedback takes the documents on top of a ranking list and adds terms appearing in these document to a new query. In this work we use the idea to add synonyms and other similar terms to query terms before the pseudo-relevance feedback. This type of expansion can be divided into two categories. The first category involves the use of ontologies or lexicons (relational knowledge). The second category is word embedding (WE). Here closed words for expansion have to be very precise, otherwise a query drift may occur, and precision and accuracy of retrieval may deteriorate. There are several avenues to further improve the similarity results.1. Use the multi-language version of the methods (e.g. Recski et al. FORMULA0 2. Use PPDB 2.0 to design the Paragram vectors BID18 3. Apply the multi-sense methods (knowledge graps) with state-of-the-art relational enriched vectors 4. Recalibrate annotation results using state-of-the-art results.
Paper provides a description of a procedure to enhance word vector space model with an evaluation of Paragram and GloVe models for Similarity Benchmarks. This paper suggests a new algorithm that adjusts GloVe word vectors and then uses a non-Euclidean similarity function between them. The authors present observations on the weaknesses of the existing vector space models and list a 6-step approach for refining existing word vectors