{"action":"created","issue":{"url":"https://api.github.com/repos/dselivanov/text2vec/issues/34","labels_url":"https://api.github.com/repos/dselivanov/text2vec/issues/34/labels{/name}","comments_url":"https://api.github.com/repos/dselivanov/text2vec/issues/34/comments","events_url":"https://api.github.com/repos/dselivanov/text2vec/issues/34/events","html_url":"https://github.com/dselivanov/text2vec/issues/34","id":113982837,"number":34,"title":"paragraph2vec","user":{"login":"dselivanov","id":5123805,"avatar_url":"https://avatars.githubusercontent.com/u/5123805?v=3","gravatar_id":"","url":"https://api.github.com/users/dselivanov","html_url":"https://github.com/dselivanov","followers_url":"https://api.github.com/users/dselivanov/followers","following_url":"https://api.github.com/users/dselivanov/following{/other_user}","gists_url":"https://api.github.com/users/dselivanov/gists{/gist_id}","starred_url":"https://api.github.com/users/dselivanov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dselivanov/subscriptions","organizations_url":"https://api.github.com/users/dselivanov/orgs","repos_url":"https://api.github.com/users/dselivanov/repos","events_url":"https://api.github.com/users/dselivanov/events{/privacy}","received_events_url":"https://api.github.com/users/dselivanov/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dselivanov/text2vec/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":{"login":"dselivanov","id":5123805,"avatar_url":"https://avatars.githubusercontent.com/u/5123805?v=3","gravatar_id":"","url":"https://api.github.com/users/dselivanov","html_url":"https://github.com/dselivanov","followers_url":"https://api.github.com/users/dselivanov/followers","following_url":"https://api.github.com/users/dselivanov/following{/other_user}","gists_url":"https://api.github.com/users/dselivanov/gists{/gist_id}","starred_url":"https://api.github.com/users/dselivanov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dselivanov/subscriptions","organizations_url":"https://api.github.com/users/dselivanov/orgs","repos_url":"https://api.github.com/users/dselivanov/repos","events_url":"https://api.github.com/users/dselivanov/events{/privacy}","received_events_url":"https://api.github.com/users/dselivanov/received_events","type":"User","site_admin":false},"milestone":null,"comments":3,"created_at":"2015-10-29T05:30:24Z","updated_at":"2015-10-30T16:12:50Z","closed_at":null,"body":"See useful **zachmayer** [comment](https://github.com/dselivanov/text2vec/pull/26#issuecomment-149903059): \r\n> \r\n1. Start with GloVe.  In this case you have a square matrix: word X word, where the rows are the words and the columns are the contexts they occur in.  This matrix is symmetrical, and in GloVe you factor this matrix to get vectors for your words.\r\n2. Now consider the `dtm` matrix you already have defined in your package.  In this case you have a rectangular matrix: document X word, where the rows are documents and the columns are the words (or contexts) that occur in those documents.  If you do SVD on this matrix, you get pretty simple (but effective) vectors for your documents.  I've actually used this approach in a lot of cases: bag-of-words + SVD gives pretty decent document vectors pretty quickly.\r\n>\r\nNow, it's time to get fancy! =D  Both of our matrices have the same number of columns: each column is a word from our bag-of-words.  So we can row bind these matrices together into a giant matrix where the rows are words / documents and the columns are contexts (either for the word, or that occur in the document).  Now we factor this matrix and find words vectors AND document vectors *at the same time*.  This simultaneous word/document vector strategy seems to give better results than doing step 1 & 2 on their own.\r\nSo in your package, you could start with create_hash_corpus with ngrams = 0 and skips = 0.  Then extract the dtm from this matrix.  Next, create_hash_corpus with skips = 1:8 and some fancy skip-gram weighting function.  Extract the cooc matrix from this matrix.\r\nNow run `final_matrix = rbind(dtm, cooc)` and then `model = irlba(final_matrix, nu=100))` and then `doc_vecs = model$u[1:nrow(dtm),]`, and you have really good document vectors!"},"comment":{"url":"https://api.github.com/repos/dselivanov/text2vec/issues/comments/152569089","html_url":"https://github.com/dselivanov/text2vec/issues/34#issuecomment-152569089","issue_url":"https://api.github.com/repos/dselivanov/text2vec/issues/34","id":152569089,"user":{"login":"zachmayer","id":581590,"avatar_url":"https://avatars.githubusercontent.com/u/581590?v=3","gravatar_id":"","url":"https://api.github.com/users/zachmayer","html_url":"https://github.com/zachmayer","followers_url":"https://api.github.com/users/zachmayer/followers","following_url":"https://api.github.com/users/zachmayer/following{/other_user}","gists_url":"https://api.github.com/users/zachmayer/gists{/gist_id}","starred_url":"https://api.github.com/users/zachmayer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zachmayer/subscriptions","organizations_url":"https://api.github.com/users/zachmayer/orgs","repos_url":"https://api.github.com/users/zachmayer/repos","events_url":"https://api.github.com/users/zachmayer/events{/privacy}","received_events_url":"https://api.github.com/users/zachmayer/received_events","type":"User","site_admin":false},"created_at":"2015-10-30T16:12:49Z","updated_at":"2015-10-30T16:12:49Z","body":"btw, I haven't been tf-idf weighting my matricies, and wow it makes a huge difference!"}}