<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Topic Modeling on NIPS Dataset Using Gaussian LDA w/ Word-Embeddings
Step1:  Loading the word_vector model with GenSim
Step2:  Sets of vocab to filter on
Step3:  Document cleaning
Tokenizing just on spaces
no lemmatization or stemming
removing non-ascci characters
removing stop words
removing words not in Glove vocab
removing non-alpha words (e.g. Letter and symbols)
removing short words < 2 characters long
lowercasing all words
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import os
from operator import itemgetter
from collections import Counter
import scipy.stats as stat
from gensim.models import Word2Vec
from nltk import corpus
import FastGaussianLDA2
Explanation: Topic Modeling on NIPS Dataset Using Gaussian LDA w/ Word-Embeddings
End of explanation
wvmodel = Word2Vec.load_word2vec_format(
    "/Users/michael/Documents/Gaussian_LDA-master/data/glove.wiki/glove.6B.50d.txt", binary=False)
print "word-vector dimension: {}".format(wvmodel.vector_size())
Explanation: Loading the word_vector model with GenSim
End of explanation
wv_vocab = set(wvmodel.vocab.keys())
stops = set(corpus.stopwords.words(fileids="english"))
Explanation: Sets of vocab to filter on: NLTK StopWords and Glove vocab
End of explanation
corpus = []
nips_path = "/Users/michael/Documents/GaussianLDA/data/"
for folder in os.listdir(nips_path)[1:]:
    for doc in os.listdir(nips_path + folder):
        with open(nips_path + folder + "/" + doc, 'r') as f:
            txt = f.read().split()
            txt = map(lambda x: x.lower(), txt)  # Lowercasing each word
            txt = filter(lambda word: [letter for letter in word if ord(letter) < 128 ], txt)  # Checking each word for ascci error
            txt = filter(lambda x: x not in stops, txt)  # Removing stop words
            txt = filter(lambda x: x.isalpha(), txt)  # Removing non-letter words (eg numbers and symbols)
            txt = filter(lambda x: len(x) > 2, txt)  # removing super short words and single letters
            txt = filter(lambda x: x in wv_vocab, txt)    
            txt = ' '.join(txt)
            corpus.append(txt)
print "Number of documents in corpus: {}".format(len(corpus))
reload(FastGaussianLDA2)
topics = 50
dim = 50
run_num = 1
outputfile = "/Users/michael/Documents/GaussianLDA/output/NIPS_{}_{}T_{}D_".format(str(run_num),
                                                                                  str(topics), 
                                                                                  str(dim))
lda = FastGaussianLDA2.Gauss_LDA(topics, corpus, word_vector_model=wvmodel, alpha=.5, outputfile=outputfile)
lda.fit(50)  # Number of samples to run
Explanation: Document cleaning
Tokenizing just on spaces
no lemmatization or stemming
removing non-ascci characters
removing stop words
removing words not in Glove vocab
removing non-alpha words (e.g. Letter and symbols)
removing short words < 2 characters long
lowercasing all words
End of explanation
<END_TASK>