<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Meetup 1
Going to parse texts for most used words.
Step1:  We want to "tokenize" the text and discard "stopwords" like 'a', 'the', 'in'. These words aren't relevant for our analysis.
To tokenize our text we're going to use regular expressions. Regular expressions are cool and you should try to use them whenever you can. To use regular expression we need to import the regular expression module re. Lets do this in the next cell!!
Step2:  We want to tokenize words. We will use \w+ regular expression to tokenize all the words.
- Lets break this down - \w will match any alphanumerica and underscore characters
- The +
Step3:  That was the easy part..... We want all the data(text) to be "normalized". The word 'Linear' is different then the word 'linear' but for our case it shouldn't be counted twice. 
Lets create a Python list container/data structure to store all of our words. For a more in depth look at Python lists and how to use them efficiently take a look at .....
Step4:  Now we must...clean the data yet more. It's like when you think you've cleaned your room but your mom tells you it ain't that clean yet.
Step5:  Now we have a Python list of stop words and a Python list of words in our text. We want to cross reference the tokens with the stop words and save those in a new list. Lets do that....
Step6:  Now comes the real fun stuff. Lets plot the word frequency histogram with two lines of actual code.
<ASSISTANT_TASK:>
Python Code:
# Lets see how many lines are in the PDF
# We can use the '!' special character to run Linux commands inside of our notebook
!wc -l test.txt
# Now lets see how many words
!wc -w test.txt
import nltk
from nltk import tokenize
# Lets open the file so we can access the ascii contents
# fd stands for file descriptor but we can use whatever name we want
# the open command returns a file descripor object, which itself isn't very useful
# so we need to read the entire contents so we have a text string we can parse
# advanced: use a context manager with open() as x:
fd = open('test.txt', 'r')
text = fd.read()
text
Explanation: Meetup 1
Going to parse texts for most used words.
End of explanation
# import the regular expression module
import re
Explanation: We want to "tokenize" the text and discard "stopwords" like 'a', 'the', 'in'. These words aren't relevant for our analysis.
To tokenize our text we're going to use regular expressions. Regular expressions are cool and you should try to use them whenever you can. To use regular expression we need to import the regular expression module re. Lets do this in the next cell!!
End of explanation
match_words = '\w+'
tokens = re.findall(match_words, text)
tokens[0:9]
# We can also use nltk to accomplish the same thing
# from nltk.tokenize import RegexpTokenizer
# tokenizer = RegexpTokenizer('\w+')
# tokenizer.tokenize(text)
Explanation: We want to tokenize words. We will use \w+ regular expression to tokenize all the words.
- Lets break this down - \w will match any alphanumerica and underscore characters
- The +
End of explanation
words = []
for word in tokens:
    words.append(word.lower())
words[0:8]
Explanation: That was the easy part..... We want all the data(text) to be "normalized". The word 'Linear' is different then the word 'linear' but for our case it shouldn't be counted twice. 
Lets create a Python list container/data structure to store all of our words. For a more in depth look at Python lists and how to use them efficiently take a look at .....
End of explanation
#Here we want a list of common stopwords but we need to download them first.
nltk.download('stopwords')
stop_words  = nltk.corpus.stopwords.words('english')
stop_words
Explanation: Now we must...clean the data yet more. It's like when you think you've cleaned your room but your mom tells you it ain't that clean yet.
End of explanation
words_nsw = []
for w in words:
    if w not in stop_words:
        words_nsw.append(w)
words_nsw[0:11]
Explanation: Now we have a Python list of stop words and a Python list of words in our text. We want to cross reference the tokens with the stop words and save those in a new list. Lets do that....
End of explanation
# lets import a graphing and data visualization library
import matplotlib.pyplot as plt
# Lets tell jupyter notebook to display images inside our notebook
# %matplotlib inline
freq_dist = nltk.FreqDist(words_nsw)
freq_dist.plot(30)
Explanation: Now comes the real fun stuff. Lets plot the word frequency histogram with two lines of actual code.
End of explanation
<END_TASK>