<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Algorithms Exercise 1
Imports
Step3:  Word counting
Write a function tokenize that takes a string of English text returns a list of words. It should also remove stop words, which are common short words that are often removed before natural language processing. Your function should have the following logic
Step4:  Write a function count_words that takes a list of words and returns a dictionary where the keys in the dictionary are the unique words in the list and the values are the word counts.
Step6:  Write a function sort_word_counts that return a list of sorted word counts
Step7:  Perform a word count analysis on Chapter 1 of Moby Dick, whose text can be found in the file mobydick_chapter1.txt
Step8:  Create a "Cleveland Style" dotplot of the counts of the top 50 words using Matplotlib. If you don't know what a dotplot is, you will have to do some research...
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
from matplotlib import pyplot as plt
import numpy as np
Explanation: Algorithms Exercise 1
Imports
End of explanation
def tokenize(s, stop_words=None, punctuation='`~!@#$%^&*()_-+={[}]|\:;"<,>.?/}\t'):
    Split a string into a list of words, removing punctuation and stop words.
    w = []
    for line in s.splitlines(): #uses the splitlines function to split each of the lines in the input
        w.extend(line.split(' '))
    w = [''.join(filter(lambda c: c not in punctuation, word)) for word in w] # filters out the punctuation
    if isinstance(stop_words, str):
        stop_words = stop_words.split(' ')
    if stop_words is not None: #calls the if statment to pull out the words if they are not nothing
        w = [word for word in w if word not in stop_words]
    w = [word.lower() for word in w if word]
    return w #ends the function
tokenize(s='no @ the range, randy .. is !a',stop_words='the', punctuation='`~!@#$%^&*()_-+={[}]|\:;"<,>.?/}\t') #my check to see if it was working
assert tokenize("This, is the way; that things will end", stop_words=['the', 'is']) == \
    ['this', 'way', 'that', 'things', 'will', 'end']
wasteland = 
APRIL is the cruellest month, breeding
Lilacs out of the dead land, mixing
Memory and desire, stirring
Dull roots with spring rain.
assert tokenize(wasteland, stop_words='is the of and') == \
    ['april','cruellest','month','breeding','lilacs','out','dead','land',
     'mixing','memory','desire','stirring','dull','roots','with','spring',
     'rain']
Explanation: Word counting
Write a function tokenize that takes a string of English text returns a list of words. It should also remove stop words, which are common short words that are often removed before natural language processing. Your function should have the following logic:
Split the string into lines using splitlines.
Split each line into a list of words and merge the lists for each line.
Use Python's builtin filter function to remove all punctuation.
If stop_words is a list, remove all occurences of the words in the list.
If stop_words is a space delimeted string of words, split them and remove them.
Remove any remaining empty words.
Make all words lowercase.
End of explanation
def count_words(data):
    wc = {}
    for word in data:
        if word in wc:
            wc[word] += 1
        else:
            wc[word] = 1
    return wc
assert count_words(tokenize('this and the this from and a a a')) == \
    {'a': 3, 'and': 2, 'from': 1, 'the': 1, 'this': 2}
Explanation: Write a function count_words that takes a list of words and returns a dictionary where the keys in the dictionary are the unique words in the list and the values are the word counts.
End of explanation
def sort_word_counts(wc):
    Return a list of 2-tuples of (word, count), sorted by count descending.
    return list(sorted(wc.items(),key=lambda x:x[1],reverse=True))
    
assert sort_word_counts(count_words(tokenize('this and a the this this and a a a'))) == \
    [('a', 4), ('this', 3), ('and', 2), ('the', 1)]
Explanation: Write a function sort_word_counts that return a list of sorted word counts:
Each element of the list should be a (word, count) tuple.
The list should be sorted by the word counts, with the higest counts coming first.
To perform this sort, look at using the sorted function with a custom key and reverse
  argument.
End of explanation
with open('mobydick_chapter1.txt') as ishmael:
    raw=ishmael.read()
l=tokenize(raw,stop_words='the of and a to in is it that as')
c=count_words(l)
swc=sort_word_counts(c)
assert swc[0]==('i',43)
assert len(swc)==848
Explanation: Perform a word count analysis on Chapter 1 of Moby Dick, whose text can be found in the file mobydick_chapter1.txt:
Read the file into a string.
Tokenize with stop words of 'the of and a to in is it that as'.
Perform a word count, the sort and save the result in a variable named swc.
End of explanation
# YOUR CODE HERE
raise NotImplementedError()
assert True # use this for grading the dotplot
Explanation: Create a "Cleveland Style" dotplot of the counts of the top 50 words using Matplotlib. If you don't know what a dotplot is, you will have to do some research...
End of explanation
<END_TASK>