<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Language Classifier
Step1:  Text Classification from Folders
Step2:  Footnote
Step3:  Bigrams/Trigrams
Step4:  specify the ngram_range - the smallest ngram to use, and the largest.  the default is (1,1), so only 1-grams are used.  this example calculates the 1-gram and the 2-gram (bi-gram)
<ASSISTANT_TASK:>
Python Code:
count,feature_names=text.count_letters('data/languages/E3.txt')
print((count,feature_names))
count,feature_names=text.count_letters('data/languages/E3.txt')
print((count,feature_names))
p=text.letter_freq('English',feature_names)
print(p)
print((sum(count*log10(p))))
C=text.LanguageFileClassifier()
result=C.loglikelihood('data/languages/E*.txt',verbose=True)
C.predict('data/languages/E*',verbose=True)
[C.target_names[i] for i in C.predict('data/languages/E*')]
Explanation: Language Classifier
End of explanation
from classy import text
train=text.load_files('data/films/train',verbose=True)
test=text.load_files('data/films/test',verbose=True)
train,test=text.text_to_vectors('data/films/train','data/films/test',verbose=True)
train.vectors
v=array(train.vectors[0,:].todense()).ravel()
v.max()
v.shape
v=array(train.vectors[0,:].todense()).ravel()
plot(v,'.')
v=array(train.vectors[10,:].todense()).ravel()
plot(v,'.')
xlabel('feature number')
ylabel('frequency of feature')
train.vectors.shape
C=text.Multinomial()
C.fit(train.vectors,train.targets)
C.predict(test.vectors)
C.percent_correct(test.vectors,test.targets)
Explanation: Text Classification from Folders
End of explanation
from classy import *
train_files=text.load_files('data/films/train',verbose=True)
test_files=text.load_files('data/films/test',verbose=True)
train_data,test_data=text.text_to_vectors(train_files,test_files,verbose=True)
train_data.vectors
vectors_to_image(train_data.vectors,binary=True)
vectors_to_image(train_data.vectors,binary=False)
Explanation: Footnote
End of explanation
from classy import text
train_files=text.load_files('data/films/train',verbose=True)
test_files=text.load_files('data/films/test',verbose=True)
train_data,test_data=text.text_to_vectors(train_files,test_files,verbose=True)
train_data.vectors
Explanation: Bigrams/Trigrams
End of explanation
train_data,test_data=text.text_to_vectors(train_files,test_files,ngram_range=(1,2),verbose=True)
train_data.vectors
print((train_data.feature_names[:100]))
Explanation: specify the ngram_range - the smallest ngram to use, and the largest.  the default is (1,1), so only 1-grams are used.  this example calculates the 1-gram and the 2-gram (bi-gram)
End of explanation
<END_TASK>