Inverse Document Frequency of a corpus
<p>I have a folder with 10 txt files. I am trying to compute IDF of a given term. But my output is differing from expected. Here is my code for idf.</p>

<p>Here s is a set which contains union of all words from those 10 files.</p>

<pre><code>def idf(term):
    i = 0
    doc_counts = 0
    totaldocs = 10
    if term in s:
        for filename in os.listdir(root_of_my_corpus):
            file = open(os.path.join(root_of_my_corpus, filename), "r", encoding='UTF-8')
            idfdoc = file.read()
            file.close() 
            idfdoc = idfdoc.lower()
            tokenidf = tokenizer.tokenize(idfdoc)
            if term in tokenidf:
                doc_counts+=1
    return(math.log(totaldocs/doc_counts))
</code></pre>