How to calculate conditional_frequency_distribution and conditional_probability_distribution for trigrams in nltk python
<p>I want to calculate <strong>Conditional Probability Distribution</strong> for my language model but I am not able to do because I need <strong>Conditional Frequency Distribution</strong> which I am not able to generate. This is my code:</p>

<pre><code># -*- coding: utf-8 -*-

import io
import nltk
from nltk.util import ngrams
from nltk.tokenize import sent_tokenize
from preprocessor import utf8_to_ascii

with io.open("mypet.txt",'r',encoding='utf8') as utf_file:
    file_content = utf_file.read()

ascii_content = utf8_to_ascii(file_content)
sentence_tokenize_list = sent_tokenize(ascii_content)

all_trigrams = []

for sentence in sentence_tokenize_list:
    sentence = sentence.rstrip('.!?')
    tokens = nltk.re.findall(r"\w+(?:[-']\w+)*|'|[-.(]+|\S\w*", sentence)
    trigrams = ngrams(tokens, 3,pad_left=True,pad_right=True,left_pad_symbol='&lt;s&gt;', right_pad_symbol="&lt;/s&gt;")
    all_trigrams.extend(trigrams)

conditional_frequency_distribution = nltk.ConditionalFreqDist(all_trigrams)
conditional_probability_distribution = nltk.ConditionalProbDist(conditional_frequency_distribution, nltk.MLEProbDist)

for trigram in all_trigrams:
    print "{0}: {1}".format(conditional_probability_distribution[trigram[0]].prob(trigram[1]), trigram)
</code></pre>

<p>But I am getting this error:</p>

<pre><code>line 23, in &lt;module&gt;
ValueError: too many values to unpack
</code></pre>

<p>This is my preprocessor.py file which is handling utf-8 chars:</p>

<pre><code># -*- coding: utf-8 -*-

import json


def utf8_to_ascii(utf8_text):
    with open("utf_to_ascii.json") as data_file:
        data = json.load(data_file)
    utf_table = data["chars"]
    for key, value in utf_table.items():
        utf8_text = utf8_text.replace(key, value)
    return utf8_text.encode('ascii')
</code></pre>

<p>And this is my utf_to_ascii.json file which I used to replace utf-8 char to ascii char:</p>

<pre><code>{
 "chars": {
          "“":"",
          "”":"",
          "’":"'",
          "—":"-",
          "–":"-"
 }
}
</code></pre>

<p>Can someone suggest How can I calculate Conditional Frequency Distribution for trigrams in NLTK?</p>