{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1316","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1316/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1316/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1316/events","html_url":"https://github.com/fchollet/keras/issues/1316","id":123159523,"number":1316,"title":"nbest in rnn","user":{"login":"versis","id":5721737,"avatar_url":"https://avatars.githubusercontent.com/u/5721737?v=3","gravatar_id":"","url":"https://api.github.com/users/versis","html_url":"https://github.com/versis","followers_url":"https://api.github.com/users/versis/followers","following_url":"https://api.github.com/users/versis/following{/other_user}","gists_url":"https://api.github.com/users/versis/gists{/gist_id}","starred_url":"https://api.github.com/users/versis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/versis/subscriptions","organizations_url":"https://api.github.com/users/versis/orgs","repos_url":"https://api.github.com/users/versis/repos","events_url":"https://api.github.com/users/versis/events{/privacy}","received_events_url":"https://api.github.com/users/versis/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-20T13:19:57Z","updated_at":"2015-12-20T16:30:30Z","closed_at":null,"body":"Hi, I'm just starting to learn how keras is working and I have few questions.\r\n\r\n1. How exactly is **preditct_prob()** working? Especially for recurrent layers.\r\nAfter training I would like to compare sequences and check which one is more probable given the model. Is this method providing this funcionality? I can't figure this out from the code :/\r\n\r\n2. Another question regarding example [**lstm_text_generation.py**] (https://github.com/fchollet/keras/blob/master/examples/lstm_text_generation.py).  How many layers this network contains? Isee 2xLSTM and 1xDense, but do we also asume that there is input layer and output layer?\r\n\r\n3. Second LSTM (in the same xample) looks like this:\r\n_model.add(LSTM(512, return_sequences=False))_\r\nWhy **return_sequence** is **False**? Don't we want the network to backpropagate for every element in sequence in each layer?\r\n\r\n4. Is there any parameter for truncated backpropagation?\r\n"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/166133826","html_url":"https://github.com/fchollet/keras/issues/1316#issuecomment-166133826","issue_url":"https://api.github.com/repos/fchollet/keras/issues/1316","id":166133826,"user":{"login":"versis","id":5721737,"avatar_url":"https://avatars.githubusercontent.com/u/5721737?v=3","gravatar_id":"","url":"https://api.github.com/users/versis","html_url":"https://github.com/versis","followers_url":"https://api.github.com/users/versis/followers","following_url":"https://api.github.com/users/versis/following{/other_user}","gists_url":"https://api.github.com/users/versis/gists{/gist_id}","starred_url":"https://api.github.com/users/versis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/versis/subscriptions","organizations_url":"https://api.github.com/users/versis/orgs","repos_url":"https://api.github.com/users/versis/repos","events_url":"https://api.github.com/users/versis/events{/privacy}","received_events_url":"https://api.github.com/users/versis/received_events","type":"User","site_admin":false},"created_at":"2015-12-20T16:30:30Z","updated_at":"2015-12-20T16:30:30Z","body":"_**Update for point 3:**_\r\nI noticed that there is a possibility to have either **Dense** or **TimeDistributedDense** layer at the end of the network. In the first case we would like to have return_sequence equal to _False_ and in the second case _True_, correct?\r\n\r\nDo we assume that when using TimeDistributedDense labels (y) are sequences of the same size as corresponding input sequences (x)?\r\n\r\nIf so, then using Dense at the end (given that I want to have RNN) means that we feed the network with sequence and expect one element at the end, correct? \r\n\r\nPutting this in context of [http://karpathy.github.io/2015/05/21/rnn-effectiveness/](Karpathy's rrn article) and his description of different types of RNN. Using Dense is like _many to one (3)_ type, and using TimeDistributedDense is like _many to many (5, not 4)_ type?\r\n\r\nWhy isn't the example from point **2** using TimeDistributedDense layer at the end? Wouldn't that work better (in theory)?"}}