{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/395","labels_url":"https://api.github.com/repos/fchollet/keras/issues/395/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/395/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/395/events","html_url":"https://github.com/fchollet/keras/issues/395","id":95211755,"number":395,"title":"is the Sequence to Sequence learning right?","user":{"login":"EderSantana","id":3902480,"avatar_url":"https://avatars.githubusercontent.com/u/3902480?v=3","gravatar_id":"","url":"https://api.github.com/users/EderSantana","html_url":"https://github.com/EderSantana","followers_url":"https://api.github.com/users/EderSantana/followers","following_url":"https://api.github.com/users/EderSantana/following{/other_user}","gists_url":"https://api.github.com/users/EderSantana/gists{/gist_id}","starred_url":"https://api.github.com/users/EderSantana/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/EderSantana/subscriptions","organizations_url":"https://api.github.com/users/EderSantana/orgs","repos_url":"https://api.github.com/users/EderSantana/repos","events_url":"https://api.github.com/users/EderSantana/events{/privacy}","received_events_url":"https://api.github.com/users/EderSantana/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":60,"created_at":"2015-07-15T14:38:31Z","updated_at":"2015-10-29T20:50:38Z","closed_at":"2015-08-13T14:33:52Z","body":"Assume we are trying to learn a sequence to sequence map. For this we can use Recurrent and TimeDistributedDense layers. Now assume that the sequences have different lengths. We should pad both input and desired sequences with zeros, right? But how will the objective function handle the  padded values? There is no choice to pass a mask to the objective function. Won't this bias the cost function?"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/152317907","html_url":"https://github.com/fchollet/keras/issues/395#issuecomment-152317907","issue_url":"https://api.github.com/repos/fchollet/keras/issues/395","id":152317907,"user":{"login":"LeavesBreathe","id":14891677,"avatar_url":"https://avatars.githubusercontent.com/u/14891677?v=3","gravatar_id":"","url":"https://api.github.com/users/LeavesBreathe","html_url":"https://github.com/LeavesBreathe","followers_url":"https://api.github.com/users/LeavesBreathe/followers","following_url":"https://api.github.com/users/LeavesBreathe/following{/other_user}","gists_url":"https://api.github.com/users/LeavesBreathe/gists{/gist_id}","starred_url":"https://api.github.com/users/LeavesBreathe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/LeavesBreathe/subscriptions","organizations_url":"https://api.github.com/users/LeavesBreathe/orgs","repos_url":"https://api.github.com/users/LeavesBreathe/repos","events_url":"https://api.github.com/users/LeavesBreathe/events{/privacy}","received_events_url":"https://api.github.com/users/LeavesBreathe/received_events","type":"User","site_admin":false},"created_at":"2015-10-29T20:50:38Z","updated_at":"2015-10-29T20:50:38Z","body":"@sergeyf @simonhughes22 Alright, I just feel stupid. Both of you are telling me the answer, and I can't understand it. \r\n\r\nI completely understand x_train. The 3d matrix of (nb_samples, timesteps, word_vectors). And I can do the same with y_train as well. But when I do model.predict, won't the model have to predict word vectors for each word? \r\n\r\nSuppose you use 32 scalars per word (when you set word2vec size = 32). This means that for each word in the sentence, you must predict 32 scalars per word correct? Not only that, you can't use a softmax. And what are the odds that the network is going predict the exact 32 scalars that correspond to a word? This is why I'm so lost. "}}