{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/98","labels_url":"https://api.github.com/repos/fchollet/keras/issues/98/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/98/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/98/events","html_url":"https://github.com/fchollet/keras/issues/98","id":73243556,"number":98,"title":"\"real time\" recurrent nets","user":{"login":"lemuriandezapada","id":11992601,"avatar_url":"https://avatars.githubusercontent.com/u/11992601?v=3","gravatar_id":"","url":"https://api.github.com/users/lemuriandezapada","html_url":"https://github.com/lemuriandezapada","followers_url":"https://api.github.com/users/lemuriandezapada/followers","following_url":"https://api.github.com/users/lemuriandezapada/following{/other_user}","gists_url":"https://api.github.com/users/lemuriandezapada/gists{/gist_id}","starred_url":"https://api.github.com/users/lemuriandezapada/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lemuriandezapada/subscriptions","organizations_url":"https://api.github.com/users/lemuriandezapada/orgs","repos_url":"https://api.github.com/users/lemuriandezapada/repos","events_url":"https://api.github.com/users/lemuriandezapada/events{/privacy}","received_events_url":"https://api.github.com/users/lemuriandezapada/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":51,"created_at":"2015-05-05T08:14:05Z","updated_at":"2015-08-12T18:12:07Z","closed_at":null,"body":"Hey guys,\r\n\r\nI was wondering how are the initial internal states in a recurrent layer dealt with? So far it appears they are reset are every run. Is there any way to preserve them?\r\n\r\nI'd like to be able to feed a .predict_proba() function data one time step at a time for a time series task, as the points come in, without also feeding the entire history all over again. Is this somehow possible? Thanks"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/130397650","html_url":"https://github.com/fchollet/keras/issues/98#issuecomment-130397650","issue_url":"https://api.github.com/repos/fchollet/keras/issues/98","id":130397650,"user":{"login":"harpone","id":5112840,"avatar_url":"https://avatars.githubusercontent.com/u/5112840?v=3","gravatar_id":"","url":"https://api.github.com/users/harpone","html_url":"https://github.com/harpone","followers_url":"https://api.github.com/users/harpone/followers","following_url":"https://api.github.com/users/harpone/following{/other_user}","gists_url":"https://api.github.com/users/harpone/gists{/gist_id}","starred_url":"https://api.github.com/users/harpone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/harpone/subscriptions","organizations_url":"https://api.github.com/users/harpone/orgs","repos_url":"https://api.github.com/users/harpone/repos","events_url":"https://api.github.com/users/harpone/events{/privacy}","received_events_url":"https://api.github.com/users/harpone/received_events","type":"User","site_admin":false},"created_at":"2015-08-12T18:12:07Z","updated_at":"2015-08-12T18:12:07Z","body":"Maybe I'm completely haywire here (don't know Theano much, Keras even less), but can't you do something like this:\r\n\r\nSo let's say we have init h0:\r\n```\r\nself.h0  = theano.shared(np.zeros((1, nh), dtype=theano.config.floatX))  # init h\r\n```\r\nyada yada then there's the scan over the training data:\r\n\r\n```\r\n[h_trn, x_trn], _ = theano.scan(fn=self.recurrence,\r\n                                   sequences=x,\r\n                                   outputs_info=[self.h0, None])\r\n```\r\n\r\nThen the usual parameter updates:\r\n```\r\nupdates = OrderedDict((p, p-lr*g) for p, g in zip(self.params, gradients))\r\n```\r\n\r\nThen we can just update the self.h0 value together with the parameter updates like this:\r\n\r\n```\r\nupdates.update({self.h0:h_trn[-1, :, :]})\r\n```"}}