{"action":"created","issue":{"url":"https://api.github.com/repos/Lasagne/Lasagne/issues/425","labels_url":"https://api.github.com/repos/Lasagne/Lasagne/issues/425/labels{/name}","comments_url":"https://api.github.com/repos/Lasagne/Lasagne/issues/425/comments","events_url":"https://api.github.com/repos/Lasagne/Lasagne/issues/425/events","html_url":"https://github.com/Lasagne/Lasagne/issues/425","id":107537894,"number":425,"title":"Provide RNN \"containers\"","user":{"login":"f0k","id":629706,"avatar_url":"https://avatars.githubusercontent.com/u/629706?v=3","gravatar_id":"","url":"https://api.github.com/users/f0k","html_url":"https://github.com/f0k","followers_url":"https://api.github.com/users/f0k/followers","following_url":"https://api.github.com/users/f0k/following{/other_user}","gists_url":"https://api.github.com/users/f0k/gists{/gist_id}","starred_url":"https://api.github.com/users/f0k/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/f0k/subscriptions","organizations_url":"https://api.github.com/users/f0k/orgs","repos_url":"https://api.github.com/users/f0k/repos","events_url":"https://api.github.com/users/f0k/events{/privacy}","received_events_url":"https://api.github.com/users/f0k/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-09-21T15:31:14Z","updated_at":"2015-09-21T16:26:25Z","closed_at":null,"body":"In https://github.com/Lasagne/Recipes/pull/23#issuecomment-141979978, @skaae brought up the idea of splitting up the recurrent layers into layers performing a single computation step and a container layer that handles the recurrence (i.e., sets up the `scan` function or explicitly unrolls it).\r\n\r\nThe use case was to provide a way to efficiently sample from an RNN that was trained to generate time sequences (e.g., text) -- the most efficient way would be to tap into the scan function, but that's impossible the way it's currently encapsulated.\r\nAnother advantage could be that it reduces the amount of code duplication (see #356), and maybe it would even make it easier to equip LSTMs or GRUs with custom connections (#357)?\r\n\r\nAs far as I've understood, the following current code:\r\n```\r\nlayer = lasagne.layers.InputLayer(shape=(N_BATCH, MAX_LENGTH, 2))\r\nlayer = lasagne.layers.RecurrentLayer(layer, N_HIDDEN)\r\nlayer = lasagne.layers.SliceLayer(layer, -1, 1)\r\nlayer = lasagne.layers.DenseLayer(layer, num_units=1, nonlinearity=lasagne.nonlinearities.tanh)\r\n```\r\nCould be turned into:\r\n```\r\nlayer = lasagne.layers.InputLayer(shape=(N_BATCH, MAX_LENGTH, 2))\r\n# single time step\r\nrnn_layer = lasagne.layers.InputLayer((N_BATCH, 2))\r\nrnn_layer = lasagne.layers.VanillaRecurrentStep(rnn_layer, N_HIDDEN)\r\nlayer = lasagne.layers.Recurrence(layer, step=rnn_layer)\r\nlayer = lasagne.layers.SliceLayer(layer, -1, 1)\r\nlayer = lasagne.layers.DenseLayer(layer, num_units=1, nonlinearity=lasagne.nonlinearities.tanh)\r\n```\r\nWhere we'd have different `Step` layers, and one of them could be sampling from a multivariate distribution, for the character generation use case.\r\n`Recurrence` could have a keyword argument telling whether it persists its recurrent states, and another telling whether it feeds back on its own output (that would be needed for character generation).\r\n\r\nDisclaimer: I've just started looking into RNNs and am not sure what's the best way to break down the code yet -- I'm just opening this issue so there is a place to discuss the idea for those who are into RNNs with possible participation by a future me :)"},"comment":{"url":"https://api.github.com/repos/Lasagne/Lasagne/issues/comments/142034327","html_url":"https://github.com/Lasagne/Lasagne/issues/425#issuecomment-142034327","issue_url":"https://api.github.com/repos/Lasagne/Lasagne/issues/425","id":142034327,"user":{"login":"skaae","id":2623134,"avatar_url":"https://avatars.githubusercontent.com/u/2623134?v=3","gravatar_id":"","url":"https://api.github.com/users/skaae","html_url":"https://github.com/skaae","followers_url":"https://api.github.com/users/skaae/followers","following_url":"https://api.github.com/users/skaae/following{/other_user}","gists_url":"https://api.github.com/users/skaae/gists{/gist_id}","starred_url":"https://api.github.com/users/skaae/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/skaae/subscriptions","organizations_url":"https://api.github.com/users/skaae/orgs","repos_url":"https://api.github.com/users/skaae/repos","events_url":"https://api.github.com/users/skaae/events{/privacy}","received_events_url":"https://api.github.com/users/skaae/received_events","type":"User","site_admin":false},"created_at":"2015-09-21T16:26:25Z","updated_at":"2015-09-21T16:26:25Z","body":"I'll try to code something later tonight"}}