{"action":"created","issue":{"url":"https://api.github.com/repos/craffel/nntools/issues/20","labels_url":"https://api.github.com/repos/craffel/nntools/issues/20/labels{/name}","comments_url":"https://api.github.com/repos/craffel/nntools/issues/20/comments","events_url":"https://api.github.com/repos/craffel/nntools/issues/20/events","html_url":"https://github.com/craffel/nntools/issues/20","id":64941991,"number":20,"title":"Speedup 20-30% RNN","user":{"login":"lpigou","id":3671037,"avatar_url":"https://avatars.githubusercontent.com/u/3671037?v=3","gravatar_id":"","url":"https://api.github.com/users/lpigou","html_url":"https://github.com/lpigou","followers_url":"https://api.github.com/users/lpigou/followers","following_url":"https://api.github.com/users/lpigou/following{/other_user}","gists_url":"https://api.github.com/users/lpigou/gists{/gist_id}","starred_url":"https://api.github.com/users/lpigou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lpigou/subscriptions","organizations_url":"https://api.github.com/users/lpigou/orgs","repos_url":"https://api.github.com/users/lpigou/repos","events_url":"https://api.github.com/users/lpigou/events{/privacy}","received_events_url":"https://api.github.com/users/lpigou/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-03-28T13:21:09Z","updated_at":"2015-04-24T13:03:42Z","closed_at":null,"body":"In the CustomRecurrentLayer, if you calculate self.input_to_hidden.get_output(layer_input) outside the step function, I observed a GPU speedup of 20-30% for my RNN (bidirectional, 3 layers deep, 256 hidden states each layer, minibatch: 20, time length: 1000, features: 180).\r\n\r\nSomething like this:\r\n\r\n        n_time_steps, n_batch = input.shape[0], input.shape[1]\r\n        input = input.reshape((n_time_steps*n_batch, input.shape[2]))\r\n        input = self.input_to_hidden.get_output(input)\r\n        input = input.reshape((n_time_steps,n_batch,-1))\r\n\r\n        # Create single recurrent computation step function\r\n        def step(layer_input, hid_previous):\r\n            return self.nonlinearity(\r\n                layer_input +\r\n                self.hidden_to_hidden.get_output(hid_previous))\r\n"},"comment":{"url":"https://api.github.com/repos/craffel/nntools/issues/comments/95926744","html_url":"https://github.com/craffel/nntools/issues/20#issuecomment-95926744","issue_url":"https://api.github.com/repos/craffel/nntools/issues/20","id":95926744,"user":{"login":"JackKelly","id":460756,"avatar_url":"https://avatars.githubusercontent.com/u/460756?v=3","gravatar_id":"","url":"https://api.github.com/users/JackKelly","html_url":"https://github.com/JackKelly","followers_url":"https://api.github.com/users/JackKelly/followers","following_url":"https://api.github.com/users/JackKelly/following{/other_user}","gists_url":"https://api.github.com/users/JackKelly/gists{/gist_id}","starred_url":"https://api.github.com/users/JackKelly/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JackKelly/subscriptions","organizations_url":"https://api.github.com/users/JackKelly/orgs","repos_url":"https://api.github.com/users/JackKelly/repos","events_url":"https://api.github.com/users/JackKelly/events{/privacy}","received_events_url":"https://api.github.com/users/JackKelly/received_events","type":"User","site_admin":false},"created_at":"2015-04-24T13:03:42Z","updated_at":"2015-04-24T13:03:42Z","body":"aha, my code runs if I don't set `gradient_steps` for the `RecurrentLayer`.\r\n\r\nis there a way to make this code make when `gradient_steps` is set?  I can't think of a way, but I'm only just starting to understand Theano!"}}