{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1014","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1014/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1014/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1014/events","html_url":"https://github.com/fchollet/keras/issues/1014","id":116976635,"number":1014,"title":"Switch to a modular backend for Theano and TensorFlow support","user":{"login":"fchollet","id":710255,"avatar_url":"https://avatars.githubusercontent.com/u/710255?v=3","gravatar_id":"","url":"https://api.github.com/users/fchollet","html_url":"https://github.com/fchollet","followers_url":"https://api.github.com/users/fchollet/followers","following_url":"https://api.github.com/users/fchollet/following{/other_user}","gists_url":"https://api.github.com/users/fchollet/gists{/gist_id}","starred_url":"https://api.github.com/users/fchollet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fchollet/subscriptions","organizations_url":"https://api.github.com/users/fchollet/orgs","repos_url":"https://api.github.com/users/fchollet/repos","events_url":"https://api.github.com/users/fchollet/events{/privacy}","received_events_url":"https://api.github.com/users/fchollet/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":31,"created_at":"2015-11-15T06:29:52Z","updated_at":"2015-11-23T01:56:22Z","closed_at":null,"body":"Here's the `backend` branch, introducing `theano_backend.py` and `tensorflow_backend.py`: https://github.com/fchollet/keras/tree/backend/keras/backend\r\n\r\nIt's still a work in progress. It contains 95% of the Theano functionality used in Keras, implemented both in Theano and in TensorFlow, and accessible through a same API. \r\n\r\nThe general usage in Keras would be as follow (just a trivial example):\r\n```python\r\nfrom keras import backend as K\r\n\r\nx = K.variable(np.random.random((3, 4)))\r\ny = K.placeholder(ndim=2)\r\nz = K.exp(x) + y\r\nf = K.function(inputs=[y], outputs=[z], updates=[(x, x * 2)])\r\n```\r\n\r\n### What isn't implemented:\r\n- `rnn`, the equivalent of Theano's `scan`.\r\n- the remaining 5%: as far as I can tell, `shape_padright`, `tensordot`, and `batched_tensordot`, the broadcasting business... we might deal with it by eliminating these from the Keras codebase.\r\n\r\n### Known issues\r\n`conv2d` and `maxpool2d` work separately in TF and Theano, but output different values. It is unclear yet why. Not a big issue unless we want trained models to be portable from one backend to another (which I would prefer, of course).\r\n\r\n### What's next\r\n\r\n- adding Theano and TF implementations for `rnn`\r\n- figuring out the `conv2d` and `maxpool2d` issue\r\n- rewriting Keras to use the backend (`K`) instead of Theano. In 95% of cases this would be a trivial search-and-replace. The remaining 5% might require extending the backend a little bit, or reimplementing old Theano functionality with existing backend primitives.\r\n\r\nIf you have cycles to spare, please consider helping out with the rewriting of Keras. It is simple and quite impactful."},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/158828127","html_url":"https://github.com/fchollet/keras/issues/1014#issuecomment-158828127","issue_url":"https://api.github.com/repos/fchollet/keras/issues/1014","id":158828127,"user":{"login":"fchollet","id":710255,"avatar_url":"https://avatars.githubusercontent.com/u/710255?v=3","gravatar_id":"","url":"https://api.github.com/users/fchollet","html_url":"https://github.com/fchollet","followers_url":"https://api.github.com/users/fchollet/followers","following_url":"https://api.github.com/users/fchollet/following{/other_user}","gists_url":"https://api.github.com/users/fchollet/gists{/gist_id}","starred_url":"https://api.github.com/users/fchollet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fchollet/subscriptions","organizations_url":"https://api.github.com/users/fchollet/orgs","repos_url":"https://api.github.com/users/fchollet/repos","events_url":"https://api.github.com/users/fchollet/events{/privacy}","received_events_url":"https://api.github.com/users/fchollet/received_events","type":"User","site_admin":false},"created_at":"2015-11-23T01:56:21Z","updated_at":"2015-11-23T01:56:21Z","body":"So I think we are looking for something like this on the Theano side...\r\n\r\n```python\r\ndef rnn(step_function, inputs, initial_states,\r\n        go_backwards=False, masking=True):\r\n    '''Iterates over the time dimension of a tensor.\r\n\r\n    Parameters\r\n    ----------\r\n    inputs: tensor of temporal data of shape (samples, time, ...)\r\n        (at least 3D).\r\n    step_function:\r\n        Parameters:\r\n            input: tensor with shape (samples, ...) (no time dimension),\r\n                representing input for the batch of samples at a certain\r\n                time step.\r\n            states: list of tensors.\r\n        Returns:\r\n            output: tensor with shape (samples, ...) (no time dimension),\r\n            new_states: list of tensors, same length and shapes\r\n                as 'states'.\r\n    initial_states: tensor with shape (samples, ...) (at least 2D),\r\n        containing the initial values for the states used in\r\n        the step function.\r\n    go_backwards: boolean. If True, do the iteration over\r\n        the time dimension in reverse order.\r\n    masking: boolean. If true, any input timestep inputs[s, i]\r\n        that is all-zeros will be skipped (states will be passed to\r\n        the next step unchanged) and the corresponding output will\r\n        be all zeros.\r\n\r\n    Returns\r\n    -------\r\n    A tuple (outputs, new_states).\r\n        outputs: tensor with shape (samples, time, ...) where each\r\n            entry outputs[s, t] is the output of the step function\r\n            at time t for sample s.\r\n        new_states: list of tensors, updated states used by\r\n            the step function.\r\n    '''\r\n    inputs = inputs.dimshuffle((1, 0, 2))\r\n    initial_states = [s.dimshuffle((1, 0, 2)) for s in initial_states]\r\n\r\n    def _step(*args):\r\n        assert len(args) >= 2\r\n        input = args[0]\r\n        states = args[1:]\r\n        if masking and not T.any(input):\r\n            # if all-zero input timestep, return\r\n            # all-zero output and unchanged states\r\n            return [T.zeros_like(states[0])] + states[1:]\r\n        output, states = step_function(input, states)\r\n        results = [output] + states\r\n        return results\r\n\r\n    results, _ = theano.scan(\r\n        _step,\r\n        sequences=inputs,\r\n        outputs_info=initial_states,\r\n        go_backwards=go_backwards)\r\n\r\n    outputs = results[0]\r\n    states = results[1:]\r\n    outputs = outputs.dimshuffle((1, 0, 2))\r\n    states = [s.dimshuffle((1, 0, 2)) for s in states]\r\n    return outputs, states\r\n```\r\n\r\nThis may well be mistaken, particularly the handling of masking. I'm still thinking about it. If you see anything fishy, please let me know."}}