{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/379","labels_url":"https://api.github.com/repos/fchollet/keras/issues/379/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/379/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/379/events","html_url":"https://github.com/fchollet/keras/issues/379","id":94410284,"number":379,"title":"merge_mode=\"concat\" behaving unexpectedly? (with the GPU backend)","user":{"login":"iskandr","id":48441,"avatar_url":"https://avatars.githubusercontent.com/u/48441?v=3","gravatar_id":"","url":"https://api.github.com/users/iskandr","html_url":"https://github.com/iskandr","followers_url":"https://api.github.com/users/iskandr/followers","following_url":"https://api.github.com/users/iskandr/following{/other_user}","gists_url":"https://api.github.com/users/iskandr/gists{/gist_id}","starred_url":"https://api.github.com/users/iskandr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/iskandr/subscriptions","organizations_url":"https://api.github.com/users/iskandr/orgs","repos_url":"https://api.github.com/users/iskandr/repos","events_url":"https://api.github.com/users/iskandr/events{/privacy}","received_events_url":"https://api.github.com/users/iskandr/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-07-10T23:05:35Z","updated_at":"2015-10-08T09:11:03Z","closed_at":"2015-07-10T23:58:14Z","body":"I'm building a network using the new Graph model which takes two sequences as inputs, feeds them through an RNN, concatenates the final output vectors, and passes that on to a dense layer. I expected that the input size of dense layer should be twice the output dimensionality of the RNNs but this doesn't seem to be working. \r\n\r\nDoes anyone know what I may be doing wrong? \r\n\r\n```\r\ngraph = Graph()\r\ngraph.add_input(name='peptide', ndim=3)\r\ngraph.add_input(name='mhc', ndim=3)\r\n\r\n# RNN for peptide sequences\r\ngraph.add_node(JZS1(input_dim=20, output_dim=5), name=\"peptide_rnn\", input=\"peptide\")\r\n                                                                                                                        \r\n# RNN for MHC sequences\r\ngraph.add_node(JZS1(input_dim=20, output_dim=5), name=\"mhc_rnn\", input=\"mhc\")\r\n\r\n# concatenate last output of both RNNs and transform them into a lower dimensional space\r\ngraph.add_node(Dense(5 * 2, 16, activation=\"relu\"), name=\"hidden\", merge_mode=\"concat\", inputs=(\"peptide_rnn\", \"mhc_rnn\"))\r\n\r\n# output prediction of affinity\r\ngraph.add_node(Dense(16, 1, activation=\"sigmoid\"), name=\"affinity\", input=\"hidden\")\r\n\r\n# weird that I have to name the output twice\r\ngraph.add_output(name='affinity_output', input='affinity')\r\ngraph.compile('rmsprop', {'affinity_output':'mse'})\r\n```\r\n\r\nThe error that I get is:\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"train-jsz-rnn.py\", line 138, in <module>\r\n    history = graph.fit({'peptide':padded_peptides, 'mhc':padded_mhc, 'affinity_output':log_target_values}, nb_epoch=1)\r\n  File \"/home/ubuntu/keras/keras/models.py\", line 537, in fit\r\n    validation_split=validation_split, val_f=val_f, val_ins=val_ins, shuffle=shuffle, metrics=metrics)\r\n  File \"/home/ubuntu/keras/keras/models.py\", line 135, in _fit\r\n    outs = f(*ins_batch)\r\n  File \"/home/ubuntu/anaconda/lib/python2.7/site-packages/theano/compile/function_module.py\", line 588, in __call__\r\n    self.fn.thunks[self.fn.position_of_error])\r\n  File \"/home/ubuntu/anaconda/lib/python2.7/site-packages/theano/compile/function_module.py\", line 579, in __call__\r\n    outputs = self.fn()\r\nValueError: dimension mismatch in args to gemm (128,5)x(10,16)->(128,16)\r\nApply node that caused the error: GpuDot22(GpuJoin.0, hidden_W)\r\nInputs shapes: [(128, 5), (10, 16)]\r\nInputs strides: [(5, 1), (16, 1)]\r\nInputs types: [CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix)]\r\n```\r\n\r\nIt seems like the two RNN outputs are not actually getting merged into a 10 dimensional vector, is that expected?\r\n\r\n*edit*: This error goes away (and the model trains successfully) when using the Theano CPU backend. \r\n\r\n*edit 2.0*: The error goes away even on the GPU if I switch the JZS1 RNN to an LSTM. \r\n\r\n*edit 3.0* Actually, the error persists with LSTMs but only if I'm using floatX=32. Current hypothesis: merging of input variables doesn't get handled correctly when calling an external matrix multiply routine. "},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/146465831","html_url":"https://github.com/fchollet/keras/issues/379#issuecomment-146465831","issue_url":"https://api.github.com/repos/fchollet/keras/issues/379","id":146465831,"user":{"login":"zhipengChen","id":13817269,"avatar_url":"https://avatars.githubusercontent.com/u/13817269?v=3","gravatar_id":"","url":"https://api.github.com/users/zhipengChen","html_url":"https://github.com/zhipengChen","followers_url":"https://api.github.com/users/zhipengChen/followers","following_url":"https://api.github.com/users/zhipengChen/following{/other_user}","gists_url":"https://api.github.com/users/zhipengChen/gists{/gist_id}","starred_url":"https://api.github.com/users/zhipengChen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhipengChen/subscriptions","organizations_url":"https://api.github.com/users/zhipengChen/orgs","repos_url":"https://api.github.com/users/zhipengChen/repos","events_url":"https://api.github.com/users/zhipengChen/events{/privacy}","received_events_url":"https://api.github.com/users/zhipengChen/received_events","type":"User","site_admin":false},"created_at":"2015-10-08T09:11:03Z","updated_at":"2015-10-08T09:11:03Z","body":"thanks "}}