{"action":"created","issue":{"url":"https://api.github.com/repos/Theano/Theano/issues/2845","labels_url":"https://api.github.com/repos/Theano/Theano/issues/2845/labels{/name}","comments_url":"https://api.github.com/repos/Theano/Theano/issues/2845/comments","events_url":"https://api.github.com/repos/Theano/Theano/issues/2845/events","html_url":"https://github.com/Theano/Theano/issues/2845","id":71806266,"number":2845,"title":"Pickle error when running multiple processes in parallel on current master branch","user":{"login":"twiecki","id":674200,"avatar_url":"https://avatars.githubusercontent.com/u/674200?v=3","gravatar_id":"","url":"https://api.github.com/users/twiecki","html_url":"https://github.com/twiecki","followers_url":"https://api.github.com/users/twiecki/followers","following_url":"https://api.github.com/users/twiecki/following{/other_user}","gists_url":"https://api.github.com/users/twiecki/gists{/gist_id}","starred_url":"https://api.github.com/users/twiecki/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twiecki/subscriptions","organizations_url":"https://api.github.com/users/twiecki/orgs","repos_url":"https://api.github.com/users/twiecki/repos","events_url":"https://api.github.com/users/twiecki/events{/privacy}","received_events_url":"https://api.github.com/users/twiecki/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-04-29T07:58:45Z","updated_at":"2015-05-21T13:29:42Z","closed_at":"2015-05-21T13:29:42Z","body":"I'm using IPython parallel to distribute the same theano graph (with different inputs) across many python processes on the same machine (i.e. the same file system). Using current theano master, I'm getting the following error on some processes:\r\n\r\n```python\r\n/opt/miniconda/lib/python2.7/site-packages/theano/gof/cmodule.pyc in check_key(self, key, key_pkl)\r\n   1156                     \"%s. Verify the __eq__ and __hash__ functions of your \"\r\n   1157                     \"Ops. The file is: %s. The key is: %s\" %\r\n-> 1158                     (msg, key_pkl, key))\r\n   1159         # Also verify that there exists no other loaded key that would be equal\r\n   1160         # to this key. In order to speed things up, we only compare to keys\r\nAssertionError: Key not found in unpickled KeyData file. Verify the __eq__ and __hash__ functions of your Ops. The file is: /home/wiecki/.theano/compiledir_Linux-3.13--generic-x86_64-with-debian-jessie-sid-x86_64-2.7.9-64/tmpjd_7GZ/key.pkl. The key is: (((11, (4,), (13, '1.9.2'), (13, '1.9.2'), ('openmp', False)), (11, 13, '1.9.2'), (11, 13, '1.9.2')), ('CLinker.cmodule_key', ('--param', '--param', '--param', '-D NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION', '-O3', '-Wno-unused-label', '-Wno-unused-variable', '-Wno-write-strings', '-fPIC', '-fno-math-errno', '-m64', '-maes', '-march=core-avx-i', '-mavx', '-mcx16', '-mf16c', '-mfsgsbase', '-mfxsr', '-mno-abm', '-mno-adx', '-mno-avx2', '-mno-bmi', '-mno-bmi2', '-mno-fma', '-mno-fma4', '-mno-hle', '-mno-lwp', '-mno-lzcnt', '-mno-movbe', '-mno-prfchw', '-mno-rdseed', '-mno-rtm', '-mno-tbm', '-mno-xop', '-mpclmul', '-mpopcnt', '-mrdrnd', '-msahf', '-msse4.1', '-msse4.2', '-mtune=core-avx-i', '-mxsave', '-mxsaveopt', 'l1-cache-line-size=64', 'l1-cache-size=32', 'l2-cache-size=25600'), (), (), 'NPY_ABI_VERSION=0x1000009', u'c_compiler_str=/usr/bin/g++ 4.8', 'md5:b53e29d0700dd6d124a13c2797487a17', (<theano.tensor.elemwise.Elemwise object at 0x7f2213430990>, ((TensorType(float64, scalar), (('30e2a936332b4d781f8b5be42400a916', 0, 0), False)),), (1, (False,)))))\r\n```"},"comment":{"url":"https://api.github.com/repos/Theano/Theano/issues/comments/104277579","html_url":"https://github.com/Theano/Theano/issues/2845#issuecomment-104277579","issue_url":"https://api.github.com/repos/Theano/Theano/issues/2845","id":104277579,"user":{"login":"twiecki","id":674200,"avatar_url":"https://avatars.githubusercontent.com/u/674200?v=3","gravatar_id":"","url":"https://api.github.com/users/twiecki","html_url":"https://github.com/twiecki","followers_url":"https://api.github.com/users/twiecki/followers","following_url":"https://api.github.com/users/twiecki/following{/other_user}","gists_url":"https://api.github.com/users/twiecki/gists{/gist_id}","starred_url":"https://api.github.com/users/twiecki/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twiecki/subscriptions","organizations_url":"https://api.github.com/users/twiecki/orgs","repos_url":"https://api.github.com/users/twiecki/repos","events_url":"https://api.github.com/users/twiecki/events{/privacy}","received_events_url":"https://api.github.com/users/twiecki/received_events","type":"User","site_admin":false},"created_at":"2015-05-21T13:29:42Z","updated_at":"2015-05-21T13:29:42Z","body":"I tried with most recent master but so far couldn't reproduce. I'll reopen if the problem re-occurs but for now it looks like it's fixed. Many thanks!"}}