{"action":"created","issue":{"url":"https://api.github.com/repos/blaze/dask/issues/717","labels_url":"https://api.github.com/repos/blaze/dask/issues/717/labels{/name}","comments_url":"https://api.github.com/repos/blaze/dask/issues/717/comments","events_url":"https://api.github.com/repos/blaze/dask/issues/717/events","html_url":"https://github.com/blaze/dask/pull/717","id":106813396,"number":717,"title":"Propagate exceptions without modification in async","user":{"login":"mrocklin","id":306380,"avatar_url":"https://avatars.githubusercontent.com/u/306380?v=3","gravatar_id":"","url":"https://api.github.com/users/mrocklin","html_url":"https://github.com/mrocklin","followers_url":"https://api.github.com/users/mrocklin/followers","following_url":"https://api.github.com/users/mrocklin/following{/other_user}","gists_url":"https://api.github.com/users/mrocklin/gists{/gist_id}","starred_url":"https://api.github.com/users/mrocklin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrocklin/subscriptions","organizations_url":"https://api.github.com/users/mrocklin/orgs","repos_url":"https://api.github.com/users/mrocklin/repos","events_url":"https://api.github.com/users/mrocklin/events{/privacy}","received_events_url":"https://api.github.com/users/mrocklin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-09-16T16:46:34Z","updated_at":"2015-09-21T17:31:00Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/blaze/dask/pulls/717","html_url":"https://github.com/blaze/dask/pull/717","diff_url":"https://github.com/blaze/dask/pull/717.diff","patch_url":"https://github.com/blaze/dask/pull/717.patch"},"body":"Previously in the async scheduler we used to put the traceback into the\r\nexception.  This caused problems because we assumed that all exceptions\r\nexpected a single message argument.  This assumption is false and,\r\nwe are unable to cleanly modify remote exceptions.\r\n\r\nNow our policy is to print out the traceback and then raise the original\r\nexception.\r\n\r\n## Before\r\n\r\n```python\r\nIn [1]: def f(x, y):\r\n    return x / y\r\n   ...: \r\n\r\nIn [2]: from dask.threaded import get\r\n\r\nIn [3]: get({'x': (f, 1, 0)}, 'x')\r\n---------------------------------------------------------------------------\r\nZeroDivisionError                         Traceback (most recent call last)\r\n<ipython-input-3-73ec8f9d1e5e> in <module>()\r\n----> 1 get({'x': (f, 1, 0)}, 'x')\r\n\r\n/home/mrocklin/workspace/dask/dask/threaded.pyc in get(dsk, result, cache, num_workers, **kwargs)\r\n     55     results = get_async(pool.apply_async, len(pool._pool), dsk, result,\r\n     56                         cache=cache, queue=queue, get_id=_thread_get_id,\r\n---> 57                         **kwargs)\r\n     58 \r\n     59     return results\r\n\r\n/home/mrocklin/workspace/dask/dask/async.py in get_async(apply_async, num_workers, dsk, result, cache, queue, get_id, raise_on_exception, rerun_exceptions_locally, callbacks, **kwargs)\r\n    494                               + 'Traceback:\\n'\r\n    495                               + '----------\\n'\r\n--> 496                               + tb)\r\n    497         state['cache'][key] = res\r\n    498         finish_task(dsk, key, state, results, keyorder.get)\r\n\r\nZeroDivisionError: \r\nRemote Exception:\r\n-----------------\r\ninteger division or modulo by zero\r\n\r\nTraceback:\r\n----------\r\n  File \"dask/async.py\", line 266, in execute_task\r\n    result = _execute_task(task, data)\r\n  File \"dask/async.py\", line 249, in _execute_task\r\n    return func(*args2)\r\n  File \"<ipython-input-1-9a7ad601d93b>\", line 2, in f\r\n    return x / y\r\n```\r\n\r\n## After\r\n\r\n```python\r\nIn [1]: def f(x, y):\r\n    return x / y\r\n   ...: \r\n\r\nIn [2]: from dask.threaded import get\r\n\r\nIn [3]: get({'x': (f, 1, 0)}, 'x')\r\nTraceback:\r\n----------\r\n  File \"dask/async.py\", line 266, in execute_task\r\n    result = _execute_task(task, data)\r\n  File \"dask/async.py\", line 249, in _execute_task\r\n    return func(*args2)\r\n  File \"<ipython-input-1-9a7ad601d93b>\", line 2, in f\r\n    return x / y\r\n\r\n---------------------------------------------------------------------------\r\nZeroDivisionError                         Traceback (most recent call last)\r\n<ipython-input-3-73ec8f9d1e5e> in <module>()\r\n----> 1 get({'x': (f, 1, 0)}, 'x')\r\n\r\n/home/mrocklin/workspace/dask/dask/threaded.pyc in get(dsk, result, cache, num_workers, **kwargs)\r\n     55     results = get_async(pool.apply_async, len(pool._pool), dsk, result,\r\n     56                         cache=cache, queue=queue, get_id=_thread_get_id,\r\n---> 57                         **kwargs)\r\n     58 \r\n     59     return results\r\n\r\n/home/mrocklin/workspace/dask/dask/async.py in get_async(apply_async, num_workers, dsk, result, cache, queue, get_id, raise_on_exception, rerun_exceptions_locally, callbacks, **kwargs)\r\n    492                       \"----------\")\r\n    493                 print(tb)\r\n--> 494                 raise(res)\r\n    495         state['cache'][key] = res\r\n    496         finish_task(dsk, key, state, results, keyorder.get)\r\n\r\nZeroDivisionError: integer division or modulo by zero\r\n```"},"comment":{"url":"https://api.github.com/repos/blaze/dask/issues/comments/142050470","html_url":"https://github.com/blaze/dask/pull/717#issuecomment-142050470","issue_url":"https://api.github.com/repos/blaze/dask/issues/717","id":142050470,"user":{"login":"mrocklin","id":306380,"avatar_url":"https://avatars.githubusercontent.com/u/306380?v=3","gravatar_id":"","url":"https://api.github.com/users/mrocklin","html_url":"https://github.com/mrocklin","followers_url":"https://api.github.com/users/mrocklin/followers","following_url":"https://api.github.com/users/mrocklin/following{/other_user}","gists_url":"https://api.github.com/users/mrocklin/gists{/gist_id}","starred_url":"https://api.github.com/users/mrocklin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrocklin/subscriptions","organizations_url":"https://api.github.com/users/mrocklin/orgs","repos_url":"https://api.github.com/users/mrocklin/repos","events_url":"https://api.github.com/users/mrocklin/events{/privacy}","received_events_url":"https://api.github.com/users/mrocklin/received_events","type":"User","site_admin":false},"created_at":"2015-09-21T17:31:00Z","updated_at":"2015-09-21T17:31:00Z","body":"At first I went with a simple `RemoteException` class that contained the base exception.  Our own test suite didn't like this though, as we often assume that the remote exception type will be the same as what we expect in local behavior.  Given that we assume this I think that others will want to assume this as well.  \r\n\r\nI've pushed a change that uses a meta-class of sorts.  We dynamically create a class that inherits from the base exception (thus satisfying `try-except` blocks) but just contains the base class as an attribute.\r\n\r\nThere is still a problem with this as we don't satisfy the API of the base exception.  I can do `getattr` magic to get past this if we want.  Further thoughts/criticism welcome."}}