{"action":"created","issue":{"url":"https://api.github.com/repos/joblib/joblib/issues/214","labels_url":"https://api.github.com/repos/joblib/joblib/issues/214/labels{/name}","comments_url":"https://api.github.com/repos/joblib/joblib/issues/214/comments","events_url":"https://api.github.com/repos/joblib/joblib/issues/214/events","html_url":"https://github.com/joblib/joblib/pull/214","id":73560817,"number":214,"title":"[MRG] mmap_mode for mixed dtype objects","user":{"login":"ogrisel","id":89061,"avatar_url":"https://avatars.githubusercontent.com/u/89061?v=3","gravatar_id":"","url":"https://api.github.com/users/ogrisel","html_url":"https://github.com/ogrisel","followers_url":"https://api.github.com/users/ogrisel/followers","following_url":"https://api.github.com/users/ogrisel/following{/other_user}","gists_url":"https://api.github.com/users/ogrisel/gists{/gist_id}","starred_url":"https://api.github.com/users/ogrisel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ogrisel/subscriptions","organizations_url":"https://api.github.com/users/ogrisel/orgs","repos_url":"https://api.github.com/users/ogrisel/repos","events_url":"https://api.github.com/users/ogrisel/events{/privacy}","received_events_url":"https://api.github.com/users/ogrisel/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/joblib/joblib/labels/Bug","name":"Bug","color":"e11d21"}],"state":"closed","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/joblib/joblib/milestones/5","html_url":"https://github.com/joblib/joblib/milestones/0.9.0","labels_url":"https://api.github.com/repos/joblib/joblib/milestones/5/labels","id":1098249,"number":5,"title":"0.9.0","description":null,"creator":{"login":"ogrisel","id":89061,"avatar_url":"https://avatars.githubusercontent.com/u/89061?v=3","gravatar_id":"","url":"https://api.github.com/users/ogrisel","html_url":"https://github.com/ogrisel","followers_url":"https://api.github.com/users/ogrisel/followers","following_url":"https://api.github.com/users/ogrisel/following{/other_user}","gists_url":"https://api.github.com/users/ogrisel/gists{/gist_id}","starred_url":"https://api.github.com/users/ogrisel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ogrisel/subscriptions","organizations_url":"https://api.github.com/users/ogrisel/orgs","repos_url":"https://api.github.com/users/ogrisel/repos","events_url":"https://api.github.com/users/ogrisel/events{/privacy}","received_events_url":"https://api.github.com/users/ogrisel/received_events","type":"User","site_admin":false},"open_issues":0,"closed_issues":1,"state":"open","created_at":"2015-05-06T08:46:09Z","updated_at":"2015-05-06T09:30:33Z","due_on":null,"closed_at":null},"comments":3,"created_at":"2015-05-06T08:47:20Z","updated_at":"2015-07-13T12:08:17Z","closed_at":"2015-05-06T09:30:33Z","pull_request":{"url":"https://api.github.com/repos/joblib/joblib/pulls/214","html_url":"https://github.com/joblib/joblib/pull/214","diff_url":"https://github.com/joblib/joblib/pull/214.diff","patch_url":"https://github.com/joblib/joblib/pull/214.patch"},"body":"Here is the kind of error message that users get when trying to load an object that has both memmap-able arrays and object-dtyped arrays in ther internal data-structures:\r\n\r\n```\r\n======================================================================\r\nERROR: joblib.test.test_numpy_pickle.test_memmap_persistence_mixed_dtypes\r\n----------------------------------------------------------------------\r\nTraceback (most recent call last):\r\n  File \"/Users/ogrisel/venvs/py34/lib/python3.4/site-packages/nose/case.py\", line 198, in runTest\r\n    self.test(*self.arg)\r\n  File \"/Users/ogrisel/code/joblib/joblib/test/test_numpy_pickle.py\", line 225, in test_memmap_persistence_mixed_dtypes\r\n    a_clone, b_clone = numpy_pickle.load(filename, mmap_mode='r')\r\n  File \"/Users/ogrisel/code/joblib/joblib/numpy_pickle.py\", line 520, in load\r\n    obj = unpickler.load()\r\n  File \"/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/pickle.py\", line 1038, in load\r\n    dispatch[key[0]](self)\r\n  File \"/Users/ogrisel/code/joblib/joblib/numpy_pickle.py\", line 386, in load_build\r\n    array = nd_array_wrapper.read(self)\r\n  File \"/Users/ogrisel/code/joblib/joblib/numpy_pickle.py\", line 136, in read\r\n    mmap_mode=unpickler.mmap_mode)\r\n  File \"/Users/ogrisel/venvs/py34/lib/python3.4/site-packages/numpy/lib/npyio.py\", line 391, in load\r\n    return format.open_memmap(file, mode=mmap_mode)\r\n  File \"/Users/ogrisel/venvs/py34/lib/python3.4/site-packages/numpy/lib/format.py\", line 725, in open_memmap\r\n    raise ValueError(msg)\r\nValueError: Array can't be memory-mapped: Python objects in dtype.\r\n```\r\n\r\nThis fix in `NDArrayWrapper` makes `joblib.load` avoid tying memmory-mapping inner arrays with `dtype=object`."},"comment":{"url":"https://api.github.com/repos/joblib/joblib/issues/comments/120904787","html_url":"https://github.com/joblib/joblib/pull/214#issuecomment-120904787","issue_url":"https://api.github.com/repos/joblib/joblib/issues/214","id":120904787,"user":{"login":"ogrisel","id":89061,"avatar_url":"https://avatars.githubusercontent.com/u/89061?v=3","gravatar_id":"","url":"https://api.github.com/users/ogrisel","html_url":"https://github.com/ogrisel","followers_url":"https://api.github.com/users/ogrisel/followers","following_url":"https://api.github.com/users/ogrisel/following{/other_user}","gists_url":"https://api.github.com/users/ogrisel/gists{/gist_id}","starred_url":"https://api.github.com/users/ogrisel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ogrisel/subscriptions","organizations_url":"https://api.github.com/users/ogrisel/orgs","repos_url":"https://api.github.com/users/ogrisel/repos","events_url":"https://api.github.com/users/ogrisel/events{/privacy}","received_events_url":"https://api.github.com/users/ogrisel/received_events","type":"User","site_admin":false},"created_at":"2015-07-13T12:08:17Z","updated_at":"2015-07-13T12:08:17Z","body":"> So is this mainly the way how people on git hub fix bugs? \r\n\r\nI cannot speak for others but this is a good way to fix bug:\r\n\r\n- first: write a new test that fails by reproducing the bug\r\n- second: change the code to fix the bug such that your new non-regress test pass while not breaking any other tests.\r\n\r\nPackage the changes as a git branch and submit a pull request to have the changed reviewed by:\r\n\r\n- automated continuous integration bots (e.g. travis-ci.org)\r\n- other project developer / maintainers.\r\n\r\n> Once a bug is reported by a user, you fix it, and write a unit test for testing the bug in unit level?\r\n\r\nIt's even better if the user can fix the bug themselves and open a pull request to contribute the fix back to the project."}}