{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/10199","labels_url":"https://api.github.com/repos/pydata/pandas/issues/10199/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/10199/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/10199/events","html_url":"https://github.com/pydata/pandas/pull/10199","id":79594577,"number":10199,"title":"PERF: releasing the GIL, #8882","user":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/Performance","name":"Performance","color":"a10c02"}],"state":"closed","locked":false,"assignee":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/pydata/pandas/milestones/34","html_url":"https://github.com/pydata/pandas/milestones/0.17.0","labels_url":"https://api.github.com/repos/pydata/pandas/milestones/34/labels","id":1033710,"number":34,"title":"0.17.0","description":"","creator":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":175,"closed_issues":59,"state":"open","created_at":"2015-03-23T10:47:38Z","updated_at":"2015-06-30T10:55:04Z","due_on":"2015-07-31T04:00:00Z","closed_at":null},"comments":20,"created_at":"2015-05-22T20:19:47Z","updated_at":"2015-06-30T14:36:49Z","closed_at":"2015-06-30T10:55:04Z","pull_request":{"url":"https://api.github.com/repos/pydata/pandas/pulls/10199","html_url":"https://github.com/pydata/pandas/pull/10199","diff_url":"https://github.com/pydata/pandas/pull/10199.diff","patch_url":"https://github.com/pydata/pandas/pull/10199.patch"},"body":"closes #8882\r\ncloses #10139 \r\n \r\nThis is now implemented for all groupbys with the factorization part as well (which is actually a big part fo the time)\r\n\r\n```\r\nIn [2]: N = 1000000\r\n\r\nIn [3]: ngroups = 1000\r\n\r\nIn [4]: np.random.seed(1234)\r\n\r\nIn [5]: df = DataFrame({'key' : np.random.randint(0,ngroups,size=N),\r\n   ...:                 'data' : np.random.randn(N) })\r\n```\r\n\r\n```\r\ndef f():\r\n    df.groupby('key')['data'].sum()\r\n\r\n# run consecutivily\r\ndef g2():\r\n    for i in range(2):\r\n        f()\r\ndef g4():\r\n    for i in range(4):\r\n        f()\r\ndef g8():\r\n    for i in range(8):\r\n        f()\r\n\r\n# run in parallel\r\n@test_parallel(num_threads=2)\r\ndef pg2():\r\n    f()\r\n\r\n@test_parallel(num_threads=4)\r\ndef pg4():\r\n    f()\r\n\r\n@test_parallel(num_threads=8)\r\ndef pg8():\r\n    f()\r\n```\r\n\r\nSo seeing a nice curve as far as addtl cores are used (as compared to some of my previous posts).\r\n```\r\nIn [19]: df\r\nOut[19]: \r\n   after  before   speedup\r\n2   27.3    52.1  1.908425\r\n4   44.8   105.0  2.343750\r\n8   83.2   213.0  2.560096\r\n```"},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/117210135","html_url":"https://github.com/pydata/pandas/pull/10199#issuecomment-117210135","issue_url":"https://api.github.com/repos/pydata/pandas/issues/10199","id":117210135,"user":{"login":"jorisvandenbossche","id":1020496,"avatar_url":"https://avatars.githubusercontent.com/u/1020496?v=3","gravatar_id":"","url":"https://api.github.com/users/jorisvandenbossche","html_url":"https://github.com/jorisvandenbossche","followers_url":"https://api.github.com/users/jorisvandenbossche/followers","following_url":"https://api.github.com/users/jorisvandenbossche/following{/other_user}","gists_url":"https://api.github.com/users/jorisvandenbossche/gists{/gist_id}","starred_url":"https://api.github.com/users/jorisvandenbossche/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jorisvandenbossche/subscriptions","organizations_url":"https://api.github.com/users/jorisvandenbossche/orgs","repos_url":"https://api.github.com/users/jorisvandenbossche/repos","events_url":"https://api.github.com/users/jorisvandenbossche/events{/privacy}","received_events_url":"https://api.github.com/users/jorisvandenbossche/received_events","type":"User","site_admin":false},"created_at":"2015-06-30T14:36:48Z","updated_at":"2015-06-30T14:36:48Z","body":"@mrocklin Can we already look forward to a new awesome blogpost? :-)"}}