{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/5233","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/5233/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/5233/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/5233/events","html_url":"https://github.com/scikit-learn/scikit-learn/pull/5233","id":105609300,"number":5233,"title":"[MRG] Proxy improvement methods added to entropy/gini","user":{"login":"jmschrei","id":3916816,"avatar_url":"https://avatars.githubusercontent.com/u/3916816?v=3","gravatar_id":"","url":"https://api.github.com/users/jmschrei","html_url":"https://github.com/jmschrei","followers_url":"https://api.github.com/users/jmschrei/followers","following_url":"https://api.github.com/users/jmschrei/following{/other_user}","gists_url":"https://api.github.com/users/jmschrei/gists{/gist_id}","starred_url":"https://api.github.com/users/jmschrei/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jmschrei/subscriptions","organizations_url":"https://api.github.com/users/jmschrei/orgs","repos_url":"https://api.github.com/users/jmschrei/repos","events_url":"https://api.github.com/users/jmschrei/events{/privacy}","received_events_url":"https://api.github.com/users/jmschrei/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-09-09T14:33:30Z","updated_at":"2015-09-10T13:37:36Z","closed_at":"2015-09-10T13:22:29Z","pull_request":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/pulls/5233","html_url":"https://github.com/scikit-learn/scikit-learn/pull/5233","diff_url":"https://github.com/scikit-learn/scikit-learn/pull/5233.diff","patch_url":"https://github.com/scikit-learn/scikit-learn/pull/5233.patch"},"body":"This pull request handles implementing explicit proxy improvement methods for entropy and gini criterion, to compliment the one added in #5203. Entropy is changed by refactoring the equation\r\n\r\n```-weighted_n_left / weighted_n_total * -sum( count_left_i / weighted_n_left * log( count_left_i / weighted_n_left ) - weighted_n_right / weighted_n_total * -sum( count_i / weighted_n_right * log(count_right_i / weighted_n_right )```\r\n\r\ninto \r\n\r\n```sum( count_left_i * log( count_left_i ) ) + sum( count_right_i * log( count_right_i ) ) - weighted_n_left * log(weighted_n_left) - weighted_n_right * log(weighted_n_right)```\r\n\r\nwhich collapses some terms, and caches the calculation involving `weighted_n_left` and `weighted_n_right`. \r\n\r\nThe gini calculation time is collapsed into \r\n\r\n```\r\nsum( count_left_i ** 2.0 ) / weighted_n_left + sum( count_right_i ** 2.0 ) / weighted_n_right\r\n```\r\n\r\nHere are time tests. Entropy seems to take basically the same amount of time, whereas gini seems to run slightly faster. I was unsure if this was worth merging, so figured I'd see what you had to say.\r\n\r\n@glouppe @arjoly @ogrisel \r\n\r\n```\r\nENTROPY\r\nBRANCH\r\nRandomForestClassifier\r\nspambase   0.94 0.081\r\nGaussian   0.922 7.972\r\nmnist      0.946 9.669\r\ncovtypes   0.948 19.574\r\n\r\nExtraTreesClassifier\r\nspambase   0.943 0.047\r\nGaussian   0.924 0.846\r\nmnist      0.952 4.678\r\ncovtypes   0.943 12.225\r\n\r\nDecisionTreeClassifier\r\nspambase   0.911 0.065\r\nGaussian   0.739 17.631\r\nmnist      0.887 30.058\r\ncovtypes   0.948 11.025\r\n\r\nMASTER\r\nRandomForestClassifier\r\nspambase   0.94 0.081\r\nGaussian   0.922 8.333\r\nmnist      0.946 9.324\r\ncovtypes   0.948 17.701\r\n\r\nExtraTreesClassifier\r\nspambase   0.943 0.047\r\nGaussian   0.924 0.79\r\nmnist      0.952 4.542\r\ncovtypes   0.943 11.81\r\n\r\nDecisionTreeClassifier\r\nspambase   0.911 0.06\r\nGaussian   0.739 16.75\r\nmnist      0.887 29.83\r\ncovtypes   0.948 10.561\r\n\r\n\r\n\r\nGINI\r\nBRANCH\r\nRandomForestClassifier\r\nGaussian   0.914 3.931\r\nspambase   0.941 0.045\r\nmnist      0.948 5.016\r\ncovtypes   0.944 13.238\r\n\r\nExtraTreesClassifier\r\nGaussian   0.918 0.652\r\nspambase   0.946 0.041\r\nmnist      0.947 4.641\r\ncovtypes   0.942 11.275\r\n\r\nDecisionTreeClassifier\r\nGaussian   0.727 8.971\r\nspambase   0.902 0.056\r\nmnist      0.873 21.287\r\ncovtypes   0.944 8.989\r\n\r\nMASTER\r\nRandomForestClassifier\r\nGaussian   0.914 4.513\r\nspambase   0.939 0.052\r\nmnist      0.949 5.27\r\ncovtypes   0.944 13.767\r\n\r\nExtraTreesClassifier\r\nGaussian   0.917 0.679\r\nspambase   0.947 0.042\r\nmnist      0.952 5.131\r\ncovtypes   0.94 11.591\r\n\r\nDecisionTreeClassifier\r\nGaussian   0.741 10.02\r\nspambase   0.898 0.052\r\nmnist      0.872 24.078\r\ncovtypes   0.943 9.836\r\n\r\n\r\n```"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/139236645","html_url":"https://github.com/scikit-learn/scikit-learn/pull/5233#issuecomment-139236645","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/5233","id":139236645,"user":{"login":"arjoly","id":1274722,"avatar_url":"https://avatars.githubusercontent.com/u/1274722?v=3","gravatar_id":"","url":"https://api.github.com/users/arjoly","html_url":"https://github.com/arjoly","followers_url":"https://api.github.com/users/arjoly/followers","following_url":"https://api.github.com/users/arjoly/following{/other_user}","gists_url":"https://api.github.com/users/arjoly/gists{/gist_id}","starred_url":"https://api.github.com/users/arjoly/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arjoly/subscriptions","organizations_url":"https://api.github.com/users/arjoly/orgs","repos_url":"https://api.github.com/users/arjoly/repos","events_url":"https://api.github.com/users/arjoly/events{/privacy}","received_events_url":"https://api.github.com/users/arjoly/received_events","type":"User","site_admin":false},"created_at":"2015-09-10T13:37:36Z","updated_at":"2015-09-10T13:37:36Z","body":"Thanks for your hard work !"}}