{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4324","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4324/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4324/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4324/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/4324","id":59569683,"number":4324,"title":"Possible bug in compute_class_weight()","user":{"login":"hannawallach","id":873454,"avatar_url":"https://avatars.githubusercontent.com/u/873454?v=3","gravatar_id":"","url":"https://api.github.com/users/hannawallach","html_url":"https://github.com/hannawallach","followers_url":"https://api.github.com/users/hannawallach/followers","following_url":"https://api.github.com/users/hannawallach/following{/other_user}","gists_url":"https://api.github.com/users/hannawallach/gists{/gist_id}","starred_url":"https://api.github.com/users/hannawallach/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hannawallach/subscriptions","organizations_url":"https://api.github.com/users/hannawallach/orgs","repos_url":"https://api.github.com/users/hannawallach/repos","events_url":"https://api.github.com/users/hannawallach/events{/privacy}","received_events_url":"https://api.github.com/users/hannawallach/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/labels/Bug","name":"Bug","color":"e10c02"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-03-02T23:31:42Z","updated_at":"2015-03-03T01:05:41Z","closed_at":null,"body":"Hi, \r\n\r\nI think there might be a bug in the compute_class_weight() method in https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/utils/class_weight.py. Lines 50 and 51 are as follows:\r\n\r\n        recip_freq = 1. / bincount(y_ind)\r\n        weight = recip_freq[le.transform(classes)] / np.mean(recip_freq)\r\n\r\nFor a two class problem, where there are N0 data points belonging to class 0 and N1 data points belonging to class 1, this is equivalent to saying weight[0] = w0 =  (1 / N0) / (0.5 * (1 / N0 + 1 / N1)) = 2 * N1 / (N0 + N1) and weight[1] = w1 = (1 / N1) / (0.5 * (1 / N0 + 1 / N1)). \r\n\r\nThese expressions for w0 and w1 do not correspond with my intuition or with the expressions given in <a href=\"http://gking.harvard.edu/files/0s.pdf\">this paper</a> by King and Zeng.\r\n\r\nIntuitively, assuming that I am trying to \"balance\" my classes using my class weights, I would expect my class weights (w0 and w1, respectively) to satisfy w0 * N0 + w1 * N1 = N0 + N1 and w0 * N0 = w1 * N1. Solving for w0 gives (N0 + N1) / (2 * N0) while solving for w1 gives (N0 + N1) / (2 * N1).\r\n\r\nThis intuition is backed up by King and Zeng's paper, which says (on pages 144--145) that if tau is the fraction of ones in the population (i.e., 0.5 for a balanced population, which is what we're assuming here) and y is the fraction of ones in the sample (i.e., N1 / (N0 + N1)) then w1 = tau / y = 0.5 / (N1 / (N0 + N1)) = (N0 + N1) / (2 * N1) and similarly w0 = (N0 + N1) / (2 * N0).\r\n\r\nIs this what you were intending to compute in lines 50 and 51? If so, then I think there is a bug."},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/76866311","html_url":"https://github.com/scikit-learn/scikit-learn/issues/4324#issuecomment-76866311","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4324","id":76866311,"user":{"login":"hannawallach","id":873454,"avatar_url":"https://avatars.githubusercontent.com/u/873454?v=3","gravatar_id":"","url":"https://api.github.com/users/hannawallach","html_url":"https://github.com/hannawallach","followers_url":"https://api.github.com/users/hannawallach/followers","following_url":"https://api.github.com/users/hannawallach/following{/other_user}","gists_url":"https://api.github.com/users/hannawallach/gists{/gist_id}","starred_url":"https://api.github.com/users/hannawallach/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hannawallach/subscriptions","organizations_url":"https://api.github.com/users/hannawallach/orgs","repos_url":"https://api.github.com/users/hannawallach/repos","events_url":"https://api.github.com/users/hannawallach/events{/privacy}","received_events_url":"https://api.github.com/users/hannawallach/received_events","type":"User","site_admin":false},"created_at":"2015-03-03T01:05:41Z","updated_at":"2015-03-03T01:05:41Z","body":"Yeah -- agreed -- I'd definitely like to know the original rationale. \r\n\r\n(Also I caught another typo in my definition of what the implemented version yields for the 3 class case. I believe it should be w0 = (1 / N0) / (1/3 * (1 / N0 + 1 / N1 + 1 / N2)) = (3 * N1 * N2) / (N1 * N2 + N0 * N2 + N0 * N1). Sorry.)\r\n\r\nI'll see if I can find out what VW does, either by digging into the code or asking JL."}}