{"action":"closed","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4144","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4144/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4144/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4144/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/4144","id":55137938,"number":4144,"title":"problem with sample_weight in SVC...","user":{"login":"SylvainTakerkart","id":7886280,"avatar_url":"https://avatars.githubusercontent.com/u/7886280?v=3","gravatar_id":"","url":"https://api.github.com/users/SylvainTakerkart","html_url":"https://github.com/SylvainTakerkart","followers_url":"https://api.github.com/users/SylvainTakerkart/followers","following_url":"https://api.github.com/users/SylvainTakerkart/following{/other_user}","gists_url":"https://api.github.com/users/SylvainTakerkart/gists{/gist_id}","starred_url":"https://api.github.com/users/SylvainTakerkart/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SylvainTakerkart/subscriptions","organizations_url":"https://api.github.com/users/SylvainTakerkart/orgs","repos_url":"https://api.github.com/users/SylvainTakerkart/repos","events_url":"https://api.github.com/users/SylvainTakerkart/events{/privacy}","received_events_url":"https://api.github.com/users/SylvainTakerkart/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/labels/Bug","name":"Bug","color":"e10c02"}],"state":"closed","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/milestones/12","html_url":"https://github.com/scikit-learn/scikit-learn/milestones/0.16","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/milestones/12/labels","id":527004,"number":12,"title":"0.16","description":"","creator":{"login":"GaelVaroquaux","id":208217,"avatar_url":"https://avatars.githubusercontent.com/u/208217?v=3","gravatar_id":"","url":"https://api.github.com/users/GaelVaroquaux","html_url":"https://github.com/GaelVaroquaux","followers_url":"https://api.github.com/users/GaelVaroquaux/followers","following_url":"https://api.github.com/users/GaelVaroquaux/following{/other_user}","gists_url":"https://api.github.com/users/GaelVaroquaux/gists{/gist_id}","starred_url":"https://api.github.com/users/GaelVaroquaux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GaelVaroquaux/subscriptions","organizations_url":"https://api.github.com/users/GaelVaroquaux/orgs","repos_url":"https://api.github.com/users/GaelVaroquaux/repos","events_url":"https://api.github.com/users/GaelVaroquaux/events{/privacy}","received_events_url":"https://api.github.com/users/GaelVaroquaux/received_events","type":"User","site_admin":false},"open_issues":43,"closed_issues":108,"state":"open","created_at":"2014-01-05T21:19:20Z","updated_at":"2015-03-10T21:43:45Z","due_on":"2015-03-15T04:00:00Z","closed_at":null},"comments":5,"created_at":"2015-01-22T10:38:20Z","updated_at":"2015-03-10T21:43:45Z","closed_at":"2015-03-10T21:43:45Z","body":"Hello, following Andy's advice(http://sourceforge.net/p/scikit-learn/mailman/message/33251002/), I'm opening this issue... Here is the description of the problem:\r\n\r\nI'm trying to use the instance weighting capability of the SVC class, and I\r\nencountered some weird behavior: when the C parameter is chosen very small\r\nand the weights are not very large, weighted SVM yields constant\r\nprediction...\r\n\r\nBelow, you'll find a  piece of code that demonstrates this. What it does:\r\n\r\n1. create a dummy dataset for binary classification, with a training and a\r\ntesting dataset\r\n2. create random weights that are very very very close to one (so close\r\nthat they should not influence what follows)\r\n3. run unweighted and weighted SVM (hereafter SVM and wSVM) with decreasing\r\nvalues of C\r\n\r\nWe expect SVM and wSVM to yield exactly the same predictions because the\r\ninstance weights are ridiculously close to one. This is exactly what\r\nhappens for large to small-ish values of C; but at some point when C gets\r\nsmaller, wSVM yields constant predictions (either all zeros, or all ones)\r\nwhile SVM still behaves normally...\r\n\r\nSylvain\r\n\r\n```python\r\nimport numpy as np\r\nfrom numpy.random import multivariate_normal\r\n\r\nfrom sklearn.svm import SVC\r\nfrom sklearn.metrics import accuracy_score\r\n\r\n\r\n# create dummy training and testing dataset...\r\nn_samples_per_class = 100\r\n\r\nX_train_plus = multivariate_normal([0,2],np.identity(2),n_samples_per_class)\r\nX_train_minus = multivariate_normal([0,3],np.identity(2),n_samples_per_class)\r\nX_test_plus = multivariate_normal([0,2],np.identity(2),n_samples_per_class)\r\nX_test_minus = multivariate_normal([0,3],np.identity(2),n_samples_per_class)\r\n\r\nX_train = np.vstack([X_train_plus, X_train_minus])\r\nX_test = np.vstack([X_test_plus, X_test_minus])\r\n\r\n# create labels\r\ny_train = np.hstack([np.zeros(n_samples_per_class), np.ones(n_samples_per_class)])\r\ny_test = np.hstack([np.zeros(n_samples_per_class), np.ones(n_samples_per_class)])\r\n\r\nn_train_samples = len(y_train)\r\n\r\n# the list of decreasing C values\r\nC_list = 10. ** np.arange(2,-5,-1)\r\n\r\n# create random sample weights (very very very close to one)\r\nmu = 1.\r\nsigma = 0.001\r\nsample_weights = np.random.normal(mu,sigma,n_train_samples)\r\n\r\nfor C in C_list:\r\n    # run SVM\r\n    svc = SVC(C=C)\r\n    svc.fit(X_train,y_train)\r\n    y_svc = svc.predict(X_test)\r\n    svc_acc = accuracy_score(y_svc,y_test)\r\n\r\n    # run weighted SVM\r\n    wsvc = SVC(C=C)\r\n    wsvc.fit(X_train,y_train,sample_weight=sample_weights)\r\n    y_wsvc = wsvc.predict(X_test)\r\n\r\n    wsvc_acc = accuracy_score(y_wsvc,y_test)\r\n    print 'Accuracies of SVM and weighted-SVM for C = %f: %.2f %.2f' % (C,svc_acc,wsvc_acc)\r\n    print 'Predictions for weighted-SVM:'\r\n    print y_wsvc\r\n    print\r\n```\r\n\r\n\r\n"}}