{"action":"created","issue":{"url":"https://api.github.com/repos/bioidiap/bob.measure/issues/3","labels_url":"https://api.github.com/repos/bioidiap/bob.measure/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/bioidiap/bob.measure/issues/3/comments","events_url":"https://api.github.com/repos/bioidiap/bob.measure/issues/3/events","html_url":"https://github.com/bioidiap/bob.measure/issues/3","id":116800995,"number":3,"title":"eer_threshold doesn't behave as expected","user":{"login":"siebenkopf","id":1398459,"avatar_url":"https://avatars.githubusercontent.com/u/1398459?v=3","gravatar_id":"","url":"https://api.github.com/users/siebenkopf","html_url":"https://github.com/siebenkopf","followers_url":"https://api.github.com/users/siebenkopf/followers","following_url":"https://api.github.com/users/siebenkopf/following{/other_user}","gists_url":"https://api.github.com/users/siebenkopf/gists{/gist_id}","starred_url":"https://api.github.com/users/siebenkopf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/siebenkopf/subscriptions","organizations_url":"https://api.github.com/users/siebenkopf/orgs","repos_url":"https://api.github.com/users/siebenkopf/repos","events_url":"https://api.github.com/users/siebenkopf/events{/privacy}","received_events_url":"https://api.github.com/users/siebenkopf/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/bioidiap/bob.measure/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/bioidiap/bob.measure/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-11-13T16:22:41Z","updated_at":"2015-11-18T16:34:35Z","closed_at":null,"body":"Hi there,\r\n\r\nI have found out that the ``eer_threshold`` function (as well as the ``min_hter_threshold`` function) does not perform as expected in case of highly unbalanced scores. Particularly, when I try to use a special low valued error indicator such as `-10000`, the returned threshold is garbage. Consider the following example:\r\n\r\n```\r\npositives = [1.]*100 + [-10000.]\r\nnegatives = [-1.]*100 + [-10000.]\r\n\r\nthreshold = bob.measure.eer_threshold(negatives, positives)\r\nprint threshold, bob.measure.farfrr(negatives, positives, threshold)\r\nprint 0, bob.measure.farfrr(negatives, positives, 0)\r\n```\r\n\r\nThe outputs of the two print commands are:\r\n\r\n```\r\n-4999.5 (0.9900990099009901, 0.009900990099009901)\r\n0 (0.0, 0.009900990099009901)\r\n```\r\nSo, the second output, where I estimated a threshold ``0`` myself, is much more suitable than the first output, which was generated by ``eer_threshold`` (the ``min_hter_threshold`` behaves exactly the same).\r\n\r\nI would suggest, we should rethink our implementation of the thresholds. I am pretty sure that there exist smarter ways (I had implemented some during my PhD...)\r\n\r\nCheers\r\nManuel"},"comment":{"url":"https://api.github.com/repos/bioidiap/bob.measure/issues/comments/157770137","html_url":"https://github.com/bioidiap/bob.measure/issues/3#issuecomment-157770137","issue_url":"https://api.github.com/repos/bioidiap/bob.measure/issues/3","id":157770137,"user":{"login":"siebenkopf","id":1398459,"avatar_url":"https://avatars.githubusercontent.com/u/1398459?v=3","gravatar_id":"","url":"https://api.github.com/users/siebenkopf","html_url":"https://github.com/siebenkopf","followers_url":"https://api.github.com/users/siebenkopf/followers","following_url":"https://api.github.com/users/siebenkopf/following{/other_user}","gists_url":"https://api.github.com/users/siebenkopf/gists{/gist_id}","starred_url":"https://api.github.com/users/siebenkopf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/siebenkopf/subscriptions","organizations_url":"https://api.github.com/users/siebenkopf/orgs","repos_url":"https://api.github.com/users/siebenkopf/repos","events_url":"https://api.github.com/users/siebenkopf/events{/privacy}","received_events_url":"https://api.github.com/users/siebenkopf/received_events","type":"User","site_admin":false},"created_at":"2015-11-18T16:34:35Z","updated_at":"2015-11-18T16:34:35Z","body":"I have one more concern with the implementation details. So far, I am copying the scores as my method needs sorted scores, and the input ``blitz::Array``'s are ``const``. Do you think it is a good idea to provide a non-const ``blitz::Array`` to the function, so that I can sort the scores in place? This would have an impact on the ``numpy.array``'s on the python side as well...\r\n\r\nOf course, I'll add proper documentation that will clarify this fact..."}}