{"action":"created","issue":{"url":"https://api.github.com/repos/NVIDIA/DIGITS/issues/137","labels_url":"https://api.github.com/repos/NVIDIA/DIGITS/issues/137/labels{/name}","comments_url":"https://api.github.com/repos/NVIDIA/DIGITS/issues/137/comments","events_url":"https://api.github.com/repos/NVIDIA/DIGITS/issues/137/events","html_url":"https://github.com/NVIDIA/DIGITS/pull/137","id":86288822,"number":137,"title":"Accuracy & confusion matrix","user":{"login":"groar","id":5939621,"avatar_url":"https://avatars.githubusercontent.com/u/5939621?v=3","gravatar_id":"","url":"https://api.github.com/users/groar","html_url":"https://github.com/groar","followers_url":"https://api.github.com/users/groar/followers","following_url":"https://api.github.com/users/groar/following{/other_user}","gists_url":"https://api.github.com/users/groar/gists{/gist_id}","starred_url":"https://api.github.com/users/groar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/groar/subscriptions","organizations_url":"https://api.github.com/users/groar/orgs","repos_url":"https://api.github.com/users/groar/repos","events_url":"https://api.github.com/users/groar/events{/privacy}","received_events_url":"https://api.github.com/users/groar/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":17,"created_at":"2015-06-08T18:48:00Z","updated_at":"2015-06-16T17:12:41Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/NVIDIA/DIGITS/pulls/137","html_url":"https://github.com/NVIDIA/DIGITS/pull/137","diff_url":"https://github.com/NVIDIA/DIGITS/pull/137.diff","patch_url":"https://github.com/NVIDIA/DIGITS/pull/137.patch"},"body":"See #17 \r\n\r\nAdds a new kind of job for performance evaluation of trained classifiers. It is now possible to visualize :\r\n* accuracy / recall curve\r\n* confusion matrix\r\n\r\nAccuracy and the confusion matrix are computed against a chosen snapshot of a training task, and against both the **validation set and testing set** (if it exists). An \"evaluate performance\" button has been added on the training view. This is currently the only way to run an evaluation job. The results are stored in the job directory in the form of two pickle files. \r\n\r\n![button](https://cloud.githubusercontent.com/assets/5939621/8041323/b2f7882a-0e17-11e5-9b3d-02ffe8471194.png)\r\n\r\n## Accuracy / recall curve\r\n\r\n![accuracy recall curve](https://cloud.githubusercontent.com/assets/5939621/8042256/58160fe2-0e1e-11e5-93a2-6cb6263ccdf5.png)\r\n\r\n\r\n## Confusion matrix\r\n\r\nI chose a very simple representation of the confusion matrix (not in the form of a matrix !), because it is more adapted to datasets with lots of classes. For each class, the top 10 most represented classes are displayed, with their respective %.   \r\n\r\n![confusion matrix](https://cloud.githubusercontent.com/assets/5939621/8042352/09d9d0b0-0e1f-11e5-825d-69fb981435cd.png)\r\n\r\n## Related jobs\r\n\r\nI added a \"Related jobs\" section on each job show view. It displays the jobs which depends on the current job. For example, models trained on a specific dataset, evaluations ran on a specific model.\r\n\r\n![Related jobs](https://cloud.githubusercontent.com/assets/5939621/8182736/8e24a58e-1432-11e5-9808-6acd7e63bb80.png)\r\n\r\n\r\n\r\nLet me know what you think, critiques and comments are more than welcome. "},"comment":{"url":"https://api.github.com/repos/NVIDIA/DIGITS/issues/comments/112500011","html_url":"https://github.com/NVIDIA/DIGITS/pull/137#issuecomment-112500011","issue_url":"https://api.github.com/repos/NVIDIA/DIGITS/issues/137","id":112500011,"user":{"login":"groar","id":5939621,"avatar_url":"https://avatars.githubusercontent.com/u/5939621?v=3","gravatar_id":"","url":"https://api.github.com/users/groar","html_url":"https://github.com/groar","followers_url":"https://api.github.com/users/groar/followers","following_url":"https://api.github.com/users/groar/following{/other_user}","gists_url":"https://api.github.com/users/groar/gists{/gist_id}","starred_url":"https://api.github.com/users/groar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/groar/subscriptions","organizations_url":"https://api.github.com/users/groar/orgs","repos_url":"https://api.github.com/users/groar/repos","events_url":"https://api.github.com/users/groar/events{/privacy}","received_events_url":"https://api.github.com/users/groar/received_events","type":"User","site_admin":false},"created_at":"2015-06-16T17:12:41Z","updated_at":"2015-06-16T17:12:41Z","body":"Maybe I just need to merge master into perf to remove conflicts ?  "}}