{"action":"created","issue":{"url":"https://api.github.com/repos/jxnl/pd2report/issues/3","labels_url":"https://api.github.com/repos/jxnl/pd2report/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/jxnl/pd2report/issues/3/comments","events_url":"https://api.github.com/repos/jxnl/pd2report/issues/3/events","html_url":"https://github.com/jxnl/pd2report/issues/3","id":102499830,"number":3,"title":"Analysis Topics","user":{"login":"jxnl","id":4852235,"avatar_url":"https://avatars.githubusercontent.com/u/4852235?v=3","gravatar_id":"","url":"https://api.github.com/users/jxnl","html_url":"https://github.com/jxnl","followers_url":"https://api.github.com/users/jxnl/followers","following_url":"https://api.github.com/users/jxnl/following{/other_user}","gists_url":"https://api.github.com/users/jxnl/gists{/gist_id}","starred_url":"https://api.github.com/users/jxnl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jxnl/subscriptions","organizations_url":"https://api.github.com/users/jxnl/orgs","repos_url":"https://api.github.com/users/jxnl/repos","events_url":"https://api.github.com/users/jxnl/events{/privacy}","received_events_url":"https://api.github.com/users/jxnl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"jxnl","id":4852235,"avatar_url":"https://avatars.githubusercontent.com/u/4852235?v=3","gravatar_id":"","url":"https://api.github.com/users/jxnl","html_url":"https://github.com/jxnl","followers_url":"https://api.github.com/users/jxnl/followers","following_url":"https://api.github.com/users/jxnl/following{/other_user}","gists_url":"https://api.github.com/users/jxnl/gists{/gist_id}","starred_url":"https://api.github.com/users/jxnl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jxnl/subscriptions","organizations_url":"https://api.github.com/users/jxnl/orgs","repos_url":"https://api.github.com/users/jxnl/repos","events_url":"https://api.github.com/users/jxnl/events{/privacy}","received_events_url":"https://api.github.com/users/jxnl/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/jxnl/pd2report/milestones/1","html_url":"https://github.com/jxnl/pd2report/milestones/Draft","labels_url":"https://api.github.com/repos/jxnl/pd2report/milestones/1/labels","id":1267189,"number":1,"title":"Draft","description":"Completed Draft for review by supervisor\r\n","creator":{"login":"jxnl","id":4852235,"avatar_url":"https://avatars.githubusercontent.com/u/4852235?v=3","gravatar_id":"","url":"https://api.github.com/users/jxnl","html_url":"https://github.com/jxnl","followers_url":"https://api.github.com/users/jxnl/followers","following_url":"https://api.github.com/users/jxnl/following{/other_user}","gists_url":"https://api.github.com/users/jxnl/gists{/gist_id}","starred_url":"https://api.github.com/users/jxnl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jxnl/subscriptions","organizations_url":"https://api.github.com/users/jxnl/orgs","repos_url":"https://api.github.com/users/jxnl/repos","events_url":"https://api.github.com/users/jxnl/events{/privacy}","received_events_url":"https://api.github.com/users/jxnl/received_events","type":"User","site_admin":false},"open_issues":3,"closed_issues":0,"state":"open","created_at":"2015-08-22T03:20:54Z","updated_at":"2015-08-22T03:28:01Z","due_on":"2015-08-24T04:00:00Z","closed_at":null},"comments":2,"created_at":"2015-08-22T03:27:50Z","updated_at":"2015-08-23T19:56:28Z","closed_at":null,"body":"# Exploration\r\n\r\n#####  popular language with great tools\r\n\r\n* One obvious reason to use python as the base language is that its popular in the open source world\r\n * Generally this means that a lot of packages have been written and open source.\r\n   * for example: [awesome python](https://github.com/vinta/awesome-python)\r\n * not only are they open but a larger community means more people who you hire will know python\r\n* lot of support exists for the tools and language in general\r\n* pretty important for technical debt is that if you have multiple cycling in and out of the project, python can be a common thread.\r\n\r\n##### ipython notebooks\r\n\r\n* One major contribution for using python for academica is the ipython notebook, something that should not be overlooked\r\n* [IPython Notebook](http://ipython.org/notebook.html)\r\n* By providing a \"workspace\" for developing code, ipython notebooks support reproducible research by having one-to-one input outputs than can be run with any machine that has the correct packages.\r\n* Its support for markdown and LaTeX also allow is to document and explain what code is doing in a more intuitive way.\r\n* notebooks have now become the base tool for teaching in the python communities.\r\n * So by using notebooks to do the most of your pipeline design, it can be easily passed on as examples for later lab members.\r\n * [Mining the Social Web](http://shop.oreilly.com/product/0636920030195.do?cmp=af-strata-books-video-product_cj_0636920030195_7133220) has done a great job of presenting the entire education book as a series of notebooks like [this](http://nbviewer.ipython.org/github/ptwobrussell/Mining-the-Social-Web-2nd-Edition/blob/master/ipynb/Chapter%201%20-%20Mining%20Twitter.ipynb)\r\n\r\n##### Data Visualization super easy. \r\n* Before jumping into building models its important to explore the data in front of you\r\n\r\n<img width=\"1126\" alt=\"screen shot 2015-08-22 at 5 12 08 pm\" src=\"https://cloud.githubusercontent.com/assets/4852235/9425891/f63fd45c-48f0-11e5-80b1-fc311340f633.png\">\r\n\r\n *  Exploratory data analysis was promoted by John Tukey to encourage statisticians to explore the data, and possibly formulate hypotheses that could lead to new data collection and experiments.\r\n\r\n* With the presence of ipython notebooks, tools like pandas and seaborn give us a whole range of transformations and visualizations to play with right out of the box. \r\n\r\n<img width=\"862\" alt=\"screen shot 2015-08-22 at 5 10 57 pm\" src=\"https://cloud.githubusercontent.com/assets/4852235/9425886/cb4de1ee-48f0-11e5-87c0-08d47337cfde.png\">\r\n\r\n* These visualizations are incredibly useful for understanding the relationships between our data and allow us ti generate new ideas on how to design our models.\r\n  * Python allows us to do this effortlessly\r\n\r\n##### Issues\r\n\r\nIt is important to note that with the split between python 2.7 and 3.4 various aspects of the python ecosystem still sit in 2.7. Its recommended that future work done in 3.4 and developed in virtual environments in order to maintain compatibility. However, luckily a majority of the packages we use for pipelineing have all been ported to 3.4"},"comment":{"url":"https://api.github.com/repos/jxnl/pd2report/issues/comments/133910809","html_url":"https://github.com/jxnl/pd2report/issues/3#issuecomment-133910809","issue_url":"https://api.github.com/repos/jxnl/pd2report/issues/3","id":133910809,"user":{"login":"jxnl","id":4852235,"avatar_url":"https://avatars.githubusercontent.com/u/4852235?v=3","gravatar_id":"","url":"https://api.github.com/users/jxnl","html_url":"https://github.com/jxnl","followers_url":"https://api.github.com/users/jxnl/followers","following_url":"https://api.github.com/users/jxnl/following{/other_user}","gists_url":"https://api.github.com/users/jxnl/gists{/gist_id}","starred_url":"https://api.github.com/users/jxnl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jxnl/subscriptions","organizations_url":"https://api.github.com/users/jxnl/orgs","repos_url":"https://api.github.com/users/jxnl/repos","events_url":"https://api.github.com/users/jxnl/events{/privacy}","received_events_url":"https://api.github.com/users/jxnl/received_events","type":"User","site_admin":false},"created_at":"2015-08-23T19:56:28Z","updated_at":"2015-08-23T19:56:28Z","body":"# Model Evaluation\r\n\r\n##### Performance Metrics\r\n* Once we've built various models we want to find the best ones\r\n * sklearn.metrics contains a tonne of different ways to measure performance.\r\n * for example, consider the problem of finding cancer. in 200 data points, you'll get once case of cancer. if our algorithm was simply \"say everyone does not have cancer\" then on average you'd have an accuracy of 99.5!, so surely other methods must exist that are more robust.\r\n * at work we often use f1_score, which is a combination of precison and recall.\r\n * instead of looking at just what we got right and wrong, we also take into consideration the number of false positives and false negatives.\r\n* Moreso, there are easy tools to visualize different properties like confusion matrices, or auc\r\n![roc](http://scikit-learn.org/stable/_images/plot_roc_001.png)\r\n* We also want to know how much better our models would be by comparing train vs test accuracy.\r\n\r\n\r\n\r\n### model evaluation\r\n\r\n### cons"}}