{"action":"created","issue":{"url":"https://api.github.com/repos/Cal-CS-61A-Staff/ok-client/issues/135","labels_url":"https://api.github.com/repos/Cal-CS-61A-Staff/ok-client/issues/135/labels{/name}","comments_url":"https://api.github.com/repos/Cal-CS-61A-Staff/ok-client/issues/135/comments","events_url":"https://api.github.com/repos/Cal-CS-61A-Staff/ok-client/issues/135/events","html_url":"https://github.com/Cal-CS-61A-Staff/ok-client/issues/135","id":117262177,"number":135,"title":"Support for IPython notebooks","user":{"login":"SamLau95","id":2468904,"avatar_url":"https://avatars.githubusercontent.com/u/2468904?v=3","gravatar_id":"","url":"https://api.github.com/users/SamLau95","html_url":"https://github.com/SamLau95","followers_url":"https://api.github.com/users/SamLau95/followers","following_url":"https://api.github.com/users/SamLau95/following{/other_user}","gists_url":"https://api.github.com/users/SamLau95/gists{/gist_id}","starred_url":"https://api.github.com/users/SamLau95/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SamLau95/subscriptions","organizations_url":"https://api.github.com/users/SamLau95/orgs","repos_url":"https://api.github.com/users/SamLau95/repos","events_url":"https://api.github.com/users/SamLau95/events{/privacy}","received_events_url":"https://api.github.com/users/SamLau95/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-11-17T01:48:55Z","updated_at":"2015-11-24T02:22:15Z","closed_at":null,"body":"We're using the ok client for all programming assignments for\r\n[DS8](http://data8.org/) and it's quite a challenge to work with.\r\n\r\nOur current workflow when a student clicks the \"Run ok tests\" button in the\r\nJupyterHub interface is:\r\n\r\n1. Export the IPython notebook as a `.py` file.\r\n2. Remove the `get_ipython` lines in the resulting file.\r\n3. Find all the cells that start with `# ok` and run them. The contents of these\r\n   cells call the `ok` client to run the corresponding tests.\r\n\r\nThis is less than ideal for the following reasons:\r\n\r\n- The `ok` cells in the notebook can't actually be run directly since the\r\n  notebook won't be exported. More confusingly, the `ok` script will be run on\r\n  old code from the last time the code was exported. This means that the ok\r\n  tests can't be run locally if the ok-extension isn't installed.\r\n- The `ok` script needs to run the entire notebook before being able to run\r\n  tests which is often slow (conceptually, each `ok` cell is testing one filled-\r\n  in student cell). This often results in the case where a student is partway\r\n  finished with a notebook and runs the `ok` tests only to have them fail\r\n  because of a later cell.\r\n- As a corollary, the `ok` script only reads in the final values of student\r\n  variables. This is not a huge issue but it causes confusion.\r\n\r\nI think these issues can be resolved if the `ok` client is able to look at cells\r\nin the notebook specifically. I'm not sure how involved this is, so any feedback\r\nwould be appreciated."},"comment":{"url":"https://api.github.com/repos/Cal-CS-61A-Staff/ok-client/issues/comments/159132807","html_url":"https://github.com/Cal-CS-61A-Staff/ok-client/issues/135#issuecomment-159132807","issue_url":"https://api.github.com/repos/Cal-CS-61A-Staff/ok-client/issues/135","id":159132807,"user":{"login":"albert12132","id":2187524,"avatar_url":"https://avatars.githubusercontent.com/u/2187524?v=3","gravatar_id":"","url":"https://api.github.com/users/albert12132","html_url":"https://github.com/albert12132","followers_url":"https://api.github.com/users/albert12132/followers","following_url":"https://api.github.com/users/albert12132/following{/other_user}","gists_url":"https://api.github.com/users/albert12132/gists{/gist_id}","starred_url":"https://api.github.com/users/albert12132/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/albert12132/subscriptions","organizations_url":"https://api.github.com/users/albert12132/orgs","repos_url":"https://api.github.com/users/albert12132/repos","events_url":"https://api.github.com/users/albert12132/events{/privacy}","received_events_url":"https://api.github.com/users/albert12132/received_events","type":"User","site_admin":false},"created_at":"2015-11-24T02:22:15Z","updated_at":"2015-11-24T02:22:15Z","body":"Where is the test 'q1' coming from (e.g. a file)? In this example, is the desired behavior supposed to reset the test's state (except for the data loaded in x) every time ok.test is called? One workaround is to have a separate python module that loads your data, then import that python module in the OK tests (this is kind of hacky because it relies on the fact that Python only imports modules once, but 61A uses it in at least the Maps project).\r\n\r\nIn any case, OK doesn't support this use case. ok-client was originally designed with the assumption that each invocation would make a single pass through the protocols. As a result, there's no API for programmatically interacting with individual tests multiple times in a single session. Fully providing this sort of support would probably require a sizable redesign of OK."}}