{"action":"closed","number":13,"pull_request":{"url":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls/13","id":37574335,"html_url":"https://github.com/ku-fpg/hermit-shell/pull/13","diff_url":"https://github.com/ku-fpg/hermit-shell/pull/13.diff","patch_url":"https://github.com/ku-fpg/hermit-shell/pull/13.patch","issue_url":"https://api.github.com/repos/ku-fpg/hermit-shell/issues/13","number":13,"state":"closed","locked":false,"title":"Basic testing framework","user":{"login":"RyanGlScott","id":2364661,"avatar_url":"https://avatars.githubusercontent.com/u/2364661?v=3","gravatar_id":"","url":"https://api.github.com/users/RyanGlScott","html_url":"https://github.com/RyanGlScott","followers_url":"https://api.github.com/users/RyanGlScott/followers","following_url":"https://api.github.com/users/RyanGlScott/following{/other_user}","gists_url":"https://api.github.com/users/RyanGlScott/gists{/gist_id}","starred_url":"https://api.github.com/users/RyanGlScott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RyanGlScott/subscriptions","organizations_url":"https://api.github.com/users/RyanGlScott/orgs","repos_url":"https://api.github.com/users/RyanGlScott/repos","events_url":"https://api.github.com/users/RyanGlScott/events{/privacy}","received_events_url":"https://api.github.com/users/RyanGlScott/received_events","type":"User","site_admin":false},"body":"Here's what I've managed to cobble together for a testing framework. This simply calls `hermit-shell` on each example and sees if it runs to completion. To run the tests, type\r\n\r\n    cabal test --show-details=streaming\r\n\r\nThe `--show-details=streaming` bit is optional, but it shows you the results of each test as it's running, which is nice.\r\n\r\nYou can filter which tests get run by searching against a pattern. For example:\r\n\r\n    cabal test --show-details=streaming --test-options=\"-p concatVanishes\"\r\n\r\nCurrent limitations:\r\n* At the moment, the only way for a test to fail is if `hermit-shell` fails to launch (e.g., if the `.hs` file it's run on isn't present). Lamentably, GHCi itself doesn't seem to have a mechanism for returning a non-zero exit code, so once GHCi starts, it's pretty much guaranteed to return `ExitSuccess`, even if something _did_ go wrong (e.g., if the script didn't typecheck). This makes it less than ideal for running on Travis at the moment, since we wouldn't readily know if something went wrong.\r\n\r\n  In the future, we should probably `diff` the output against a golden file like the [HERMIT tests](https://github.com/ku-fpg/hermit/blob/master/tests/Main.hs) do.\r\n* The `cabal test` options are pretty annoying to type. If my `make`-fu were stronger, I could alias the above commands with something more memorable.","created_at":"2015-06-12T15:19:36Z","updated_at":"2015-06-13T19:01:42Z","closed_at":"2015-06-13T19:01:42Z","merged_at":"2015-06-13T19:01:42Z","merge_commit_sha":"f42e8c4eef1510cc7c93ddf4117ec552549d3f49","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls/13/commits","review_comments_url":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls/13/comments","review_comment_url":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ku-fpg/hermit-shell/issues/13/comments","statuses_url":"https://api.github.com/repos/ku-fpg/hermit-shell/statuses/987753e9b392ccebd50d50952fc0e13f55984e7a","head":{"label":"ku-fpg:testing-123","ref":"testing-123","sha":"987753e9b392ccebd50d50952fc0e13f55984e7a","user":{"login":"ku-fpg","id":760894,"avatar_url":"https://avatars.githubusercontent.com/u/760894?v=3","gravatar_id":"","url":"https://api.github.com/users/ku-fpg","html_url":"https://github.com/ku-fpg","followers_url":"https://api.github.com/users/ku-fpg/followers","following_url":"https://api.github.com/users/ku-fpg/following{/other_user}","gists_url":"https://api.github.com/users/ku-fpg/gists{/gist_id}","starred_url":"https://api.github.com/users/ku-fpg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ku-fpg/subscriptions","organizations_url":"https://api.github.com/users/ku-fpg/orgs","repos_url":"https://api.github.com/users/ku-fpg/repos","events_url":"https://api.github.com/users/ku-fpg/events{/privacy}","received_events_url":"https://api.github.com/users/ku-fpg/received_events","type":"Organization","site_admin":false},"repo":{"id":29884149,"name":"hermit-shell","full_name":"ku-fpg/hermit-shell","owner":{"login":"ku-fpg","id":760894,"avatar_url":"https://avatars.githubusercontent.com/u/760894?v=3","gravatar_id":"","url":"https://api.github.com/users/ku-fpg","html_url":"https://github.com/ku-fpg","followers_url":"https://api.github.com/users/ku-fpg/followers","following_url":"https://api.github.com/users/ku-fpg/following{/other_user}","gists_url":"https://api.github.com/users/ku-fpg/gists{/gist_id}","starred_url":"https://api.github.com/users/ku-fpg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ku-fpg/subscriptions","organizations_url":"https://api.github.com/users/ku-fpg/orgs","repos_url":"https://api.github.com/users/ku-fpg/repos","events_url":"https://api.github.com/users/ku-fpg/events{/privacy}","received_events_url":"https://api.github.com/users/ku-fpg/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ku-fpg/hermit-shell","description":"HERMIT with GHCi shell","fork":false,"url":"https://api.github.com/repos/ku-fpg/hermit-shell","forks_url":"https://api.github.com/repos/ku-fpg/hermit-shell/forks","keys_url":"https://api.github.com/repos/ku-fpg/hermit-shell/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ku-fpg/hermit-shell/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ku-fpg/hermit-shell/teams","hooks_url":"https://api.github.com/repos/ku-fpg/hermit-shell/hooks","issue_events_url":"https://api.github.com/repos/ku-fpg/hermit-shell/issues/events{/number}","events_url":"https://api.github.com/repos/ku-fpg/hermit-shell/events","assignees_url":"https://api.github.com/repos/ku-fpg/hermit-shell/assignees{/user}","branches_url":"https://api.github.com/repos/ku-fpg/hermit-shell/branches{/branch}","tags_url":"https://api.github.com/repos/ku-fpg/hermit-shell/tags","blobs_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/refs{/sha}","trees_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ku-fpg/hermit-shell/statuses/{sha}","languages_url":"https://api.github.com/repos/ku-fpg/hermit-shell/languages","stargazers_url":"https://api.github.com/repos/ku-fpg/hermit-shell/stargazers","contributors_url":"https://api.github.com/repos/ku-fpg/hermit-shell/contributors","subscribers_url":"https://api.github.com/repos/ku-fpg/hermit-shell/subscribers","subscription_url":"https://api.github.com/repos/ku-fpg/hermit-shell/subscription","commits_url":"https://api.github.com/repos/ku-fpg/hermit-shell/commits{/sha}","git_commits_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/commits{/sha}","comments_url":"https://api.github.com/repos/ku-fpg/hermit-shell/comments{/number}","issue_comment_url":"https://api.github.com/repos/ku-fpg/hermit-shell/issues/comments{/number}","contents_url":"https://api.github.com/repos/ku-fpg/hermit-shell/contents/{+path}","compare_url":"https://api.github.com/repos/ku-fpg/hermit-shell/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ku-fpg/hermit-shell/merges","archive_url":"https://api.github.com/repos/ku-fpg/hermit-shell/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ku-fpg/hermit-shell/downloads","issues_url":"https://api.github.com/repos/ku-fpg/hermit-shell/issues{/number}","pulls_url":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls{/number}","milestones_url":"https://api.github.com/repos/ku-fpg/hermit-shell/milestones{/number}","notifications_url":"https://api.github.com/repos/ku-fpg/hermit-shell/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ku-fpg/hermit-shell/labels{/name}","releases_url":"https://api.github.com/repos/ku-fpg/hermit-shell/releases{/id}","created_at":"2015-01-26T22:07:19Z","updated_at":"2015-05-29T10:09:09Z","pushed_at":"2015-06-13T19:01:42Z","git_url":"git://github.com/ku-fpg/hermit-shell.git","ssh_url":"git@github.com:ku-fpg/hermit-shell.git","clone_url":"https://github.com/ku-fpg/hermit-shell.git","svn_url":"https://github.com/ku-fpg/hermit-shell","homepage":null,"size":908,"stargazers_count":1,"watchers_count":1,"language":"Haskell","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":10,"forks":0,"open_issues":10,"watchers":1,"default_branch":"master"}},"base":{"label":"ku-fpg:master","ref":"master","sha":"2d50394020f75c0702064db477b6f5a18bd6b9a7","user":{"login":"ku-fpg","id":760894,"avatar_url":"https://avatars.githubusercontent.com/u/760894?v=3","gravatar_id":"","url":"https://api.github.com/users/ku-fpg","html_url":"https://github.com/ku-fpg","followers_url":"https://api.github.com/users/ku-fpg/followers","following_url":"https://api.github.com/users/ku-fpg/following{/other_user}","gists_url":"https://api.github.com/users/ku-fpg/gists{/gist_id}","starred_url":"https://api.github.com/users/ku-fpg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ku-fpg/subscriptions","organizations_url":"https://api.github.com/users/ku-fpg/orgs","repos_url":"https://api.github.com/users/ku-fpg/repos","events_url":"https://api.github.com/users/ku-fpg/events{/privacy}","received_events_url":"https://api.github.com/users/ku-fpg/received_events","type":"Organization","site_admin":false},"repo":{"id":29884149,"name":"hermit-shell","full_name":"ku-fpg/hermit-shell","owner":{"login":"ku-fpg","id":760894,"avatar_url":"https://avatars.githubusercontent.com/u/760894?v=3","gravatar_id":"","url":"https://api.github.com/users/ku-fpg","html_url":"https://github.com/ku-fpg","followers_url":"https://api.github.com/users/ku-fpg/followers","following_url":"https://api.github.com/users/ku-fpg/following{/other_user}","gists_url":"https://api.github.com/users/ku-fpg/gists{/gist_id}","starred_url":"https://api.github.com/users/ku-fpg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ku-fpg/subscriptions","organizations_url":"https://api.github.com/users/ku-fpg/orgs","repos_url":"https://api.github.com/users/ku-fpg/repos","events_url":"https://api.github.com/users/ku-fpg/events{/privacy}","received_events_url":"https://api.github.com/users/ku-fpg/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ku-fpg/hermit-shell","description":"HERMIT with GHCi shell","fork":false,"url":"https://api.github.com/repos/ku-fpg/hermit-shell","forks_url":"https://api.github.com/repos/ku-fpg/hermit-shell/forks","keys_url":"https://api.github.com/repos/ku-fpg/hermit-shell/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ku-fpg/hermit-shell/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ku-fpg/hermit-shell/teams","hooks_url":"https://api.github.com/repos/ku-fpg/hermit-shell/hooks","issue_events_url":"https://api.github.com/repos/ku-fpg/hermit-shell/issues/events{/number}","events_url":"https://api.github.com/repos/ku-fpg/hermit-shell/events","assignees_url":"https://api.github.com/repos/ku-fpg/hermit-shell/assignees{/user}","branches_url":"https://api.github.com/repos/ku-fpg/hermit-shell/branches{/branch}","tags_url":"https://api.github.com/repos/ku-fpg/hermit-shell/tags","blobs_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/refs{/sha}","trees_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ku-fpg/hermit-shell/statuses/{sha}","languages_url":"https://api.github.com/repos/ku-fpg/hermit-shell/languages","stargazers_url":"https://api.github.com/repos/ku-fpg/hermit-shell/stargazers","contributors_url":"https://api.github.com/repos/ku-fpg/hermit-shell/contributors","subscribers_url":"https://api.github.com/repos/ku-fpg/hermit-shell/subscribers","subscription_url":"https://api.github.com/repos/ku-fpg/hermit-shell/subscription","commits_url":"https://api.github.com/repos/ku-fpg/hermit-shell/commits{/sha}","git_commits_url":"https://api.github.com/repos/ku-fpg/hermit-shell/git/commits{/sha}","comments_url":"https://api.github.com/repos/ku-fpg/hermit-shell/comments{/number}","issue_comment_url":"https://api.github.com/repos/ku-fpg/hermit-shell/issues/comments{/number}","contents_url":"https://api.github.com/repos/ku-fpg/hermit-shell/contents/{+path}","compare_url":"https://api.github.com/repos/ku-fpg/hermit-shell/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ku-fpg/hermit-shell/merges","archive_url":"https://api.github.com/repos/ku-fpg/hermit-shell/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ku-fpg/hermit-shell/downloads","issues_url":"https://api.github.com/repos/ku-fpg/hermit-shell/issues{/number}","pulls_url":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls{/number}","milestones_url":"https://api.github.com/repos/ku-fpg/hermit-shell/milestones{/number}","notifications_url":"https://api.github.com/repos/ku-fpg/hermit-shell/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ku-fpg/hermit-shell/labels{/name}","releases_url":"https://api.github.com/repos/ku-fpg/hermit-shell/releases{/id}","created_at":"2015-01-26T22:07:19Z","updated_at":"2015-05-29T10:09:09Z","pushed_at":"2015-06-13T19:01:42Z","git_url":"git://github.com/ku-fpg/hermit-shell.git","ssh_url":"git@github.com:ku-fpg/hermit-shell.git","clone_url":"https://github.com/ku-fpg/hermit-shell.git","svn_url":"https://github.com/ku-fpg/hermit-shell","homepage":null,"size":908,"stargazers_count":1,"watchers_count":1,"language":"Haskell","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":10,"forks":0,"open_issues":10,"watchers":1,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls/13"},"html":{"href":"https://github.com/ku-fpg/hermit-shell/pull/13"},"issue":{"href":"https://api.github.com/repos/ku-fpg/hermit-shell/issues/13"},"comments":{"href":"https://api.github.com/repos/ku-fpg/hermit-shell/issues/13/comments"},"review_comments":{"href":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls/13/comments"},"review_comment":{"href":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ku-fpg/hermit-shell/pulls/13/commits"},"statuses":{"href":"https://api.github.com/repos/ku-fpg/hermit-shell/statuses/987753e9b392ccebd50d50952fc0e13f55984e7a"}},"merged":true,"mergeable":null,"mergeable_state":"unknown","merged_by":{"login":"RyanGlScott","id":2364661,"avatar_url":"https://avatars.githubusercontent.com/u/2364661?v=3","gravatar_id":"","url":"https://api.github.com/users/RyanGlScott","html_url":"https://github.com/RyanGlScott","followers_url":"https://api.github.com/users/RyanGlScott/followers","following_url":"https://api.github.com/users/RyanGlScott/following{/other_user}","gists_url":"https://api.github.com/users/RyanGlScott/gists{/gist_id}","starred_url":"https://api.github.com/users/RyanGlScott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RyanGlScott/subscriptions","organizations_url":"https://api.github.com/users/RyanGlScott/orgs","repos_url":"https://api.github.com/users/RyanGlScott/repos","events_url":"https://api.github.com/users/RyanGlScott/events{/privacy}","received_events_url":"https://api.github.com/users/RyanGlScott/received_events","type":"User","site_admin":false},"comments":1,"review_comments":0,"commits":10,"additions":231,"deletions":55,"changed_files":17}}