{"action":"created","comment":{"url":"https://api.github.com/repos/mozilla/fjord/pulls/comments/39560163","id":39560163,"diff_hunk":"@@ -167,19 +170,191 @@ the relevant place and then create a new ``test_FILENAMEHERE.js`` file\n with your QUnit tests.\n \n \n-Changing tests\n-==============\n+Smoketests\n+==========\n+\n+We have a smoketest suite. For more details, see that README:\n+\n+https://github.com/mozilla/fjord/tree/master/smoketests\n+\n+\n+Comprehensive test plan\n+=======================\n+\n+Sometimes, we need to make substantive changes to the site that touch a lot of\n+parts. This test plan covers all the things you should do (at a minimum) to make\n+sure those parts are still working.\n+\n+This is a good thing to do after doing a Django upgrade.\n+\n+.. Note::\n+\n+   These tests aren't run frequently and they're probably out of date.\n+\n+   Run through the test plan with your code **before** you make your\n+   changes and update parts that have changed. Make sure to add\n+   sections for new functionality.\n+\n+.. Note::\n+\n+   This is labeled \"comprehensive\", but probably leave some stuff out. We should\n+   improve it as we use it.\n+\n+.. Note::\n+\n+   This test plan is very terse. You'll need to know your way around Fjord and\n+   Input for this to make a lot of sense. Sorry about that.\n+\n+\n+Unit tests\n+----------\n+\n+1. run ``./py.test`` to run the unit tests with a clean database\n+\n+\n+Smoketests\n+----------\n+\n+1. run ``./manage.py runserver`` in one terminal and launch the smoketests\n+   in another terminal\n+\n+\n+Testing collectstatic\n+---------------------\n+\n+1. delete everything in ``static/``\n+2. run ``./manage.py collectstatic`` and verify no errors and ``.js`` and ``.css``\n+   files got built\n+\n+\n+Testing migrations\n+------------------\n+\n+1. run ``./manage.py makemigrations`` -- it shouldn't create any new\n+   migrations\n+2. run ``./manage.py migrate`` with a db dump\n+\n+\n+Django check\n+------------\n+\n+1. run ``./manage.py check`` and verify no errors\n+\n+\n+Testing Elasticsearch and indexing\n+----------------------------------\n+\n+1. run ``./manage.py esstatus``\n+2. run ``./manage.py esreindex --percent=5`` to create a new index and\n+   index some stuff\n+3. run ``./manage.py esstatus`` to make sure the new index is there\n+4. run ``./manage.py esdelete <index>`` to delete that index\n+5. run ``./manage.py esstatus`` to make sure the index was created","path":"docs/testing.rst","position":128,"original_position":128,"commit_id":"468751ae03094afa97a15ed039672f7bb5e85a0b","original_commit_id":"468751ae03094afa97a15ed039672f7bb5e85a0b","user":{"login":"mythmon","id":305049,"avatar_url":"https://avatars.githubusercontent.com/u/305049?v=3","gravatar_id":"","url":"https://api.github.com/users/mythmon","html_url":"https://github.com/mythmon","followers_url":"https://api.github.com/users/mythmon/followers","following_url":"https://api.github.com/users/mythmon/following{/other_user}","gists_url":"https://api.github.com/users/mythmon/gists{/gist_id}","starred_url":"https://api.github.com/users/mythmon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mythmon/subscriptions","organizations_url":"https://api.github.com/users/mythmon/orgs","repos_url":"https://api.github.com/users/mythmon/repos","events_url":"https://api.github.com/users/mythmon/events{/privacy}","received_events_url":"https://api.github.com/users/mythmon/received_events","type":"User","site_admin":false},"body":"Should this say \"to make sure the index was deleted\"?","created_at":"2015-09-15T20:26:09Z","updated_at":"2015-09-15T20:26:09Z","html_url":"https://github.com/mozilla/fjord/pull/680#discussion_r39560163","pull_request_url":"https://api.github.com/repos/mozilla/fjord/pulls/680","_links":{"self":{"href":"https://api.github.com/repos/mozilla/fjord/pulls/comments/39560163"},"html":{"href":"https://github.com/mozilla/fjord/pull/680#discussion_r39560163"},"pull_request":{"href":"https://api.github.com/repos/mozilla/fjord/pulls/680"}}},"pull_request":{"url":"https://api.github.com/repos/mozilla/fjord/pulls/680","id":45089824,"html_url":"https://github.com/mozilla/fjord/pull/680","diff_url":"https://github.com/mozilla/fjord/pull/680.diff","patch_url":"https://github.com/mozilla/fjord/pull/680.patch","issue_url":"https://api.github.com/repos/mozilla/fjord/issues/680","number":680,"state":"open","locked":false,"title":"[bug 1205013] Add comprehensive test plan","user":{"login":"willkg","id":820826,"avatar_url":"https://avatars.githubusercontent.com/u/820826?v=3","gravatar_id":"","url":"https://api.github.com/users/willkg","html_url":"https://github.com/willkg","followers_url":"https://api.github.com/users/willkg/followers","following_url":"https://api.github.com/users/willkg/following{/other_user}","gists_url":"https://api.github.com/users/willkg/gists{/gist_id}","starred_url":"https://api.github.com/users/willkg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/willkg/subscriptions","organizations_url":"https://api.github.com/users/willkg/orgs","repos_url":"https://api.github.com/users/willkg/repos","events_url":"https://api.github.com/users/willkg/events{/privacy}","received_events_url":"https://api.github.com/users/willkg/received_events","type":"User","site_admin":false},"body":"Periodically, we do substantive changes that affect the whole codebase.\r\nWe have a unit test suite and a set of smoketests, but there's a lot of\r\nstuff that isn't covered by those.\r\n\r\nInstead of writing up a test plan every time we do a substantive change,\r\nthis is an attempt at writing a comprehensive test plan covering all the\r\nimportant things that we can reuse every time we do a substantive\r\nchange.\r\n\r\nr?","created_at":"2015-09-15T19:12:10Z","updated_at":"2015-09-15T20:26:09Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bd750a8cfe663fe0db3bd77900e76d91c00c6ba8","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/mozilla/fjord/pulls/680/commits","review_comments_url":"https://api.github.com/repos/mozilla/fjord/pulls/680/comments","review_comment_url":"https://api.github.com/repos/mozilla/fjord/pulls/comments{/number}","comments_url":"https://api.github.com/repos/mozilla/fjord/issues/680/comments","statuses_url":"https://api.github.com/repos/mozilla/fjord/statuses/468751ae03094afa97a15ed039672f7bb5e85a0b","head":{"label":"willkg:1205013-test-plan","ref":"1205013-test-plan","sha":"468751ae03094afa97a15ed039672f7bb5e85a0b","user":{"login":"willkg","id":820826,"avatar_url":"https://avatars.githubusercontent.com/u/820826?v=3","gravatar_id":"","url":"https://api.github.com/users/willkg","html_url":"https://github.com/willkg","followers_url":"https://api.github.com/users/willkg/followers","following_url":"https://api.github.com/users/willkg/following{/other_user}","gists_url":"https://api.github.com/users/willkg/gists{/gist_id}","starred_url":"https://api.github.com/users/willkg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/willkg/subscriptions","organizations_url":"https://api.github.com/users/willkg/orgs","repos_url":"https://api.github.com/users/willkg/repos","events_url":"https://api.github.com/users/willkg/events{/privacy}","received_events_url":"https://api.github.com/users/willkg/received_events","type":"User","site_admin":false},"repo":{"id":5204498,"name":"fjord","full_name":"willkg/fjord","owner":{"login":"willkg","id":820826,"avatar_url":"https://avatars.githubusercontent.com/u/820826?v=3","gravatar_id":"","url":"https://api.github.com/users/willkg","html_url":"https://github.com/willkg","followers_url":"https://api.github.com/users/willkg/followers","following_url":"https://api.github.com/users/willkg/following{/other_user}","gists_url":"https://api.github.com/users/willkg/gists{/gist_id}","starred_url":"https://api.github.com/users/willkg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/willkg/subscriptions","organizations_url":"https://api.github.com/users/willkg/orgs","repos_url":"https://api.github.com/users/willkg/repos","events_url":"https://api.github.com/users/willkg/events{/privacy}","received_events_url":"https://api.github.com/users/willkg/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/willkg/fjord","description":"Software that runs the new input.mozilla.org","fork":true,"url":"https://api.github.com/repos/willkg/fjord","forks_url":"https://api.github.com/repos/willkg/fjord/forks","keys_url":"https://api.github.com/repos/willkg/fjord/keys{/key_id}","collaborators_url":"https://api.github.com/repos/willkg/fjord/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/willkg/fjord/teams","hooks_url":"https://api.github.com/repos/willkg/fjord/hooks","issue_events_url":"https://api.github.com/repos/willkg/fjord/issues/events{/number}","events_url":"https://api.github.com/repos/willkg/fjord/events","assignees_url":"https://api.github.com/repos/willkg/fjord/assignees{/user}","branches_url":"https://api.github.com/repos/willkg/fjord/branches{/branch}","tags_url":"https://api.github.com/repos/willkg/fjord/tags","blobs_url":"https://api.github.com/repos/willkg/fjord/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/willkg/fjord/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/willkg/fjord/git/refs{/sha}","trees_url":"https://api.github.com/repos/willkg/fjord/git/trees{/sha}","statuses_url":"https://api.github.com/repos/willkg/fjord/statuses/{sha}","languages_url":"https://api.github.com/repos/willkg/fjord/languages","stargazers_url":"https://api.github.com/repos/willkg/fjord/stargazers","contributors_url":"https://api.github.com/repos/willkg/fjord/contributors","subscribers_url":"https://api.github.com/repos/willkg/fjord/subscribers","subscription_url":"https://api.github.com/repos/willkg/fjord/subscription","commits_url":"https://api.github.com/repos/willkg/fjord/commits{/sha}","git_commits_url":"https://api.github.com/repos/willkg/fjord/git/commits{/sha}","comments_url":"https://api.github.com/repos/willkg/fjord/comments{/number}","issue_comment_url":"https://api.github.com/repos/willkg/fjord/issues/comments{/number}","contents_url":"https://api.github.com/repos/willkg/fjord/contents/{+path}","compare_url":"https://api.github.com/repos/willkg/fjord/compare/{base}...{head}","merges_url":"https://api.github.com/repos/willkg/fjord/merges","archive_url":"https://api.github.com/repos/willkg/fjord/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/willkg/fjord/downloads","issues_url":"https://api.github.com/repos/willkg/fjord/issues{/number}","pulls_url":"https://api.github.com/repos/willkg/fjord/pulls{/number}","milestones_url":"https://api.github.com/repos/willkg/fjord/milestones{/number}","notifications_url":"https://api.github.com/repos/willkg/fjord/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/willkg/fjord/labels{/name}","releases_url":"https://api.github.com/repos/willkg/fjord/releases{/id}","created_at":"2012-07-27T12:53:19Z","updated_at":"2015-04-27T14:46:28Z","pushed_at":"2015-09-15T19:38:49Z","git_url":"git://github.com/willkg/fjord.git","ssh_url":"git@github.com:willkg/fjord.git","clone_url":"https://github.com/willkg/fjord.git","svn_url":"https://github.com/willkg/fjord","homepage":null,"size":70168,"stargazers_count":1,"watchers_count":1,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":1,"default_branch":"master"}},"base":{"label":"mozilla:master","ref":"master","sha":"14eca41fb96e00ffc584a5e582c5921525dba6cc","user":{"login":"mozilla","id":131524,"avatar_url":"https://avatars.githubusercontent.com/u/131524?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla","html_url":"https://github.com/mozilla","followers_url":"https://api.github.com/users/mozilla/followers","following_url":"https://api.github.com/users/mozilla/following{/other_user}","gists_url":"https://api.github.com/users/mozilla/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla/subscriptions","organizations_url":"https://api.github.com/users/mozilla/orgs","repos_url":"https://api.github.com/users/mozilla/repos","events_url":"https://api.github.com/users/mozilla/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla/received_events","type":"Organization","site_admin":false},"repo":{"id":5197539,"name":"fjord","full_name":"mozilla/fjord","owner":{"login":"mozilla","id":131524,"avatar_url":"https://avatars.githubusercontent.com/u/131524?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla","html_url":"https://github.com/mozilla","followers_url":"https://api.github.com/users/mozilla/followers","following_url":"https://api.github.com/users/mozilla/following{/other_user}","gists_url":"https://api.github.com/users/mozilla/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla/subscriptions","organizations_url":"https://api.github.com/users/mozilla/orgs","repos_url":"https://api.github.com/users/mozilla/repos","events_url":"https://api.github.com/users/mozilla/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/mozilla/fjord","description":"Software that runs the new input.mozilla.org","fork":false,"url":"https://api.github.com/repos/mozilla/fjord","forks_url":"https://api.github.com/repos/mozilla/fjord/forks","keys_url":"https://api.github.com/repos/mozilla/fjord/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mozilla/fjord/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mozilla/fjord/teams","hooks_url":"https://api.github.com/repos/mozilla/fjord/hooks","issue_events_url":"https://api.github.com/repos/mozilla/fjord/issues/events{/number}","events_url":"https://api.github.com/repos/mozilla/fjord/events","assignees_url":"https://api.github.com/repos/mozilla/fjord/assignees{/user}","branches_url":"https://api.github.com/repos/mozilla/fjord/branches{/branch}","tags_url":"https://api.github.com/repos/mozilla/fjord/tags","blobs_url":"https://api.github.com/repos/mozilla/fjord/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mozilla/fjord/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mozilla/fjord/git/refs{/sha}","trees_url":"https://api.github.com/repos/mozilla/fjord/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mozilla/fjord/statuses/{sha}","languages_url":"https://api.github.com/repos/mozilla/fjord/languages","stargazers_url":"https://api.github.com/repos/mozilla/fjord/stargazers","contributors_url":"https://api.github.com/repos/mozilla/fjord/contributors","subscribers_url":"https://api.github.com/repos/mozilla/fjord/subscribers","subscription_url":"https://api.github.com/repos/mozilla/fjord/subscription","commits_url":"https://api.github.com/repos/mozilla/fjord/commits{/sha}","git_commits_url":"https://api.github.com/repos/mozilla/fjord/git/commits{/sha}","comments_url":"https://api.github.com/repos/mozilla/fjord/comments{/number}","issue_comment_url":"https://api.github.com/repos/mozilla/fjord/issues/comments{/number}","contents_url":"https://api.github.com/repos/mozilla/fjord/contents/{+path}","compare_url":"https://api.github.com/repos/mozilla/fjord/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mozilla/fjord/merges","archive_url":"https://api.github.com/repos/mozilla/fjord/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mozilla/fjord/downloads","issues_url":"https://api.github.com/repos/mozilla/fjord/issues{/number}","pulls_url":"https://api.github.com/repos/mozilla/fjord/pulls{/number}","milestones_url":"https://api.github.com/repos/mozilla/fjord/milestones{/number}","notifications_url":"https://api.github.com/repos/mozilla/fjord/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mozilla/fjord/labels{/name}","releases_url":"https://api.github.com/repos/mozilla/fjord/releases{/id}","created_at":"2012-07-26T21:25:00Z","updated_at":"2015-08-11T18:19:44Z","pushed_at":"2015-09-15T19:38:51Z","git_url":"git://github.com/mozilla/fjord.git","ssh_url":"git@github.com:mozilla/fjord.git","clone_url":"https://github.com/mozilla/fjord.git","svn_url":"https://github.com/mozilla/fjord","homepage":null,"size":76146,"stargazers_count":20,"watchers_count":20,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":40,"mirror_url":null,"open_issues_count":1,"forks":40,"open_issues":1,"watchers":20,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/mozilla/fjord/pulls/680"},"html":{"href":"https://github.com/mozilla/fjord/pull/680"},"issue":{"href":"https://api.github.com/repos/mozilla/fjord/issues/680"},"comments":{"href":"https://api.github.com/repos/mozilla/fjord/issues/680/comments"},"review_comments":{"href":"https://api.github.com/repos/mozilla/fjord/pulls/680/comments"},"review_comment":{"href":"https://api.github.com/repos/mozilla/fjord/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/mozilla/fjord/pulls/680/commits"},"statuses":{"href":"https://api.github.com/repos/mozilla/fjord/statuses/468751ae03094afa97a15ed039672f7bb5e85a0b"}}}}