{"action":"created","comment":{"url":"https://api.github.com/repos/edx/edx-submissions/pulls/comments/29473605","id":29473605,"diff_hunk":"@@ -373,6 +383,40 @@ def get_submissions(student_item_dict, limit=None):\n     return SubmissionSerializer(submission_models, many=True).data\n \n \n+def get_all_submissions(course_id, item_id, item_type, read_replica=True):\n+    \"\"\"\n+    For the given item, get the most recent submission for every student who has submitted.\n+\n+    This may return a very large result set! It is implemented as a generator for efficiency.\n+    It yields namedtuples with the following fields:\n+        student_id\n+        attempt_number\n+        submitted_at\n+        created_at\n+        answer\n+\n+    Cannot fail unless there's a database error, but may return an empty list.\n+    \"\"\"\n+    query = Submission.objects.filter(\n+        student_item__course_id=course_id,\n+        student_item__item_id=item_id,\n+        student_item__item_type=item_type,\n+    ).values_list(\n+        'student_item__student_id', 'attempt_number', 'submitted_at', 'created_at', 'raw_answer'\n+    ).order_by('student_item__student_id', '-created_at')","path":"submissions/api.py","position":48,"original_position":48,"commit_id":"4337dc1ac23f5b776cfe20aba2f819a00ce9da48","original_commit_id":"4337dc1ac23f5b776cfe20aba2f819a00ce9da48","user":{"login":"ormsbee","id":41625,"avatar_url":"https://avatars.githubusercontent.com/u/41625?v=3","gravatar_id":"","url":"https://api.github.com/users/ormsbee","html_url":"https://github.com/ormsbee","followers_url":"https://api.github.com/users/ormsbee/followers","following_url":"https://api.github.com/users/ormsbee/following{/other_user}","gists_url":"https://api.github.com/users/ormsbee/gists{/gist_id}","starred_url":"https://api.github.com/users/ormsbee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ormsbee/subscriptions","organizations_url":"https://api.github.com/users/ormsbee/orgs","repos_url":"https://api.github.com/users/ormsbee/repos","events_url":"https://api.github.com/users/ormsbee/events{/privacy}","received_events_url":"https://api.github.com/users/ormsbee/received_events","type":"User","site_admin":false},"body":"Please make sure:\r\n\r\n1. We have a test that asserts that only one query is happening here (so we're not doing n+1 fetches for the join)\r\n2. The query plan for this looks sane for a large data set. If it's going to have to do a file sort or something like that, it might be best to just do the tracking of \"this was the most recent entry for this user\" in memory. You can still make sure that you only show only the most recent by doing an order_by on the `-id` for the Submission model, and only showing the first one that pops up. This wouldn't give us a nice ordering by student ID, but I think that's a worthwhile tradeoff if it makes the query a lot faster.\r\n","created_at":"2015-04-30T21:43:21Z","updated_at":"2015-04-30T21:43:21Z","html_url":"https://github.com/edx/edx-submissions/pull/14#discussion_r29473605","pull_request_url":"https://api.github.com/repos/edx/edx-submissions/pulls/14","_links":{"self":{"href":"https://api.github.com/repos/edx/edx-submissions/pulls/comments/29473605"},"html":{"href":"https://github.com/edx/edx-submissions/pull/14#discussion_r29473605"},"pull_request":{"href":"https://api.github.com/repos/edx/edx-submissions/pulls/14"}}},"pull_request":{"url":"https://api.github.com/repos/edx/edx-submissions/pulls/14","id":34098926,"html_url":"https://github.com/edx/edx-submissions/pull/14","diff_url":"https://github.com/edx/edx-submissions/pull/14.diff","patch_url":"https://github.com/edx/edx-submissions/pull/14.patch","issue_url":"https://api.github.com/repos/edx/edx-submissions/issues/14","number":14,"state":"open","locked":false,"title":"Add get_all_submissions method to the API","user":{"login":"bradenmacdonald","id":945577,"avatar_url":"https://avatars.githubusercontent.com/u/945577?v=3","gravatar_id":"","url":"https://api.github.com/users/bradenmacdonald","html_url":"https://github.com/bradenmacdonald","followers_url":"https://api.github.com/users/bradenmacdonald/followers","following_url":"https://api.github.com/users/bradenmacdonald/following{/other_user}","gists_url":"https://api.github.com/users/bradenmacdonald/gists{/gist_id}","starred_url":"https://api.github.com/users/bradenmacdonald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bradenmacdonald/subscriptions","organizations_url":"https://api.github.com/users/bradenmacdonald/orgs","repos_url":"https://api.github.com/users/bradenmacdonald/repos","events_url":"https://api.github.com/users/bradenmacdonald/events{/privacy}","received_events_url":"https://api.github.com/users/bradenmacdonald/received_events","type":"User","site_admin":false},"body":"WIP. Currently only a prototype. Not meant to be merged.\r\n\r\nThis adds a new API method to edx-submissions:\r\n```python\r\nget_all_submissions(course_id, item_id, item_type, read_replica=True)\r\n```\r\n\r\nThis method can be used to fetch all student answers to a particular courseware item, e.g. for CSV export.","created_at":"2015-04-25T02:21:19Z","updated_at":"2015-04-30T21:43:21Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bfb8eac075a6417a09a1854c699154b07069afaf","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/edx/edx-submissions/pulls/14/commits","review_comments_url":"https://api.github.com/repos/edx/edx-submissions/pulls/14/comments","review_comment_url":"https://api.github.com/repos/edx/edx-submissions/pulls/comments{/number}","comments_url":"https://api.github.com/repos/edx/edx-submissions/issues/14/comments","statuses_url":"https://api.github.com/repos/edx/edx-submissions/statuses/4337dc1ac23f5b776cfe20aba2f819a00ce9da48","head":{"label":"open-craft:data-export","ref":"data-export","sha":"4337dc1ac23f5b776cfe20aba2f819a00ce9da48","user":{"login":"open-craft","id":7414786,"avatar_url":"https://avatars.githubusercontent.com/u/7414786?v=3","gravatar_id":"","url":"https://api.github.com/users/open-craft","html_url":"https://github.com/open-craft","followers_url":"https://api.github.com/users/open-craft/followers","following_url":"https://api.github.com/users/open-craft/following{/other_user}","gists_url":"https://api.github.com/users/open-craft/gists{/gist_id}","starred_url":"https://api.github.com/users/open-craft/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/open-craft/subscriptions","organizations_url":"https://api.github.com/users/open-craft/orgs","repos_url":"https://api.github.com/users/open-craft/repos","events_url":"https://api.github.com/users/open-craft/events{/privacy}","received_events_url":"https://api.github.com/users/open-craft/received_events","type":"Organization","site_admin":false},"repo":{"id":34551305,"name":"edx-submissions","full_name":"open-craft/edx-submissions","owner":{"login":"open-craft","id":7414786,"avatar_url":"https://avatars.githubusercontent.com/u/7414786?v=3","gravatar_id":"","url":"https://api.github.com/users/open-craft","html_url":"https://github.com/open-craft","followers_url":"https://api.github.com/users/open-craft/followers","following_url":"https://api.github.com/users/open-craft/following{/other_user}","gists_url":"https://api.github.com/users/open-craft/gists{/gist_id}","starred_url":"https://api.github.com/users/open-craft/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/open-craft/subscriptions","organizations_url":"https://api.github.com/users/open-craft/orgs","repos_url":"https://api.github.com/users/open-craft/repos","events_url":"https://api.github.com/users/open-craft/events{/privacy}","received_events_url":"https://api.github.com/users/open-craft/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/open-craft/edx-submissions","description":"API for creating submissions and scores","fork":true,"url":"https://api.github.com/repos/open-craft/edx-submissions","forks_url":"https://api.github.com/repos/open-craft/edx-submissions/forks","keys_url":"https://api.github.com/repos/open-craft/edx-submissions/keys{/key_id}","collaborators_url":"https://api.github.com/repos/open-craft/edx-submissions/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/open-craft/edx-submissions/teams","hooks_url":"https://api.github.com/repos/open-craft/edx-submissions/hooks","issue_events_url":"https://api.github.com/repos/open-craft/edx-submissions/issues/events{/number}","events_url":"https://api.github.com/repos/open-craft/edx-submissions/events","assignees_url":"https://api.github.com/repos/open-craft/edx-submissions/assignees{/user}","branches_url":"https://api.github.com/repos/open-craft/edx-submissions/branches{/branch}","tags_url":"https://api.github.com/repos/open-craft/edx-submissions/tags","blobs_url":"https://api.github.com/repos/open-craft/edx-submissions/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/open-craft/edx-submissions/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/open-craft/edx-submissions/git/refs{/sha}","trees_url":"https://api.github.com/repos/open-craft/edx-submissions/git/trees{/sha}","statuses_url":"https://api.github.com/repos/open-craft/edx-submissions/statuses/{sha}","languages_url":"https://api.github.com/repos/open-craft/edx-submissions/languages","stargazers_url":"https://api.github.com/repos/open-craft/edx-submissions/stargazers","contributors_url":"https://api.github.com/repos/open-craft/edx-submissions/contributors","subscribers_url":"https://api.github.com/repos/open-craft/edx-submissions/subscribers","subscription_url":"https://api.github.com/repos/open-craft/edx-submissions/subscription","commits_url":"https://api.github.com/repos/open-craft/edx-submissions/commits{/sha}","git_commits_url":"https://api.github.com/repos/open-craft/edx-submissions/git/commits{/sha}","comments_url":"https://api.github.com/repos/open-craft/edx-submissions/comments{/number}","issue_comment_url":"https://api.github.com/repos/open-craft/edx-submissions/issues/comments{/number}","contents_url":"https://api.github.com/repos/open-craft/edx-submissions/contents/{+path}","compare_url":"https://api.github.com/repos/open-craft/edx-submissions/compare/{base}...{head}","merges_url":"https://api.github.com/repos/open-craft/edx-submissions/merges","archive_url":"https://api.github.com/repos/open-craft/edx-submissions/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/open-craft/edx-submissions/downloads","issues_url":"https://api.github.com/repos/open-craft/edx-submissions/issues{/number}","pulls_url":"https://api.github.com/repos/open-craft/edx-submissions/pulls{/number}","milestones_url":"https://api.github.com/repos/open-craft/edx-submissions/milestones{/number}","notifications_url":"https://api.github.com/repos/open-craft/edx-submissions/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/open-craft/edx-submissions/labels{/name}","releases_url":"https://api.github.com/repos/open-craft/edx-submissions/releases{/id}","created_at":"2015-04-25T02:11:22Z","updated_at":"2015-04-25T02:11:23Z","pushed_at":"2015-04-25T02:11:51Z","git_url":"git://github.com/open-craft/edx-submissions.git","ssh_url":"git@github.com:open-craft/edx-submissions.git","clone_url":"https://github.com/open-craft/edx-submissions.git","svn_url":"https://github.com/open-craft/edx-submissions","homepage":null,"size":559,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"edx:master","ref":"master","sha":"8fb070d2a3087dd7656d27022e550d12e3b85ba3","user":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"repo":{"id":20536064,"name":"edx-submissions","full_name":"edx/edx-submissions","owner":{"login":"edx","id":3179841,"avatar_url":"https://avatars.githubusercontent.com/u/3179841?v=3","gravatar_id":"","url":"https://api.github.com/users/edx","html_url":"https://github.com/edx","followers_url":"https://api.github.com/users/edx/followers","following_url":"https://api.github.com/users/edx/following{/other_user}","gists_url":"https://api.github.com/users/edx/gists{/gist_id}","starred_url":"https://api.github.com/users/edx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edx/subscriptions","organizations_url":"https://api.github.com/users/edx/orgs","repos_url":"https://api.github.com/users/edx/repos","events_url":"https://api.github.com/users/edx/events{/privacy}","received_events_url":"https://api.github.com/users/edx/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/edx/edx-submissions","description":"API for creating submissions and scores","fork":false,"url":"https://api.github.com/repos/edx/edx-submissions","forks_url":"https://api.github.com/repos/edx/edx-submissions/forks","keys_url":"https://api.github.com/repos/edx/edx-submissions/keys{/key_id}","collaborators_url":"https://api.github.com/repos/edx/edx-submissions/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/edx/edx-submissions/teams","hooks_url":"https://api.github.com/repos/edx/edx-submissions/hooks","issue_events_url":"https://api.github.com/repos/edx/edx-submissions/issues/events{/number}","events_url":"https://api.github.com/repos/edx/edx-submissions/events","assignees_url":"https://api.github.com/repos/edx/edx-submissions/assignees{/user}","branches_url":"https://api.github.com/repos/edx/edx-submissions/branches{/branch}","tags_url":"https://api.github.com/repos/edx/edx-submissions/tags","blobs_url":"https://api.github.com/repos/edx/edx-submissions/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/edx/edx-submissions/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/edx/edx-submissions/git/refs{/sha}","trees_url":"https://api.github.com/repos/edx/edx-submissions/git/trees{/sha}","statuses_url":"https://api.github.com/repos/edx/edx-submissions/statuses/{sha}","languages_url":"https://api.github.com/repos/edx/edx-submissions/languages","stargazers_url":"https://api.github.com/repos/edx/edx-submissions/stargazers","contributors_url":"https://api.github.com/repos/edx/edx-submissions/contributors","subscribers_url":"https://api.github.com/repos/edx/edx-submissions/subscribers","subscription_url":"https://api.github.com/repos/edx/edx-submissions/subscription","commits_url":"https://api.github.com/repos/edx/edx-submissions/commits{/sha}","git_commits_url":"https://api.github.com/repos/edx/edx-submissions/git/commits{/sha}","comments_url":"https://api.github.com/repos/edx/edx-submissions/comments{/number}","issue_comment_url":"https://api.github.com/repos/edx/edx-submissions/issues/comments{/number}","contents_url":"https://api.github.com/repos/edx/edx-submissions/contents/{+path}","compare_url":"https://api.github.com/repos/edx/edx-submissions/compare/{base}...{head}","merges_url":"https://api.github.com/repos/edx/edx-submissions/merges","archive_url":"https://api.github.com/repos/edx/edx-submissions/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/edx/edx-submissions/downloads","issues_url":"https://api.github.com/repos/edx/edx-submissions/issues{/number}","pulls_url":"https://api.github.com/repos/edx/edx-submissions/pulls{/number}","milestones_url":"https://api.github.com/repos/edx/edx-submissions/milestones{/number}","notifications_url":"https://api.github.com/repos/edx/edx-submissions/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/edx/edx-submissions/labels{/name}","releases_url":"https://api.github.com/repos/edx/edx-submissions/releases{/id}","created_at":"2014-06-05T18:19:35Z","updated_at":"2015-04-27T16:13:22Z","pushed_at":"2015-04-27T16:13:21Z","git_url":"git://github.com/edx/edx-submissions.git","ssh_url":"git@github.com:edx/edx-submissions.git","clone_url":"https://github.com/edx/edx-submissions.git","svn_url":"https://github.com/edx/edx-submissions","homepage":null,"size":559,"stargazers_count":4,"watchers_count":4,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":17,"mirror_url":null,"open_issues_count":2,"forks":17,"open_issues":2,"watchers":4,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/edx/edx-submissions/pulls/14"},"html":{"href":"https://github.com/edx/edx-submissions/pull/14"},"issue":{"href":"https://api.github.com/repos/edx/edx-submissions/issues/14"},"comments":{"href":"https://api.github.com/repos/edx/edx-submissions/issues/14/comments"},"review_comments":{"href":"https://api.github.com/repos/edx/edx-submissions/pulls/14/comments"},"review_comment":{"href":"https://api.github.com/repos/edx/edx-submissions/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/edx/edx-submissions/pulls/14/commits"},"statuses":{"href":"https://api.github.com/repos/edx/edx-submissions/statuses/4337dc1ac23f5b776cfe20aba2f819a00ce9da48"}}}}