{"action":"created","issue":{"url":"https://api.github.com/repos/edx/edx-platform/issues/6492","labels_url":"https://api.github.com/repos/edx/edx-platform/issues/6492/labels{/name}","comments_url":"https://api.github.com/repos/edx/edx-platform/issues/6492/comments","events_url":"https://api.github.com/repos/edx/edx-platform/issues/6492/events","html_url":"https://github.com/edx/edx-platform/pull/6492","id":53705443,"number":6492,"title":"Content libraries analytics enhancements (SOL-121)","user":{"login":"bradenmacdonald","id":945577,"avatar_url":"https://avatars.githubusercontent.com/u/945577?v=3","gravatar_id":"","url":"https://api.github.com/users/bradenmacdonald","html_url":"https://github.com/bradenmacdonald","followers_url":"https://api.github.com/users/bradenmacdonald/followers","following_url":"https://api.github.com/users/bradenmacdonald/following{/other_user}","gists_url":"https://api.github.com/users/bradenmacdonald/gists{/gist_id}","starred_url":"https://api.github.com/users/bradenmacdonald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bradenmacdonald/subscriptions","organizations_url":"https://api.github.com/users/bradenmacdonald/orgs","repos_url":"https://api.github.com/users/bradenmacdonald/repos","events_url":"https://api.github.com/users/bradenmacdonald/events{/privacy}","received_events_url":"https://api.github.com/users/bradenmacdonald/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-01-08T01:46:32Z","updated_at":"2015-01-08T22:05:05Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/edx/edx-platform/pulls/6492","html_url":"https://github.com/edx/edx-platform/pull/6492","diff_url":"https://github.com/edx/edx-platform/pull/6492.diff","patch_url":"https://github.com/edx/edx-platform/pull/6492.patch"},"body":"**Background**: This PR implements three changes to support better analytics when content libraries are used:\r\n\r\n* Adds a new field to the `edit_info` of blocks in split mongo's structures table, in order to track the original usage key of blocks copied from a library.\r\n* Adds extra data to LMS events like `problem_check` such as the library usage ID of the problem (builds on #2696)\r\n* Emits new `assigned` and `removed` events when students are randomly assigned problems from a library\r\n\r\n**Discussion**: Has largely taken place on [this Confluence wiki page](https://openedx.atlassian.net/wiki/display/SOL/Content+Libraries) in the comments and the section near the top entitled \"Analytics Considerations\".\r\n\r\n**Dependencies:** Depends on the PRs already merged into the `content-libraries` feature branch.\r\n\r\n**Sandbox URL**: TBD\r\n\r\n**Prior code reviews**: https://github.com/open-craft/edx-platform/pull/26\r\n\r\nThis implements support for analytics of content libraries, built to implement the ideas of Brian Wilson and Don Mitchell described on [the Confluence wiki page](https://openedx.atlassian.net/wiki/display/SOL/Content+Libraries) in the section near the top entitled \"Analytics Considerations\".\r\n\r\n## New tracking events:\r\nTwo new events will appear in the tracking logs (see example below):\r\n\r\n* `edx.librarycontentblock.content.assigned` - indicates that a student has been assigned their subset of blocks. Data includes:\r\n  - `location` The BlockUsageLocator of the LibraryContentModule emitting the event\r\n  - `added`: a list of data about the blocks added (usage keys in both the course and the library, the block's \"update_version\", and a *flat* list of the same information about any descendants the block has)\r\n  - `result`: a list of the complete set of blocks now assigned to this student (will always equal `added` at first, but if `max_count` is increased, `added` may be different)\r\n* `edx.librarycontentblock.content.removed` - indicates that a previously-assigned block is no longer being shown to this student. This should be very rare. Data includes:\r\n  - `location` The BlockUsageLocator of the LibraryContentModule emitting the event\r\n  - `removed`: a list of the blocks that were removed (same format as \"added\", above **but may not have the library usage key and version in some cases**)\r\n  - `result`: a list of the complete set of remaining blocks now assigned to this student\r\n  - `reason`: Either `overlimit` (`max_count` was decreased by the course author) or `invalid` (Block was deleted from library or library setting was changed and no longer includes this block)\r\n\r\nTo test: As the vagrant user you can run `sudo tail -f /edx/var/log/tracking/tracking.log` to watch the tracking log. If you sign in to the LMS as a student and view a course containing a LibraryContent block, the first time that you view the courseware should result in an \"edx.librarycontentblock.content.assigned\" entry appearing in the logs. This entry should appear as below.\r\n\r\nAlso the Django admin interface at http://localhost:8000/admin/courseware/studentmodule/ is helpful for resetting the student state for your test student user.\r\n\r\n## Example tracking log entries\r\nFirst, I made a library with two main blocks - one without children and one with a hierarchy. This is what it looks like:\r\n![screen shot 2015-01-05 at 10 07 52 pm](https://cloud.githubusercontent.com/assets/945577/5625058/9d87a4ca-9527-11e4-8c8a-6c6a75a05e4c.png)\r\n\r\nNext, if a student sees a library content block using this library that has recently been edited to go from showing 1 block to showing 2, they will generate this analytics \"added\" event:\r\n\r\n```json\r\n{\r\n    \"username\": \"honor\",\r\n    \"host\": \"precise64\",\r\n    \"event_source\": \"server\",\r\n    \"event_type\": \"edx.librarycontentblock.content.assigned\",\r\n    \"context\": {\r\n        \"course_user_tags\": {},\r\n        \"user_id\": 1,\r\n        \"org_id\": \"BradenX\",\r\n        \"course_id\": \"course-v1:BradenX+TEST+1\",\r\n        \"path\": \"/courses/course-v1:BradenX+TEST+1/courseware/f61bfa6228e948f59ac1107f6077e00c/1aac970a9128469ea78c8ef1c1d0a965/\"\r\n    },\r\n    \"time\": \"2015-01-06T05:47:09.501831+00:00\",\r\n    \"ip\": \"10.0.2.2\",\r\n    \"event\": {\r\n        \"added\": [\r\n            {\r\n                \"usage_key\": \"block-v1:BradenX+TEST+1+type@vertical+block@2a983fb828ee8bf6e34b\",\r\n                \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@vertical+block@108c4492010d49c2b3844eebdd99626b\",\r\n                \"original_usage_version\": \"54ab715f56c02c5a88bb5df9\",\r\n                \"descendants\": [\r\n                    {\r\n                        \"original_usage_version\": \"54ab716d56c02c5a88bb5dfb\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@vertical+block@975fbb4d7cde4642e81e\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@vertical+block@d14de4dfc5fc4fc3a2b1e5c08de3c6e8\"\r\n                    },\r\n                    {\r\n                        \"original_usage_version\": \"54ab717b56c02c5a88bb5dfd\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@html+block@af83d86bd23f3385cb8d\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@html+block@120657bdb6714a3096518e4fc020b1a9\"\r\n                    },\r\n                    {\r\n                        \"original_usage_version\": \"54ab6fb356c02c591fefcdb9\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@html+block@2779f12e43582c7e0789\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@html+block@8d79a3991a2d467cb9095bd1fa365e71\"\r\n                    }\r\n                ]\r\n            }\r\n        ],\r\n        \"location\": \"block-v1:BradenX+TEST+1+type@library_content+block@f3ebf2bfd1d047c284deebbff984e4e3\",\r\n        \"result\": [\r\n            {\r\n                \"usage_key\": \"block-v1:BradenX+TEST+1+type@problem+block@c279e825df7fb020cb2c\",\r\n                \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@problem+block@4b0043c672b54878b4600f7b1655435c\",\r\n                \"original_usage_version\": \"54ab718c56c02c5a88bb5dff\",\r\n                \"descendants\": []\r\n            },\r\n            {\r\n                \"usage_key\": \"block-v1:BradenX+TEST+1+type@vertical+block@2a983fb828ee8bf6e34b\",\r\n                \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@vertical+block@108c4492010d49c2b3844eebdd99626b\",\r\n                \"original_usage_version\": \"54ab715f56c02c5a88bb5df9\",\r\n                \"descendants\": [\r\n                    {\r\n                        \"original_usage_version\": \"54ab716d56c02c5a88bb5dfb\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@vertical+block@975fbb4d7cde4642e81e\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@vertical+block@d14de4dfc5fc4fc3a2b1e5c08de3c6e8\"\r\n                    },\r\n                    {\r\n                        \"original_usage_version\": \"54ab717b56c02c5a88bb5dfd\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@html+block@af83d86bd23f3385cb8d\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@html+block@120657bdb6714a3096518e4fc020b1a9\"\r\n                    },\r\n                    {\r\n                        \"original_usage_version\": \"54ab6fb356c02c591fefcdb9\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@html+block@2779f12e43582c7e0789\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@html+block@8d79a3991a2d467cb9095bd1fa365e71\"\r\n                    }\r\n                ]\r\n            }\r\n        ]\r\n    },\r\n    \"agent\": \"Mozilla/5.0 ...\",\r\n    \"page\": \"x_module\"\r\n}\r\n```\r\n\r\nAnd if I then edit the course and change the \"count\" back to 1, it will generate this analytics removed event when the student next views the course:\r\n\r\n```json\r\n{\r\n    \"username\": \"honor\",\r\n    \"host\": \"precise64\",\r\n    \"event_source\": \"server\",\r\n    \"event_type\": \"edx.librarycontentblock.content.removed\",\r\n    \"context\": {\r\n        \"course_user_tags\": {},\r\n        \"user_id\": 1,\r\n        \"org_id\": \"BradenX\",\r\n        \"course_id\": \"course-v1:BradenX+TEST+1\",\r\n        \"path\": \"/courses/course-v1:BradenX+TEST+1/courseware/f61bfa6228e948f59ac1107f6077e00c/1aac970a9128469ea78c8ef1c1d0a965/\"\r\n    },\r\n    \"time\": \"2015-01-06T05:36:15.601560+00:00\",\r\n    \"ip\": \"10.0.2.2\",\r\n    \"event\": {\r\n        \"reason\": \"overlimit\",\r\n        \"removed\": [\r\n            {\r\n                \"original_usage_version\": \"54ab718c56c02c5a88bb5dff\",\r\n                \"descendants\": [],\r\n                \"usage_key\": \"block-v1:BradenX+TEST+1+type@problem+block@c279e825df7fb020cb2c\",\r\n                \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@problem+block@4b0043c672b54878b4600f7b1655435c\"\r\n            }\r\n        ],\r\n        \"location\": \"block-v1:BradenX+TEST+1+type@library_content+block@f3ebf2bfd1d047c284deebbff984e4e3\",\r\n        \"result\": [\r\n            {\r\n                \"usage_key\": \"block-v1:BradenX+TEST+1+type@vertical+block@2a983fb828ee8bf6e34b\",\r\n                \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@vertical+block@108c4492010d49c2b3844eebdd99626b\",\r\n                \"original_usage_version\": \"54ab715f56c02c5a88bb5df9\",\r\n                \"descendants\": [\r\n                    {\r\n                        \"original_usage_version\": \"54ab716d56c02c5a88bb5dfb\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@vertical+block@975fbb4d7cde4642e81e\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@vertical+block@d14de4dfc5fc4fc3a2b1e5c08de3c6e8\"\r\n                    },\r\n                    {\r\n                        \"original_usage_version\": \"54ab717b56c02c5a88bb5dfd\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@html+block@af83d86bd23f3385cb8d\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@html+block@120657bdb6714a3096518e4fc020b1a9\"\r\n                    },\r\n                    {\r\n                        \"original_usage_version\": \"54ab6fb356c02c591fefcdb9\",\r\n                        \"usage_key\": \"block-v1:BradenX+TEST+1+type@html+block@2779f12e43582c7e0789\",\r\n                        \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@html+block@8d79a3991a2d467cb9095bd1fa365e71\"\r\n                    }\r\n                ]\r\n            }\r\n        ]\r\n    },\r\n    \"agent\": \"Mozilla/5.0 ...\",\r\n    \"page\": \"x_module\"\r\n}\r\n```\r\n\r\n## Information added to other events\r\nThis PR also adds tracking context data (`usage_key`, `original_usage_key`, and `original_usage_version`) to some normal events like `problem_check`:\r\n\r\n```json\r\n{\r\n    \"username\": \"honor\",\r\n    \"event_source\": \"server\",\r\n    \"event_type\": \"problem_check\",\r\n    \"context\": {\r\n        \"course_user_tags\": {},\r\n        \"user_id\": 1,\r\n        \"org_id\": \"BradenX\",\r\n        \"module\": {\r\n            \"usage_key\": \"block-v1:BradenX+OVRD_TEST+1+type@problem+block@c279e825df7fb020cb2c\",\r\n            \"original_usage_version\": \"54a2234656c02c0fdaa54f57\",\r\n            \"display_name\": \"A Friendly Test Problem\",\r\n            \"original_usage_key\": \"lib-block-v1:BradenX+OVRD+type@problem+block@4b0043c672b54878b4600f7b1655435c\"\r\n        },\r\n        \"course_id\": \"course-v1:BradenX+OVRD_TEST+1\",\r\n        \"path\": \"/courses/course-v1:BradenX+OVRD_TEST+1/xblock/block-v1:BradenX+OVRD_TEST+1+type@problem+block@c279e825df7fb020cb2c/handler/xmodule_handler/problem_check\"\r\n    },\r\n    \"time\": \"2015-01-02T22:08:54.091389+00:00\",\r\n    \"event\": {\r\n        \"submission\": {},\r\n        \"success\": \"incorrect\"\r\n    }\r\n}\r\n```\r\n## TODO:\r\n* When viewing the course \"Progress\" page, spurious `assigned` events are emitted, because the XBlock assigns children but isn't able to save its state to the database - so in the future, children will be re-assigned. Addressing this issue appears to be too involved to achieve as part of this PR."},"comment":{"url":"https://api.github.com/repos/edx/edx-platform/issues/comments/69258238","html_url":"https://github.com/edx/edx-platform/pull/6492#issuecomment-69258238","issue_url":"https://api.github.com/repos/edx/edx-platform/issues/6492","id":69258238,"user":{"login":"bradenmacdonald","id":945577,"avatar_url":"https://avatars.githubusercontent.com/u/945577?v=3","gravatar_id":"","url":"https://api.github.com/users/bradenmacdonald","html_url":"https://github.com/bradenmacdonald","followers_url":"https://api.github.com/users/bradenmacdonald/followers","following_url":"https://api.github.com/users/bradenmacdonald/following{/other_user}","gists_url":"https://api.github.com/users/bradenmacdonald/gists{/gist_id}","starred_url":"https://api.github.com/users/bradenmacdonald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bradenmacdonald/subscriptions","organizations_url":"https://api.github.com/users/bradenmacdonald/orgs","repos_url":"https://api.github.com/users/bradenmacdonald/repos","events_url":"https://api.github.com/users/bradenmacdonald/events{/privacy}","received_events_url":"https://api.github.com/users/bradenmacdonald/received_events","type":"User","site_admin":false},"created_at":"2015-01-08T22:05:05Z","updated_at":"2015-01-08T22:05:05Z","body":"Test failure is unrelated (bok choy discussion pagination issue fixed on master via 1b7d5f347d7a6636a53e27f8e2d9aaeb0b5f317a)."}}