{"action":"closed","number":779,"pull_request":{"url":"https://api.github.com/repos/4teamwork/opengever.core/pulls/779","id":30667126,"html_url":"https://github.com/4teamwork/opengever.core/pull/779","diff_url":"https://github.com/4teamwork/opengever.core/pull/779.diff","patch_url":"https://github.com/4teamwork/opengever.core/pull/779.patch","issue_url":"https://api.github.com/repos/4teamwork/opengever.core/issues/779","number":779,"state":"closed","locked":false,"title":"Automatic CSRF protection with newest plone.protect","user":{"login":"jone","id":7469,"avatar_url":"https://avatars.githubusercontent.com/u/7469?v=3","gravatar_id":"","url":"https://api.github.com/users/jone","html_url":"https://github.com/jone","followers_url":"https://api.github.com/users/jone/followers","following_url":"https://api.github.com/users/jone/following{/other_user}","gists_url":"https://api.github.com/users/jone/gists{/gist_id}","starred_url":"https://api.github.com/users/jone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jone/subscriptions","organizations_url":"https://api.github.com/users/jone/orgs","repos_url":"https://api.github.com/users/jone/repos","events_url":"https://api.github.com/users/jone/events{/privacy}","received_events_url":"https://api.github.com/users/jone/received_events","type":"User","site_admin":false},"body":"This PR updates to the newest plone.protect version, which has automatic CSRF protection.\r\nCloses #754\r\n\r\nHow plone.protect works:\r\n- All \"write\" requests are validated: a `_authenticator` request param is required.\r\n- All `text/html` requests are parsed and for each `<form>` a hidden input field with the authenticator token is injected.\r\n- The `X-Frame-Options` header is set, so that all contents from Opengever can no longer be included as frame in a page from another domain.\r\n- ``request.SESSION`` should never be used when possible: it changes a lot of objects on each access, even when not changed!\r\n\r\n\r\n**Opengever adjustements of plone.protect**\r\nIn `opengever.base.protect` the automatic CSRF protection is extended:\r\n- A mechanism is added for whitelisting changed objects.\r\n  - `portal_memberdata._members` is changed often and is therefore globally whitelisted for all requests.\r\n- Patch: when there is no response content, don't try to parse it (saves us from a lot of errors in the log).\r\n- Patch: when the response charset is set, the content is decoded (e.g. `ISO-8859-15` for some mails)\r\n\r\n\r\n**Changes overview**\r\n\r\n- The redirector is rewritten to use cookies instead of sessions (separate PR #771).\r\n\r\n- Various counters and storages are whitelisted on access.\r\n  Often all annotations are whitelisted so that we can add stuff.\r\n  - reference number prefix adapter\r\n  - sequence number adapter\r\n  - OGDS sync stamp\r\n  - tree portlet favorites\r\n  - task response container\r\n\r\n- Quickupload is extended / patched:\r\n  - CSRF is disabled for JS initialization requests since the `SESSION` is accessed.\r\n  - JS is injected for updating the quickuploader params with a token.\r\n  - The file upload view is patched for moving the authenticator param to `request.form` (non-multipart file post).\r\n\r\n- ``ftw.contentmenu`` is updated:\r\n  - Transition links are required to have an `_authenticator` param since they are usually writes ( https://github.com/4teamwork/ftw.contentmenu/pull/10 )\r\n  - The update have made the \"advanced\" link to appear (https://github.com/4teamwork/ftw.contentmenu/pull/5) which we remove in a customization now.\r\n\r\n- `/edit` is always a write because of locking; CSRF is disabled for theese requests\r\n\r\n- Write actions are extended with an authenticator:\r\n  - Checkin / checkout actions on documents\r\n  - Transition controller redirects\r\n  - Journalized file downloads\r\n  - Reference prefix manager\r\n\r\n- dictstorage / tabbedview:\r\n  - Enable sqlalchemy on tabebdview in order to make dictstorage not use annotations so that we don't have a change.\r\n\r\n- Minor changes\r\n  - testing: added logging stream handler so that it logs when the plone.protect confirmation page is rendered.\r\n  - autocomplete widget test: CSRF disabled because the ftw.testbrowser testing layer is used.\r\n\r\n\r\n**Developer infos**\r\n\r\n- Always add a token to links which are supposed to make writes:\r\n  ```python\r\n  from plone.protect.utils import addTokenToUrl\r\n  url = addTokenToUrl(url)\r\n  ```\r\n  ```xml\r\n  <a tal:attributes=\"href string:${object/absolute_url}/@@view?_authenticator=${context/@@authenticator/token}\" />\r\n  ```\r\n\r\n- Whitelist objects which may be written on GET requests but are not actual changes:\r\n  ```python\r\n  from opengever.base.protect import unprotected_write\r\n\r\n  annotations = unprotected_write(IAnnotations(context))\r\n  ```\r\n  This example whitelists the annotations (actually the BTree of the annotations) but not its children.\r\n\r\n- Do not use `SESSION`.\r\n\r\n- Be aware that unexpected writes might trigger CSRF.\r\n\r\n- When confused, open ``opengever.core.testing`` and switch the loggers to ``logging.DEBUG``.\r\n\r\n- Do not disable CSRF whenever possible. When necessary, you can do it like this:\r\n  ```python\r\n  from plone.protect.interfaces import IDisableCSRFProtection\r\n  from zope.interface import alsoProvides\r\n  alsoProvides(self.request, IDisableCSRFProtection)\r\n  ```\r\n\r\n\r\n**Discussion**\r\n\r\n- JavaScript is not tested and might have problems I didn't discover. Please help to test every AJAX request!\r\n\r\n- Please test external tools carefully.\r\n\r\n- *Placeholder for something important I wanted to write but forgot what it is*\r\n\r\n**Todo**\r\n- [x] Make Plone site creation work again.\r\n- [x] Test external editor, probably not working.\r\n- [x] Test mail in. *tested, works [lg]*\r\n- [x] Test docugate (?) *the whole docugate story is on hold, will probably never make it into production [lg]*","created_at":"2015-03-06T18:21:15Z","updated_at":"2015-03-17T17:02:50Z","closed_at":"2015-03-17T17:02:50Z","merged_at":"2015-03-17T17:02:50Z","merge_commit_sha":"da082f519ed1be1e1dba80cd92cb40cb79f0cfc6","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/4teamwork/opengever.core/pulls/779/commits","review_comments_url":"https://api.github.com/repos/4teamwork/opengever.core/pulls/779/comments","review_comment_url":"https://api.github.com/repos/4teamwork/opengever.core/pulls/comments{/number}","comments_url":"https://api.github.com/repos/4teamwork/opengever.core/issues/779/comments","statuses_url":"https://api.github.com/repos/4teamwork/opengever.core/statuses/7f7010fe69247185ef0e8cab9dc04a1b3ffe7806","head":{"label":"4teamwork:jone-csrf","ref":"jone-csrf","sha":"7f7010fe69247185ef0e8cab9dc04a1b3ffe7806","user":{"login":"4teamwork","id":681821,"avatar_url":"https://avatars.githubusercontent.com/u/681821?v=3","gravatar_id":"","url":"https://api.github.com/users/4teamwork","html_url":"https://github.com/4teamwork","followers_url":"https://api.github.com/users/4teamwork/followers","following_url":"https://api.github.com/users/4teamwork/following{/other_user}","gists_url":"https://api.github.com/users/4teamwork/gists{/gist_id}","starred_url":"https://api.github.com/users/4teamwork/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/4teamwork/subscriptions","organizations_url":"https://api.github.com/users/4teamwork/orgs","repos_url":"https://api.github.com/users/4teamwork/repos","events_url":"https://api.github.com/users/4teamwork/events{/privacy}","received_events_url":"https://api.github.com/users/4teamwork/received_events","type":"Organization","site_admin":false},"repo":{"id":9788097,"name":"opengever.core","full_name":"4teamwork/opengever.core","owner":{"login":"4teamwork","id":681821,"avatar_url":"https://avatars.githubusercontent.com/u/681821?v=3","gravatar_id":"","url":"https://api.github.com/users/4teamwork","html_url":"https://github.com/4teamwork","followers_url":"https://api.github.com/users/4teamwork/followers","following_url":"https://api.github.com/users/4teamwork/following{/other_user}","gists_url":"https://api.github.com/users/4teamwork/gists{/gist_id}","starred_url":"https://api.github.com/users/4teamwork/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/4teamwork/subscriptions","organizations_url":"https://api.github.com/users/4teamwork/orgs","repos_url":"https://api.github.com/users/4teamwork/repos","events_url":"https://api.github.com/users/4teamwork/events{/privacy}","received_events_url":"https://api.github.com/users/4teamwork/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/4teamwork/opengever.core","description":"OneGov GEVER core package","fork":false,"url":"https://api.github.com/repos/4teamwork/opengever.core","forks_url":"https://api.github.com/repos/4teamwork/opengever.core/forks","keys_url":"https://api.github.com/repos/4teamwork/opengever.core/keys{/key_id}","collaborators_url":"https://api.github.com/repos/4teamwork/opengever.core/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/4teamwork/opengever.core/teams","hooks_url":"https://api.github.com/repos/4teamwork/opengever.core/hooks","issue_events_url":"https://api.github.com/repos/4teamwork/opengever.core/issues/events{/number}","events_url":"https://api.github.com/repos/4teamwork/opengever.core/events","assignees_url":"https://api.github.com/repos/4teamwork/opengever.core/assignees{/user}","branches_url":"https://api.github.com/repos/4teamwork/opengever.core/branches{/branch}","tags_url":"https://api.github.com/repos/4teamwork/opengever.core/tags","blobs_url":"https://api.github.com/repos/4teamwork/opengever.core/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/4teamwork/opengever.core/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/4teamwork/opengever.core/git/refs{/sha}","trees_url":"https://api.github.com/repos/4teamwork/opengever.core/git/trees{/sha}","statuses_url":"https://api.github.com/repos/4teamwork/opengever.core/statuses/{sha}","languages_url":"https://api.github.com/repos/4teamwork/opengever.core/languages","stargazers_url":"https://api.github.com/repos/4teamwork/opengever.core/stargazers","contributors_url":"https://api.github.com/repos/4teamwork/opengever.core/contributors","subscribers_url":"https://api.github.com/repos/4teamwork/opengever.core/subscribers","subscription_url":"https://api.github.com/repos/4teamwork/opengever.core/subscription","commits_url":"https://api.github.com/repos/4teamwork/opengever.core/commits{/sha}","git_commits_url":"https://api.github.com/repos/4teamwork/opengever.core/git/commits{/sha}","comments_url":"https://api.github.com/repos/4teamwork/opengever.core/comments{/number}","issue_comment_url":"https://api.github.com/repos/4teamwork/opengever.core/issues/comments{/number}","contents_url":"https://api.github.com/repos/4teamwork/opengever.core/contents/{+path}","compare_url":"https://api.github.com/repos/4teamwork/opengever.core/compare/{base}...{head}","merges_url":"https://api.github.com/repos/4teamwork/opengever.core/merges","archive_url":"https://api.github.com/repos/4teamwork/opengever.core/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/4teamwork/opengever.core/downloads","issues_url":"https://api.github.com/repos/4teamwork/opengever.core/issues{/number}","pulls_url":"https://api.github.com/repos/4teamwork/opengever.core/pulls{/number}","milestones_url":"https://api.github.com/repos/4teamwork/opengever.core/milestones{/number}","notifications_url":"https://api.github.com/repos/4teamwork/opengever.core/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/4teamwork/opengever.core/labels{/name}","releases_url":"https://api.github.com/repos/4teamwork/opengever.core/releases{/id}","created_at":"2013-05-01T08:28:16Z","updated_at":"2015-03-17T14:35:10Z","pushed_at":"2015-03-17T17:02:50Z","git_url":"git://github.com/4teamwork/opengever.core.git","ssh_url":"git@github.com:4teamwork/opengever.core.git","clone_url":"https://github.com/4teamwork/opengever.core.git","svn_url":"https://github.com/4teamwork/opengever.core","homepage":"","size":25755,"stargazers_count":6,"watchers_count":6,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":4,"mirror_url":null,"open_issues_count":80,"forks":4,"open_issues":80,"watchers":6,"default_branch":"master"}},"base":{"label":"4teamwork:master","ref":"master","sha":"6e3e118e50213c6425da8df8ba34cfccaa4a4711","user":{"login":"4teamwork","id":681821,"avatar_url":"https://avatars.githubusercontent.com/u/681821?v=3","gravatar_id":"","url":"https://api.github.com/users/4teamwork","html_url":"https://github.com/4teamwork","followers_url":"https://api.github.com/users/4teamwork/followers","following_url":"https://api.github.com/users/4teamwork/following{/other_user}","gists_url":"https://api.github.com/users/4teamwork/gists{/gist_id}","starred_url":"https://api.github.com/users/4teamwork/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/4teamwork/subscriptions","organizations_url":"https://api.github.com/users/4teamwork/orgs","repos_url":"https://api.github.com/users/4teamwork/repos","events_url":"https://api.github.com/users/4teamwork/events{/privacy}","received_events_url":"https://api.github.com/users/4teamwork/received_events","type":"Organization","site_admin":false},"repo":{"id":9788097,"name":"opengever.core","full_name":"4teamwork/opengever.core","owner":{"login":"4teamwork","id":681821,"avatar_url":"https://avatars.githubusercontent.com/u/681821?v=3","gravatar_id":"","url":"https://api.github.com/users/4teamwork","html_url":"https://github.com/4teamwork","followers_url":"https://api.github.com/users/4teamwork/followers","following_url":"https://api.github.com/users/4teamwork/following{/other_user}","gists_url":"https://api.github.com/users/4teamwork/gists{/gist_id}","starred_url":"https://api.github.com/users/4teamwork/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/4teamwork/subscriptions","organizations_url":"https://api.github.com/users/4teamwork/orgs","repos_url":"https://api.github.com/users/4teamwork/repos","events_url":"https://api.github.com/users/4teamwork/events{/privacy}","received_events_url":"https://api.github.com/users/4teamwork/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/4teamwork/opengever.core","description":"OneGov GEVER core package","fork":false,"url":"https://api.github.com/repos/4teamwork/opengever.core","forks_url":"https://api.github.com/repos/4teamwork/opengever.core/forks","keys_url":"https://api.github.com/repos/4teamwork/opengever.core/keys{/key_id}","collaborators_url":"https://api.github.com/repos/4teamwork/opengever.core/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/4teamwork/opengever.core/teams","hooks_url":"https://api.github.com/repos/4teamwork/opengever.core/hooks","issue_events_url":"https://api.github.com/repos/4teamwork/opengever.core/issues/events{/number}","events_url":"https://api.github.com/repos/4teamwork/opengever.core/events","assignees_url":"https://api.github.com/repos/4teamwork/opengever.core/assignees{/user}","branches_url":"https://api.github.com/repos/4teamwork/opengever.core/branches{/branch}","tags_url":"https://api.github.com/repos/4teamwork/opengever.core/tags","blobs_url":"https://api.github.com/repos/4teamwork/opengever.core/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/4teamwork/opengever.core/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/4teamwork/opengever.core/git/refs{/sha}","trees_url":"https://api.github.com/repos/4teamwork/opengever.core/git/trees{/sha}","statuses_url":"https://api.github.com/repos/4teamwork/opengever.core/statuses/{sha}","languages_url":"https://api.github.com/repos/4teamwork/opengever.core/languages","stargazers_url":"https://api.github.com/repos/4teamwork/opengever.core/stargazers","contributors_url":"https://api.github.com/repos/4teamwork/opengever.core/contributors","subscribers_url":"https://api.github.com/repos/4teamwork/opengever.core/subscribers","subscription_url":"https://api.github.com/repos/4teamwork/opengever.core/subscription","commits_url":"https://api.github.com/repos/4teamwork/opengever.core/commits{/sha}","git_commits_url":"https://api.github.com/repos/4teamwork/opengever.core/git/commits{/sha}","comments_url":"https://api.github.com/repos/4teamwork/opengever.core/comments{/number}","issue_comment_url":"https://api.github.com/repos/4teamwork/opengever.core/issues/comments{/number}","contents_url":"https://api.github.com/repos/4teamwork/opengever.core/contents/{+path}","compare_url":"https://api.github.com/repos/4teamwork/opengever.core/compare/{base}...{head}","merges_url":"https://api.github.com/repos/4teamwork/opengever.core/merges","archive_url":"https://api.github.com/repos/4teamwork/opengever.core/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/4teamwork/opengever.core/downloads","issues_url":"https://api.github.com/repos/4teamwork/opengever.core/issues{/number}","pulls_url":"https://api.github.com/repos/4teamwork/opengever.core/pulls{/number}","milestones_url":"https://api.github.com/repos/4teamwork/opengever.core/milestones{/number}","notifications_url":"https://api.github.com/repos/4teamwork/opengever.core/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/4teamwork/opengever.core/labels{/name}","releases_url":"https://api.github.com/repos/4teamwork/opengever.core/releases{/id}","created_at":"2013-05-01T08:28:16Z","updated_at":"2015-03-17T14:35:10Z","pushed_at":"2015-03-17T17:02:50Z","git_url":"git://github.com/4teamwork/opengever.core.git","ssh_url":"git@github.com:4teamwork/opengever.core.git","clone_url":"https://github.com/4teamwork/opengever.core.git","svn_url":"https://github.com/4teamwork/opengever.core","homepage":"","size":25755,"stargazers_count":6,"watchers_count":6,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":4,"mirror_url":null,"open_issues_count":80,"forks":4,"open_issues":80,"watchers":6,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/4teamwork/opengever.core/pulls/779"},"html":{"href":"https://github.com/4teamwork/opengever.core/pull/779"},"issue":{"href":"https://api.github.com/repos/4teamwork/opengever.core/issues/779"},"comments":{"href":"https://api.github.com/repos/4teamwork/opengever.core/issues/779/comments"},"review_comments":{"href":"https://api.github.com/repos/4teamwork/opengever.core/pulls/779/comments"},"review_comment":{"href":"https://api.github.com/repos/4teamwork/opengever.core/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/4teamwork/opengever.core/pulls/779/commits"},"statuses":{"href":"https://api.github.com/repos/4teamwork/opengever.core/statuses/7f7010fe69247185ef0e8cab9dc04a1b3ffe7806"}},"merged":true,"mergeable":null,"mergeable_state":"unknown","merged_by":{"login":"deiferni","id":736583,"avatar_url":"https://avatars.githubusercontent.com/u/736583?v=3","gravatar_id":"","url":"https://api.github.com/users/deiferni","html_url":"https://github.com/deiferni","followers_url":"https://api.github.com/users/deiferni/followers","following_url":"https://api.github.com/users/deiferni/following{/other_user}","gists_url":"https://api.github.com/users/deiferni/gists{/gist_id}","starred_url":"https://api.github.com/users/deiferni/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/deiferni/subscriptions","organizations_url":"https://api.github.com/users/deiferni/orgs","repos_url":"https://api.github.com/users/deiferni/repos","events_url":"https://api.github.com/users/deiferni/events{/privacy}","received_events_url":"https://api.github.com/users/deiferni/received_events","type":"User","site_admin":false},"comments":13,"review_comments":7,"commits":30,"additions":423,"deletions":83,"changed_files":39}}