{"action":"created","issue":{"url":"https://api.github.com/repos/scrapy/scrapy/issues/1586","labels_url":"https://api.github.com/repos/scrapy/scrapy/issues/1586/labels{/name}","comments_url":"https://api.github.com/repos/scrapy/scrapy/issues/1586/comments","events_url":"https://api.github.com/repos/scrapy/scrapy/issues/1586/events","html_url":"https://github.com/scrapy/scrapy/pull/1586","id":115523803,"number":1586,"title":"[MRG+1] Backwards compatible per key priorities","user":{"login":"jdemaeyer","id":10531844,"avatar_url":"https://avatars.githubusercontent.com/u/10531844?v=3","gravatar_id":"","url":"https://api.github.com/users/jdemaeyer","html_url":"https://github.com/jdemaeyer","followers_url":"https://api.github.com/users/jdemaeyer/followers","following_url":"https://api.github.com/users/jdemaeyer/following{/other_user}","gists_url":"https://api.github.com/users/jdemaeyer/gists{/gist_id}","starred_url":"https://api.github.com/users/jdemaeyer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jdemaeyer/subscriptions","organizations_url":"https://api.github.com/users/jdemaeyer/orgs","repos_url":"https://api.github.com/users/jdemaeyer/repos","events_url":"https://api.github.com/users/jdemaeyer/events{/privacy}","received_events_url":"https://api.github.com/users/jdemaeyer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/scrapy/scrapy/milestones/7","html_url":"https://github.com/scrapy/scrapy/milestones/Scrapy%201.1","labels_url":"https://api.github.com/repos/scrapy/scrapy/milestones/7/labels","id":1008005,"number":7,"title":"Scrapy 1.1","description":"","creator":{"login":"pablohoffman","id":185212,"avatar_url":"https://avatars.githubusercontent.com/u/185212?v=3","gravatar_id":"","url":"https://api.github.com/users/pablohoffman","html_url":"https://github.com/pablohoffman","followers_url":"https://api.github.com/users/pablohoffman/followers","following_url":"https://api.github.com/users/pablohoffman/following{/other_user}","gists_url":"https://api.github.com/users/pablohoffman/gists{/gist_id}","starred_url":"https://api.github.com/users/pablohoffman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pablohoffman/subscriptions","organizations_url":"https://api.github.com/users/pablohoffman/orgs","repos_url":"https://api.github.com/users/pablohoffman/repos","events_url":"https://api.github.com/users/pablohoffman/events{/privacy}","received_events_url":"https://api.github.com/users/pablohoffman/received_events","type":"User","site_admin":false},"open_issues":5,"closed_issues":7,"state":"open","created_at":"2015-03-05T18:14:46Z","updated_at":"2015-11-10T13:08:32Z","due_on":null,"closed_at":null},"comments":12,"created_at":"2015-11-06T15:17:04Z","updated_at":"2015-11-11T06:14:51Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/scrapy/scrapy/pulls/1586","html_url":"https://github.com/scrapy/scrapy/pull/1586","diff_url":"https://github.com/scrapy/scrapy/pull/1586.diff","patch_url":"https://github.com/scrapy/scrapy/pull/1586.patch"},"body":"This is a partial reversal of #1149 that provides per-key priorities in a backwards compatible fashion.\r\n\r\nThe backwards incompatible API change in #1149 was that when writing to dictionary settings, their contents were updated, not completely overwritten. This PR brings back the old behaviour of completely discarding the previous contents of the dictionary. If the dictionary setting written to is a `BaseSettings` instance, it is not overwritten but cleared and then updated (resulting in the same contents, but conserving per-key priorities).\r\n\r\nThe price for this is:\r\n  - we cannot get rid of the `_BASE` settings\r\n  - it will again be much harder to disable single components from the command line or from a spider's `custom_settings` (because disabling a single component requires writing down the complete dictionary without that component). @kmike had the idea that we could add some helper magic for updating (not overwriting) dictionaries, e.g. allow people to do `scrapy ... -s update:EXTENSIONS={\"blah\": null}`, or `custom_settings: { \"update:EXTENSIONS\": {\"blah\": None}}`. I'll implement that in a follow-up PR to discuss it and see if it makes unforeseen trouble.\r\n\r\nAs a bonus, I added backwards compatibility to the `build_component_list()` utility function, albeit it not being public. @rolando maybe you could check whether it is indeed backwards compatible with how you use it? Also take a look at the now-public [`BaseSettings.getcomposite()` method](https://github.com/jdemaeyer/scrapy/blob/fix/backwards-compatible-per-key-priorities/scrapy/settings/__init__.py#L197). It helps in separating a utility function that's meant to convert a dictionary into a sorted list from how that dictionary is created."},"comment":{"url":"https://api.github.com/repos/scrapy/scrapy/issues/comments/155679323","html_url":"https://github.com/scrapy/scrapy/pull/1586#issuecomment-155679323","issue_url":"https://api.github.com/repos/scrapy/scrapy/issues/1586","id":155679323,"user":{"login":"curita","id":213781,"avatar_url":"https://avatars.githubusercontent.com/u/213781?v=3","gravatar_id":"","url":"https://api.github.com/users/curita","html_url":"https://github.com/curita","followers_url":"https://api.github.com/users/curita/followers","following_url":"https://api.github.com/users/curita/following{/other_user}","gists_url":"https://api.github.com/users/curita/gists{/gist_id}","starred_url":"https://api.github.com/users/curita/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/curita/subscriptions","organizations_url":"https://api.github.com/users/curita/orgs","repos_url":"https://api.github.com/users/curita/repos","events_url":"https://api.github.com/users/curita/events{/privacy}","received_events_url":"https://api.github.com/users/curita/received_events","type":"User","site_admin":false},"created_at":"2015-11-11T06:14:51Z","updated_at":"2015-11-11T06:14:51Z","body":"Not sure what's the long term plan for *_BASE settings now, but if we plan to keep them for good we could implement `DEFAULT_REQUEST_HEADERS_BASE` as Paul suggested."}}