{"action":"created","issue":{"url":"https://api.github.com/repos/ckan/ckan/issues/2261","labels_url":"https://api.github.com/repos/ckan/ckan/issues/2261/labels{/name}","comments_url":"https://api.github.com/repos/ckan/ckan/issues/2261/comments","events_url":"https://api.github.com/repos/ckan/ckan/issues/2261/events","html_url":"https://github.com/ckan/ckan/issues/2261","id":56359945,"number":2261,"title":"Index packages of all status (except 'deleted') not just the 'active' ones","user":{"login":"clementmouchet","id":1419916,"avatar_url":"https://avatars.githubusercontent.com/u/1419916?v=3","gravatar_id":"","url":"https://api.github.com/users/clementmouchet","html_url":"https://github.com/clementmouchet","followers_url":"https://api.github.com/users/clementmouchet/followers","following_url":"https://api.github.com/users/clementmouchet/following{/other_user}","gists_url":"https://api.github.com/users/clementmouchet/gists{/gist_id}","starred_url":"https://api.github.com/users/clementmouchet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clementmouchet/subscriptions","organizations_url":"https://api.github.com/users/clementmouchet/orgs","repos_url":"https://api.github.com/users/clementmouchet/repos","events_url":"https://api.github.com/users/clementmouchet/events{/privacy}","received_events_url":"https://api.github.com/users/clementmouchet/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"wardi","id":153258,"avatar_url":"https://avatars.githubusercontent.com/u/153258?v=3","gravatar_id":"","url":"https://api.github.com/users/wardi","html_url":"https://github.com/wardi","followers_url":"https://api.github.com/users/wardi/followers","following_url":"https://api.github.com/users/wardi/following{/other_user}","gists_url":"https://api.github.com/users/wardi/gists{/gist_id}","starred_url":"https://api.github.com/users/wardi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wardi/subscriptions","organizations_url":"https://api.github.com/users/wardi/orgs","repos_url":"https://api.github.com/users/wardi/repos","events_url":"https://api.github.com/users/wardi/events{/privacy}","received_events_url":"https://api.github.com/users/wardi/received_events","type":"User","site_admin":false},"milestone":null,"comments":1,"created_at":"2015-02-03T10:33:16Z","updated_at":"2015-02-03T13:41:11Z","closed_at":null,"body":"Hi all,\r\n\r\nI wonder if there is a reason for indexing only active packages?\r\nI understand that there might be a performance overhead from indexing drafts & custom states but I think this would be very handy in many cases. (@benjaminlaot, @andreivicentiuhincu and I are using custom states for packages to enable a sort of validation workflow)\r\n\r\nHaving the ability to do package search using the index to get drafts & custom states would be better than just active ones. \r\n\r\nI played with it very briefly and changed the `state == 'active'` to `state != 'deleted'` in the two places below.\r\n\r\n``` diff\r\ndiff --git a/ckan/lib/search/__init__.py b/ckan/lib/search/__init__.py\r\nindex d4cb59e..ea28b04 100644\r\n--- a/ckan/lib/search/__init__.py\r\n+++ b/ckan/lib/search/__init__.py\r\n@@ -165,7 +165,7 @@ def rebuild(package_id=None, only_missing=False, force=False, refresh=False, def\r\n             package_index.update_dict(pkg_dict, True)\r\n     else:\r\n         package_ids = [r[0] for r in model.Session.query(model.Package.id).\r\n-                       filter(model.Package.state == 'active').all()]\r\n+                       filter(model.Package.state != 'deleted').all()]\r\n         if only_missing:\r\n             log.info('Indexing only missing packages...')\r\n             package_query = query_for(model.Package)\r\ndiff --git a/ckan/lib/search/index.py b/ckan/lib/search/index.py\r\nindex 3acea3c..f4f7506 100644\r\n--- a/ckan/lib/search/index.py\r\n+++ b/ckan/lib/search/index.py\r\n@@ -126,7 +126,7 @@ class PackageSearchIndex(SearchIndex):\r\n         if title:\r\n             pkg_dict['title_string'] = title\r\n \r\n-        if (not pkg_dict.get('state')) or ('active' not in pkg_dict.get('state')):\r\n+        if (not pkg_dict.get('state')) or ('deleted' in pkg_dict.get('state')):\r\n             return self.delete_package(pkg_dict)\r\n \r\n         index_fields = RESERVED_FIELDS + pkg_dict.keys()\r\n```\r\n\r\nIt doesn't break existing behaviour because all the places using package search filter the index by `state == 'active'` anyway.\r\n\r\nWhat do you think? \r\nHow does that fit with your road map? \r\nWould you take a PR that allows indexing other states as above, or using config options?\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/ckan/ckan/issues/comments/72651604","html_url":"https://github.com/ckan/ckan/issues/2261#issuecomment-72651604","issue_url":"https://api.github.com/repos/ckan/ckan/issues/2261","id":72651604,"user":{"login":"wardi","id":153258,"avatar_url":"https://avatars.githubusercontent.com/u/153258?v=3","gravatar_id":"","url":"https://api.github.com/users/wardi","html_url":"https://github.com/wardi","followers_url":"https://api.github.com/users/wardi/followers","following_url":"https://api.github.com/users/wardi/following{/other_user}","gists_url":"https://api.github.com/users/wardi/gists{/gist_id}","starred_url":"https://api.github.com/users/wardi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wardi/subscriptions","organizations_url":"https://api.github.com/users/wardi/orgs","repos_url":"https://api.github.com/users/wardi/repos","events_url":"https://api.github.com/users/wardi/events{/privacy}","received_events_url":"https://api.github.com/users/wardi/received_events","type":"User","site_admin":false},"created_at":"2015-02-03T13:41:11Z","updated_at":"2015-02-03T13:41:11Z","body":"We were just discussing this one on the developer call. My biggest concern is this changes the behaviour of draft datasets. Currently draft datasets aren't indexed in solr and are visible only to their owner. My second concern is that your custom states might overlap with the way CKAN is already using the state field - why wouldn't you want draft or deleted versions of datasets in custom states?\r\n\r\nI suggest using an extra field to store your state value instead of the one ckan uses.\r\n\r\nWould you tell us what your state values and their meanings are?\r\n\r\nThis might also be related to the workflow ideas tickets like https://github.com/ckan/ideas-and-roadmap/issues/108 consider contributing your use cases to the discussion there."}}