{"action":"created","issue":{"url":"https://api.github.com/repos/ckan/ckanext-harvest/issues/162","labels_url":"https://api.github.com/repos/ckan/ckanext-harvest/issues/162/labels{/name}","comments_url":"https://api.github.com/repos/ckan/ckanext-harvest/issues/162/comments","events_url":"https://api.github.com/repos/ckan/ckanext-harvest/issues/162/events","html_url":"https://github.com/ckan/ckanext-harvest/issues/162","id":111925048,"number":162,"title":"Hanging job / database error","user":{"login":"neothemachine","id":530988,"avatar_url":"https://avatars.githubusercontent.com/u/530988?v=3","gravatar_id":"","url":"https://api.github.com/users/neothemachine","html_url":"https://github.com/neothemachine","followers_url":"https://api.github.com/users/neothemachine/followers","following_url":"https://api.github.com/users/neothemachine/following{/other_user}","gists_url":"https://api.github.com/users/neothemachine/gists{/gist_id}","starred_url":"https://api.github.com/users/neothemachine/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/neothemachine/subscriptions","organizations_url":"https://api.github.com/users/neothemachine/orgs","repos_url":"https://api.github.com/users/neothemachine/repos","events_url":"https://api.github.com/users/neothemachine/events{/privacy}","received_events_url":"https://api.github.com/users/neothemachine/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-10-16T22:48:49Z","updated_at":"2015-10-17T21:33:44Z","closed_at":null,"body":"I just had a job stuck in running, so I wanted to clear everything to rerun it. By the way, the data got successfully imported, but for some reason the job didn't finish properly. This is probably connected to the error I get when I press \"Clear\" to redo it:\r\n\r\nAn error occurred: [(IntegrityError) update or delete on table \"package\" violates foreign key constraint \"harvest_object_package_id_fkey\" on table \"harvest_object\" DETAIL: Key (id)=(1e85a8ef-efcc-4fa1-a40a-3cc1bec5c8bc) is still referenced from table \"harvest_object\". \r\n\r\n```sql\r\nbegin; \r\nupdate package set state = 'to_delete' where id in (select package_id from harvest_object where harvest_source_id = 'fd15d7cd-b576-4577-acbb-7246eee9b41e');\r\ndelete from resource_view where resource_id in (select id from resource where package_id in (select id from package where state = 'to_delete' ));\r\ndelete from resource_revision where package_id in (select id from package where state = 'to_delete' );\r\ndelete from resource where package_id in (select id from package where state = 'to_delete' );    \r\ndelete from harvest_object_error where harvest_object_id in (select id from harvest_object where harvest_source_id = 'fd15d7cd-b576-4577-acbb-7246eee9b41e');\r\ndelete from harvest_object_extra where harvest_object_id in (select id from harvest_object where harvest_source_id = 'fd15d7cd-b576-4577-acbb-7246eee9b41e');\r\ndelete from harvest_object where harvest_source_id = 'fd15d7cd-b576-4577-acbb-7246eee9b41e';\r\ndelete from harvest_gather_error where harvest_job_id in (select id from harvest_job where source_id = 'fd15d7cd-b576-4577-acbb-7246eee9b41e');\r\ndelete from harvest_job where source_id = 'fd15d7cd-b576-4577-acbb-7246eee9b41e';\r\ndelete from package_role where package_id in (select id from package where state = 'to_delete' );\r\ndelete from user_object_role where id not in (select user_object_role_id from package_role) and context = 'Package';\r\ndelete from package_tag_revision where package_id in (select id from package where state = 'to_delete');\r\ndelete from member_revision where table_id in (select id from package where state = 'to_delete');\r\ndelete from package_extra_revision where package_id in (select id from package where state = 'to_delete');\r\ndelete from package_revision where id in (select id from package where state = 'to_delete');\r\ndelete from package_tag where package_id in (select id from package where state = 'to_delete');\r\ndelete from package_extra where package_id in (select id from package where state = 'to_delete');\r\ndelete from package_relationship_revision where subject_package_id in (select id from package where state = 'to_delete');\r\ndelete from package_relationship_revision where object_package_id in (select id from package where state = 'to_delete');\r\ndelete from package_relationship where subject_package_id in (select id from package where state = 'to_delete');\r\ndelete from package_relationship where object_package_id in (select id from package where state = 'to_delete');\r\ndelete from member where table_id in (select id from package where state = 'to_delete');\r\ndelete from related_dataset where dataset_id in (select id from package where state = 'to_delete');\r\ndelete from related where id in ('');\r\ndelete from package where id in (select id from package where state = 'to_delete');\r\ncommit;\r\n\r\n```\r\n\r\nIt's a fresh CKAN install / server and I'm just doing some first testing, but this doesn't look too promising. Any idea?"},"comment":{"url":"https://api.github.com/repos/ckan/ckanext-harvest/issues/comments/148953607","html_url":"https://github.com/ckan/ckanext-harvest/issues/162#issuecomment-148953607","issue_url":"https://api.github.com/repos/ckan/ckanext-harvest/issues/162","id":148953607,"user":{"login":"neothemachine","id":530988,"avatar_url":"https://avatars.githubusercontent.com/u/530988?v=3","gravatar_id":"","url":"https://api.github.com/users/neothemachine","html_url":"https://github.com/neothemachine","followers_url":"https://api.github.com/users/neothemachine/followers","following_url":"https://api.github.com/users/neothemachine/following{/other_user}","gists_url":"https://api.github.com/users/neothemachine/gists{/gist_id}","starred_url":"https://api.github.com/users/neothemachine/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/neothemachine/subscriptions","organizations_url":"https://api.github.com/users/neothemachine/orgs","repos_url":"https://api.github.com/users/neothemachine/repos","events_url":"https://api.github.com/users/neothemachine/events{/privacy}","received_events_url":"https://api.github.com/users/neothemachine/received_events","type":"User","site_admin":false},"created_at":"2015-10-17T21:33:43Z","updated_at":"2015-10-17T21:33:43Z","body":"Well, I certainly can try to debug it, but my first results worry me a bit:\r\n\r\n```\r\nselect harvest_source_id, package_id from harvest_object;\r\n          harvest_source_id           |              package_id\r\n--------------------------------------+--------------------------------------\r\n fd15d7cd-b576-4577-acbb-7246eee9b41e | b37d3c77-354a-40ab-aae4-1b43e20b12de\r\n 041fbcb6-fb0d-4488-bbaa-b5baa674fe8e | 1e85a8ef-efcc-4fa1-a40a-3cc1bec5c8bc\r\n fd15d7cd-b576-4577-acbb-7246eee9b41e |\r\n fd15d7cd-b576-4577-acbb-7246eee9b41e | 1e85a8ef-efcc-4fa1-a40a-3cc1bec5c8bc\r\n fd15d7cd-b576-4577-acbb-7246eee9b41e | b37d3c77-354a-40ab-aae4-1b43e20b12de\r\n fd15d7cd-b576-4577-acbb-7246eee9b41e | 1e85a8ef-efcc-4fa1-a40a-3cc1bec5c8bc\r\n fd15d7cd-b576-4577-acbb-7246eee9b41e | b37d3c77-354a-40ab-aae4-1b43e20b12de\r\n(7 rows)\r\n```\r\n\r\nTwo different harvest_source_id's reference the same package with id \"1e85a8ef-efcc-4fa1-a40a-3cc1bec5c8bc\". How can this happen? Also, in one case the package_id is empty, not sure if that's supposed to be like that."}}