{"action":"created","issue":{"url":"https://api.github.com/repos/Wikidata/Wikidata-Toolkit/issues/121","labels_url":"https://api.github.com/repos/Wikidata/Wikidata-Toolkit/issues/121/labels{/name}","comments_url":"https://api.github.com/repos/Wikidata/Wikidata-Toolkit/issues/121/comments","events_url":"https://api.github.com/repos/Wikidata/Wikidata-Toolkit/issues/121/events","html_url":"https://github.com/Wikidata/Wikidata-Toolkit/issues/121","id":58517224,"number":121,"title":"WARNING: org.json.JSONException: JSONObject[\"claims\"] is not a JSONArray.","user":{"login":"alismayilov","id":1295876,"avatar_url":"https://avatars.githubusercontent.com/u/1295876?v=3","gravatar_id":"","url":"https://api.github.com/users/alismayilov","html_url":"https://github.com/alismayilov","followers_url":"https://api.github.com/users/alismayilov/followers","following_url":"https://api.github.com/users/alismayilov/following{/other_user}","gists_url":"https://api.github.com/users/alismayilov/gists{/gist_id}","starred_url":"https://api.github.com/users/alismayilov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alismayilov/subscriptions","organizations_url":"https://api.github.com/users/alismayilov/orgs","repos_url":"https://api.github.com/users/alismayilov/repos","events_url":"https://api.github.com/users/alismayilov/events{/privacy}","received_events_url":"https://api.github.com/users/alismayilov/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-02-22T20:32:26Z","updated_at":"2015-02-24T16:35:18Z","closed_at":null,"body":"We use wikidata toolkit for wikidata extraction. It works fine for most of the pages. However, for some pages, I got \"org.json.JSONException: JSONObject[\"claims\"] is not a JSONArray.\" error. I don't get this error for very old dumps (August - 2014). I got it when I use newer dumps. "},"comment":{"url":"https://api.github.com/repos/Wikidata/Wikidata-Toolkit/issues/comments/75791345","html_url":"https://github.com/Wikidata/Wikidata-Toolkit/issues/121#issuecomment-75791345","issue_url":"https://api.github.com/repos/Wikidata/Wikidata-Toolkit/issues/121","id":75791345,"user":{"login":"mkroetzsch","id":110881,"avatar_url":"https://avatars.githubusercontent.com/u/110881?v=3","gravatar_id":"","url":"https://api.github.com/users/mkroetzsch","html_url":"https://github.com/mkroetzsch","followers_url":"https://api.github.com/users/mkroetzsch/followers","following_url":"https://api.github.com/users/mkroetzsch/following{/other_user}","gists_url":"https://api.github.com/users/mkroetzsch/gists{/gist_id}","starred_url":"https://api.github.com/users/mkroetzsch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mkroetzsch/subscriptions","organizations_url":"https://api.github.com/users/mkroetzsch/orgs","repos_url":"https://api.github.com/users/mkroetzsch/repos","events_url":"https://api.github.com/users/mkroetzsch/events{/privacy}","received_events_url":"https://api.github.com/users/mkroetzsch/received_events","type":"User","site_admin":false},"created_at":"2015-02-24T16:35:18Z","updated_at":"2015-02-24T16:35:18Z","body":"So do we agree now that this is simply malformed JSON content, and that WDTK correctly recognizes this error and rejects the broken data as it should? I don't think it would make sense for us to work around such errors if they are simply known Wikibase bugs that are expected to be fixed sometime soon (fingers crossed). If this is the conclusion, I suggest we simply close this issue (and ideally also link to the relevant Wikibase bug that can be watched).\r\n\r\n> @mkroetzsch What do you think about automating the recognition of such problems and fixing them on the fly to imporove robustness?\r\n\r\nOur regular JSON file processor already is robust to arbitrary JSON errors in the sense that it will only ignore the broken item and continue with the remaining data without crashing. Adding special handling to fix individual glitches seems not the best use of our resources unless the errors become so frequent again that it really becomes an issue. Last time I checked, such problems affected less than 10 out of 16 million items, which seems acceptable, esp. considering the fact that affected items usually had zero claims (so are irrelevant for all processing steps that look at the claims).\r\n"}}