{"action":"created","comment":{"url":"https://api.github.com/repos/sckott/oai/pulls/comments/41329987","id":41329987,"diff_hunk":"@@ -0,0 +1,67 @@\n+# Read XML while fishing-out some errors\n+\n+# Function `tryCatch` looks at conditions thrown by `read_xml`. If the condition\n+# is of class \"Rcpp::exception\" a handler function `handle_rcpp_exception` is\n+# executed. This function looks at the message of the original condition. If it\n+# looks like \"PCDATA invalid Char value\" it extracts the value of the invalid\n+# character and the error number (in square brackets in the original message). A\n+# new condition object is created of class `\"invalid_char_value\"` which extends\n+# the original class `\"Rcpp::exception\"`. Extracted offending character value\n+# and error number are added as attributes to the new condition. Finally, the\n+# condition is signalled.\n+#\n+# Conditions not caught by `tryCatch`, so all others apart from\n+# `\"Rcpp::exception\"`s, are signaled as usual.\n+#\n+# This allows for special handling of different conditions somewhere else\n+\n+read_xml_with_errors <- function(x, ...) {\n+  # Handle Rcpp exceptions,\n+  # which include libxml errors\n+  handle_rcpp_exception <- function(cond) {\n+    # better error message\n+    cond$message <- paste(\"xml2::read_xml says:\", cond$message)\n+    # enhance for selected errors\n+    if(grepl(\"PCDATA invalid Char value\", cond$message)) {\n+      cond <- condition(\n+        subclass=c(\"invalid_char_value\", class(cond)),\n+        message = cond$message,\n+        call=cond$call,\n+        error_no = as.numeric(stringr::str_extract(cond$message, \"(?<=\\\\[)[0-9]+(?=\\\\]$)\" )),\n+        char_value = as.numeric(stringr::str_extract(cond$message, \"(?<=value )[0-9]+(?= \\\\[)\" ))\n+      )\n+    }\n+    stop(cond)\n+  }\n+\n+  # Catch!\n+  tryCatch( xml2::read_xml(x, ...),\n+            \"Rcpp::exception\" = handle_rcpp_exception\n+  )\n+}\n+\n+\n+\n+\n+# Read XML safely\n+#\n+# Removes illegal characters\n+# @param invalid_as NA or character. If not NA, replace invalid characters with `invalid_as`\n+read_xml_safely <- function(x, ..., xml_invalid_as=getOption(\"oai.xml_invalid_as\", \"\") ) {","path":"R/read_xml_safely.R","position":50,"original_position":50,"commit_id":"72bf319a9cbc5c275df9cb322dedf61cb264ea84","original_commit_id":"72bf319a9cbc5c275df9cb322dedf61cb264ea84","user":{"login":"mbojan","id":636574,"avatar_url":"https://avatars.githubusercontent.com/u/636574?v=3","gravatar_id":"","url":"https://api.github.com/users/mbojan","html_url":"https://github.com/mbojan","followers_url":"https://api.github.com/users/mbojan/followers","following_url":"https://api.github.com/users/mbojan/following{/other_user}","gists_url":"https://api.github.com/users/mbojan/gists{/gist_id}","starred_url":"https://api.github.com/users/mbojan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbojan/subscriptions","organizations_url":"https://api.github.com/users/mbojan/orgs","repos_url":"https://api.github.com/users/mbojan/repos","events_url":"https://api.github.com/users/mbojan/events{/privacy}","received_events_url":"https://api.github.com/users/mbojan/received_events","type":"User","site_admin":false},"body":"Yes. There is a default value which can be changed by setting the option.","created_at":"2015-10-06T22:16:14Z","updated_at":"2015-10-06T22:16:14Z","html_url":"https://github.com/sckott/oai/pull/27#discussion_r41329987","pull_request_url":"https://api.github.com/repos/sckott/oai/pulls/27","_links":{"self":{"href":"https://api.github.com/repos/sckott/oai/pulls/comments/41329987"},"html":{"href":"https://github.com/sckott/oai/pull/27#discussion_r41329987"},"pull_request":{"href":"https://api.github.com/repos/sckott/oai/pulls/27"}}},"pull_request":{"url":"https://api.github.com/repos/sckott/oai/pulls/27","id":46884994,"html_url":"https://github.com/sckott/oai/pull/27","diff_url":"https://github.com/sckott/oai/pull/27.diff","patch_url":"https://github.com/sckott/oai/pull/27.patch","issue_url":"https://api.github.com/repos/sckott/oai/issues/27","number":27,"state":"open","locked":false,"title":"Handling XML faults","user":{"login":"mbojan","id":636574,"avatar_url":"https://avatars.githubusercontent.com/u/636574?v=3","gravatar_id":"","url":"https://api.github.com/users/mbojan","html_url":"https://github.com/mbojan","followers_url":"https://api.github.com/users/mbojan/followers","following_url":"https://api.github.com/users/mbojan/following{/other_user}","gists_url":"https://api.github.com/users/mbojan/gists{/gist_id}","starred_url":"https://api.github.com/users/mbojan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbojan/subscriptions","organizations_url":"https://api.github.com/users/mbojan/orgs","repos_url":"https://api.github.com/users/mbojan/repos","events_url":"https://api.github.com/users/mbojan/events{/privacy}","received_events_url":"https://api.github.com/users/mbojan/received_events","type":"User","site_admin":false},"body":"Addresses #24 .\r\n\r\nIn short the flow is to:\r\n\r\n- try to parse with `read_xml`\r\n- if that fails, try `read_html`\r\n- `resumptionToken` is extracted in both cases above\r\n- if `as=\"raw\"` proceed as usual in both cases\r\n- if `as != \"raw\"` save raw XML and proceed with the token (if any)\r\n\r\nIt is getting complicated and I think we should use some package options. In this PR I have already used one in `read_xml_safely`. We might use `options` or have some object stored in package namespace (like in knitr `opts_chunk` etc.)\r\n\r\nAt this moment this is used only in `while_oai` and functions that use it. `read_xml_safely` could be called  instead of `read_xml` in other functions too.","created_at":"2015-10-06T10:20:49Z","updated_at":"2015-10-06T22:16:15Z","closed_at":null,"merged_at":null,"merge_commit_sha":"04adcf24ff6ac940cf23ae5e7aa5b73d829e312e","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/sckott/oai/pulls/27/commits","review_comments_url":"https://api.github.com/repos/sckott/oai/pulls/27/comments","review_comment_url":"https://api.github.com/repos/sckott/oai/pulls/comments{/number}","comments_url":"https://api.github.com/repos/sckott/oai/issues/27/comments","statuses_url":"https://api.github.com/repos/sckott/oai/statuses/72bf319a9cbc5c275df9cb322dedf61cb264ea84","head":{"label":"mbojan:xml_errors","ref":"xml_errors","sha":"72bf319a9cbc5c275df9cb322dedf61cb264ea84","user":{"login":"mbojan","id":636574,"avatar_url":"https://avatars.githubusercontent.com/u/636574?v=3","gravatar_id":"","url":"https://api.github.com/users/mbojan","html_url":"https://github.com/mbojan","followers_url":"https://api.github.com/users/mbojan/followers","following_url":"https://api.github.com/users/mbojan/following{/other_user}","gists_url":"https://api.github.com/users/mbojan/gists{/gist_id}","starred_url":"https://api.github.com/users/mbojan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbojan/subscriptions","organizations_url":"https://api.github.com/users/mbojan/orgs","repos_url":"https://api.github.com/users/mbojan/repos","events_url":"https://api.github.com/users/mbojan/events{/privacy}","received_events_url":"https://api.github.com/users/mbojan/received_events","type":"User","site_admin":false},"repo":{"id":42313978,"name":"oai","full_name":"mbojan/oai","owner":{"login":"mbojan","id":636574,"avatar_url":"https://avatars.githubusercontent.com/u/636574?v=3","gravatar_id":"","url":"https://api.github.com/users/mbojan","html_url":"https://github.com/mbojan","followers_url":"https://api.github.com/users/mbojan/followers","following_url":"https://api.github.com/users/mbojan/following{/other_user}","gists_url":"https://api.github.com/users/mbojan/gists{/gist_id}","starred_url":"https://api.github.com/users/mbojan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbojan/subscriptions","organizations_url":"https://api.github.com/users/mbojan/orgs","repos_url":"https://api.github.com/users/mbojan/repos","events_url":"https://api.github.com/users/mbojan/events{/privacy}","received_events_url":"https://api.github.com/users/mbojan/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/mbojan/oai","description":"OAI-PMH client","fork":true,"url":"https://api.github.com/repos/mbojan/oai","forks_url":"https://api.github.com/repos/mbojan/oai/forks","keys_url":"https://api.github.com/repos/mbojan/oai/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mbojan/oai/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mbojan/oai/teams","hooks_url":"https://api.github.com/repos/mbojan/oai/hooks","issue_events_url":"https://api.github.com/repos/mbojan/oai/issues/events{/number}","events_url":"https://api.github.com/repos/mbojan/oai/events","assignees_url":"https://api.github.com/repos/mbojan/oai/assignees{/user}","branches_url":"https://api.github.com/repos/mbojan/oai/branches{/branch}","tags_url":"https://api.github.com/repos/mbojan/oai/tags","blobs_url":"https://api.github.com/repos/mbojan/oai/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mbojan/oai/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mbojan/oai/git/refs{/sha}","trees_url":"https://api.github.com/repos/mbojan/oai/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mbojan/oai/statuses/{sha}","languages_url":"https://api.github.com/repos/mbojan/oai/languages","stargazers_url":"https://api.github.com/repos/mbojan/oai/stargazers","contributors_url":"https://api.github.com/repos/mbojan/oai/contributors","subscribers_url":"https://api.github.com/repos/mbojan/oai/subscribers","subscription_url":"https://api.github.com/repos/mbojan/oai/subscription","commits_url":"https://api.github.com/repos/mbojan/oai/commits{/sha}","git_commits_url":"https://api.github.com/repos/mbojan/oai/git/commits{/sha}","comments_url":"https://api.github.com/repos/mbojan/oai/comments{/number}","issue_comment_url":"https://api.github.com/repos/mbojan/oai/issues/comments{/number}","contents_url":"https://api.github.com/repos/mbojan/oai/contents/{+path}","compare_url":"https://api.github.com/repos/mbojan/oai/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mbojan/oai/merges","archive_url":"https://api.github.com/repos/mbojan/oai/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mbojan/oai/downloads","issues_url":"https://api.github.com/repos/mbojan/oai/issues{/number}","pulls_url":"https://api.github.com/repos/mbojan/oai/pulls{/number}","milestones_url":"https://api.github.com/repos/mbojan/oai/milestones{/number}","notifications_url":"https://api.github.com/repos/mbojan/oai/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mbojan/oai/labels{/name}","releases_url":"https://api.github.com/repos/mbojan/oai/releases{/id}","created_at":"2015-09-11T14:42:33Z","updated_at":"2015-09-24T14:47:33Z","pushed_at":"2015-10-06T13:07:16Z","git_url":"git://github.com/mbojan/oai.git","ssh_url":"git@github.com:mbojan/oai.git","clone_url":"https://github.com/mbojan/oai.git","svn_url":"https://github.com/mbojan/oai","homepage":"","size":639,"stargazers_count":0,"watchers_count":0,"language":"R","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"sckott:master","ref":"master","sha":"94d1d3c013121ce639d27f4afcc9a4952fa0cb48","user":{"login":"sckott","id":577668,"avatar_url":"https://avatars.githubusercontent.com/u/577668?v=3","gravatar_id":"","url":"https://api.github.com/users/sckott","html_url":"https://github.com/sckott","followers_url":"https://api.github.com/users/sckott/followers","following_url":"https://api.github.com/users/sckott/following{/other_user}","gists_url":"https://api.github.com/users/sckott/gists{/gist_id}","starred_url":"https://api.github.com/users/sckott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sckott/subscriptions","organizations_url":"https://api.github.com/users/sckott/orgs","repos_url":"https://api.github.com/users/sckott/repos","events_url":"https://api.github.com/users/sckott/events{/privacy}","received_events_url":"https://api.github.com/users/sckott/received_events","type":"User","site_admin":false},"repo":{"id":37342428,"name":"oai","full_name":"sckott/oai","owner":{"login":"sckott","id":577668,"avatar_url":"https://avatars.githubusercontent.com/u/577668?v=3","gravatar_id":"","url":"https://api.github.com/users/sckott","html_url":"https://github.com/sckott","followers_url":"https://api.github.com/users/sckott/followers","following_url":"https://api.github.com/users/sckott/following{/other_user}","gists_url":"https://api.github.com/users/sckott/gists{/gist_id}","starred_url":"https://api.github.com/users/sckott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sckott/subscriptions","organizations_url":"https://api.github.com/users/sckott/orgs","repos_url":"https://api.github.com/users/sckott/repos","events_url":"https://api.github.com/users/sckott/events{/privacy}","received_events_url":"https://api.github.com/users/sckott/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/sckott/oai","description":"OAI-PMH client","fork":false,"url":"https://api.github.com/repos/sckott/oai","forks_url":"https://api.github.com/repos/sckott/oai/forks","keys_url":"https://api.github.com/repos/sckott/oai/keys{/key_id}","collaborators_url":"https://api.github.com/repos/sckott/oai/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/sckott/oai/teams","hooks_url":"https://api.github.com/repos/sckott/oai/hooks","issue_events_url":"https://api.github.com/repos/sckott/oai/issues/events{/number}","events_url":"https://api.github.com/repos/sckott/oai/events","assignees_url":"https://api.github.com/repos/sckott/oai/assignees{/user}","branches_url":"https://api.github.com/repos/sckott/oai/branches{/branch}","tags_url":"https://api.github.com/repos/sckott/oai/tags","blobs_url":"https://api.github.com/repos/sckott/oai/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/sckott/oai/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/sckott/oai/git/refs{/sha}","trees_url":"https://api.github.com/repos/sckott/oai/git/trees{/sha}","statuses_url":"https://api.github.com/repos/sckott/oai/statuses/{sha}","languages_url":"https://api.github.com/repos/sckott/oai/languages","stargazers_url":"https://api.github.com/repos/sckott/oai/stargazers","contributors_url":"https://api.github.com/repos/sckott/oai/contributors","subscribers_url":"https://api.github.com/repos/sckott/oai/subscribers","subscription_url":"https://api.github.com/repos/sckott/oai/subscription","commits_url":"https://api.github.com/repos/sckott/oai/commits{/sha}","git_commits_url":"https://api.github.com/repos/sckott/oai/git/commits{/sha}","comments_url":"https://api.github.com/repos/sckott/oai/comments{/number}","issue_comment_url":"https://api.github.com/repos/sckott/oai/issues/comments{/number}","contents_url":"https://api.github.com/repos/sckott/oai/contents/{+path}","compare_url":"https://api.github.com/repos/sckott/oai/compare/{base}...{head}","merges_url":"https://api.github.com/repos/sckott/oai/merges","archive_url":"https://api.github.com/repos/sckott/oai/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/sckott/oai/downloads","issues_url":"https://api.github.com/repos/sckott/oai/issues{/number}","pulls_url":"https://api.github.com/repos/sckott/oai/pulls{/number}","milestones_url":"https://api.github.com/repos/sckott/oai/milestones{/number}","notifications_url":"https://api.github.com/repos/sckott/oai/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/sckott/oai/labels{/name}","releases_url":"https://api.github.com/repos/sckott/oai/releases{/id}","created_at":"2015-06-12T20:14:33Z","updated_at":"2015-09-11T14:10:28Z","pushed_at":"2015-10-06T10:20:49Z","git_url":"git://github.com/sckott/oai.git","ssh_url":"git@github.com:sckott/oai.git","clone_url":"https://github.com/sckott/oai.git","svn_url":"https://github.com/sckott/oai","homepage":"","size":485,"stargazers_count":3,"watchers_count":3,"language":"R","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":6,"forks":1,"open_issues":6,"watchers":3,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/sckott/oai/pulls/27"},"html":{"href":"https://github.com/sckott/oai/pull/27"},"issue":{"href":"https://api.github.com/repos/sckott/oai/issues/27"},"comments":{"href":"https://api.github.com/repos/sckott/oai/issues/27/comments"},"review_comments":{"href":"https://api.github.com/repos/sckott/oai/pulls/27/comments"},"review_comment":{"href":"https://api.github.com/repos/sckott/oai/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/sckott/oai/pulls/27/commits"},"statuses":{"href":"https://api.github.com/repos/sckott/oai/statuses/72bf319a9cbc5c275df9cb322dedf61cb264ea84"}}}}