{"action":"created","comment":{"url":"https://api.github.com/repos/JuliaLang/julia/pulls/comments/31625812","id":31625812,"diff_hunk":"@@ -0,0 +1,51 @@\n+# This file is a part of Julia. License is MIT: http://julialang.org/license\n+\n+#=\n+@doc \"\"\"\n+@brief      Error messages for Unicode / UTF support\n+\"\"\" ->\n+=#\n+\n+const UTF_ERR_SHORT = 1\n+const UTF_ERR_CONT  = 2\n+const UTF_ERR_LONG  = 3\n+const UTF_ERR_NOT_LEAD = 4\n+const UTF_ERR_NOT_TRAIL = 5\n+const UTF_ERR_NOT_SURROGATE = 6\n+const UTF_ERR_MISSING_SURROGATE = 7\n+const UTF_ERR_INVALID = 8\n+const UTF_ERR_SURROGATE = 9\n+const UTF_ERR_NULL_16_TERMINATE = 10\n+const UTF_ERR_NULL_32_TERMINATE = 11\n+const UTF_ERR_MAX = 11\n+\n+const errMsgs = [\n+    \"invalid UTF-8 sequence starting at index <<1>> (0x<<2>>) missing one or more continuation bytes)\",\n+    \"invalid UTF-8 sequence starting at index <<1>> (0x<<2>> is not a continuation byte)\",\n+    \"invalid UTF-8 sequence, overlong encoding starting at index <<1>> (0x<<2>>)\",\n+    \"not a leading Unicode surrogate code unit at index <<1>> (0x<<2>>)\",\n+    \"not a trailing Unicode surrogate code unit at index <<1>> (0x<<2>>)\",\n+    \"not a valid Unicode surrogate code unit at index <<1>> (0x<<2>>\",\n+    \"missing trailing Unicode surrogate code unit after index <<1>> (0x<<2>>)\",\n+    \"invalid Unicode character starting at index <<1>> (0x<<2>> > 0x10ffff)\",\n+    \"surrogate encoding not allowed in UTF-8 or UTF-32, at index <<1>> (0x<<2>>)\",\n+    \"UTF16String data must be NULL-terminated\",\n+    \"UTF32String data must be NULL-terminated\"\n+]\n+#=\n+@doc \"\"\"\n+@brief      Throws ArgumentError with information about the specific error, location, and character\n+\n+@param[in]  errcode Error code for Unicode error (one of UTF_ERR_*)\n+@param[in]  charpos Index of invalid byte or character\n+@param[in]  invchar Invalid byte or character\n+\n+@throws never returns, always throws ArgumentError\n+\"\"\" ->\n+=#\n+function utf_errfunc(errcode::Integer, charpos, invchar)","path":"base/utferror.jl","position":null,"original_position":46,"commit_id":"58cc0268f3ac3abda740604f8e098f82dd79af28","original_commit_id":"0b6619a6034a9890a84f4e22fe5ed3cb1bf9874b","user":{"login":"ScottPJones","id":11798751,"avatar_url":"https://avatars.githubusercontent.com/u/11798751?v=3","gravatar_id":"","url":"https://api.github.com/users/ScottPJones","html_url":"https://github.com/ScottPJones","followers_url":"https://api.github.com/users/ScottPJones/followers","following_url":"https://api.github.com/users/ScottPJones/following{/other_user}","gists_url":"https://api.github.com/users/ScottPJones/gists{/gist_id}","starred_url":"https://api.github.com/users/ScottPJones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ScottPJones/subscriptions","organizations_url":"https://api.github.com/users/ScottPJones/orgs","repos_url":"https://api.github.com/users/ScottPJones/repos","events_url":"https://api.github.com/users/ScottPJones/events{/privacy}","received_events_url":"https://api.github.com/users/ScottPJones/received_events","type":"User","site_admin":false},"body":"About the micro benchmarking... I know that the way it just runs things and uses `@time` and everything is really crappy... I really haven't had time to learn how best to collect all of the information that I wish to and save it for later processing... I miss having a high speed database always available just by adding a `^` character to the associative array!  (my changes to gc.c that already got merged actually help collect the data, but storing and retrieving)\r\nAny pointers to how better to benchmark things like this, and produce a nice spreadsheet or graph in julia, would be *greatly* appreciated!","created_at":"2015-06-03T14:09:36Z","updated_at":"2015-06-03T14:09:36Z","html_url":"https://github.com/JuliaLang/julia/pull/11551#discussion_r31625812","pull_request_url":"https://api.github.com/repos/JuliaLang/julia/pulls/11551","_links":{"self":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/comments/31625812"},"html":{"href":"https://github.com/JuliaLang/julia/pull/11551#discussion_r31625812"},"pull_request":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/11551"}}},"pull_request":{"url":"https://api.github.com/repos/JuliaLang/julia/pulls/11551","id":36790807,"html_url":"https://github.com/JuliaLang/julia/pull/11551","diff_url":"https://github.com/JuliaLang/julia/pull/11551.diff","patch_url":"https://github.com/JuliaLang/julia/pull/11551.patch","issue_url":"https://api.github.com/repos/JuliaLang/julia/issues/11551","number":11551,"state":"open","locked":false,"title":"Fix #10959 bugs with UTF-16/UTF-32 conversions","user":{"login":"ScottPJones","id":11798751,"avatar_url":"https://avatars.githubusercontent.com/u/11798751?v=3","gravatar_id":"","url":"https://api.github.com/users/ScottPJones","html_url":"https://github.com/ScottPJones","followers_url":"https://api.github.com/users/ScottPJones/followers","following_url":"https://api.github.com/users/ScottPJones/following{/other_user}","gists_url":"https://api.github.com/users/ScottPJones/gists{/gist_id}","starred_url":"https://api.github.com/users/ScottPJones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ScottPJones/subscriptions","organizations_url":"https://api.github.com/users/ScottPJones/orgs","repos_url":"https://api.github.com/users/ScottPJones/repos","events_url":"https://api.github.com/users/ScottPJones/events{/privacy}","received_events_url":"https://api.github.com/users/ScottPJones/received_events","type":"User","site_admin":false},"body":"Rewrote a number of the conversions between ASCIIString, UTF8String,\r\nUTF16String, and UTF32String, and also rewrote length() for\r\nUTF16String().\r\n\r\nAdded validation functions that can be used in many other places, which also fix a number of Unicode handling bugs, such as detecting overlong encodings, out of range characters, singleton surrogate characters, and missing continuation characters, among other problems.\r\n\r\nAdded over 150 lines of testing code to detect the above conversion problems\r\n\r\nAdded (in a gist) code to show other conversion problems not yet fixed:\r\nhttps://gist.github.com/ScottPJones/4e6e8938f0559998f9fc\r\n\r\nAdded (in a gist) code to benchmark the performance, to ensure that adding the extra validity\r\nchecking did not adversely affect performance (in fact, performance was greatly improved).\r\nhttps://gist.github.com/ScottPJones/79ed895f05f85f333d84","created_at":"2015-06-03T01:22:27Z","updated_at":"2015-06-03T14:09:36Z","closed_at":null,"merged_at":null,"merge_commit_sha":"4fa589a9decdf3f48253be57a4ddacccc850b2e7","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/JuliaLang/julia/pulls/11551/commits","review_comments_url":"https://api.github.com/repos/JuliaLang/julia/pulls/11551/comments","review_comment_url":"https://api.github.com/repos/JuliaLang/julia/pulls/comments{/number}","comments_url":"https://api.github.com/repos/JuliaLang/julia/issues/11551/comments","statuses_url":"https://api.github.com/repos/JuliaLang/julia/statuses/58cc0268f3ac3abda740604f8e098f82dd79af28","head":{"label":"ScottPJones:spj/fixutf","ref":"spj/fixutf","sha":"58cc0268f3ac3abda740604f8e098f82dd79af28","user":{"login":"ScottPJones","id":11798751,"avatar_url":"https://avatars.githubusercontent.com/u/11798751?v=3","gravatar_id":"","url":"https://api.github.com/users/ScottPJones","html_url":"https://github.com/ScottPJones","followers_url":"https://api.github.com/users/ScottPJones/followers","following_url":"https://api.github.com/users/ScottPJones/following{/other_user}","gists_url":"https://api.github.com/users/ScottPJones/gists{/gist_id}","starred_url":"https://api.github.com/users/ScottPJones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ScottPJones/subscriptions","organizations_url":"https://api.github.com/users/ScottPJones/orgs","repos_url":"https://api.github.com/users/ScottPJones/repos","events_url":"https://api.github.com/users/ScottPJones/events{/privacy}","received_events_url":"https://api.github.com/users/ScottPJones/received_events","type":"User","site_admin":false},"repo":{"id":34334720,"name":"julia","full_name":"ScottPJones/julia","owner":{"login":"ScottPJones","id":11798751,"avatar_url":"https://avatars.githubusercontent.com/u/11798751?v=3","gravatar_id":"","url":"https://api.github.com/users/ScottPJones","html_url":"https://github.com/ScottPJones","followers_url":"https://api.github.com/users/ScottPJones/followers","following_url":"https://api.github.com/users/ScottPJones/following{/other_user}","gists_url":"https://api.github.com/users/ScottPJones/gists{/gist_id}","starred_url":"https://api.github.com/users/ScottPJones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ScottPJones/subscriptions","organizations_url":"https://api.github.com/users/ScottPJones/orgs","repos_url":"https://api.github.com/users/ScottPJones/repos","events_url":"https://api.github.com/users/ScottPJones/events{/privacy}","received_events_url":"https://api.github.com/users/ScottPJones/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/ScottPJones/julia","description":"The Julia Language: A fresh approach to technical computing.","fork":true,"url":"https://api.github.com/repos/ScottPJones/julia","forks_url":"https://api.github.com/repos/ScottPJones/julia/forks","keys_url":"https://api.github.com/repos/ScottPJones/julia/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ScottPJones/julia/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ScottPJones/julia/teams","hooks_url":"https://api.github.com/repos/ScottPJones/julia/hooks","issue_events_url":"https://api.github.com/repos/ScottPJones/julia/issues/events{/number}","events_url":"https://api.github.com/repos/ScottPJones/julia/events","assignees_url":"https://api.github.com/repos/ScottPJones/julia/assignees{/user}","branches_url":"https://api.github.com/repos/ScottPJones/julia/branches{/branch}","tags_url":"https://api.github.com/repos/ScottPJones/julia/tags","blobs_url":"https://api.github.com/repos/ScottPJones/julia/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ScottPJones/julia/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ScottPJones/julia/git/refs{/sha}","trees_url":"https://api.github.com/repos/ScottPJones/julia/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ScottPJones/julia/statuses/{sha}","languages_url":"https://api.github.com/repos/ScottPJones/julia/languages","stargazers_url":"https://api.github.com/repos/ScottPJones/julia/stargazers","contributors_url":"https://api.github.com/repos/ScottPJones/julia/contributors","subscribers_url":"https://api.github.com/repos/ScottPJones/julia/subscribers","subscription_url":"https://api.github.com/repos/ScottPJones/julia/subscription","commits_url":"https://api.github.com/repos/ScottPJones/julia/commits{/sha}","git_commits_url":"https://api.github.com/repos/ScottPJones/julia/git/commits{/sha}","comments_url":"https://api.github.com/repos/ScottPJones/julia/comments{/number}","issue_comment_url":"https://api.github.com/repos/ScottPJones/julia/issues/comments{/number}","contents_url":"https://api.github.com/repos/ScottPJones/julia/contents/{+path}","compare_url":"https://api.github.com/repos/ScottPJones/julia/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ScottPJones/julia/merges","archive_url":"https://api.github.com/repos/ScottPJones/julia/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ScottPJones/julia/downloads","issues_url":"https://api.github.com/repos/ScottPJones/julia/issues{/number}","pulls_url":"https://api.github.com/repos/ScottPJones/julia/pulls{/number}","milestones_url":"https://api.github.com/repos/ScottPJones/julia/milestones{/number}","notifications_url":"https://api.github.com/repos/ScottPJones/julia/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ScottPJones/julia/labels{/name}","releases_url":"https://api.github.com/repos/ScottPJones/julia/releases{/id}","created_at":"2015-04-21T15:12:29Z","updated_at":"2015-05-08T00:23:59Z","pushed_at":"2015-06-03T11:47:02Z","git_url":"git://github.com/ScottPJones/julia.git","ssh_url":"git@github.com:ScottPJones/julia.git","clone_url":"https://github.com/ScottPJones/julia.git","svn_url":"https://github.com/ScottPJones/julia","homepage":"http://julialang.org/","size":73384,"stargazers_count":0,"watchers_count":0,"language":"Julia","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"JuliaLang:master","ref":"master","sha":"a031f938385c19de2c217748e8c2adeb632ce125","user":{"login":"JuliaLang","id":743164,"avatar_url":"https://avatars.githubusercontent.com/u/743164?v=3","gravatar_id":"","url":"https://api.github.com/users/JuliaLang","html_url":"https://github.com/JuliaLang","followers_url":"https://api.github.com/users/JuliaLang/followers","following_url":"https://api.github.com/users/JuliaLang/following{/other_user}","gists_url":"https://api.github.com/users/JuliaLang/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaLang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaLang/subscriptions","organizations_url":"https://api.github.com/users/JuliaLang/orgs","repos_url":"https://api.github.com/users/JuliaLang/repos","events_url":"https://api.github.com/users/JuliaLang/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaLang/received_events","type":"Organization","site_admin":false},"repo":{"id":1644196,"name":"julia","full_name":"JuliaLang/julia","owner":{"login":"JuliaLang","id":743164,"avatar_url":"https://avatars.githubusercontent.com/u/743164?v=3","gravatar_id":"","url":"https://api.github.com/users/JuliaLang","html_url":"https://github.com/JuliaLang","followers_url":"https://api.github.com/users/JuliaLang/followers","following_url":"https://api.github.com/users/JuliaLang/following{/other_user}","gists_url":"https://api.github.com/users/JuliaLang/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaLang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaLang/subscriptions","organizations_url":"https://api.github.com/users/JuliaLang/orgs","repos_url":"https://api.github.com/users/JuliaLang/repos","events_url":"https://api.github.com/users/JuliaLang/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaLang/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/JuliaLang/julia","description":"The Julia Language: A fresh approach to technical computing.","fork":false,"url":"https://api.github.com/repos/JuliaLang/julia","forks_url":"https://api.github.com/repos/JuliaLang/julia/forks","keys_url":"https://api.github.com/repos/JuliaLang/julia/keys{/key_id}","collaborators_url":"https://api.github.com/repos/JuliaLang/julia/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/JuliaLang/julia/teams","hooks_url":"https://api.github.com/repos/JuliaLang/julia/hooks","issue_events_url":"https://api.github.com/repos/JuliaLang/julia/issues/events{/number}","events_url":"https://api.github.com/repos/JuliaLang/julia/events","assignees_url":"https://api.github.com/repos/JuliaLang/julia/assignees{/user}","branches_url":"https://api.github.com/repos/JuliaLang/julia/branches{/branch}","tags_url":"https://api.github.com/repos/JuliaLang/julia/tags","blobs_url":"https://api.github.com/repos/JuliaLang/julia/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/JuliaLang/julia/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/JuliaLang/julia/git/refs{/sha}","trees_url":"https://api.github.com/repos/JuliaLang/julia/git/trees{/sha}","statuses_url":"https://api.github.com/repos/JuliaLang/julia/statuses/{sha}","languages_url":"https://api.github.com/repos/JuliaLang/julia/languages","stargazers_url":"https://api.github.com/repos/JuliaLang/julia/stargazers","contributors_url":"https://api.github.com/repos/JuliaLang/julia/contributors","subscribers_url":"https://api.github.com/repos/JuliaLang/julia/subscribers","subscription_url":"https://api.github.com/repos/JuliaLang/julia/subscription","commits_url":"https://api.github.com/repos/JuliaLang/julia/commits{/sha}","git_commits_url":"https://api.github.com/repos/JuliaLang/julia/git/commits{/sha}","comments_url":"https://api.github.com/repos/JuliaLang/julia/comments{/number}","issue_comment_url":"https://api.github.com/repos/JuliaLang/julia/issues/comments{/number}","contents_url":"https://api.github.com/repos/JuliaLang/julia/contents/{+path}","compare_url":"https://api.github.com/repos/JuliaLang/julia/compare/{base}...{head}","merges_url":"https://api.github.com/repos/JuliaLang/julia/merges","archive_url":"https://api.github.com/repos/JuliaLang/julia/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/JuliaLang/julia/downloads","issues_url":"https://api.github.com/repos/JuliaLang/julia/issues{/number}","pulls_url":"https://api.github.com/repos/JuliaLang/julia/pulls{/number}","milestones_url":"https://api.github.com/repos/JuliaLang/julia/milestones{/number}","notifications_url":"https://api.github.com/repos/JuliaLang/julia/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/JuliaLang/julia/labels{/name}","releases_url":"https://api.github.com/repos/JuliaLang/julia/releases{/id}","created_at":"2011-04-21T07:01:50Z","updated_at":"2015-06-03T12:41:36Z","pushed_at":"2015-06-03T13:51:36Z","git_url":"git://github.com/JuliaLang/julia.git","ssh_url":"git@github.com:JuliaLang/julia.git","clone_url":"https://github.com/JuliaLang/julia.git","svn_url":"https://github.com/JuliaLang/julia","homepage":"http://julialang.org/","size":217490,"stargazers_count":5624,"watchers_count":5624,"language":"Julia","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1179,"mirror_url":null,"open_issues_count":1283,"forks":1179,"open_issues":1283,"watchers":5624,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/11551"},"html":{"href":"https://github.com/JuliaLang/julia/pull/11551"},"issue":{"href":"https://api.github.com/repos/JuliaLang/julia/issues/11551"},"comments":{"href":"https://api.github.com/repos/JuliaLang/julia/issues/11551/comments"},"review_comments":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/11551/comments"},"review_comment":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/11551/commits"},"statuses":{"href":"https://api.github.com/repos/JuliaLang/julia/statuses/58cc0268f3ac3abda740604f8e098f82dd79af28"}}}}