{"action":"created","issue":{"url":"https://api.github.com/repos/aoh/radamsa/issues/7","labels_url":"https://api.github.com/repos/aoh/radamsa/issues/7/labels{/name}","comments_url":"https://api.github.com/repos/aoh/radamsa/issues/7/comments","events_url":"https://api.github.com/repos/aoh/radamsa/issues/7/events","html_url":"https://github.com/aoh/radamsa/issues/7","id":123608792,"number":7,"title":"unit tests sometimes have sporadic failures","user":{"login":"brarcher","id":5264535,"avatar_url":"https://avatars.githubusercontent.com/u/5264535?v=3","gravatar_id":"","url":"https://api.github.com/users/brarcher","html_url":"https://github.com/brarcher","followers_url":"https://api.github.com/users/brarcher/followers","following_url":"https://api.github.com/users/brarcher/following{/other_user}","gists_url":"https://api.github.com/users/brarcher/gists{/gist_id}","starred_url":"https://api.github.com/users/brarcher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brarcher/subscriptions","organizations_url":"https://api.github.com/users/brarcher/orgs","repos_url":"https://api.github.com/users/brarcher/repos","events_url":"https://api.github.com/users/brarcher/events{/privacy}","received_events_url":"https://api.github.com/users/brarcher/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-12-23T06:30:01Z","updated_at":"2015-12-24T07:35:00Z","closed_at":null,"body":"It has been observed at least on OSX that some of the unit tests sometimes have sporadic failures. Following are some example failures as output by tests/run:\r\n```\r\n-n o tests/ts1.sh: \r\nsort: string comparison failed: Illegal byte sequence\r\nsort: Set LC_ALL='C' to work around the problem.\r\nsort: The strings compared were `\\'v\\217\\347' and `\\376\"i\\3317.\\345\\250V\\333w>\\346\\311\\203\\034\\316=\\337~\\233n\\320\\325\\005\\371\\320Sp\\301|\\247\"\\036\\024\\221\\247\\016\\213\\222;\\256=<c&\\3224'.\r\n```\r\n\r\n```\r\n-n  o tests/tr2.sh: \r\nsort: string comparison failed: Illegal byte sequence\r\nsort: Set LC_ALL='C' to work around the problem.\r\nsort: The strings compared were `6\\030v\\317\\3133о\\366|\\263htv9\\0173\\340\\2421\\275F\\a\\232\\360DJ\\017\\233\\037:)\\241\\023\\375\\350.\\272\\r<6\\201\\002\\330\\203+\\005\\221#\\355$\\343\\321F\\357T\\036\\264g[>]\\344\\200Ę\\265s\\236\\031E\\302-\\220ǰܺob<\\210\\004\\32415\\246\\300{ˏ\\030\\270xrژ\\335/\\243_ވ8\\255y\\a\\177\\362\\234!\\251N\\336\\322\\371\\325p\\024\\f\\241\\353&#6\\371\\204\\313\\020V\\031\\311\\210V\\302\\004\\\\\\237\\374\\316\\215!i\\357s\\231,P\\373+\\346\\303\\310tX\\300\\355\\177\\247R\\347u:3bA-\\2148\\03114\\361\\271k\\241\\376\\247/\\033\\271S\\\\|,\\a#\\200w\\237\\374\\002\\232!\\024\\316\\346\\371C\\017\\370\\354˕\\343\\241\\301\\244\\025\\2763\\000iÜP\\340\\021.\\001\\301\\246\\304\\363\\233\\266\\022!\\030\\232L\\024\\204\\311K\\030\\340\\3249.\\310\\354\\a_\\t\\374{j.$0\\021q\\267\\252<\\021\\023\\260\\301Z\\235m\\005\\330H\\342~\\016\\t\\242\\310\\303Oڏ\\210S\\311\\177\\275\\240\\345AwQ g\\334\\370\\302\\336\\021\\207\\r}`;8\\326Ҵ\\270.\\363q6\\325J,\\234(\\253QƼ\\226V\\310\\301$W\\231A\\273\\033\\000\\251\\274ѥ\\321\\322\\027\\320\\000뚦{@\\277~-\\205\\343݅E\\200\\341\\032\\203\\240\\027\\3338\\366z\\351CM6\\177C\\201\\312(N\\273\\346\\201d\\200\\032\\177\\371*\\177\\sort: string comparison failed: Illegal byte sequence\r\nsort: Set LC_ALL='C' to work around the problem.\r\nsort: The strings compared were `\\rX\\266\\204(a) (b)' and `\\rX\\266\\204'.\r\n```\r\n\r\nSome of the unit tests when they fail do not emit output to help diagnose the failure. Here is an example invoking tests/ab.sh directly:\r\n```\r\n$ rc=0\r\n$ attempt=1\r\n$ while [ $rc -eq 0 ]; do tests/ab.sh bin/radamsa ; rc=$?; echo $attempt; attempt=$((attempt+1)); done\r\n1\r\n2\r\n3\r\n4\r\n5\r\n6\r\n7\r\n8\r\n9\r\n10\r\n11\r\n12\r\n$\r\n```\r\nAfter 12 attempts a failure was observed, but the reason for the failure is not emitted.\r\n\r\nLikely it is the expectation that the unit test results be consistent. If the current revision in git is under development and the sporadic failure is expected or some cleanup is still underway, kindly ignore. I was unable to determine if release v0.4's unit tests encountered sporadic failures as issue https://github.com/aoh/radamsa/issues/5 affects the v0.4 release.\r\n\r\nAs a comparison, release v0.3 had consistently passing unit tests.\r\n\r\n(As a side note, at least on OSX the built-in echo command in sh does not support the -n option. This is the reason that \"-n\" is printed before all of the tests in tests/run . Consider reworking when echo is used in that script so that the -n option is unnecessary, if relevant).\r\n"},"comment":{"url":"https://api.github.com/repos/aoh/radamsa/issues/comments/167061158","html_url":"https://github.com/aoh/radamsa/issues/7#issuecomment-167061158","issue_url":"https://api.github.com/repos/aoh/radamsa/issues/7","id":167061158,"user":{"login":"aoh","id":320972,"avatar_url":"https://avatars.githubusercontent.com/u/320972?v=3","gravatar_id":"","url":"https://api.github.com/users/aoh","html_url":"https://github.com/aoh","followers_url":"https://api.github.com/users/aoh/followers","following_url":"https://api.github.com/users/aoh/following{/other_user}","gists_url":"https://api.github.com/users/aoh/gists{/gist_id}","starred_url":"https://api.github.com/users/aoh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aoh/subscriptions","organizations_url":"https://api.github.com/users/aoh/orgs","repos_url":"https://api.github.com/users/aoh/repos","events_url":"https://api.github.com/users/aoh/events{/privacy}","received_events_url":"https://api.github.com/users/aoh/received_events","type":"User","site_admin":false},"created_at":"2015-12-24T07:35:00Z","updated_at":"2015-12-24T07:35:00Z","body":"Makes sense. There is now a separate test build target, which runs the tests if necessary. I also added a thanks-section to readme.md."}}