{"action":"created","issue":{"url":"https://api.github.com/repos/codeclimate/codeclimate-duplication/issues/60","labels_url":"https://api.github.com/repos/codeclimate/codeclimate-duplication/issues/60/labels{/name}","comments_url":"https://api.github.com/repos/codeclimate/codeclimate-duplication/issues/60/comments","events_url":"https://api.github.com/repos/codeclimate/codeclimate-duplication/issues/60/events","html_url":"https://github.com/codeclimate/codeclimate-duplication/pull/60","id":124119541,"number":60,"title":"Use .capture3 not .popen3 for external parsers","user":{"login":"wfleming","id":188402,"avatar_url":"https://avatars.githubusercontent.com/u/188402?v=3","gravatar_id":"","url":"https://api.github.com/users/wfleming","html_url":"https://github.com/wfleming","followers_url":"https://api.github.com/users/wfleming/followers","following_url":"https://api.github.com/users/wfleming/following{/other_user}","gists_url":"https://api.github.com/users/wfleming/gists{/gist_id}","starred_url":"https://api.github.com/users/wfleming/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wfleming/subscriptions","organizations_url":"https://api.github.com/users/wfleming/orgs","repos_url":"https://api.github.com/users/wfleming/repos","events_url":"https://api.github.com/users/wfleming/events{/privacy}","received_events_url":"https://api.github.com/users/wfleming/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-12-28T20:25:49Z","updated_at":"2015-12-28T20:30:38Z","closed_at":"2015-12-28T20:30:38Z","pull_request":{"url":"https://api.github.com/repos/codeclimate/codeclimate-duplication/pulls/60","html_url":"https://github.com/codeclimate/codeclimate-duplication/pull/60","diff_url":"https://github.com/codeclimate/codeclimate-duplication/pull/60.diff","patch_url":"https://github.com/codeclimate/codeclimate-duplication/pull/60.patch"},"body":"In my testing, I believe this will fix the frequent timeouts we've seen\r\nhappen on specific files.\r\n\r\nLike other issues we'd been seeing, these were actually\r\nnon-deterministic, they just happened to *almost* always fail. But I did\r\nsee them succeed occasionally: this may be partially why I didn't catch\r\nthis in testing before the prior release, since I ran it against some of\r\nthe same projects we saw failing. I just got lucky the first time\r\naround.\r\n\r\nWhat I saw locally was that sometimes the status thread given by `.open3` would\r\nbe dead before it was asked for its value, and in those cases the call\r\nseemed to succeed. If the thread was still alive when asked for its\r\nvalue, it seemed to then hang indefinitely.\r\n\r\nLooking at the code for [`.capture3`][1], the biggest difference seems\r\nto be the use of threads for capturing stdout/stderr. There's even a\r\nwarning in [stdlib docs about deadlocking from not doing this][2]! So I\r\nthink the previous implementation was effectively equivalent to a poor,\r\nbuggy implementation of `.capture3`, and we should just use that. It's\r\nunclear to me if the older usage of `IO.popen` also suffered from this\r\nbehavior and we just happened to be swallowing it, or if the pipe buffer\r\nbehavior is a little different there.\r\n\r\nI put together [this test repo](https://github.com/codeclimate-testing/will-scratch) to stress the duplication engine across all languages. It reliably fails on `master` of this repo & succeeds on this branch.\r\n\r\nThoughts, @codeclimate/review ?\r\n\r\n[1]: https://github.com/jruby/jruby/blob/3dbb634f2764b24bca9e92ecd3d52d859cc0e847/lib/ruby/stdlib/open3.rb#L258-L274\r\n[2]: http://ruby-doc.org/stdlib-2.2.3/libdoc/open3/rdoc/Open3.html#method-c-popen3"},"comment":{"url":"https://api.github.com/repos/codeclimate/codeclimate-duplication/issues/comments/167648001","html_url":"https://github.com/codeclimate/codeclimate-duplication/pull/60#issuecomment-167648001","issue_url":"https://api.github.com/repos/codeclimate/codeclimate-duplication/issues/60","id":167648001,"user":{"login":"GordonDiggs","id":245083,"avatar_url":"https://avatars.githubusercontent.com/u/245083?v=3","gravatar_id":"","url":"https://api.github.com/users/GordonDiggs","html_url":"https://github.com/GordonDiggs","followers_url":"https://api.github.com/users/GordonDiggs/followers","following_url":"https://api.github.com/users/GordonDiggs/following{/other_user}","gists_url":"https://api.github.com/users/GordonDiggs/gists{/gist_id}","starred_url":"https://api.github.com/users/GordonDiggs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GordonDiggs/subscriptions","organizations_url":"https://api.github.com/users/GordonDiggs/orgs","repos_url":"https://api.github.com/users/GordonDiggs/repos","events_url":"https://api.github.com/users/GordonDiggs/events{/privacy}","received_events_url":"https://api.github.com/users/GordonDiggs/received_events","type":"User","site_admin":false},"created_at":"2015-12-28T20:30:37Z","updated_at":"2015-12-28T20:30:37Z","body":"@wfleming I think you have a typo in your commit message:\r\n\r\n> buggy implementation of .capture3, and we should just use that. It's"}}