{"action":"created","issue":{"url":"https://api.github.com/repos/ReScience/ReScience/issues/19","labels_url":"https://api.github.com/repos/ReScience/ReScience/issues/19/labels{/name}","comments_url":"https://api.github.com/repos/ReScience/ReScience/issues/19/comments","events_url":"https://api.github.com/repos/ReScience/ReScience/issues/19/events","html_url":"https://github.com/ReScience/ReScience/issues/19","id":107035406,"number":19,"title":"Submit the replication of my own research not allowed","user":{"login":"fabiansinz","id":7535854,"avatar_url":"https://avatars.githubusercontent.com/u/7535854?v=3","gravatar_id":"","url":"https://api.github.com/users/fabiansinz","html_url":"https://github.com/fabiansinz","followers_url":"https://api.github.com/users/fabiansinz/followers","following_url":"https://api.github.com/users/fabiansinz/following{/other_user}","gists_url":"https://api.github.com/users/fabiansinz/gists{/gist_id}","starred_url":"https://api.github.com/users/fabiansinz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fabiansinz/subscriptions","organizations_url":"https://api.github.com/users/fabiansinz/orgs","repos_url":"https://api.github.com/users/fabiansinz/repos","events_url":"https://api.github.com/users/fabiansinz/events{/privacy}","received_events_url":"https://api.github.com/users/fabiansinz/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ReScience/ReScience/labels/Question","name":"Question","color":"5319e7"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-09-17T17:40:10Z","updated_at":"2015-09-18T05:33:33Z","closed_at":null,"body":"I'd like to strongly suggest to rethink that restriction. I really like the idea of this journal, but I fear that this restriction will be a substantial limiting factor for the number of submissions and the usefulness of the journal.\r\n\r\nA very obvious reason for this is that time is limited for all of us. Therefore, we usually never reproduce the results of an entire paper, but only the parts that are relevant to us. Would you then accept code that reproduces parts of that paper?\r\n\r\nMore importantly, I don't think that this restriction really promotes reproducible research. By only allowing reproduced results from people other than the authors, you are basically checking whether the methods part of a paper is sufficient to reproduce the results. While that should be the case, it is unfortunately only true in rare situations. There are various reasons for this:\r\n\r\n- Journals have word limitations.\r\n- Condensing a complex analysis into a few paragraphs in a methods part is a daunting task and it is very easy to fail.\r\n- The data has not been published.\r\n\r\nThe best reproduction of an analysis is its code. This is why code should always be published with a paper. Unfortunately, this is not common practice and even if the code is published, I haven't seen a single instance in which it was reviewed. I thought that ReScience could fill this gap perfectly and who would be better suited to supply it than then authors?\r\n\r\nLet's say you allowed authors to publish their analysis. This would mean that they would have to provide the data (because you need to be able to run it) and you would have at least one running instance that reproduces the results. It might have mistakes and biases, but at least it is one entire running instance. Others who are insterested in reproducing the paper could look at the code and, with the methods part of the paper, figure out what the authors did. If they find a error or a particular bias in the analysis, they could still submit an issue to the code repository. The time required to understand and reproduce the results, however, would be drastically reduced. I feel this would serve reproducibility much more than the hope that people spend significant time to reproduce an entire paper.\r\n"},"comment":{"url":"https://api.github.com/repos/ReScience/ReScience/issues/comments/141349984","html_url":"https://github.com/ReScience/ReScience/issues/19#issuecomment-141349984","issue_url":"https://api.github.com/repos/ReScience/ReScience/issues/19","id":141349984,"user":{"login":"rougier","id":327203,"avatar_url":"https://avatars.githubusercontent.com/u/327203?v=3","gravatar_id":"","url":"https://api.github.com/users/rougier","html_url":"https://github.com/rougier","followers_url":"https://api.github.com/users/rougier/followers","following_url":"https://api.github.com/users/rougier/following{/other_user}","gists_url":"https://api.github.com/users/rougier/gists{/gist_id}","starred_url":"https://api.github.com/users/rougier/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rougier/subscriptions","organizations_url":"https://api.github.com/users/rougier/orgs","repos_url":"https://api.github.com/users/rougier/repos","events_url":"https://api.github.com/users/rougier/events{/privacy}","received_events_url":"https://api.github.com/users/rougier/received_events","type":"User","site_admin":false},"created_at":"2015-09-18T05:33:33Z","updated_at":"2015-09-18T05:33:33Z","body":"@fabiansinz ReScience cannot fix the whole reproducibility problem on its own. I agree with most for your arguments but still, ReScience may not  be the place to do that yet. I'm not saying that things won't change in the future, but until we get some submissions, it might be better to keep ReScience focused. \r\n\r\n@ctb Yes, but I fear this won't be the case anytime soon. Reviewing code is hard, especially for an original research where readability is not the main point. But when a paper is replicated, we can expect better code (at least) and more or less easier control of the code regarding expected results.\r\n\r\n\r\n"}}