{"action":"created","issue":{"url":"https://api.github.com/repos/ds4se/chapters/issues/52","labels_url":"https://api.github.com/repos/ds4se/chapters/issues/52/labels{/name}","comments_url":"https://api.github.com/repos/ds4se/chapters/issues/52/comments","events_url":"https://api.github.com/repos/ds4se/chapters/issues/52/events","html_url":"https://github.com/ds4se/chapters/issues/52","id":111765181,"number":52,"title":"./meinagappan/Diversity_in_SE_Research.md","user":{"login":"timm","id":29195,"avatar_url":"https://avatars.githubusercontent.com/u/29195?v=3","gravatar_id":"","url":"https://api.github.com/users/timm","html_url":"https://github.com/timm","followers_url":"https://api.github.com/users/timm/followers","following_url":"https://api.github.com/users/timm/following{/other_user}","gists_url":"https://api.github.com/users/timm/gists{/gist_id}","starred_url":"https://api.github.com/users/timm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timm/subscriptions","organizations_url":"https://api.github.com/users/timm/orgs","repos_url":"https://api.github.com/users/timm/repos","events_url":"https://api.github.com/users/timm/events{/privacy}","received_events_url":"https://api.github.com/users/timm/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ds4se/chapters/labels/2.reviewTwo","name":"2.reviewTwo","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-10-16T05:54:48Z","updated_at":"2015-12-13T17:37:34Z","closed_at":null,"body":"After review, relabel to 'reviewTwo'. After second review, relabel to 'EditorsComment'."},"comment":{"url":"https://api.github.com/repos/ds4se/chapters/issues/comments/164279392","html_url":"https://github.com/ds4se/chapters/issues/52#issuecomment-164279392","issue_url":"https://api.github.com/repos/ds4se/chapters/issues/52","id":164279392,"user":{"login":"timm","id":29195,"avatar_url":"https://avatars.githubusercontent.com/u/29195?v=3","gravatar_id":"","url":"https://api.github.com/users/timm","html_url":"https://github.com/timm","followers_url":"https://api.github.com/users/timm/followers","following_url":"https://api.github.com/users/timm/following{/other_user}","gists_url":"https://api.github.com/users/timm/gists{/gist_id}","starred_url":"https://api.github.com/users/timm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timm/subscriptions","organizations_url":"https://api.github.com/users/timm/orgs","repos_url":"https://api.github.com/users/timm/repos","events_url":"https://api.github.com/users/timm/events{/privacy}","received_events_url":"https://api.github.com/users/timm/received_events","type":"User","site_admin":false},"created_at":"2015-12-13T17:37:33Z","updated_at":"2015-12-13T17:37:33Z","body":"Review from martin shepperd\r\n\r\n# Martin Shepperd Review \r\n\r\n## Title of chapter\r\n\r\nDiversity in Software Engineering Research\r\n\r\n## URL to the chapter\r\n\r\nhttps://github.com/ds4se/chapters/meinagappan/Diversity_in_SE_Research.md\r\n\r\n## Message?\r\n\r\nEmpirical researchers need to think carefully about how choose samples of projects to investigate.\r\n\r\n## Accessible?\r\n\r\nThe chapter isn't particularly technically demanding so it would be ok for a generalist audience, however it's not very self-contained and various concepts are not explained nor is there sufficient rationale.  \r\n\r\n## Size?\r\n\r\nI suggest the chapter could be extended to define terms such as neighbourhood, relate the ideas to more traditional inferential statistics and sampling theory and most of all provide a more explicit motivation for these ideas.\r\n\r\n## Gotta Mantra?\r\n\r\nCan't think of anything better.  Sorry\r\n\r\n## Best Points (and some general commentary)\r\n\r\nThe authors tackle a very important but rarely discussed matter so that alone makes the chapter welcome. How we sample and how we generalise are really fundamental questions.\r\n\r\nHowever, I wonder if the major issue isn’t more that the sample is very far from random which hugely undermines our ability to make inferences about the population.  This could be more problematic than small samples *per se*.\r\n\r\nI think the approach you propose might be termed stratified sampling but such that each subsample is ~equal size. The problems you use to motivate seem to me to be the consequence of a highly non-random i.e. arbitrary sampling procedure. The other usual motivation for a stratified sample is when some sub-groups are rare but you don’t wish to run the risk of missing them altogether.\r\n\r\nThere are two reasons why normally we’d care a lot about the randomness of the sampling.  First, inferential statistics are entirely dependent upon it.  Second, if the population is not well understood then the interesting sub-groups may not be knowable at the time of the empirical research so effectively the researcher relies on serendipitous randomness.\r\n\r\nI can’t quite figure where you sit in terms of the above. You talk about generalisation and coverage so I’m guessing your reasoning is if the researcher has covered the universe then generalisations are safe.  However there’s the issue of how large neighbourhoods are plus how well a region is covered.  The more traditional statistical view means you can think in terms of estimators and we can use a good deal of theory to quantify our confidence in the estimators.  So are you saying your approach is a necessary evil because random (stratified) sampling is seldom possible or that it’s better than conventional approaches. If the latter I’m going to struggle.  As a simple example, suppose we wish to model how my confidence in the estimate improves as the sample size increases I can use theory including the Central Limit Theorem etc etc to do this.  As long as I’m sampling randomly.  In your case I’m not sure you can even assert monotonicity.\r\n\r\nThe notion of similarity has been further developed by researchers especially in case-based reasoning.  The issues include (i) categorical dimensions or features (ii) combining multiple dimensions - using the product will have some strange effects - there are many well understood measures obviously starting with Euclidean but it might be Manhattan would suit for your purposes (iii) finding efficient feature subsets and (iv) how you standardise for units (but either max-min methods or z-scores are vulnerable to finding new observations which can change the similarities between them perhaps quite dramatically.\r\n\r\nA point you might make more strongly is that seldom much effort define the population / universe of discourse.\r\n\r\nI appreciate I’ve simplified both viewpoints so I’m sure you you will be able to make more nuanced responses to these comments.\r\n\r\n**MINOR:**\r\n\r\npara1: Make clear whether you mean sampling large scale artefacts or sampling large numbers of artefacts.\r\n\r\nYou do not define what you mean by neighbourhood. This would be useful.\r\n\r\n**TYPOS:**\r\n\r\n\"large (or all if possible) number of software projects.”???\r\nA very small point but you use the SE acronym a line before defining it.  (Para1)\r\n\"ap- plications” and \"Ex- apples”  \"de- fined\"\r\n\"Takingideasfromdiversity,identifytheneighborhoods(subgroups)ofallprojects\""}}