{"action":"created","issue":{"url":"https://api.github.com/repos/UniversalDependencies/docs/issues/227","labels_url":"https://api.github.com/repos/UniversalDependencies/docs/issues/227/labels{/name}","comments_url":"https://api.github.com/repos/UniversalDependencies/docs/issues/227/comments","events_url":"https://api.github.com/repos/UniversalDependencies/docs/issues/227/events","html_url":"https://github.com/UniversalDependencies/docs/issues/227","id":114905239,"number":227,"title":"Intersections between datasets","user":{"login":"laufrant","id":15638007,"avatar_url":"https://avatars.githubusercontent.com/u/15638007?v=3","gravatar_id":"","url":"https://api.github.com/users/laufrant","html_url":"https://github.com/laufrant","followers_url":"https://api.github.com/users/laufrant/followers","following_url":"https://api.github.com/users/laufrant/following{/other_user}","gists_url":"https://api.github.com/users/laufrant/gists{/gist_id}","starred_url":"https://api.github.com/users/laufrant/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laufrant/subscriptions","organizations_url":"https://api.github.com/users/laufrant/orgs","repos_url":"https://api.github.com/users/laufrant/repos","events_url":"https://api.github.com/users/laufrant/events{/privacy}","received_events_url":"https://api.github.com/users/laufrant/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-11-03T20:41:47Z","updated_at":"2015-11-04T09:52:49Z","closed_at":null,"body":"Hi,\r\n\r\nI ran some tests to check for intersections between train/dev/test datasets and I was surprised by the large numbers I found. Some are duplicates inside train or very short sentences, but there are also many long sentences present in both train and test sets (around 8% of the Spanish test set). More importantly, in many cases the train and the test occurrences have different annotations.\r\n\r\nI have uploaded here listings for the duplicates I have found so far in ud-1.1, with X markers for annotation divergences: http://pastebin.com/raw.php?i=fkPU2uzN\r\n\r\nI found similar issues in the earlier versions. Had this been already reported?\r\n\r\nThanks for looking into this,\r\nLauriane\r\n"},"comment":{"url":"https://api.github.com/repos/UniversalDependencies/docs/issues/comments/153664336","html_url":"https://github.com/UniversalDependencies/docs/issues/227#issuecomment-153664336","issue_url":"https://api.github.com/repos/UniversalDependencies/docs/issues/227","id":153664336,"user":{"login":"laufrant","id":15638007,"avatar_url":"https://avatars.githubusercontent.com/u/15638007?v=3","gravatar_id":"","url":"https://api.github.com/users/laufrant","html_url":"https://github.com/laufrant","followers_url":"https://api.github.com/users/laufrant/followers","following_url":"https://api.github.com/users/laufrant/following{/other_user}","gists_url":"https://api.github.com/users/laufrant/gists{/gist_id}","starred_url":"https://api.github.com/users/laufrant/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laufrant/subscriptions","organizations_url":"https://api.github.com/users/laufrant/orgs","repos_url":"https://api.github.com/users/laufrant/repos","events_url":"https://api.github.com/users/laufrant/events{/privacy}","received_events_url":"https://api.github.com/users/laufrant/received_events","type":"User","site_admin":false},"created_at":"2015-11-04T09:52:49Z","updated_at":"2015-11-04T09:52:49Z","body":"I changed slightly my output, it should be easier to check the sentences now : http://pastebin.com/raw.php?i=A8BW3Pj1\r\nIt is not easy to give you my script, since it is basically a small tweak in my own pipeline. Sorry for that.\r\nDo you need any other output?\r\n\r\nAt least de/es/fr/it seem to me like true overlaps. For English they are mostly redundancies inside each set, but with annotation divergences though (for instance ADJ becomes PROPN in sentence at lines 7035 and 7627 in train set, change of root for sentence at lines 30629 and 32034…). For the other languages I cannot say."}}