{"action":"created","issue":{"url":"https://api.github.com/repos/ARTFL-Project/PhiloLogic4/issues/412","labels_url":"https://api.github.com/repos/ARTFL-Project/PhiloLogic4/issues/412/labels{/name}","comments_url":"https://api.github.com/repos/ARTFL-Project/PhiloLogic4/issues/412/comments","events_url":"https://api.github.com/repos/ARTFL-Project/PhiloLogic4/issues/412/events","html_url":"https://github.com/ARTFL-Project/PhiloLogic4/issues/412","id":80365811,"number":412,"title":"Tokenizing issue in collocations with hyphen tags","user":{"login":"clovis","id":357350,"avatar_url":"https://avatars.githubusercontent.com/u/357350?v=3","gravatar_id":"","url":"https://api.github.com/users/clovis","html_url":"https://github.com/clovis","followers_url":"https://api.github.com/users/clovis/followers","following_url":"https://api.github.com/users/clovis/following{/other_user}","gists_url":"https://api.github.com/users/clovis/gists{/gist_id}","starred_url":"https://api.github.com/users/clovis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clovis/subscriptions","organizations_url":"https://api.github.com/users/clovis/orgs","repos_url":"https://api.github.com/users/clovis/repos","events_url":"https://api.github.com/users/clovis/events{/privacy}","received_events_url":"https://api.github.com/users/clovis/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ARTFL-Project/PhiloLogic4/labels/bug","name":"bug","color":"fc2929"}],"state":"closed","locked":false,"assignee":{"login":"clovis","id":357350,"avatar_url":"https://avatars.githubusercontent.com/u/357350?v=3","gravatar_id":"","url":"https://api.github.com/users/clovis","html_url":"https://github.com/clovis","followers_url":"https://api.github.com/users/clovis/followers","following_url":"https://api.github.com/users/clovis/following{/other_user}","gists_url":"https://api.github.com/users/clovis/gists{/gist_id}","starred_url":"https://api.github.com/users/clovis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clovis/subscriptions","organizations_url":"https://api.github.com/users/clovis/orgs","repos_url":"https://api.github.com/users/clovis/repos","events_url":"https://api.github.com/users/clovis/events{/privacy}","received_events_url":"https://api.github.com/users/clovis/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2015-05-25T03:17:22Z","updated_at":"2015-06-23T21:06:38Z","closed_at":"2015-06-23T21:06:28Z","body":"The ECCO database has many in-word tags. This breaks tokenization in collocations.\r\n\r\nWe could fix this by removing the element that causes the issue. In the case of ECCO, it's \r\n```<span type=\"hyphenInWord\" rend=\"hidden\"></span>````\r\n\r\nRichard, how do you feel about using tweaking <a href=\"https://github.com/ARTFL-Project/PhiloLogic4/blob/master/www/reports/collocation.py#L147\">the current tokenizing regex</a> we use for collocations to detec in-word tags? "},"comment":{"url":"https://api.github.com/repos/ARTFL-Project/PhiloLogic4/issues/comments/114641953","html_url":"https://github.com/ARTFL-Project/PhiloLogic4/issues/412#issuecomment-114641953","issue_url":"https://api.github.com/repos/ARTFL-Project/PhiloLogic4/issues/412","id":114641953,"user":{"login":"clovis","id":357350,"avatar_url":"https://avatars.githubusercontent.com/u/357350?v=3","gravatar_id":"","url":"https://api.github.com/users/clovis","html_url":"https://github.com/clovis","followers_url":"https://api.github.com/users/clovis/followers","following_url":"https://api.github.com/users/clovis/following{/other_user}","gists_url":"https://api.github.com/users/clovis/gists{/gist_id}","starred_url":"https://api.github.com/users/clovis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clovis/subscriptions","organizations_url":"https://api.github.com/users/clovis/orgs","repos_url":"https://api.github.com/users/clovis/repos","events_url":"https://api.github.com/users/clovis/events{/privacy}","received_events_url":"https://api.github.com/users/clovis/received_events","type":"User","site_admin":false},"created_at":"2015-06-23T21:06:38Z","updated_at":"2015-06-23T21:06:38Z","body":"Fixed in new collocation code"}}