{"action":"opened","issue":{"url":"https://api.github.com/repos/brettc/partitionfinder/issues/18","labels_url":"https://api.github.com/repos/brettc/partitionfinder/issues/18/labels{/name}","comments_url":"https://api.github.com/repos/brettc/partitionfinder/issues/18/comments","events_url":"https://api.github.com/repos/brettc/partitionfinder/issues/18/events","html_url":"https://github.com/brettc/partitionfinder/issues/18","id":54578619,"number":18,"title":"generate likelihoods for \"fabricated subsets\" in the kmeans algorithm","user":{"login":"pbfrandsen","id":1823345,"avatar_url":"https://avatars.githubusercontent.com/u/1823345?v=3","gravatar_id":"","url":"https://api.github.com/users/pbfrandsen","html_url":"https://github.com/pbfrandsen","followers_url":"https://api.github.com/users/pbfrandsen/followers","following_url":"https://api.github.com/users/pbfrandsen/following{/other_user}","gists_url":"https://api.github.com/users/pbfrandsen/gists{/gist_id}","starred_url":"https://api.github.com/users/pbfrandsen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pbfrandsen/subscriptions","organizations_url":"https://api.github.com/users/pbfrandsen/orgs","repos_url":"https://api.github.com/users/pbfrandsen/repos","events_url":"https://api.github.com/users/pbfrandsen/events{/privacy}","received_events_url":"https://api.github.com/users/pbfrandsen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-01-16T14:06:00Z","updated_at":"2015-01-16T14:06:00Z","closed_at":null,"body":"The \"fabricated subsets\" feature requires that some sort of BIC score be assigned to subsets that we cannot analyze. To do this we must estimate the log likelihood for the subset as a whole. Since the definition of the fabricated subset is that raxml/phyml cannot analyze it, we don't have the subset log likelihood. In the first version of kmeans, we simply added up the site log likelihoods that we had conveniently generated for the clustering step, i.e.:\r\n\r\n![screen shot 2015-01-16 at 8 59 46 am](https://cloud.githubusercontent.com/assets/1823345/5777222/3b48a4f0-9d5e-11e4-9312-b202e70426a3.png)\r\n\r\nThis is no longer viable since we now use TIGER site rates rather than site likelihoods.\r\n\r\nShall we:\r\n\r\n1. Since we switched to TIGER rates, I haven't seen a dataset that required fabricated subsets yet, should we get rid of the fabricated subset function altogether and throw an error if the subsets get to small?\r\n\r\n2. Keep the BIC of the unsplit subset and set the BIC of the problematic subset to a value that makes it, plus the new subset one BIC point better than the unsplit subset so that algorithm keeps going?\r\n\r\n3. Other ideas? "}}