{"action":"created","issue":{"url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31","labels_url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31/labels{/name}","comments_url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31/comments","events_url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31/events","html_url":"https://github.com/GuangchuangYu/clusterProfiler/issues/31","id":113507782,"number":31,"title":"Comparison clusterProfiler::gseGO with GSEA-P","user":{"login":"mevers","id":9429788,"avatar_url":"https://avatars.githubusercontent.com/u/9429788?v=3","gravatar_id":"","url":"https://api.github.com/users/mevers","html_url":"https://github.com/mevers","followers_url":"https://api.github.com/users/mevers/followers","following_url":"https://api.github.com/users/mevers/following{/other_user}","gists_url":"https://api.github.com/users/mevers/gists{/gist_id}","starred_url":"https://api.github.com/users/mevers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mevers/subscriptions","organizations_url":"https://api.github.com/users/mevers/orgs","repos_url":"https://api.github.com/users/mevers/repos","events_url":"https://api.github.com/users/mevers/events{/privacy}","received_events_url":"https://api.github.com/users/mevers/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":24,"created_at":"2015-10-27T05:07:32Z","updated_at":"2015-10-28T20:16:21Z","closed_at":"2015-10-28T02:54:39Z","body":"Dear Guangchuang.\r\n\r\nI've been running some side-by-side comparisons of GSEA-P and gseGO from clusterProfiler, using the same ranked-by-logFC list of genes.\r\n\r\nI am linking to two corresponding analysis files for the ontology \"cellular component\".\r\n\r\nTwo things struck me, maybe you can provide some insight.\r\n1. If I compare terms (positive phenotype correlation) that are enriched in both methods, overlap is very poor. While there are some overlapping terms, their enrichment scores/p-values are very different.\r\n2. P-values/adjusted p-values/q-values from gseGO are generally smaller and don't show a lot of variation. Corresponding values from GSEA-P usually show a larger dynamic range (from the threshold all the way to 1.0).\r\n\r\n[GSEA analysis using clusterProfiler::gseGO](http://s000.tinyupload.com/?file_id=78851688815127428757)\r\n[GSEA analysis using GSEA-P (positive phenotype correlation only)](http://s000.tinyupload.com/?file_id=00995488015610901858)\r\n\r\nAny help in clarifying this situation would be greatly appreciated.\r\n\r\nBest,\r\nMaurits\r\n\r\nPS.\r\nParameter sets for both methods should be consistent, see below.\r\n\r\nI used gseGO with the following parameters:\r\n```R\r\nres.GSEA.GO<-gseGO(geneList = geneList,\r\n                   organism = \"human\",\r\n                   exponent = 1,\r\n                   ont = ont[i],\r\n                   nPerm = 1000,\r\n                   minGSSize = 15,\r\n                   pvalueCutoff = 0.25,\r\n                   verbose = TRUE);\r\n t<-summary(res.GSEA.GO);\r\n# Exclude terms that are larger than 500\r\n# (default in GSEA-P)\r\n t<-t[which(t$setSize<500),];\r\n```\r\nGSEA-P was run with the following parameters:\r\n```\r\nproducer_class\txtools.gsea.GseaPreranked\r\nproducer_timestamp\t1445920145899\r\nparam\tcollapse\ttrue\r\nparam\tplot_top_x\t20\r\nparam\trnk\t/home/maurits/Desktop/clusterProfiler/list.rnk\r\nparam\tnorm\tmeandiv\r\nparam\tscoring_scheme\tweighted\r\nparam\tmake_sets\ttrue\r\nparam\tmode\tMax_probe\r\nparam\tgmx\tgseaftp.broadinstitute.org://pub/gsea/gene_sets/c5.cc.v5.0.symbols.gmt\r\nparam\tgui\tfalse\r\nparam\tchip\tgseaftp.broadinstitute.org://pub/gsea/annotations/GENE_SYMBOL.chip\r\nparam\trpt_label\tmy_analysis\r\nparam\thelp\tfalse\r\nparam\tout\t/home/maurits/gsea_home/output/oct27\r\nparam\tinclude_only_symbols\ttrue\r\nparam\tset_min\t15\r\nparam\tnperm\t1000\r\nparam\trnd_seed\ttimestamp\r\nparam\tzip_report\tfalse\r\nparam\tset_max\t500\r\n```"},"comment":{"url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/comments/151977381","html_url":"https://github.com/GuangchuangYu/clusterProfiler/issues/31#issuecomment-151977381","issue_url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31","id":151977381,"user":{"login":"mevers","id":9429788,"avatar_url":"https://avatars.githubusercontent.com/u/9429788?v=3","gravatar_id":"","url":"https://api.github.com/users/mevers","html_url":"https://github.com/mevers","followers_url":"https://api.github.com/users/mevers/followers","following_url":"https://api.github.com/users/mevers/following{/other_user}","gists_url":"https://api.github.com/users/mevers/gists{/gist_id}","starred_url":"https://api.github.com/users/mevers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mevers/subscriptions","organizations_url":"https://api.github.com/users/mevers/orgs","repos_url":"https://api.github.com/users/mevers/repos","events_url":"https://api.github.com/users/mevers/events{/privacy}","received_events_url":"https://api.github.com/users/mevers/received_events","type":"User","site_admin":false},"created_at":"2015-10-28T20:16:21Z","updated_at":"2015-10-28T20:16:21Z","body":"> For each geneList, we can calculate the observed ES, and then perform permutation to generate a null ES distribution. The pvalue = sum(ES >= permES)/nPerm for greater side as an example. So sum(ES >= permES) is k and nPerm is N. This is indeed the way GSEA-P calculate pvalue. \r\n\r\nI completely agree with you, so we're on the same page. \r\n\r\n> N != nPerm in GSEA-P, they separate positive ES and negative ES and treat them separately. This is reasonable since the distribution of ES is bimodal. This is also why pvalues produced by GSEA-P are more dynamic, since the denominator is not fixed.\r\n\r\nYes, this is discussed in the Appendix of Subramaniam et al. under section \"Enrichment scores\".\r\n\r\nExcellent, so it seems that the issue is resolved. Thanks Guangchuang for looking into this, and seeing this through this lengthy communication. I appreciate it. As I said before, I think you created a very nice and user-friendly tool.\r\n\r\nBest,\r\nMaurits   \r\n"}}