{"action":"created","issue":{"url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31","labels_url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31/labels{/name}","comments_url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31/comments","events_url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31/events","html_url":"https://github.com/GuangchuangYu/clusterProfiler/issues/31","id":113507782,"number":31,"title":"Comparison clusterProfiler::gseGO with GSEA-P","user":{"login":"mevers","id":9429788,"avatar_url":"https://avatars.githubusercontent.com/u/9429788?v=3","gravatar_id":"","url":"https://api.github.com/users/mevers","html_url":"https://github.com/mevers","followers_url":"https://api.github.com/users/mevers/followers","following_url":"https://api.github.com/users/mevers/following{/other_user}","gists_url":"https://api.github.com/users/mevers/gists{/gist_id}","starred_url":"https://api.github.com/users/mevers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mevers/subscriptions","organizations_url":"https://api.github.com/users/mevers/orgs","repos_url":"https://api.github.com/users/mevers/repos","events_url":"https://api.github.com/users/mevers/events{/privacy}","received_events_url":"https://api.github.com/users/mevers/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":21,"created_at":"2015-10-27T05:07:32Z","updated_at":"2015-10-28T06:21:40Z","closed_at":"2015-10-28T02:54:39Z","body":"Dear Guangchuang.\r\n\r\nI've been running some side-by-side comparisons of GSEA-P and gseGO from clusterProfiler, using the same ranked-by-logFC list of genes.\r\n\r\nI am linking to two corresponding analysis files for the ontology \"cellular component\".\r\n\r\nTwo things struck me, maybe you can provide some insight.\r\n1. If I compare terms (positive phenotype correlation) that are enriched in both methods, overlap is very poor. While there are some overlapping terms, their enrichment scores/p-values are very different.\r\n2. P-values/adjusted p-values/q-values from gseGO are generally smaller and don't show a lot of variation. Corresponding values from GSEA-P usually show a larger dynamic range (from the threshold all the way to 1.0).\r\n\r\n[GSEA analysis using clusterProfiler::gseGO](http://s000.tinyupload.com/?file_id=78851688815127428757)\r\n[GSEA analysis using GSEA-P (positive phenotype correlation only)](http://s000.tinyupload.com/?file_id=00995488015610901858)\r\n\r\nAny help in clarifying this situation would be greatly appreciated.\r\n\r\nBest,\r\nMaurits\r\n\r\nPS.\r\nParameter sets for both methods should be consistent, see below.\r\n\r\nI used gseGO with the following parameters:\r\n```R\r\nres.GSEA.GO<-gseGO(geneList = geneList,\r\n                   organism = \"human\",\r\n                   exponent = 1,\r\n                   ont = ont[i],\r\n                   nPerm = 1000,\r\n                   minGSSize = 15,\r\n                   pvalueCutoff = 0.25,\r\n                   verbose = TRUE);\r\n t<-summary(res.GSEA.GO);\r\n# Exclude terms that are larger than 500\r\n# (default in GSEA-P)\r\n t<-t[which(t$setSize<500),];\r\n```\r\nGSEA-P was run with the following parameters:\r\n```\r\nproducer_class\txtools.gsea.GseaPreranked\r\nproducer_timestamp\t1445920145899\r\nparam\tcollapse\ttrue\r\nparam\tplot_top_x\t20\r\nparam\trnk\t/home/maurits/Desktop/clusterProfiler/list.rnk\r\nparam\tnorm\tmeandiv\r\nparam\tscoring_scheme\tweighted\r\nparam\tmake_sets\ttrue\r\nparam\tmode\tMax_probe\r\nparam\tgmx\tgseaftp.broadinstitute.org://pub/gsea/gene_sets/c5.cc.v5.0.symbols.gmt\r\nparam\tgui\tfalse\r\nparam\tchip\tgseaftp.broadinstitute.org://pub/gsea/annotations/GENE_SYMBOL.chip\r\nparam\trpt_label\tmy_analysis\r\nparam\thelp\tfalse\r\nparam\tout\t/home/maurits/gsea_home/output/oct27\r\nparam\tinclude_only_symbols\ttrue\r\nparam\tset_min\t15\r\nparam\tnperm\t1000\r\nparam\trnd_seed\ttimestamp\r\nparam\tzip_report\tfalse\r\nparam\tset_max\t500\r\n```"},"comment":{"url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/comments/151737650","html_url":"https://github.com/GuangchuangYu/clusterProfiler/issues/31#issuecomment-151737650","issue_url":"https://api.github.com/repos/GuangchuangYu/clusterProfiler/issues/31","id":151737650,"user":{"login":"mevers","id":9429788,"avatar_url":"https://avatars.githubusercontent.com/u/9429788?v=3","gravatar_id":"","url":"https://api.github.com/users/mevers","html_url":"https://github.com/mevers","followers_url":"https://api.github.com/users/mevers/followers","following_url":"https://api.github.com/users/mevers/following{/other_user}","gists_url":"https://api.github.com/users/mevers/gists{/gist_id}","starred_url":"https://api.github.com/users/mevers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mevers/subscriptions","organizations_url":"https://api.github.com/users/mevers/orgs","repos_url":"https://api.github.com/users/mevers/repos","events_url":"https://api.github.com/users/mevers/events{/privacy}","received_events_url":"https://api.github.com/users/mevers/received_events","type":"User","site_admin":false},"created_at":"2015-10-28T06:21:39Z","updated_at":"2015-10-28T06:21:39Z","body":">If you are fine with (k+1)/(N+1) with k = 0, you should be fine with k equals to other integer. This is the reason of non-zero pvalues are identical.\r\n\r\nI think you misunderstand how p-values are calculated in the GSEA formalism. Are you telling me you calculate p-values as (k+1)/(N+1) for some integer k? What is that integer k?\r\nEither way, that is not the correct way. And that is also not what GSEA-P does.\r\n\r\nAs stated in the PNAS publication (see the above quote I give, and my PPS on the permutation test), you determine the p-value by calculating the area corresponding to the tail (bounded by the score of the unpermuted enrichment score) of the empirical enrichment score distribution that you get from the permutation-derived enrichment scores (your null distribution).\r\n\r\nThe 1/N is just an upper bound that GSEA-P gives *if* the p-value is zero, to account for the number of permutation tests. In doing so GSEA-P is indeed more conservative.\r\n\r\n**The enrichment p-value is definitely not (k+1)/(N+1) (or k/N for that matter)**. \r\n\r\n  "}}