{"action":"created","comment":{"url":"https://api.github.com/repos/kennycason/kumo/pulls/comments/36489961","id":36489961,"diff_hunk":"@@ -95,7 +99,39 @@\n         return sorted.subList(0, Math.min(sorted.size(), wordFrequencesToReturn));\n     }\n \n-    public void setStopWords(Collection<String> stopWords) {\n+    public List<WordFrequency> load(final List<String> texts,FileType fileType) {\n+    \tif(fileType == FileType.REGULAR)\n+    \t\treturn load(texts);\n+    \t\n+        final List<WordFrequency> wordFrequencies = new ArrayList<>();\n+        // generate all word counts\n+        final Map<String, Integer> cloud = calculateCloud(texts, wordTokenizer, fileType);\n+\n+        for(String key : cloud.keySet()) {\n+            if(key.length() >= minWordLength\n+                    && key.length() < MAX_LENGTH) {\n+                wordFrequencies.add(new WordFrequency(key, cloud.get(key)));\n+            }\n+        }\n+        return takeTopFrequencies(wordFrequencies);\n+    }\n+    \n+    private Map<String, Integer> calculateCloud(List<String> texts, WordTokenizer tokenizer, FileType fileType) {\n+    \tfinal Map<String, Integer> cloud = new HashMap<>();\n+        for(String text : texts) {\n+            final List<String> words = Lambda.filter(stopWordFilter, tokenizer.tokenize(sanitizer.sanitize(text)));\n+            int sz = words.size();\n+            if(sz > 2)","path":"src/main/java/wordcloud/nlp/FrequencyAnalizer.java","position":38,"original_position":38,"commit_id":"2f01b8358901016f9de3ffcbb04058825da2b7a9","original_commit_id":"2f01b8358901016f9de3ffcbb04058825da2b7a9","user":{"login":"kennycason","id":2858428,"avatar_url":"https://avatars.githubusercontent.com/u/2858428?v=3","gravatar_id":"","url":"https://api.github.com/users/kennycason","html_url":"https://github.com/kennycason","followers_url":"https://api.github.com/users/kennycason/followers","following_url":"https://api.github.com/users/kennycason/following{/other_user}","gists_url":"https://api.github.com/users/kennycason/gists{/gist_id}","starred_url":"https://api.github.com/users/kennycason/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kennycason/subscriptions","organizations_url":"https://api.github.com/users/kennycason/orgs","repos_url":"https://api.github.com/users/kennycason/repos","events_url":"https://api.github.com/users/kennycason/events{/privacy}","received_events_url":"https://api.github.com/users/kennycason/received_events","type":"User","site_admin":false},"body":"same comment with the brackets.","created_at":"2015-08-07T04:17:11Z","updated_at":"2015-08-07T04:17:11Z","html_url":"https://github.com/kennycason/kumo/pull/13#discussion_r36489961","pull_request_url":"https://api.github.com/repos/kennycason/kumo/pulls/13","_links":{"self":{"href":"https://api.github.com/repos/kennycason/kumo/pulls/comments/36489961"},"html":{"href":"https://github.com/kennycason/kumo/pull/13#discussion_r36489961"},"pull_request":{"href":"https://api.github.com/repos/kennycason/kumo/pulls/13"}}},"pull_request":{"url":"https://api.github.com/repos/kennycason/kumo/pulls/13","id":41856613,"html_url":"https://github.com/kennycason/kumo/pull/13","diff_url":"https://github.com/kennycason/kumo/pull/13.diff","patch_url":"https://github.com/kennycason/kumo/pull/13.patch","issue_url":"https://api.github.com/repos/kennycason/kumo/issues/13","number":13,"state":"open","locked":false,"title":"Pull request : Word cloud for the files containing word-frequency pair","user":{"login":"satishsaley","id":3267004,"avatar_url":"https://avatars.githubusercontent.com/u/3267004?v=3","gravatar_id":"","url":"https://api.github.com/users/satishsaley","html_url":"https://github.com/satishsaley","followers_url":"https://api.github.com/users/satishsaley/followers","following_url":"https://api.github.com/users/satishsaley/following{/other_user}","gists_url":"https://api.github.com/users/satishsaley/gists{/gist_id}","starred_url":"https://api.github.com/users/satishsaley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/satishsaley/subscriptions","organizations_url":"https://api.github.com/users/satishsaley/orgs","repos_url":"https://api.github.com/users/satishsaley/repos","events_url":"https://api.github.com/users/satishsaley/events{/privacy}","received_events_url":"https://api.github.com/users/satishsaley/received_events","type":"User","site_admin":false},"body":"Hi Kenny,\r\n\r\nI have added functionality to create word cloud if file already contains word-frequency pairs.\r\n\r\nIf the data from which we want to create a word cloud is large, we can run simple MapReduce job to calculate word-frequency pairs. Later, we will process the output of MapReduce job using kumo library to generate a word cloud.\r\n\r\nLet me know how you find my changes. Please convey me any suggestions you have to make code more clean and robust.","created_at":"2015-08-07T03:23:46Z","updated_at":"2015-08-07T04:17:11Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/kennycason/kumo/pulls/13/commits","review_comments_url":"https://api.github.com/repos/kennycason/kumo/pulls/13/comments","review_comment_url":"https://api.github.com/repos/kennycason/kumo/pulls/comments{/number}","comments_url":"https://api.github.com/repos/kennycason/kumo/issues/13/comments","statuses_url":"https://api.github.com/repos/kennycason/kumo/statuses/2f01b8358901016f9de3ffcbb04058825da2b7a9","head":{"label":"satishsaley:master","ref":"master","sha":"2f01b8358901016f9de3ffcbb04058825da2b7a9","user":{"login":"satishsaley","id":3267004,"avatar_url":"https://avatars.githubusercontent.com/u/3267004?v=3","gravatar_id":"","url":"https://api.github.com/users/satishsaley","html_url":"https://github.com/satishsaley","followers_url":"https://api.github.com/users/satishsaley/followers","following_url":"https://api.github.com/users/satishsaley/following{/other_user}","gists_url":"https://api.github.com/users/satishsaley/gists{/gist_id}","starred_url":"https://api.github.com/users/satishsaley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/satishsaley/subscriptions","organizations_url":"https://api.github.com/users/satishsaley/orgs","repos_url":"https://api.github.com/users/satishsaley/repos","events_url":"https://api.github.com/users/satishsaley/events{/privacy}","received_events_url":"https://api.github.com/users/satishsaley/received_events","type":"User","site_admin":false},"repo":{"id":37897269,"name":"kumo","full_name":"satishsaley/kumo","owner":{"login":"satishsaley","id":3267004,"avatar_url":"https://avatars.githubusercontent.com/u/3267004?v=3","gravatar_id":"","url":"https://api.github.com/users/satishsaley","html_url":"https://github.com/satishsaley","followers_url":"https://api.github.com/users/satishsaley/followers","following_url":"https://api.github.com/users/satishsaley/following{/other_user}","gists_url":"https://api.github.com/users/satishsaley/gists{/gist_id}","starred_url":"https://api.github.com/users/satishsaley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/satishsaley/subscriptions","organizations_url":"https://api.github.com/users/satishsaley/orgs","repos_url":"https://api.github.com/users/satishsaley/repos","events_url":"https://api.github.com/users/satishsaley/events{/privacy}","received_events_url":"https://api.github.com/users/satishsaley/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/satishsaley/kumo","description":"Kumo - Java Word Cloud ","fork":true,"url":"https://api.github.com/repos/satishsaley/kumo","forks_url":"https://api.github.com/repos/satishsaley/kumo/forks","keys_url":"https://api.github.com/repos/satishsaley/kumo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/satishsaley/kumo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/satishsaley/kumo/teams","hooks_url":"https://api.github.com/repos/satishsaley/kumo/hooks","issue_events_url":"https://api.github.com/repos/satishsaley/kumo/issues/events{/number}","events_url":"https://api.github.com/repos/satishsaley/kumo/events","assignees_url":"https://api.github.com/repos/satishsaley/kumo/assignees{/user}","branches_url":"https://api.github.com/repos/satishsaley/kumo/branches{/branch}","tags_url":"https://api.github.com/repos/satishsaley/kumo/tags","blobs_url":"https://api.github.com/repos/satishsaley/kumo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/satishsaley/kumo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/satishsaley/kumo/git/refs{/sha}","trees_url":"https://api.github.com/repos/satishsaley/kumo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/satishsaley/kumo/statuses/{sha}","languages_url":"https://api.github.com/repos/satishsaley/kumo/languages","stargazers_url":"https://api.github.com/repos/satishsaley/kumo/stargazers","contributors_url":"https://api.github.com/repos/satishsaley/kumo/contributors","subscribers_url":"https://api.github.com/repos/satishsaley/kumo/subscribers","subscription_url":"https://api.github.com/repos/satishsaley/kumo/subscription","commits_url":"https://api.github.com/repos/satishsaley/kumo/commits{/sha}","git_commits_url":"https://api.github.com/repos/satishsaley/kumo/git/commits{/sha}","comments_url":"https://api.github.com/repos/satishsaley/kumo/comments{/number}","issue_comment_url":"https://api.github.com/repos/satishsaley/kumo/issues/comments{/number}","contents_url":"https://api.github.com/repos/satishsaley/kumo/contents/{+path}","compare_url":"https://api.github.com/repos/satishsaley/kumo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/satishsaley/kumo/merges","archive_url":"https://api.github.com/repos/satishsaley/kumo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/satishsaley/kumo/downloads","issues_url":"https://api.github.com/repos/satishsaley/kumo/issues{/number}","pulls_url":"https://api.github.com/repos/satishsaley/kumo/pulls{/number}","milestones_url":"https://api.github.com/repos/satishsaley/kumo/milestones{/number}","notifications_url":"https://api.github.com/repos/satishsaley/kumo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/satishsaley/kumo/labels{/name}","releases_url":"https://api.github.com/repos/satishsaley/kumo/releases{/id}","created_at":"2015-06-23T04:33:47Z","updated_at":"2015-06-23T04:33:48Z","pushed_at":"2015-06-26T15:42:44Z","git_url":"git://github.com/satishsaley/kumo.git","ssh_url":"git@github.com:satishsaley/kumo.git","clone_url":"https://github.com/satishsaley/kumo.git","svn_url":"https://github.com/satishsaley/kumo","homepage":null,"size":17142,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"kennycason:master","ref":"master","sha":"e4c6d730ce2e7acbebd2bdb92ef56feb67c1565b","user":{"login":"kennycason","id":2858428,"avatar_url":"https://avatars.githubusercontent.com/u/2858428?v=3","gravatar_id":"","url":"https://api.github.com/users/kennycason","html_url":"https://github.com/kennycason","followers_url":"https://api.github.com/users/kennycason/followers","following_url":"https://api.github.com/users/kennycason/following{/other_user}","gists_url":"https://api.github.com/users/kennycason/gists{/gist_id}","starred_url":"https://api.github.com/users/kennycason/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kennycason/subscriptions","organizations_url":"https://api.github.com/users/kennycason/orgs","repos_url":"https://api.github.com/users/kennycason/repos","events_url":"https://api.github.com/users/kennycason/events{/privacy}","received_events_url":"https://api.github.com/users/kennycason/received_events","type":"User","site_admin":false},"repo":{"id":21477166,"name":"kumo","full_name":"kennycason/kumo","owner":{"login":"kennycason","id":2858428,"avatar_url":"https://avatars.githubusercontent.com/u/2858428?v=3","gravatar_id":"","url":"https://api.github.com/users/kennycason","html_url":"https://github.com/kennycason","followers_url":"https://api.github.com/users/kennycason/followers","following_url":"https://api.github.com/users/kennycason/following{/other_user}","gists_url":"https://api.github.com/users/kennycason/gists{/gist_id}","starred_url":"https://api.github.com/users/kennycason/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kennycason/subscriptions","organizations_url":"https://api.github.com/users/kennycason/orgs","repos_url":"https://api.github.com/users/kennycason/repos","events_url":"https://api.github.com/users/kennycason/events{/privacy}","received_events_url":"https://api.github.com/users/kennycason/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/kennycason/kumo","description":"Kumo - Java Word Cloud ","fork":false,"url":"https://api.github.com/repos/kennycason/kumo","forks_url":"https://api.github.com/repos/kennycason/kumo/forks","keys_url":"https://api.github.com/repos/kennycason/kumo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kennycason/kumo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kennycason/kumo/teams","hooks_url":"https://api.github.com/repos/kennycason/kumo/hooks","issue_events_url":"https://api.github.com/repos/kennycason/kumo/issues/events{/number}","events_url":"https://api.github.com/repos/kennycason/kumo/events","assignees_url":"https://api.github.com/repos/kennycason/kumo/assignees{/user}","branches_url":"https://api.github.com/repos/kennycason/kumo/branches{/branch}","tags_url":"https://api.github.com/repos/kennycason/kumo/tags","blobs_url":"https://api.github.com/repos/kennycason/kumo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kennycason/kumo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kennycason/kumo/git/refs{/sha}","trees_url":"https://api.github.com/repos/kennycason/kumo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kennycason/kumo/statuses/{sha}","languages_url":"https://api.github.com/repos/kennycason/kumo/languages","stargazers_url":"https://api.github.com/repos/kennycason/kumo/stargazers","contributors_url":"https://api.github.com/repos/kennycason/kumo/contributors","subscribers_url":"https://api.github.com/repos/kennycason/kumo/subscribers","subscription_url":"https://api.github.com/repos/kennycason/kumo/subscription","commits_url":"https://api.github.com/repos/kennycason/kumo/commits{/sha}","git_commits_url":"https://api.github.com/repos/kennycason/kumo/git/commits{/sha}","comments_url":"https://api.github.com/repos/kennycason/kumo/comments{/number}","issue_comment_url":"https://api.github.com/repos/kennycason/kumo/issues/comments{/number}","contents_url":"https://api.github.com/repos/kennycason/kumo/contents/{+path}","compare_url":"https://api.github.com/repos/kennycason/kumo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kennycason/kumo/merges","archive_url":"https://api.github.com/repos/kennycason/kumo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kennycason/kumo/downloads","issues_url":"https://api.github.com/repos/kennycason/kumo/issues{/number}","pulls_url":"https://api.github.com/repos/kennycason/kumo/pulls{/number}","milestones_url":"https://api.github.com/repos/kennycason/kumo/milestones{/number}","notifications_url":"https://api.github.com/repos/kennycason/kumo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kennycason/kumo/labels{/name}","releases_url":"https://api.github.com/repos/kennycason/kumo/releases{/id}","created_at":"2014-07-03T21:14:53Z","updated_at":"2015-07-09T10:47:28Z","pushed_at":"2015-08-07T03:23:47Z","git_url":"git://github.com/kennycason/kumo.git","ssh_url":"git@github.com:kennycason/kumo.git","clone_url":"https://github.com/kennycason/kumo.git","svn_url":"https://github.com/kennycason/kumo","homepage":null,"size":18177,"stargazers_count":19,"watchers_count":19,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":12,"mirror_url":null,"open_issues_count":4,"forks":12,"open_issues":4,"watchers":19,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/kennycason/kumo/pulls/13"},"html":{"href":"https://github.com/kennycason/kumo/pull/13"},"issue":{"href":"https://api.github.com/repos/kennycason/kumo/issues/13"},"comments":{"href":"https://api.github.com/repos/kennycason/kumo/issues/13/comments"},"review_comments":{"href":"https://api.github.com/repos/kennycason/kumo/pulls/13/comments"},"review_comment":{"href":"https://api.github.com/repos/kennycason/kumo/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/kennycason/kumo/pulls/13/commits"},"statuses":{"href":"https://api.github.com/repos/kennycason/kumo/statuses/2f01b8358901016f9de3ffcbb04058825da2b7a9"}}}}