{"action":"created","issue":{"url":"https://api.github.com/repos/Zintinio/word2vec/issues/38","labels_url":"https://api.github.com/repos/Zintinio/word2vec/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/Zintinio/word2vec/issues/38/comments","events_url":"https://api.github.com/repos/Zintinio/word2vec/issues/38/events","html_url":"https://github.com/Zintinio/word2vec/issues/38","id":101758510,"number":38,"title":"Look into mmaping the corpus","user":{"login":"Zintinio","id":648208,"avatar_url":"https://avatars.githubusercontent.com/u/648208?v=3","gravatar_id":"","url":"https://api.github.com/users/Zintinio","html_url":"https://github.com/Zintinio","followers_url":"https://api.github.com/users/Zintinio/followers","following_url":"https://api.github.com/users/Zintinio/following{/other_user}","gists_url":"https://api.github.com/users/Zintinio/gists{/gist_id}","starred_url":"https://api.github.com/users/Zintinio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Zintinio/subscriptions","organizations_url":"https://api.github.com/users/Zintinio/orgs","repos_url":"https://api.github.com/users/Zintinio/repos","events_url":"https://api.github.com/users/Zintinio/events{/privacy}","received_events_url":"https://api.github.com/users/Zintinio/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-08-18T21:57:49Z","updated_at":"2015-08-18T22:07:09Z","closed_at":null,"body":"For files of a few hundred GB, there should be no issue to ```mmap``` with ```PROT_READ```. Check if improves performance."},"comment":{"url":"https://api.github.com/repos/Zintinio/word2vec/issues/comments/132370202","html_url":"https://github.com/Zintinio/word2vec/issues/38#issuecomment-132370202","issue_url":"https://api.github.com/repos/Zintinio/word2vec/issues/38","id":132370202,"user":{"login":"Zintinio","id":648208,"avatar_url":"https://avatars.githubusercontent.com/u/648208?v=3","gravatar_id":"","url":"https://api.github.com/users/Zintinio","html_url":"https://github.com/Zintinio","followers_url":"https://api.github.com/users/Zintinio/followers","following_url":"https://api.github.com/users/Zintinio/following{/other_user}","gists_url":"https://api.github.com/users/Zintinio/gists{/gist_id}","starred_url":"https://api.github.com/users/Zintinio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Zintinio/subscriptions","organizations_url":"https://api.github.com/users/Zintinio/orgs","repos_url":"https://api.github.com/users/Zintinio/repos","events_url":"https://api.github.com/users/Zintinio/events{/privacy}","received_events_url":"https://api.github.com/users/Zintinio/received_events","type":"User","site_admin":false},"created_at":"2015-08-18T22:07:09Z","updated_at":"2015-08-18T22:07:09Z","body":"In [https://github.com/Zintinio/word2vec/blob/master/word2vec.c#L74](https://github.com/Zintinio/word2vec/blob/master/word2vec.c#L74):\r\n```c\r\nvoid ReadWord(char *word, FILE *fin) {\r\n  int a = 0, ch;\r\n  while (!feof(fin)) {\r\n    ch = fgetc(fin);\r\n    if (ch == 13) continue;\r\n    if ((ch == ' ') || (ch == '\\t') || (ch == '\\n')) {\r\n      if (a > 0) {\r\n        if (ch == '\\n') ungetc(ch, fin);\r\n        break;\r\n      }\r\n      if (ch == '\\n') {\r\n        strcpy(word, (char *)\"</s>\");\r\n        return;\r\n      } else continue;\r\n    }\r\n    word[a] = ch;\r\n    a++;\r\n    if (a >= MAX_STRING - 1) a--;   // Truncate too long words\r\n  }\r\n  word[a] = 0;\r\n}\r\n```\r\n"}}