{"action":"created","issue":{"url":"https://api.github.com/repos/mne-tools/mne-python/issues/2165","labels_url":"https://api.github.com/repos/mne-tools/mne-python/issues/2165/labels{/name}","comments_url":"https://api.github.com/repos/mne-tools/mne-python/issues/2165/comments","events_url":"https://api.github.com/repos/mne-tools/mne-python/issues/2165/events","html_url":"https://github.com/mne-tools/mne-python/issues/2165","id":82365487,"number":2165,"title":"FIX/ENH: permutation_t_test inefficient for many perumations and and big input matrices","user":{"login":"dengemann","id":1908618,"avatar_url":"https://avatars.githubusercontent.com/u/1908618?v=3","gravatar_id":"","url":"https://api.github.com/users/dengemann","html_url":"https://github.com/dengemann","followers_url":"https://api.github.com/users/dengemann/followers","following_url":"https://api.github.com/users/dengemann/following{/other_user}","gists_url":"https://api.github.com/users/dengemann/gists{/gist_id}","starred_url":"https://api.github.com/users/dengemann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dengemann/subscriptions","organizations_url":"https://api.github.com/users/dengemann/orgs","repos_url":"https://api.github.com/users/dengemann/repos","events_url":"https://api.github.com/users/dengemann/events{/privacy}","received_events_url":"https://api.github.com/users/dengemann/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-05-29T10:13:49Z","updated_at":"2015-05-29T10:14:01Z","closed_at":null,"body":"I suspect the reason is here:\r\n\r\nhttps://github.com/mne-tools/mne-python/blob/master/mne/stats/permutations.py#L54\r\n\r\nTo give a simplified example:\r\nIf you have 50000 permutations, 5 jobs, and fsaverage source space data for a group of subject these dot products become heavy as via array split `perms` would be of size 10000, n_samples \r\n\r\nSymptoms:\r\n- computers die from memory and swapping limits\r\n\r\nWe should maybe change the parallelization and re-evaluate speed/memory tradeoffs."},"comment":{"url":"https://api.github.com/repos/mne-tools/mne-python/issues/comments/106765450","html_url":"https://github.com/mne-tools/mne-python/issues/2165#issuecomment-106765450","issue_url":"https://api.github.com/repos/mne-tools/mne-python/issues/2165","id":106765450,"user":{"login":"dengemann","id":1908618,"avatar_url":"https://avatars.githubusercontent.com/u/1908618?v=3","gravatar_id":"","url":"https://api.github.com/users/dengemann","html_url":"https://github.com/dengemann","followers_url":"https://api.github.com/users/dengemann/followers","following_url":"https://api.github.com/users/dengemann/following{/other_user}","gists_url":"https://api.github.com/users/dengemann/gists{/gist_id}","starred_url":"https://api.github.com/users/dengemann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dengemann/subscriptions","organizations_url":"https://api.github.com/users/dengemann/orgs","repos_url":"https://api.github.com/users/dengemann/repos","events_url":"https://api.github.com/users/dengemann/events{/privacy}","received_events_url":"https://api.github.com/users/dengemann/received_events","type":"User","site_admin":false},"created_at":"2015-05-29T10:14:01Z","updated_at":"2015-05-29T10:14:01Z","body":"cc @Eric89GXL @agramfort "}}