{"action":"created","issue":{"url":"https://api.github.com/repos/psychopy/psychopy/issues/950","labels_url":"https://api.github.com/repos/psychopy/psychopy/issues/950/labels{/name}","comments_url":"https://api.github.com/repos/psychopy/psychopy/issues/950/comments","events_url":"https://api.github.com/repos/psychopy/psychopy/issues/950/events","html_url":"https://github.com/psychopy/psychopy/issues/950","id":103225720,"number":950,"title":"Extremely large .psydat files","user":{"login":"ralfer","id":1465806,"avatar_url":"https://avatars.githubusercontent.com/u/1465806?v=3","gravatar_id":"","url":"https://api.github.com/users/ralfer","html_url":"https://github.com/ralfer","followers_url":"https://api.github.com/users/ralfer/followers","following_url":"https://api.github.com/users/ralfer/following{/other_user}","gists_url":"https://api.github.com/users/ralfer/gists{/gist_id}","starred_url":"https://api.github.com/users/ralfer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ralfer/subscriptions","organizations_url":"https://api.github.com/users/ralfer/orgs","repos_url":"https://api.github.com/users/ralfer/repos","events_url":"https://api.github.com/users/ralfer/events{/privacy}","received_events_url":"https://api.github.com/users/ralfer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-08-26T09:47:59Z","updated_at":"2015-09-11T17:22:55Z","closed_at":null,"body":"Sometimes PsychoPy (v 1.82.01) generates extremely large psydat files. For 150kb csv I have pickled data with sizes over 100 Mb. Interestingly, 7zip manages to compress them to really small sizes effortlessly. Here is an example: https://copy.com/0QIVaCIgOVtTRDSh - 180 Mb psydat compressed to 355 Kb 7z.\r\n\r\nThis issue has been reported in association with condition loops here: https://groups.google.com/forum/#!searchin/psychopy-users/psydat/psychopy-users/TPudxceQ1ws/zEwIEqLv7mUJ\r\n\r\nNot sure if it is the case for my own data. Although there are some conditional branches, I also experienced this issue (but not to such extent) when working with scripts written without builder that were pretty straightforward. Here is an example from such script: https://copy.com/zO1CNkHRUGxvhNbf 10 Mb psydat compressed to 1 Mb 7zip. But maybe in this case the size is more reasonable due to the large number of objects and trials. "},"comment":{"url":"https://api.github.com/repos/psychopy/psychopy/issues/comments/139606086","html_url":"https://github.com/psychopy/psychopy/issues/950#issuecomment-139606086","issue_url":"https://api.github.com/repos/psychopy/psychopy/issues/950","id":139606086,"user":{"login":"peircej","id":380582,"avatar_url":"https://avatars.githubusercontent.com/u/380582?v=3","gravatar_id":"","url":"https://api.github.com/users/peircej","html_url":"https://github.com/peircej","followers_url":"https://api.github.com/users/peircej/followers","following_url":"https://api.github.com/users/peircej/following{/other_user}","gists_url":"https://api.github.com/users/peircej/gists{/gist_id}","starred_url":"https://api.github.com/users/peircej/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/peircej/subscriptions","organizations_url":"https://api.github.com/users/peircej/orgs","repos_url":"https://api.github.com/users/peircej/repos","events_url":"https://api.github.com/users/peircej/events{/privacy}","received_events_url":"https://api.github.com/users/peircej/received_events","type":"User","site_admin":false},"created_at":"2015-09-11T17:22:55Z","updated_at":"2015-09-11T17:22:55Z","body":"My thoughts on this didn't pan out - having multiple \"copies\" of the experiment saved in the data file wasn't the issue because, actually, they're just pointers of a single object and they're saved correctly that way too.\r\n\r\nSo now I suspect that the issue is actually to do with the fact that we preallocate space for saving data when you create your loops. Maybe you have nested loops(?) and even if you don't \"use\" all the trials in each one by aborting them early the space is created for each."}}