{"action":"created","issue":{"url":"https://api.github.com/repos/MDAnalysis/mdanalysis/issues/506","labels_url":"https://api.github.com/repos/MDAnalysis/mdanalysis/issues/506/labels{/name}","comments_url":"https://api.github.com/repos/MDAnalysis/mdanalysis/issues/506/comments","events_url":"https://api.github.com/repos/MDAnalysis/mdanalysis/issues/506/events","html_url":"https://github.com/MDAnalysis/mdanalysis/issues/506","id":112900023,"number":506,"title":"replace netCDF4 with scipy.io.netcdf for the Amber NCDFReader and NCDFWriter","user":{"login":"orbeckst","id":237980,"avatar_url":"https://avatars.githubusercontent.com/u/237980?v=3","gravatar_id":"","url":"https://api.github.com/users/orbeckst","html_url":"https://github.com/orbeckst","followers_url":"https://api.github.com/users/orbeckst/followers","following_url":"https://api.github.com/users/orbeckst/following{/other_user}","gists_url":"https://api.github.com/users/orbeckst/gists{/gist_id}","starred_url":"https://api.github.com/users/orbeckst/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/orbeckst/subscriptions","organizations_url":"https://api.github.com/users/orbeckst/orgs","repos_url":"https://api.github.com/users/orbeckst/repos","events_url":"https://api.github.com/users/orbeckst/events{/privacy}","received_events_url":"https://api.github.com/users/orbeckst/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/MDAnalysis/mdanalysis/labels/Component-Readers","name":"Component-Readers","color":"0052cc"},{"url":"https://api.github.com/repos/MDAnalysis/mdanalysis/labels/Component-Writers","name":"Component-Writers","color":"ddddff"},{"url":"https://api.github.com/repos/MDAnalysis/mdanalysis/labels/Format-Amber","name":"Format-Amber","color":"006b75"},{"url":"https://api.github.com/repos/MDAnalysis/mdanalysis/labels/performance","name":"performance","color":"f7c6c7"},{"url":"https://api.github.com/repos/MDAnalysis/mdanalysis/labels/question","name":"question","color":"cc317c"},{"url":"https://api.github.com/repos/MDAnalysis/mdanalysis/labels/usability","name":"usability","color":"5319e7"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-10-22T21:57:17Z","updated_at":"2015-10-22T22:24:43Z","closed_at":null,"body":"As discussed in #488, it is possible to read Amber netcdf trajectories with the `scipy.io.netcdf` reader. According to @swails and others, reading performance is better with the scipy reader. Additionally, the scipy implementation is pure python (based on [pupynere](https://bitbucket.org/robertodealmeida/pupynere/)) and does *not* require [installation of the netcdf libraries](/MDAnalysis/mdanalysis/wiki/netcdf).\r\n\r\n@orbeckst and @tylerjereddy make PR #503 available, in which netCDF4 was replaced with scipy.io.netcdf.  The question is: *Should we replace netCDF4 with scipy.io.netcdf?*\r\n\r\nPoints to consider:\r\n- We do not install `scipy` as part of the installation (although we list it as an optional dependency for analysis – and `gridDataFormats` also wants it); it is likely that anyone installing MDAnalysis will also install scipy.\r\n- scipy.io.netcdf does not read netcdf4 files; Amber files are netcdf3 so that's not a problem but should we ever want to read version 4, we would need to switch again\r\n- reading performance of Amber trajectories with scipy.io.netcdf is reportedly better than netCDF4 but we need to do our own benchmarks (both for reading and writing, and with `mmap=True` and `mmap=False`, see next point)\r\n- By default, the scipy `netcdf_file` uses `mmap()` so the trajectory has to remain open while arrays are accessed. This might make it more difficult to implement e.g. #239 (and might confuse users). Using `mmap=False` could be an option if performance is not degraded (but need to check).\r\n- As a fall back, we could bundle a copy of the scipy netcdf.py file with MDAnalysis and load it if scipy is not available.\r\n- The [netCDF4 API](http://unidata.github.io/netcdf4-python/) differs from the [scipy.io.netcdf API](http://docs.scipy.org/doc/scipy-0.16.0/reference/generated/scipy.io.netcdf.netcdf_file.html) (which implements the `Scientific.IO.NetCDF` API) so maintaining both libraries is cumbersome (but possible).\r\n"},"comment":{"url":"https://api.github.com/repos/MDAnalysis/mdanalysis/issues/comments/150372780","html_url":"https://github.com/MDAnalysis/mdanalysis/issues/506#issuecomment-150372780","issue_url":"https://api.github.com/repos/MDAnalysis/mdanalysis/issues/506","id":150372780,"user":{"login":"swails","id":779022,"avatar_url":"https://avatars.githubusercontent.com/u/779022?v=3","gravatar_id":"","url":"https://api.github.com/users/swails","html_url":"https://github.com/swails","followers_url":"https://api.github.com/users/swails/followers","following_url":"https://api.github.com/users/swails/following{/other_user}","gists_url":"https://api.github.com/users/swails/gists{/gist_id}","starred_url":"https://api.github.com/users/swails/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/swails/subscriptions","organizations_url":"https://api.github.com/users/swails/orgs","repos_url":"https://api.github.com/users/swails/repos","events_url":"https://api.github.com/users/swails/events{/privacy}","received_events_url":"https://api.github.com/users/swails/received_events","type":"User","site_admin":false},"created_at":"2015-10-22T22:24:43Z","updated_at":"2015-10-22T22:24:43Z","body":"FWIW, I originally had an implementation of NetCDF readers and writers that was backend-agnostic (supporting scipy, netCDF4, and ScientificPython).  The main annoyance there was actually in Python 3, where some packages returned `str` and others `bytes` for the string attributes.\r\n\r\nI have since ripped out that code and replaced it with the scipy backend (and I pulled out `netcdf.py` and included it directly in my package).  But the git history contains the (unit-tested) code that read and wrote both NetCDF trajectories and restarts from Amber: https://github.com/ParmEd/ParmEd/blob/2.0.5/parmed/amber/netcdffiles.py"}}