{"action":"created","issue":{"url":"https://api.github.com/repos/astropy/astropy/issues/3883","labels_url":"https://api.github.com/repos/astropy/astropy/issues/3883/labels{/name}","comments_url":"https://api.github.com/repos/astropy/astropy/issues/3883/comments","events_url":"https://api.github.com/repos/astropy/astropy/issues/3883/events","html_url":"https://github.com/astropy/astropy/issues/3883","id":90703795,"number":3883,"title":"Checksum inconsistent when BITPIX changes (io.fits)","user":{"login":"nparley","id":6215361,"avatar_url":"https://avatars.githubusercontent.com/u/6215361?v=3","gravatar_id":"","url":"https://api.github.com/users/nparley","html_url":"https://github.com/nparley","followers_url":"https://api.github.com/users/nparley/followers","following_url":"https://api.github.com/users/nparley/following{/other_user}","gists_url":"https://api.github.com/users/nparley/gists{/gist_id}","starred_url":"https://api.github.com/users/nparley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nparley/subscriptions","organizations_url":"https://api.github.com/users/nparley/orgs","repos_url":"https://api.github.com/users/nparley/repos","events_url":"https://api.github.com/users/nparley/events{/privacy}","received_events_url":"https://api.github.com/users/nparley/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/astropy/astropy/labels/io.fits","name":"io.fits","color":"c7def8"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-06-24T15:09:37Z","updated_at":"2015-06-25T06:27:27Z","closed_at":null,"body":"The checksum being calculated for a fits file I am testing is inconsistent. For example with a fits file test.fits which does not have a checksum:\r\n\r\n``` \r\n$ fitscheck -f -w -i test.fits\r\n$ fitscheck test.fits\r\nBAD 'test.fits' .. verification failed for HDU ('PRIMARY', 1).\r\n1 errors\r\n```\r\n\r\nThe problem seems to come from the fact that the datasum is calculated differently before and after the image data is loaded.\r\n\r\n```\r\n>>> from astropy.io import fits\r\n>>> hdu = fits.open('test.fits')\r\n>>> hdu[0]._calculate_datasum('standard')\r\n3044153422\r\n>>> data = hdu[0].data\r\n>>> hdu[0]._calculate_datasum('standard')\r\n2977440024\r\n```\r\n\r\nThis fits file has an original BITPIX of 16 and once updated by writeto has a BITPIX of -32 but the checksum is being calculated using the original data.  I am surprised that BITPIX is changing as I thought the image data would just be directly copied if the data is not loaded. But it seems the data is loaded in the writeto mehod and thus changed to -32.\r\n\r\n```\r\n>>> from astropy.io import fits\r\n>>> hdu = fits.open('test.fits')\r\n>>> hdulist[0]._data_loaded\r\nFalse\r\n>>> hdulist.writeto('test2.fits', clobber=True)\r\n>>> hdulist[0]._data_loaded\r\nTrue\r\n```\r\n\r\nI think the probable solution, if the data is going to be changed to a float on writeto, is changing the order of the checksum calculations to make sure it is calculated on the -32 data. I.e. this works with out problems:\r\n\r\n```\r\n>>> hdulist = fits.open('test_org.fits')\r\n>>> data = hdulist[0].data\r\n>>> hdulist.writeto('test.fits', clobber=True, checksum=True)\r\n\r\n$ fitscheck test.fits\r\n$\r\n```\r\n\r\n I have put the test fits file for this here: https://www.dropbox.com/s/mkvuohp7e2nhpe0/test_org.fits?dl=0 \r\nCheers, Neiil"},"comment":{"url":"https://api.github.com/repos/astropy/astropy/issues/comments/115122647","html_url":"https://github.com/astropy/astropy/issues/3883#issuecomment-115122647","issue_url":"https://api.github.com/repos/astropy/astropy/issues/3883","id":115122647,"user":{"login":"nparley","id":6215361,"avatar_url":"https://avatars.githubusercontent.com/u/6215361?v=3","gravatar_id":"","url":"https://api.github.com/users/nparley","html_url":"https://github.com/nparley","followers_url":"https://api.github.com/users/nparley/followers","following_url":"https://api.github.com/users/nparley/following{/other_user}","gists_url":"https://api.github.com/users/nparley/gists{/gist_id}","starred_url":"https://api.github.com/users/nparley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nparley/subscriptions","organizations_url":"https://api.github.com/users/nparley/orgs","repos_url":"https://api.github.com/users/nparley/repos","events_url":"https://api.github.com/users/nparley/events{/privacy}","received_events_url":"https://api.github.com/users/nparley/received_events","type":"User","site_admin":false},"created_at":"2015-06-25T06:27:27Z","updated_at":"2015-06-25T06:27:27Z","body":"Sounds good.\r\n\r\nLooking at the code this is what I think is happening with regard to the check sum and image scaling when the data has not been hit.\r\n\r\n- ``_ImageBaseHDU._prewriteto``: Updates the header scaling values but I don't think the data has been scaled yet then calls base method\r\n- ``_Base._prewriteto``: Calls ``_update_checksum(checksum)`` but at this point the data still has not been hit so the checksum will calculate the value using the raw data\r\n- ``_Base._writeto``: Calls ``_ImageBaseHDU._writedata_internal`` which then hits the self.data property which will call ``_ImageBaseHDU._get_scaled_image_data`` and the data will be updated and the headers will be also. \r\n\r\nSo the easy fit for this is just to hit the data in ``_ImageBaseHDU._prewriteto`` so that the checksum gets updated correctly. "}}