{"action":"created","issue":{"url":"https://api.github.com/repos/astropy/astropy/issues/3746","labels_url":"https://api.github.com/repos/astropy/astropy/issues/3746/labels{/name}","comments_url":"https://api.github.com/repos/astropy/astropy/issues/3746/comments","events_url":"https://api.github.com/repos/astropy/astropy/issues/3746/events","html_url":"https://github.com/astropy/astropy/issues/3746","id":72224676,"number":3746,"title":"HIERARCH keywords don't work with CONTINUE cards","user":{"login":"Knusper","id":4704406,"avatar_url":"https://avatars.githubusercontent.com/u/4704406?v=3","gravatar_id":"","url":"https://api.github.com/users/Knusper","html_url":"https://github.com/Knusper","followers_url":"https://api.github.com/users/Knusper/followers","following_url":"https://api.github.com/users/Knusper/following{/other_user}","gists_url":"https://api.github.com/users/Knusper/gists{/gist_id}","starred_url":"https://api.github.com/users/Knusper/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Knusper/subscriptions","organizations_url":"https://api.github.com/users/Knusper/orgs","repos_url":"https://api.github.com/users/Knusper/repos","events_url":"https://api.github.com/users/Knusper/events{/privacy}","received_events_url":"https://api.github.com/users/Knusper/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/astropy/astropy/labels/Docs","name":"Docs","color":"02d7e1"},{"url":"https://api.github.com/repos/astropy/astropy/labels/io.fits","name":"io.fits","color":"c7def8"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-04-30T17:18:58Z","updated_at":"2015-05-01T11:08:44Z","closed_at":null,"body":"The `astropy.io.fits` manual states, that we can use header keywords longer than 8-characters. In this case [HIERARCH cards will be created][1]. The manual also states, that if we want to store keyword-value pairs longer than 80-characters, [continue cards will automatically be created][2].\r\n\r\nHowever, in practice it seems that both definitions work only mutually exclusive, i.e. we can not create a FITS file containing a keyword value pair, where the keyword is longer than 8-characters (i.e. a HIERARCH keyword) and the value is a  very long string.\r\n\r\nAn example:\r\n\r\n    from astropy.io import fits\r\n    \r\n    header1 = fits.Header()\r\n    header2 = fits.Header()\r\n    header3 = fits.Header()\r\n    \r\n    header1['TEST'] = 'superlongstring'*10\r\n    header2['TEST TEST'] = 'superlongstring'\r\n    header3['TEST TEST'] = 'superlongstring'*10\r\n\r\nHere `header1` and `header2` will be correct, but when calling `repr(header3)` or attempting to save a FITS file with such a header, the error `ValueError: The keyword TEST TEST with its value is too long` is raised.\r\n\r\n\r\n[1]: http://docs.astropy.org/en/stable/io/fits/usag/headers.html#hierarch-cards\r\n[2]: http://docs.astropy.org/en/stable/io/fits/usag/headers.html#continue-cards\r\n\r\n\r\nLooking at the `astropy` source code, it is explicitly ruled out to use CONTINUE cards in a value belonging to a HIERARCH keyword.\r\n\r\nIn the source code we find ([astropy/io/fits/card.py#L1227-1236](https://github.com/astropy/astropy/blob/v1.0.x/astropy/io/fits/card.py#L1227-1236)):\r\n\r\n    keywordvalue_length = len(keyword) + len(delimiter) + len(value)\r\n    if (keywordvalue_length > self.length and\r\n            keyword.startswith('HIERARCH')):\r\n        if (keywordvalue_length == self.length + 1 and keyword[-1] == ' '):\r\n            output = ''.join([keyword[:-1], delimiter, value, comment])\r\n        else:\r\n            # I guess the HIERARCH card spec is incompatible with CONTINUE\r\n            # cards\r\n            raise ValueError('The keyword %s with its value is too long' %\r\n\t                         self.keyword)\r\n\r\n\r\nThe comment is from 2011, and was put there during a redesign of pyfits. Before that, pyfits could also read or write exlusively either HIERARCH cards or cards with CONTINUE statement, and apparently it was kept like that. The relevant [old PyFITS code is](https://github.com/spacetelescope/PyFITS/blob/2ae676b84060d78fd59e6b843995dfec74b5c0a7/lib/pyfits/card.py#L383-387):\r\n\r\n    if cardimage[:8].upper() == 'HIERARCH':\r\n\t\tcard = _HierarchCard()\r\n\t\t# for card image longer than 80, assume it contains CONTINUE card(s).\r\n    elif len(cardimage) > Card.length:\r\n\t\tcard = _ContinueCard()\r\n\r\nBoth, the HIERARCH and the CONTINUE keywords are not part of the official [FITS standard (Pence, W.D. et al. 2010, Astronomy & Astrophysics Vol. 524, A42)](http://dx.doi.org/10.1051/0004-6361/201015362). The HIERARCH keyword convention is given in [Wiecencec, A. et al. 2009, \"The ESO HIERARCH Keyword Convention\")](http://fits.gsfc.nasa.gov/registry/hierarch_keyword.html) and the CONTINUE convention is given by the [HEASARC FITS Working Group, 2007. \"The CONTINUE Long String Keyword Convention\"](http://fits.gsfc.nasa.gov/registry/continue/continue.pdf). Reading carefully both of these definitions, I see absolutely no reason why they should be mutually exclusive. Hence, `astropy` should support `HIERARCH` keywords with `CONTINUE` values.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/astropy/astropy/issues/comments/98101212","html_url":"https://github.com/astropy/astropy/issues/3746#issuecomment-98101212","issue_url":"https://api.github.com/repos/astropy/astropy/issues/3746","id":98101212,"user":{"login":"olebole","id":397223,"avatar_url":"https://avatars.githubusercontent.com/u/397223?v=3","gravatar_id":"","url":"https://api.github.com/users/olebole","html_url":"https://github.com/olebole","followers_url":"https://api.github.com/users/olebole/followers","following_url":"https://api.github.com/users/olebole/following{/other_user}","gists_url":"https://api.github.com/users/olebole/gists{/gist_id}","starred_url":"https://api.github.com/users/olebole/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olebole/subscriptions","organizations_url":"https://api.github.com/users/olebole/orgs","repos_url":"https://api.github.com/users/olebole/repos","events_url":"https://api.github.com/users/olebole/events{/privacy}","received_events_url":"https://api.github.com/users/olebole/received_events","type":"User","site_admin":false},"created_at":"2015-05-01T11:08:44Z","updated_at":"2015-05-01T11:08:44Z","body":"I agree that strictly spoken the `CONTINUE` extension is to be applied (only) to string values, while `HIERARCH` headers have no value (in the standard FITS meaning) at all.\r\n\r\nHowever, when reading both conventions ([`HIERARCH`](http://fits.gsfc.nasa.gov/registry/hierarch/hierarch.pdf) and [`CONTINUE`](http://fits.gsfc.nasa.gov/registry/continue/continue.pdf)), I would guess it is more an extension to `CONTINUE` than to `HIERARCH`. However, in my opinion the `CONTINUE` extension is made in a way that extending it would be compatible with all possible cases of FITS reader. Specifically, the `CONTINUE` extension says:\r\n\r\n>If a FITS reader encounters a CONTINUE keyword that is not preceded by a string keyword (or another CONTINUE keyword) whose value string ends with the `&' character, then that CONTINUE keyword should be ignored (i.e., it should be interpreted the same as a COMMENT keyword).\r\n\r\nLets look for a possible header:\r\n```\r\nHIERARCH ASTROPY EXAMPLE = 'this is a very &'\r\nCONTINUE 'long string value'\r\n```\r\n\r\nFor the possible cases of standard conform FITS readers this means:\r\n\r\n1. FITS reader that don't support `HIERARCH` and also not `CONTINUE`: They will ignore all `HIERARCH` and `CONTINUE` values, as expected;\r\n2. FITS reader that support `HIERARCH`, but not `CONTINUE`: they will shorten all strings (be them `HIERARCH` or not) to the first line, which is what one would expect,\r\n3. FITS reader that support `CONTINUE`, but not `HIERARCH`: They will ignore the `CONTINUE` line after a `HIERARCH` line (and the `HIERARCH` line itself), which is again what expected\r\n4. FITS reader that support mutually both keywords mutually exclusive: They will read the first line for `HIERARCH` keywords, but ignore a following `CONTINUE` card, again what one would expect\r\n5. A hypothetical FITS reader that supports both keywords at the same time, would get the full string.\r\n\r\nAlso, the extension to support `CONTINUE` cards after `HIERARCH` cards would be quite natural, there is not much danger that someone comes up with a alternative idea. Therefore, I could imagine that it would be not a mistake to just support this (maybe when a special flag is set). There is the danger that the long value may not be read by other readers, but there is no danger of misinterpretation."}}