Tiled Image Compression

The CFITSIO interface for reading and writing FITS files (and hence most programs that use it) supports a new image compression format in which the image is divided into a grid of rectangular tiles and then each tile of pixels is individually compressed. The compressed tiles are stored in rows of a variable length array column in a FITS binary table, but CFITSIO recognizes that the binary table extension contains an image and treats it as if it were an IMAGE extension. This tile-compressed format is especially well suited for compressing very large images because a) the FITS header keywords remain uncompressed for rapid read access, and because b) it is possible to extract and uncompress sections of the image without having to uncompress the entire image. This format is also much more effective in compressing floating point images (using a lossy compression algorithm) than simply compressing the image using gzip or compress.

A detailed description of this format is available at: http://heasarc.gsfc.nasa.gov/docs/software/fitsio/compression/compress_image.html

The N-dimensional FITS image can be divided into any desired rectangular grid of compression tiles. By default the tiles are chosen to correspond to the rows of the image, each containing NAXIS1 pixels. For example, a 800 x 800 x 4 pixel data cube would be divided in to 3200 tiles containing 800 pixels each by default. Alternatively, this data cube could be divided into 256 tiles that are each 100 X 100 X 1 pixels in size, or 4 tiles containing 800 x 800 X 1 pixels, or a single tile containing the entire data cube. Note that the image dimensions are not required to be an integer multiple of the tile dimensions, so, for example, this data cube could also be divided into 250 X 200 pixel tiles, in which case the last tile in each row would only contain 50 X 200 pixels.

Currently, 3 image compression algorithms are supported: Rice, GZIP, and PLIO. Rice and GZIP are general purpose algorithms that can be used to compress almost any image. The PLIO algorithm, on the other hand, is more specialized and was developed for use in IRAF to store pixel data quality masks. It is designed to only work on images containing positive integers with values up to about 2**24. Other image compression algorithms may be supported in the future.

The 3 supported image compression algorithms are all 'loss-less' when applied to integer FITS images; the pixel values are preserved exactly with no loss of information during the compression and uncompression process. Floating point FITS images (which have BITPIX = -32 or -64) are first quantized into scaled integer pixel values before being compressed. This technique produces much higher compression factors than simply using GZIP to compress the image, but it also means that the original floating value pixel values may not be precisely returned when the image is uncompressed. When done properly, this only discards the 'noise' from the floating point values without losing any significant information. The amount of noise that is discarded can be controlled by the 'noise_bits' compression parameter.

No special action is required to read tile-compressed images because all the CFITSIO routines that read normal uncompressed FITS images can also read images in the tile-compressed format; CFITSIO essentially treats the binary table that contains the compressed tiles as if it were an IMAGE extension.

When a program creates and writes a new output image with CFITSIO, the user can indicate that images should be written in tile-compressed format by enclosing the compression parameters in square brackets following the root disk file name. The `imcopy' example program that included with the CFITSIO distribution can be used for this purpose to compress or uncompress images. Here are some examples of the extended file name syntax for specifying tile-compressed output images:

    myfile.fit[compress]    - use the default compression algorithm (Rice)
                              and the default tile size (row by row)

    myfile.fit[compress GZIP] - use the specified compression algorithm;
    myfile.fit[compress Rice]     only the first letter of the algorithm 
    myfile.fit[compress PLIO]     name is required.

    myfile.fit[compress R 100,100]   - use Rice compression and 
                                       100 x 100 pixel tile size

    myfile.fit[compress R 100,100;2] - as above, and also use noisebits = 2

The fimgzip and fimgunzip tasks in the HEASOFT software package can be used to compress or uncompress images in this tiled compression format.


filenames, imfilter, colfilter, rowfilter, binfilter, calc_express,