Thursday, 27 November 2014

Cassini - Getting the image data and something about the file format

the next few posts will cover images from the Cassini probe.

Getting the data

The main repository I'm grabbing data from is http://pds-imaging.jpl.nasa.gov/volumes/iss.html, and specifically the "Saturn EDR Data Sets" (EDR is "Experimental Data Record").

Note that the early versions of these volumes are primarily calibration data, and include Earth, Venus and Jupiter data sets. Spinning forward to a recent version then let's take
wget http://pds-imaging.jpl.nasa.gov/data/cassini/cassini_orbiter/coiss_2087.tar.gz
This gets us the 367M file coiss_2087.tar.gz, and this contains a bunch of Vicar format files. Vicar file formats date back to the 70's, and the Wikipedia Page contains some links to the data format, and converters.

Going into a bit more detail....

Vicar files and the detached Label file

The files under data/ come in pairs - one is the image file, and one is a detached header label, for example:
N1767153499_1.IMG
N1767153499_1.LBL
(The "'N" means this is from the Narrow Angle camera, and the leading 10 digit number is the spacecraft clock time at which the image was taken. The "_1" part is a revision label, which can be modified when the data is updated).

The Detached Label (.LBL) file should look familiar after parsing the Voyager object records - Similar to our Voyager header records this is basically a set of tag=value entries, with support for some specific per-Object namespaces. It also contains references to associated EDR file and the description of the file for the Vicar format as well (relative directory paths).

The files are described in the document/edrsis.txt file, which references the NASA specifications, but basically for the Vicar format image file then the format is as described in http://www-mipl.jpl.nasa.gov/PAG/public/vug/vug9.html.

So, when parsing into the Vicar file itself, we have the basic areas:
  • The VICAR Label
  • The Binary Label Header
  • The Image region
  • The End of Data Set label
Only the VICAR Label and Image region need to be in the file, and the other fields are optional.

The Image region contains lines of Pixel data, which may be prefixed with a Binary Prefix. This prefix is optional, but can be used to store per-line information.

The VICAR label must be parsed to get the image data format and layout. As with the Voyager files this is basically Key/Value pairs, however unlike Voyager they are not explicitly length delimited. In this case the key/value pairs are seperated by spaces. This has the complication that some quoted sequences may contain spaces, such as
  GAIN_MODE_ID='29 ELECTRONS PER DN'

For the files we're examining the field separators are actually multiple spaces, but to be slightly more robust we can look at the quoting.

Also the System Items in a VICAR header are in a fixed order, which should be enough to give us all the information we need to parse the image section. The first item should be a "LBLSIZE" entry, which will tell us the size of the label field, and we can go on and parse from there. However because we're going to be dealing with Venus express PDS files, which embed Vicar headers, then we won't be strict about this being the first item in the file (or the ordering).