10B format.

To save 10 bits data a special format is needed. This is a first attempt to explain the format. Note: C-based. Please let me know if something is not clear.

Header.

The header is 16000 bytes. This sounds much but is still a very small fraction of a typical HRPT file. Most of the space is reserved for color maps for all channels (not used yet).

Here is the header description in C.
Note:

char=8 bytes
short=16 bytes
long=32 bytes
Order of bytes in multi-byte type: lower to higher significant (INTEL order, so-called 'little-endian'). E.g.: short: first byte is LSBytes, second is MSByte.
The fill bytes show where most systems would add bytes anyway, even if these fill bytes are not there. This is because on these systems the following holds for words in C-structures:
1. 2-byte shorts start at even addresses
2. 4-byte longs and floats start at addresses multiple of 4
3. 1-byte chars can start anywhere
The program was originally designed on a DOS-system. Here, each type can start anywhere in a struct. In the DOS-program this header is written and read by means of fwrite and fread commands.
To keep it as portable as possible the header as defined here has to be read in 4 parts (also in DOS!):
```
fread(hrpthdr,131,1,fp);
fread((char *)&hrpthdr->offset_lon,42,1,fp);
fread((char *)&hrpthdr->kr[0],105,1,fp);
fread((char *)&hrpthdr->widthmax,15722,1,fp);
```
The same holds for writing:
```
fwrite(hrpthdr,131,1,fp);
fwrite((char *)&hrpthdr->offset_lon,42,1,fp);
fwrite((char *)&hrpthdr->kr[0],105,1,fp);
fwrite((char *)&hrpthdr->widthmax,15722,1,fp);
```


#define HRPTSIGN "HRPT"
#define CHRPTSIGN "CHRP"
#define SEAWSIGN "SEAW"
#define ENDHDR "END_HDR_"
#define CMAPLEN10B 1024
#define FRAMESTART "FRAME"
#define LENSATNAME 16
#define LENSIGN 4
#define ENDFLG 8

/* Sub-record for time, similar to "tm", but with 2-bytes shorts 
   (important for portability to e.g. Linux) */
struct tm_sh
{
  short tm_sec,tm_min,tm_hour;
  short tm_mday,tm_mon,tm_year,tm_wday,tm_yday;
  short tm_isdst;
};

/* Sub-record with Kepler info */
typedef struct 
{
  long noradnr;
  long internat;
  long year;
  float daynr;
  float decayrate;
  float inclination;
  float raan;
  float exentricity;
  float perigee;
  float anomaly;
  float revs;
  long rev_orbit;
} KEPLER;


The actual header of the file:

typedef struct
{
  char sign[LENSIGN];
  char satname[LENSATNAME];
  char chan_tot;
  char filetype;
  short width,height;              /* width < 2048 if zoomed  */
  short depth;
  char offset_gmt,offset_sec;
  struct tm_sh tm_comp;            /* 48 */
  KEPLER kepler;

  char hdr_version[ENDFLG];           /*  */
  short hdrlen;                    /* length of header */
  short x_offset,y_offset;         /* Offset from original if zoomed  */
  char satdir,reserve2;            /*  */

  struct tm_sh tm_satstrt;
  char tm_comp_sat_vlg;
  char fill1;                       /* See note! */

  float offset_lon;
  short dmin[8],dmax[8];
  short ro,go,bo;
  short fill2;                      /* See note! */

  float kr[8],kg[8],kb[8];          /* linear combination of channels */
  char channel[9];                  /* selected channels; 0 = header recorded */
  char fill3;                       /* See note!  */

  short widthmax;
  float alphamax;
  char r[5][CMAPLEN10B],g[5][CMAPLEN10B],b[5][CMAPLEN10B];
  char reserve[348];                /* reserved-bytes */
  char endhdr[ENDFLG];              /* 15786 */
} HRPTHDR;

Note: The element:

  short hdrlen;                    /* length of header */

gives the possibility to change header lengths without compability problems between vesrions of software used to record and versions used to process.

Data part.

Data is grouped in pieces of 4 words:

word nr. 1, bits 1-8
word nr. 2, bits 1-8
word nr. 3, bits 1-8
word nr. 4, bits 1-8
LSB of previous 4 words:
- word 1 bits 9-10
- word 2 bits 9-10
- word 3 bits 9-10
- word 4 bits 9-10

E.g.: The first 4 words in a frame are part of the 6 words synchronisation part. These are in hex:

284 16F 35C 19D

In bits:


10 1000 0100 regrouped: 1010 0001 00
01 0110 1111 regrouped: 0101 1011 11
11 0101 1100 regrouped: 1101 0111 00
01 1001 1101 regrouped: 0110 0111 01

In the 10B format this is:


1010 0001 (first 8 bits of first word)
0101 1011 (first 8 bits of second word)
1101 0111 (first 8 bits of third word)
0110 0111 (first 8 bits of fourth word)
0011 0001 (combined LSBs of previous 4 words)

So, in hex:

A1 5B D7 67 31

Each data frame is separated by a sub-header of 7 bytes:

5 bytes, ascii F R A M E (if in-sync) or F R A M e (if out-sync)
2 bytes forming a short, giving the number of bytes for 1 frame (= 1 line)

The 2-bytes word doesn't include the 7 bytes of the sub-header.
Example: If the file contains all channels including frame header, and if it contains all 2048 pixels per channel per horizontal line the number is 13865 (hex 3629). So:

F R A M E 29 36 [13865 bytes] F R A M E 29 36 [13865 bytes] ....

etc. Calculation this number can be done as:

(nr_x*nr_ch + 852)*5/4

If no frame header is recorded then ommit the number '852'.
(Note: The frame header contains 750 words; each line contains 100 extra "auxiliarly sync words'. Together 850; however, this doesn't give a integer after multiplying with 5/4. Therefore 2 extra words are added.)

Order of words are the same as how they are transmitted. So, if 3 channels are recorded:

8 MSbits of channel 1
8 MSbits of channel 2
8 MSbits of channel 3
8 MSbits of channel 1
4x 2 LSBs of previous 4 words:
- 2 bits of channel 1
- 2 bits of channel 2
- 2 bits of channel 3
- 2 bits of channel 1
8 MSbits of channel 2
8 MSbits of channel 3
8 MSbits of channel 1
8 MSbits of channel 2
4x 2 LSBs of previous 4 words:
- 2 bits of channel 2
- 2 bits of channel 3
- 2 bits of channel 1
- 2 bits of channel 2

Etc.

The complete 10B format looks like:

16000 bytes header
7 bytes sub-header
data of first frame (with selected channels)
7 bytes sub-header
data of second frame (with selected channels)
...
7 bytes sub-header
data of last frame (with selected channels)

Note that, to extract a channel from this format, it is necessary to know how much channels are recorded. This can be found in the header in:

char channel[9];

Meaning:

channel[0] != 0: header is recorded
channel[1] != 0: channel 1 recorded
channel[2] != 0: channel 2 recorded
channel[3] != 0: channel 3 recorded
channel[4] != 0: channel 4 recorded
channel[5] != 0: channel 5 recorded
channel[6]-[8]: same meaning; extension for Seastar

CHRPT-header

The only difference between the HRPT and CHRPT-header is the last part of the header.


typedef struct
{
  char sign[LENSIGN];
  char satname[LENSATNAME];
  char chan_tot;
  char filetype;
  short width,height;
  short depth;
  char offset_gmt,offset_sec;
  struct tm_sh tm_comp;
  KEPLER kepler;

  char hdr_version[ENDFLG];
  short hdrlen;
  short x_offset,y_offset;
  char satdir,reserve2;

  struct tm_sh tm_satstrt;
  char tm_comp_sat_vlg;
  char fill1;                         /* See note!  */

  float offset_lon;
  short dmin[10],dmax[10];
  short ro,go,bo;
  short fill2;                        /* See note!  */

  float kr[10],kg[10],kb[10];
  char channel[11];
  char fill3;                         /* See note!  */

  short widthmax;
  char fill4[2];                      /* See note!  */

  float alphamax;
  char r[5][CMAPLEN10B],g[5][CMAPLEN10B],b[5][CMAPLEN10B];
  char reserve[330];
  char endhdr[ENDFLG];
} CHRPTHDR;