fact.io.hdureader

Class ZFITSHeapReader

java.lang.Object

fact.io.hdureader.ZFITSHeapReader

All Implemented Interfaces:

Reader, Iterable<OptionalTypesMap<String,Serializable>>, Iterator<OptionalTypesMap<String,Serializable>>

public final class ZFITSHeapReader
extends Object
implements Reader

This reader can interpret the bytes in the heap of a bintable that has been stored in the ZFits format. Some specification of that format can be found on arxiv : https://arxiv.org/pdf/1506.06045v1.pdf

This class implements the iterable and iterator interfaces which allow for such nice things as

for(OptionalTypesMap p : heapReader){ assertTrue(p.containsKey("Data")); assertTrue(p.size() == 9); }

The next() and the getNextRow() methods deliver a map with the data from one row in the heap.

Reading the heap in a ZFits bintable works as follows:

The heap data starts at the offset stored in the 'ZHEAPTR' variable in the header. This is wrong according to the FITS standard. It should be the 'THEAP' variable.

Each heap contains one TileHeader for each row with one BlockHeader for each column in the row. Information about the data in the columns can be read from the 'ZFORMn' keywords in the header.

Data in the tiles is in Little Endian byte order.

The data from one tile is read completely into one bytebuffer.

See Also:

Then for each column the appropriate compression type is parsed from the block header.,

When a column is huffman compressed, the definition of the huffmann tree will be stored in the first couple of bytes in that row. This is documented exactly nowhere!

First we start with

[(int) numberOfCompressedBytes, (long) numberOfShortsAfterDecompression, (long) numberOfSymbolsUsedInCompression]

after that for each symbol (numberOfSymbolsUsedInCompression)

[(short) symbol, (byte) codeLengthInBits, the codeword stored in codeLengthInBits-rounded-up-to-bytes bytes (max 4 bytes)]

From these symbols and codewords the tree can be build see the ByteWiseHuffmanTree for details.,

Created by mackaiver on 14/11/16.

Method Summary

Modifier and Type	Method and Description
static ZFITSHeapReader	forTable(BinTable binTable) Creates a ZFITSHeapReader for a given binary table.
OptionalTypesMap<String,Serializable>	getNextRow() Get the data from the next row.
OptionalTypesMap<String,Serializable>	getNextRow(boolean ignoreWrongTileHeader) Get the data from the next row.
boolean	hasNext() Check whether there is another row to return from this heap
void	skipRows(int amount) Skips the given number of rows.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface fact.io.hdureader.Reader

iterator, next

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Method Detail

hasNext

public boolean hasNext()

Check whether there is another row to return from this heap

Specified by:

hasNext in interface Reader

Specified by:

hasNext in interface Iterator<OptionalTypesMap<String,Serializable>>

Returns:

true iff another row can be read.

forTable

public static ZFITSHeapReader forTable(BinTable binTable)

Creates a ZFITSHeapReader for a given binary table. This reader is iterable and provides a @see next() method which will yield the column data for each row.

Parameters:

binTable - the binary table which contains a zfits heap.

Returns:

the ZFITSHeapReader for the given table.

skipRows

public void skipRows(int amount)
              throws IOException

Skips the given number of rows.

Specified by:

skipRows in interface Reader

Parameters:

amount - The amount of rows to skip.

Throws:

IOException

getNextRow

public OptionalTypesMap<String,Serializable> getNextRow()
                                                 throws IOException

Get the data from the next row. The columns in the row can be accessed by their name in the resulting map that is returned by this method. The resulting map comes with convenience methods for accessing data of predefined types.

Specified by:

getNextRow in interface Reader

Returns:

a map containing the the data from the rows columns.

Throws:

NoSuchElementException - iff hasNext() is false

IOException - if some IO error occurs

getNextRow

public OptionalTypesMap<String,Serializable> getNextRow(boolean ignoreWrongTileHeader)
                                                 throws IOException

Get the data from the next row. The columns in the row can be accessed by their name in the resulting map that is returned by this method. The resulting map comes with convenience methods for accessing data of predefined types.

Some zfits file apparently contain a wrong tile header in the heap storing the ZDrsCellOffsets. I haven't figures out to which files this applies. Files from 2013 seem to have that problem. But also newer ones. Some files from 2016 are fine however.

The old ZFits implementation by M.Bulinski ignored the problem by default.

Parameters:

ignoreWrongTileHeader - if true, ignores wrong tileheader information.

Returns:

a map containing the the data from the rows columns.

Throws:

NoSuchElementException - iff hasNext() is false

IOException