fact.io

Class FactFileListMultiStream

java.lang.Object

stream.io.AbstractStream

stream.io.multi.AbstractMultiStream

fact.io.FactFileListMultiStream

All Implemented Interfaces:

stream.io.multi.MultiStream, stream.io.Source, stream.io.Stream

public class FactFileListMultiStream
extends stream.io.multi.AbstractMultiStream

Takes a json file of the form

[ { "drs_path":"\/fact\/raw\/2014\/10\/02\/20141002_193.drs.fits.gz", "data_path":"\/fact\/raw\/2014\/10\/02\/20141002_185.fits.fz", ... }, {...} ]

and creates a single stream for the files listed. The dictionaries in the json may of course contain more keys. The 'drsPathKey' and `dataPathKey` parameter define the names of the keys to the file paths. So in the example above they would need to be set to "drs_path" and "data_path" which are the default values. A key called '@drsFile' will be injected into the DataStream by this multistream. That means when you're using this stream you don't need to set the `url` parameter of the DrsCalibration processor. If the underlying stream throws an IOexception in case of missing files, the next file will be tried when the skipErrors flag is set.

Created by mackaiver on 4/10/15.

Field Summary

Fields

Modifier and Type	Field and Description
String	dataPathKey
String	drsPathKey
BlockingQueue<fact.io.FactFileListMultiStream.DataDrsPair>	fileQueue
boolean	skipEmpty
boolean	skipErrors
stream.io.SourceURL	url

Fields inherited from class stream.io.multi.AbstractMultiStream

activate, activeWrapper, additionOrder, streams

Fields inherited from class stream.io.AbstractStream

closed, count, id, in, limit, prefix, seqId, sequenceKey

Fields inherited from interface stream.io.Stream

SOURCE_KEY

Constructor Summary

Constructors

Constructor and Description
FactFileListMultiStream(stream.io.SourceURL url)

Method Summary

Modifier and Type	Method and Description
void	close()
void	init() Read the json file provided by the url parameter and build a queue of File objects to be analyzed
stream.Data	readNext()
void	setUrl(stream.io.SourceURL url)

Methods inherited from class stream.io.multi.AbstractMultiStream

addStream, getActivate, getStreams, setActivate

Methods inherited from class stream.io.AbstractStream

getId, getInputStream, getLimit, getPrefix, getSequenceKey, getUrl, read, setId, setLimit, setPrefix, setSequenceKey

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface stream.io.Stream

getLimit, setLimit

Methods inherited from interface stream.io.Source

getId, read, setId

Field Detail

fileQueue

public final BlockingQueue<fact.io.FactFileListMultiStream.DataDrsPair> fileQueue

url

@Parameter(required=true,
           description="A file containing a json array of dicts with the paths to the files.")
public stream.io.SourceURL url

skipErrors

@Parameter(required=false,
           description="Flag indicating whether next file should be tried in case of errors in underlying stream.",
           defaultValue="false")
public boolean skipErrors

skipEmpty

@Parameter(required=false,
           description="Similar to skipErrors but will only skip the MissingHDU exception, i.e. empty files",
           defaultValue="false")
public boolean skipEmpty

drsPathKey

@Parameter(required=false,
           defaultValue="drs_path")
public String drsPathKey

dataPathKey

@Parameter(required=false,
           defaultValue="data_path")
public String dataPathKey

Constructor Detail

FactFileListMultiStream

public FactFileListMultiStream(stream.io.SourceURL url)

Method Detail

init

public void init()
          throws Exception

Read the json file provided by the url parameter and build a queue of File objects to be analyzed

Specified by:

init in interface stream.io.Source

Overrides:

init in class stream.io.multi.AbstractMultiStream

Throws:

Exception - might be thrown in case the json cannot be read.

readNext

public stream.Data readNext()
                     throws Exception

Specified by:

readNext in class stream.io.AbstractStream

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface stream.io.Source

Overrides:

close in class stream.io.multi.AbstractMultiStream

Throws:

Exception

setUrl

public void setUrl(stream.io.SourceURL url)

Overrides:

setUrl in class stream.io.AbstractStream