fact.io

Class RecursiveDirectoryStream

java.lang.Object

stream.io.AbstractStream

stream.io.multi.AbstractMultiStream

fact.io.RecursiveDirectoryStream

All Implemented Interfaces:

stream.io.multi.MultiStream, stream.io.Source, stream.io.Stream

public class RecursiveDirectoryStream
extends stream.io.multi.AbstractMultiStream

Given a glob like a pattern describing files a folder _relative_ to the provided url, starts a stream working on all files sequentially.

For example:

finds all files below (recursively) /some/folder with their names matching the pattern. See unix globs for more information on pattern syntax

Created by mackaiver on 12/15/15.

Field Summary

Fields

Modifier and Type	Field and Description
BlockingQueue<Path>	files
int	maxDepth
String	pattern
boolean	skipErrors

Fields inherited from class stream.io.multi.AbstractMultiStream

activate, activeWrapper, additionOrder, streams

Fields inherited from class stream.io.AbstractStream

closed, count, id, in, limit, prefix, seqId, sequenceKey, url

Fields inherited from interface stream.io.Stream

SOURCE_KEY

Constructor Summary

Constructors

Constructor and Description
RecursiveDirectoryStream(stream.io.SourceURL url)

Method Summary

Modifier and Type	Method and Description
void	close()
void	init()
stream.Data	readNext()

Methods inherited from class stream.io.multi.AbstractMultiStream

addStream, getActivate, getStreams, setActivate

Methods inherited from class stream.io.AbstractStream

getId, getInputStream, getLimit, getPrefix, getSequenceKey, getUrl, read, setId, setLimit, setPrefix, setSequenceKey, setUrl

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface stream.io.Stream

getLimit, setLimit

Methods inherited from interface stream.io.Source

getId, read, setId

Field Detail

files

public BlockingQueue<Path> files

maxDepth

@Parameter(required=false,
           description="Maximum depth of folders to traverse",
           defaultValue="6")
public int maxDepth

pattern

@Parameter(required=true,
           description="The pattern to filter files by. Understands usual glob syntax")
public String pattern

skipErrors

@Parameter(required=false,
           description="If false the reading of a broken file throws an exception and the process is aborted, if true the next file will be processed",
           defaultValue="true")
public boolean skipErrors

Constructor Detail

RecursiveDirectoryStream

public RecursiveDirectoryStream(stream.io.SourceURL url)

Method Detail

init

public void init()
          throws Exception

Specified by:

init in interface stream.io.Source

Overrides:

init in class stream.io.multi.AbstractMultiStream

Throws:

Exception

readNext

public stream.Data readNext()
                     throws Exception

Specified by:

readNext in class stream.io.AbstractStream

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface stream.io.Source

Overrides:

close in class stream.io.multi.AbstractMultiStream

Throws:

Exception