fact.io

Class JSONWriter

java.lang.Object

fact.io.Writer

fact.io.JSONWriter

All Implemented Interfaces:

stream.Processor, stream.StatefulProcessor

public class JSONWriter
extends Writer
implements stream.StatefulProcessor

Writes a keys from the data item to .json files. The format will be:

 [
   {"key1": value, ...},
   ...,
   {"key1": value, ...}
 ]
 

 

keys is evaluated using the stream.Keys class, so wild cards *, ? and negations with ! are possible. This will write all keys ending with `Pointing` to the json file but not AzPointing.

 <fact.io.JSONWriter keys="*Pointing,!AzPointing" url="file:test.json" />
 

The writer also supports the .jsonl format. http://jsonlines.org/ To use the .jsonl format provide the key jsonl="true" in the xml.

In this case the format will be:

 {"key1": value, ...}
 ...
 {"key1": value, ...}
 

To be able to store special float values we use the extension of the json standard found in most implementations. E. g. Google's gson, most JavaScript parsers and python's json module. So we are using Infinity, -Infinity and NaN by default. python's pandas das not support this format directly, so use json to load the data and then create the DataFrame:

 import json
 import pandas as pd
 with open('test.json', 'r') as f:
     data = json.load(f)
 df = pd.DataFrame(data)
 

If you do not want this behaviour, you can use specialDoubleValuesAsString="true" to convert these values to json compatible strings containing "inf", "-inf" or "nan"

fact.container.PixelSet is converted to an array of chids by default, if you want to have the full output of this container, set pixelSetsAsInt="false"

The following keys are added by default to the output: EventNum, TriggerType, NROI, NPIX

By default, the JSONWriter overwrites an existing file, if you want to append (which actually only makes sense if jsonl="true), you can use: append="true"

The JSONWriter can also add gzip compression on the fly. Use the gzip Option to directly write gzip compressed files. append="true"

Created by bruegge on 7/30/14. Refactored by maxnoe on 2/2/2016

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	JSONWriter.DateTimeAdapter
class	JSONWriter.PixelSetAdapter
class	JSONWriter.SignDigitsAdapter
class	JSONWriter.SpecialDoubleValuesAdapter

Field Summary

Fields

Modifier and Type	Field and Description
boolean	allowNullKeys
boolean	append
boolean	caseSensitive
Integer	doubleSignDigits
boolean	gzip
boolean	jsonl
stream.Keys	keys
boolean	pixelSetsAsInt
boolean	specialDoubleValuesAsString
URL	url

Constructor Summary

Constructors

Constructor and Description
JSONWriter()

Method Summary

Modifier and Type	Method and Description
void	finish()
void	init(stream.ProcessContext processContext)
stream.Data	process(stream.Data data)
void	resetState()

Methods inherited from class fact.io.Writer

getDefaultKeys, isSimulated, testKeys

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keys

@Parameter(description="Keys to save to the outputfile, if not given, the default keys for observations and simulations are stored, taken from the default/settings.properties file")
public stream.Keys keys

doubleSignDigits

@Parameter(required=false,
           description="Defines how many significant digits are used for double values",
           defaultValue="null")
public Integer doubleSignDigits

jsonl

@Parameter(required=false,
           description="If true, use jsonl format instead of json format",
           defaultValue="false")
public boolean jsonl

append

@Parameter(required=false,
           description="If true, append to existing file else overwrite",
           defaultValue="false")
public boolean append

pixelSetsAsInt

@Parameter(required=false,
           description="If true, PixelSets are written out as int arrays of chids",
           defaultValue="true")
public boolean pixelSetsAsInt

specialDoubleValuesAsString

@Parameter(required=false,
           description="If true, Infinity, -Infinity and NaN are converted to strings \'inf\', \'-inf\' and \'nan\'",
           defaultValue="false")
public boolean specialDoubleValuesAsString

gzip

@Parameter(required=false,
           description="If true, use gzip compression")
public boolean gzip

allowNullKeys

@Parameter(required=false,
           description="Set if you want to allow empty keys.")
public boolean allowNullKeys

caseSensitive

@Parameter(required=false,
           description="If the matching of the keys should be done case sensitive")
public boolean caseSensitive

url

@Parameter(required=true)
public URL url

Constructor Detail

JSONWriter

public JSONWriter()

Method Detail

process

public stream.Data process(stream.Data data)

Specified by:

process in interface stream.Processor

init

public void init(stream.ProcessContext processContext)
          throws Exception

Specified by:

init in interface stream.StatefulProcessor

Throws:

Exception

resetState

public void resetState()
                throws Exception

Specified by:

resetState in interface stream.StatefulProcessor

Throws:

Exception

finish

public void finish()
            throws Exception

Specified by:

finish in interface stream.StatefulProcessor

Throws:

Exception