Ingest JSON
Many systems export information in JSON format and CFS has features to help you convert JSON into IDOL documents.
JSON files are parsed as follows:
- Any document that has the metadata field
AUTN_JSON_PARSING_SECTION
is parsed using the settings specified in the section of the CFS configuration file that matches the field value. - All other JSON documents are parsed using the settings specified in the
[JSONParsing]
section of the CFS configuration file.
For information about the JSON parsing configuration parameters, refer to the Connector Framework Server Reference.
The CFS JSON reader accepts a stream of JSON document objects:
{"DREREFERENCE": "one"}{"DREREFERENCE": "two"}{"DREREFERENCE": "three"}
Or an array of JSON document objects:
[{"DREREFERENCE": "one"},{"DREREFERENCE": "two"},{"DREREFERENCE": "three"}]
Example
Consider the following JSON:
{ "ref": "12345abcd", "someField": [ "value1", "value2" ], "otherfield": { "field_name": "myfield", "@attribute": true, "attrs": { "attr1": "fish", "attr2": "chips", "attr3": "peas" }, "field_value": { "attr_name": "key", "attr_value": "value", "interesting": 7 }, "someField": [ 12.9 ] }, "content_stuff": { "1": "page1", "2": [ "page2", "page3" ] } }
To ingest this JSON, you might use the following configuration:
[JSONParsing] ReferencePath=ref ContentPaths=content_stuff/^.*$ FieldSections=SomeField,OtherField ExpandArrays=true [SomeField] Path=someField/^.*$ [OtherField] Path=otherfield NamePath=field_name ValuePath=field_value/interesting FieldSections=SomeField AttributeSections=OtherFieldAttr,OtherFieldAttr2,OtherFieldAttr3 [OtherFieldAttr] Path=^@(attribute)$ [OtherFieldAttr2] NamePath=field_value/attr_name ValuePath=field_value/attr_value [OtherFieldAttr3] Path=attrs/^.*$
To ingest the JSON, send the ingest action to CFS:
http://localhost:7000/action=ingest&adds=%3Cadds%3E%3Cadd%3E%3Csource%20 filename%3D%22file.json%22%20 lifetime%3D%22permanent%22%20%2F%3E %3C%2Fadd%3E%3C%2Fadds%3E
This would produce the following document:
#DREREFERENCE 12345abcd #DREFIELD myfield="7" #DREFIELD myfield/@attribute="true" #DREFIELD myfield/@key="value" #DREFIELD myfield/@attr1="fish" #DREFIELD myfield/@attr2="chips" #DREFIELD myfield/@attr3="peas" #DREFIELD myfield/someField="12.9" #DREFIELD someField="value1" #DREFIELD someField="value2" #DRESECTION 0 #DRECONTENT page1 #DREENDDOC #DREREFERENCE 12345abcd #DRESECTION 1 #DRECONTENT page2 #DREENDDOC #DREREFERENCE 12345abcd #DRESECTION 2 #DRECONTENT page3 #DREENDDOC