jsonstreamreader 0.0.1

A new Flutter package project.

Json Stream Reader Beta

A Flutter Json Stream Reader

Getting Started

This project is processes a local json file by splitting it into smaller chunks, with automatic garbage collecting.

1. Create file and streamer instance

File file = new File(path.path + '/citylots.json');

Streamer s = new Streamer(file);

/* or Streamer streamer = new Streamer(file,chunksize=200); */

default chunksize is 100(100kb).

2. Create Reader instance

Reader reader = new Reader(streamer);

3. Register a function to recieve data, your options are
   • filter(String expression,void (dynamic value, String key)) - value is either an JsonObject or a String
   • trailing(void (dynamic value, String key))
   • pipe(void (dynamic value, String key)) - With pipe only array values(example in [3] array value is 3) or completed - objects(example {"3":4,"5":6} will be processed but {"3":4, will never be, in which case you should use trailing.
4. Example

getApplicationDocumentsDirectory().then((Directory path) {

File file = new File(path.path + '/citylots.json');

Streamer streamer = new Streamer(file);

Reader r = new Reader(streamer);

r.filter("\$.*", (dynamic value, String key) {

>     //items are received here

> 
>    print(value);
>

})

.done(() {//called when all items have been processed

>     print("Completed");

> 

}).fail((err) {//called when and if processing fails

>     print(err);

});

}).catchError((error) {

>     print(error);

>

});

How are keys Represented?

Root is represented as /

1. Example 1

{
    "glossary": {
        "title": "example glossary",
		"GlossDiv": {
            "title": "S",
			"GlossList": {
                "GlossEntry": {
                    "ID": "SGML",
					"SortAs": "SGML",
					"GlossTerm": "Standard Generalized Markup Language",
					"Acronym": "SGML",
					"Abbrev": "ISO 8879:1986",
					"GlossDef": {
                        "para": "A meta-markup language, used to create markup languages such as DocBook.",
						"GlossSeeAlso": ["GML", "XML"]
                    },
					"GlossSee": "markup"
                }
            }
        }
    }
}

• object['glossary'] has a key of /
• object['glossary']['title'] has a key of /glossary/
• object['glossary']['GlossDiv'] has a key of /glossary/
• object['glossary']['GlossDiv']['title'] has a key of /glossary/title/
• object['glossary']['GlossDiv']['GlossList']['GlossEntry'] has a key of /glossary/GlossDiv/GlossList

So from Root(/) key is Root + key/

2. Example 2

[ {"age":12}, 2, 3, 4 ]

• object[0]['age'] has a key of /0/age/
• object[1] has a key of /1/

Using Expression in filter

reader.filter(Expression, (dynamic value, String key) {

>     //items are received here

> 
>    print(value);
>

})

1. If you start your expression with "/" normal regular expressions rules apply so for object[0]['age'] I can use "\/0\/age\/". You have to escape "/".

   reader.filter("\/0\/age\/", (dynamic value, String key) {

   //items are received here

   print(value);

   })

2. For all children of object['glossary']['GlossDiv'] I can use "\/glossary\/GlossDiv\/.*".

   reader.filter("\/glossary\/GlossDiv\/.*", (dynamic value, String key) {

   //items are received here

   print(value);

   })

Using JsonPath "Hack" in filter

Expression	Path
$.	^\/$
[0:9]	[0-9]\/$
[0:9]+	[0-9]+\/$
..	\/.*\/.*\/$
...	\/.*\/.*\/.*\/$
.*	.*
[*]	.*

1. so for object[0]['age'] I can use "$.0.age" which is equivalent to "^\/0\/age\/$".

   reader.filter("$.0.age", (dynamic value, String key) {     //items are received here     print(value); })

2. For all children of object['glossary']['GlossDiv'] I can use $.glossary.GlossDiv.* which is equivalent ot "\/glossary\/GlossDiv\/.*".

   reader.filter("$.glossary.GlossDiv.*", (dynamic value, String key) {

   //items are received here

   print(value);

   })

This project was tested using

1. https://github.com/zemirco/sf-city-lots-json/blob/master/citylots.json
2. https://github.com/thaiwsa/aws-speed/blob/master/JsonProcess/jsondata/02.json
3. http://www.carqueryapi.com/api/0.3/?callback=?&cmd=getMakes&year=1970&sold_in_us=1&utm_medium=referral&utm_campaign=ZEEF&utm_source=https%3A%2F%2Fjson-datasets.zeef.com%2Fjdorfman