1 | # JSONStream
|
2 |
|
3 | streaming JSON.parse and stringify
|
4 |
|
5 | <img src=https://secure.travis-ci.org/dominictarr/JSONStream.png?branch=master>
|
6 |
|
7 | ## example
|
8 |
|
9 | ``` js
|
10 |
|
11 | var request = require('request')
|
12 | , JSONStream = require('JSONStream')
|
13 | , es = require('event-stream')
|
14 |
|
15 | request({url: 'http://isaacs.couchone.com/registry/_all_docs'})
|
16 | .pipe(JSONStream.parse('rows.*'))
|
17 | .pipe(es.mapSync(function (data) {
|
18 | console.error(data)
|
19 | return data
|
20 | }))
|
21 | ```
|
22 |
|
23 | ## JSONStream.parse(path)
|
24 |
|
25 | parse stream of values that match a path
|
26 |
|
27 | ``` js
|
28 | JSONStream.parse('rows.*.doc')
|
29 | ```
|
30 |
|
31 | The `..` operator is the recursive descent operator from [JSONPath](http://goessner.net/articles/JsonPath/), which will match a child at any depth (see examples below).
|
32 |
|
33 | If your keys have keys that include `.` or `*` etc, use an array instead.
|
34 | `['row', true, /^doc/]`.
|
35 |
|
36 | If you use an array, `RegExp`s, booleans, and/or functions. The `..` operator is also available in array representation, using `{recurse: true}`.
|
37 | any object that matches the path will be emitted as 'data' (and `pipe`d down stream)
|
38 |
|
39 | If `path` is empty or null, no 'data' events are emitted.
|
40 |
|
41 | ### Examples
|
42 |
|
43 | query a couchdb view:
|
44 |
|
45 | ``` bash
|
46 | curl -sS localhost:5984/tests/_all_docs&include_docs=true
|
47 | ```
|
48 | you will get something like this:
|
49 |
|
50 | ``` js
|
51 | {"total_rows":129,"offset":0,"rows":[
|
52 | { "id":"change1_0.6995461115147918"
|
53 | , "key":"change1_0.6995461115147918"
|
54 | , "value":{"rev":"1-e240bae28c7bb3667f02760f6398d508"}
|
55 | , "doc":{
|
56 | "_id": "change1_0.6995461115147918"
|
57 | , "_rev": "1-e240bae28c7bb3667f02760f6398d508","hello":1}
|
58 | },
|
59 | { "id":"change2_0.6995461115147918"
|
60 | , "key":"change2_0.6995461115147918"
|
61 | , "value":{"rev":"1-13677d36b98c0c075145bb8975105153"}
|
62 | , "doc":{
|
63 | "_id":"change2_0.6995461115147918"
|
64 | , "_rev":"1-13677d36b98c0c075145bb8975105153"
|
65 | , "hello":2
|
66 | }
|
67 | },
|
68 | ]}
|
69 |
|
70 | ```
|
71 |
|
72 | we are probably most interested in the `rows.*.docs`
|
73 |
|
74 | create a `Stream` that parses the documents from the feed like this:
|
75 |
|
76 | ``` js
|
77 | var stream = JSONStream.parse(['rows', true, 'doc']) //rows, ANYTHING, doc
|
78 |
|
79 | stream.on('data', function(data) {
|
80 | console.log('received:', data);
|
81 | });
|
82 |
|
83 | stream.on('root', function(root, count) {
|
84 | if (!count) {
|
85 | console.log('no matches found:', root);
|
86 | }
|
87 | });
|
88 | ```
|
89 | awesome!
|
90 |
|
91 | ### recursive patterns (..)
|
92 |
|
93 | `JSONStream.parser('docs..value')`
|
94 | (or `JSONStream.parser(['docs', {recurse: true}, 'value'])` using an array)
|
95 | will emit every `value` object that is a child, grand-child, etc. of the
|
96 | `docs` object. In this example, it will match exactly 5 times at various depth
|
97 | levels, emitting 0, 1, 2, 3 and 4 as results.
|
98 |
|
99 | ```js
|
100 | {
|
101 | "total": 5,
|
102 | "docs": [
|
103 | {
|
104 | "key": {
|
105 | "value": 0,
|
106 | "some": "property"
|
107 | }
|
108 | },
|
109 | {"value": 1},
|
110 | {"value": 2},
|
111 | {"blbl": [{}, {"a":0, "b":1, "value":3}, 10]},
|
112 | {"value": 4}
|
113 | ]
|
114 | }
|
115 | ```
|
116 |
|
117 | ## JSONStream.parse(pattern, map)
|
118 |
|
119 | provide a function that can be used to map or filter
|
120 | the json output. `map` is passed the value at that node of the pattern,
|
121 | if `map` return non-nullish (anything but `null` or `undefined`)
|
122 | that value will be emitted in the stream. If it returns a nullish value,
|
123 | nothing will be emitted.
|
124 |
|
125 | ## JSONStream.stringify(open, sep, close)
|
126 |
|
127 | Create a writable stream.
|
128 |
|
129 | you may pass in custom `open`, `close`, and `seperator` strings.
|
130 | But, by default, `JSONStream.stringify()` will create an array,
|
131 | (with default options `open='[\n', sep='\n,\n', close='\n]\n'`)
|
132 |
|
133 | If you call `JSONStream.stringify(false)`
|
134 | the elements will only be seperated by a newline.
|
135 |
|
136 | If you only write one item this will be valid JSON.
|
137 |
|
138 | If you write many items,
|
139 | you can use a `RegExp` to split it into valid chunks.
|
140 |
|
141 | ## JSONStream.stringifyObject(open, sep, close)
|
142 |
|
143 | Very much like `JSONStream.stringify`,
|
144 | but creates a writable stream for objects instead of arrays.
|
145 |
|
146 | Accordingly, `open='{\n', sep='\n,\n', close='\n}\n'`.
|
147 |
|
148 | When you `.write()` to the stream you must supply an array with `[ key, data ]`
|
149 | as the first argument.
|
150 |
|
151 | ## unix tool
|
152 |
|
153 | query npm to see all the modules that browserify has ever depended on.
|
154 |
|
155 | ``` bash
|
156 | curl https://registry.npmjs.org/browserify | JSONStream 'versions.*.dependencies'
|
157 | ```
|
158 |
|
159 | ## numbers
|
160 |
|
161 | There are occasional problems parsing and unparsing very precise numbers.
|
162 |
|
163 | I have opened an issue here:
|
164 |
|
165 | https://github.com/creationix/jsonparse/issues/2
|
166 |
|
167 | +1
|
168 |
|
169 | ## Acknowlegements
|
170 |
|
171 | this module depends on https://github.com/creationix/jsonparse
|
172 | by Tim Caswell
|
173 | and also thanks to Florent Jaby for teaching me about parsing with:
|
174 | https://github.com/Floby/node-json-streams
|
175 |
|
176 | ## license
|
177 |
|
178 | MIT / APACHE2
|