Quantcast
Channel: Sleepless Dev
Viewing all articles
Browse latest Browse all 217

Groovy and Boon provide the fastest JSON parser for the JVM | Java JSON Benchmarks for Jackson vs. Boon/Groovy

$
0
0
Boon Home | Boon Source | If you are new to boon, you might want to start here. Simple opinionated Java for the novice to expert level Java Programmer. Low Ceremony. High Productivity. A real boon to Java to developers!

Boon and Groovy 2.3 provide the fastest JSON parser on the JVM. 


Jackson is consistently faster than GSON and JSONSmart. Boon JSON parser and the new Groovy 2.3 JSON parser are faster than Jackson. They are faster with InputStream, Reader, reading files, byte[], and char[] and String.



Groovy JSON support and the Boon JSON parser are up to 3x to 5x faster than Jackson at parsing JSON from String and char[], and 2x to 4x faster at parsing byte[].

Groovy JSON support and Boon JSON support are also faster than Jackson at encoding JSON strings. Boon is faster than Jackson at serializing/de-serializing Java instances to/fro JSON. The core of the Boon JSON parser has been forked into Groovy 2.3 (now in Beta). In the process Boon JSON support was improved and further enhanced. Groovy and Boon JSON parsers speeds are equivalent. Groovy now has the fastest JSON parser on the JVM. 

The new Groovy JSON parser based on Boon JSON parser is 20x faster than the previous Groovy JSON parser circa Groovy 2.2.

If you are using Groovy, just upgrade to Groovy 2.3 and should see the speedup. If you are using Java and you can't or wont use Groovy then follow this guide 10 minutes guide to using Boon JSON support (Boon is a Java lib). You can also use Boon with Groovy or Scala or ... :)

See the full benchmarks at Boon JSON Parser versus Jackson with source code.








Java Boon/Groovy vs Jackson - JSON Benchmarks

Boon comes with a very fast JSON parser which has now been ported / forked into Groovy 2.3.

JSON Benchmarks

We used JMH which is what OpenJDK uses. We used the json files from json.org for parsing.
JMH is a Java harness for building, running, and analyzing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.
JMH is Maven-driven, hence having Maven installed will bring the best experience. Also, the working JDK installation should be available to build the project, the build will require the JAVA_HOME set to the correct JDK location.

Environment

OSX MacBook Pro, JDK 1.7, 16GB RAM and 512GB SSD drive.

Summary

Jackson is consistently faster than GSON and JSONSmart. 
Boon JSON parser is faster than Jackson with Readerreading filesbyte[], and char[] and String.
Boon is 3x to 5x faster than Jackson at parsing JSON from String and char[], and 2x to 4x faster at parsing byte[].
Boon and Jackson speeds are much closer with InputStream based parsing. Boon is usually faster at InputStream, but the margins are much smaller.
Boon and Jackson are close at handling InputStream if the JSON stream is small. Once the JSON stream gets over 1KB to 2KB, Boon wins consistently.
Boon is faster at encoding JSON strings, and serializing/de-serializing Java instances to/from JSON than Jackson.

Quick FAQ

Why don't you test GSON and JSONSmart?

Jackson is faster than GSON and JSONSmart. We have tests for all four. We only included the tests for Jackson because Jackson is faster than the other two. Anyone can download the benchmark and run all of the tests.

Why don't you test Jackson AST?

We do. We see no real difference between readTree and readValue. We left out AST because it is redundant. The tests are still there and easy to run. I find AST +/- 5% of non AST.

Who wrote the benchmarks?

They first version was written by Stephane Landelle of Gatling fame (Gatling is the best stress test tool on the JVM). Later they were added to by Andrey Bloschetsov for Groovy serialization. Rick Hightower added new tests as well.

Relationship to Groovy JSON parser for 2.3?

Andrey Bloschetsov and Rick Hightower, with the help of the Groovy lead developers, took the Boon JSON Parser and forked it into Groovy 2.3 which is in BETA. Andrey started looking into JSON speed up for serialization and Guillaume Laforge drafted Rick Hightower who kept bragging about his fast JSON parser on Facebook.
We did this mainly for parsing, but we also improved the JSON serialization speed.
Boon and Groovy parsing are unsurprisingly very comparable.
Groovy JSON parsing is a bit faster in a few use cases, and Boon parser is a bit faster in a few uses cases, but mostly they are neck in neck. Currently they are largely the same so you can expect similar results from Groovy.
Boon JSON de-serialization (from JSON into instances of Classes) is still quite a bit faster than Groovy. This will change in Groovy 2.4 or 2.5.
The new Groovy JSON parser based on Boon is 20x faster than the Groovy 2.2 parser. And the new Groovy Parser is faster than any main stream JSON parser on the JVM.

JSON

Most JSON samples are taken from json.org sample files. The idea was to be fair.

See JSON Samples from JSON.org

InputStream Parsing

Boon parsing is the weakest with InputStreams. Boon and Jackson are the closest in performance with InputStream parsing.

Run JMH as follows:
 java -jar target/microbenchmarks.jar ... -wi 2 -i 3 -f 2 -t 16
The above does Two warm-ups. Two forks. 16 threads. Three iterations. Jackson and Boon are completely warmed up after the first iteration. Running it longer does not help.
(Later we add more warm-ups.)
Boon.
privateObjectparse(InputStreaminputStream)throwsException{
returnparser.parse(inputStream);
}

Jackson.
privateObjectparse(InputStreaminputStream)throwsException{

returnJACKSON_MAPPER.readValue(inputStream,Map.class);
}

InputStream actionLabel.json 872 bytes

Benchmark                                                    Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.inputStream.MainBoonBenchmark.actionLabel thrpt 16 6 1 866021.511 152818.015 ops/s
i.g.j.inputStream.MainJacksonObjectBenchmark.actionLabel thrpt 16 6 1 662051.786 70960.055 ops/s
Boon is faster.

InputStream citm_catalog.json 1.7 MB (1,700,000 bytes)

i.g.j.inputStream.MainBoonBenchmark.citmCatalog             thrpt  16         6    1      997.894      106.665    ops/s
i.g.j.inputStream.MainJacksonObjectBenchmark.citmCatalog thrpt 16 6 1 464.647 82.287 ops/s
Boon is a lot faster.

InputStream medium.json 2 KB (2,000 bytes)

i.g.j.inputStream.MainBoonBenchmark.medium                  thrpt  16         6    1   729649.386    32040.721    ops/s
i.g.j.inputStream.MainJacksonObjectBenchmark.medium thrpt 16 6 1 419224.947 21098.385 ops/s
Boon wins.

InputStream menu.json 256 bytes

i.g.j.inputStream.MainJacksonObjectBenchmark.menu           thrpt  16         6    1  1985725.125   181813.950    ops/s
i.g.j.inputStream.MainBoonBenchmark.menu thrpt 16 6 1 1505494.119 24945.816 ops/s
Jackson wins. (Can't win them all.) For this small of a payload you might need to tweak the default buffer sizes for Boon.

InputStream sgml.json 705 bytes

i.g.j.inputStream.MainBoonBenchmark.sgml                    thrpt  16         6    1  1350396.608    31969.601    ops/s
i.g.j.inputStream.MainJacksonObjectBenchmark.sgml thrpt 16 6 1 1209915.253 122783.831 ops/s
Boon barely wins.

InputStream webxml.json 4K bytes

i.g.j.inputStream.MainBoonBenchmark.webxml                  thrpt  16         6    1   431055.661    31544.139    ops/s
i.g.j.inputStream.MainJacksonObjectBenchmark.webxml thrpt 16 6 1 189635.644 104334.060 ops/s
Boon wins by a large margin.

InputStream widget.json 761 bytes

i.g.j.inputStream.MainBoonBenchmark.widget                  thrpt  16         6    1  1066325.400   108550.520    ops/s
i.g.j.inputStream.MainJacksonObjectBenchmark.widget thrpt 16 6 1 699439.433 130830.543 ops/s
Boon wins by a large margin.


InputStream JSON Parsing Jackson vs. Boon 



Higher is better. 100 represents 100% of the fastest. 
Boon wins 6 out of 7, and when it wins the margins are higher.

Caveat

Boon JSON parser is a parser optimized for REST and websocket style services.
Boon JSON parser is optimized to work in a reactive style environment like Akka, Vert.x, etc. Boon JSON parser would work well with MessageDrivenBeans (EJB or Spring).
For some use cases it runs stateless faster than Jackson runs with shared buffers.
Jackson can handle very large JSON files and very large JSON streams. This support was added to Boon JSON as part of the fork/port to Groovy JSON. With larger files the speeds are much more equivalent, and in fact Jackson might be 10% faster in some cases and Groovy/Boon 10% faster in other cases.
Benchmark first. Don't assume.
It would take some extra effort to get superior results in a Servlet environment (it can be done). Framework providers like Grails could include fast Groovy based parsing with minimal impact on web developers. 
Jackson is mature and solid. Don't assume ripping out Jackson for Boon will buy you anything. Jackson has tooling that makes it work really well in a Java EE environment, and is more fool proof.

Jackson is hands down the most popular JSON Java framework in the Java ecosystem. You can find it everywhere from Vertx.io to ElasticSearch to just about every JAX-RS implementation on the planet to Spring. If you are doing or have done JSON with Java then you have used Jackson either directly or indirectly. It is well designed, full of features and very fast.

File system read

Both Boon and Jackson have the ability to read straight from the file system. It seems Jackson needed more warm-ups so we added them. We don't think non-warmed up code proves anything. Both Jackson and Boon were all warmed-up after three warm-ups.
 java -jar target/microbenchmarks.jar ".*file.*"  -wi 3 -i 3 -f 2 -t 16
Jackson File.
privateObjectparse(StringfileName)throwsException{
returnJACKSON_MAPPER.readTree(newFile(fileName));
}

Boon File.
privateObjectparse(StringfileName)throwsException{
returnparser.parseFile(Map.class,fileName);
}

File actionLabel.json 872 bytes

i.g.j.f.BoonBenchMark.actionLabel          thrpt  16         6    1   253893.017    14314.289    ops/s
i.g.j.f.JacksonASTBenchmark.actionLabel thrpt 16 6 1 231613.819 25364.317 ops/s
Call it a tie. Boon barely wins.

File citm_catalog.json 1.7 MB (1,700,000 bytes)

i.g.j.f.BoonBenchMark.citmCatalog          thrpt  16         6    1      862.778      106.362    ops/s
i.g.j.f.JacksonASTBenchmark.citmCatalog thrpt 16 6 1 451.617 59.926 ops/s
Boon wins by a wide margin.

File medium.json 2 KB (2,000 bytes)

Benchmark                                   Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.f.BoonBenchMark.medium thrpt 16 6 1 249553.100 19159.077 ops/s
i.g.j.f.JacksonASTBenchmark.medium thrpt 16 6 1 197056.072 21201.261 ops/s
Boon wins.

File menu.json 256 bytes

Benchmark                                   Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.f.BoonBenchMark.menu thrpt 16 6 1 271678.375 24672.358 ops/s
i.g.j.f.JacksonASTBenchmark.menu thrpt 16 6 1 284756.122 88111.175 ops/s
Call it a tie. Jackson barely wins.

File sgml.json 705 bytes

Benchmark                                   Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.f.BoonBenchMark.sgml thrpt 16 6 1 258278.919 9886.144 ops/s
i.g.j.f.JacksonASTBenchmark.sgml thrpt 16 6 1 253126.067 20414.849 ops/s
Call it a tie. Boon barely wins.

File webxml.json 4K bytes

Benchmark                                   Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.f.BoonBenchMark.webxml thrpt 16 6 1 208778.647 15219.752 ops/s
i.g.j.f.JacksonASTBenchmark.webxml thrpt 16 6 1 127184.025 17270.754 ops/s
Boon wins by a large margin.

File widget.json 761 bytes

Benchmark                                   Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.f.BoonBenchMark.widget thrpt 16 6 1 268296.542 21856.333 ops/s
i.g.j.f.JacksonASTBenchmark.widget thrpt 16 6 1 222440.689 39514.739 ops/s
Boon wins.

File Boon vs. Jackson


Boon wins 5 out of 7 tests. And it wins by larger margins.

Reader

Boon and Jackson can use a reader to read a JSON stream.
 java -jar target/microbenchmarks.jar ".*reader.*"  -wi 3 -i 3 -f 2 -t 16

Reader results (altogether)

Benchmark                                          Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.r.MainBoonBenchmark.actionLabel thrpt 16 6 1 1482160.136 127335.188 ops/s
i.g.j.r.MainJacksonObjectBenchmark.actionLabel thrpt 16 6 1 528522.919 70537.978 ops/s

Boon 3x faster

i.g.j.r.MainBoonBenchmark.citmCatalog thrpt 16 6 1 1102.425 272.243 ops/s
i.g.j.r.MainJacksonObjectBenchmark.citmCatalog thrpt 16 6 1 396.981 28.491 ops/s

Boon over 2x faster

i.g.j.r.MainBoonBenchmark.medium thrpt 16 6 1 1164408.836 71138.499 ops/s
i.g.j.r.MainJacksonObjectBenchmark.medium thrpt 16 6 1 365940.803 15780.746 ops/s

Boon 2.7x faster.

i.g.j.r.MainBoonBenchmark.menu thrpt 16 6 1 4730577.939 206541.736 ops/s
i.g.j.r.MainJacksonObjectBenchmark.menu thrpt 16 6 1 1581462.192 289868.322 ops/s

Boon over 2x faster.

i.g.j.r.MainBoonBenchmark.sgml thrpt 16 6 1 4813803.211 157129.919 ops/s
i.g.j.r.MainJacksonObjectBenchmark.sgml thrpt 16 6 1 1643288.822 80243.617 ops/s

Boon over 2x faster.

i.g.j.r.MainBoonBenchmark.webxml thrpt 16 6 1 646994.644 101870.656 ops/s
i.g.j.r.MainJacksonObjectBenchmark.webxml thrpt 16 6 1 203544.281 15564.338 ops/s

Boon over 3x faster.

i.g.j.r.MainBoonBenchmark.widget thrpt 16 6 1 1922759.581 360131.835 ops/s
i.g.j.r.MainJacksonObjectBenchmark.widget thrpt 16 6 1 639264.142 36270.570 ops/s

Boon almost 3x faster.

Parsing JSON from Reader Boon vs. Jackson

Boon wins 7 out of 7 tests and almost all close to 3x faster.

Parse From byte[] array

Boon and Jackson can parse direct from a byte[].
 java -jar target/microbenchmarks.jar ".*byte.*"  -wi 3 -i 3 -f 2 -t 16

Results from parsing direct from a byte array

Benchmark                                          Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.b.MainBoonBenchmark.actionLabel thrpt 16 6 1 1221958.008 30506.924 ops/s
i.g.j.b.MainJacksonObjectBenchmark.actionLabel thrpt 16 6 1 636254.022 33273.771 ops/s

Boon wins by 2x

i.g.j.b.MainBoonBenchmark.citmCatalog thrpt 16 6 1 1000.572 79.712 ops/s
i.g.j.b.MainJacksonObjectBenchmark.citmCatalog thrpt 16 6 1 537.311 61.683 ops/s


Boon wins by almost 2x

i.g.j.b.MainBoonBenchmark.medium thrpt 16 6 1 854426.764 84604.024 ops/s
i.g.j.b.MainJacksonObjectBenchmark.medium thrpt 16 6 1 436113.169 34620.702 ops/s

Boon wins by almost 2x

i.g.j.b.MainBoonBenchmark.menu thrpt 16 6 1 3740629.358 570942.364 ops/s
i.g.j.b.MainJacksonObjectBenchmark.menu thrpt 16 6 1 1842253.614 50144.188 ops/s

Boon wins by 2x.

i.g.j.b.MainBoonBenchmark.sgml thrpt 16 6 1 2377758.261 162251.992 ops/s
i.g.j.b.MainJacksonObjectBenchmark.sgml thrpt 16 6 1 1190724.547 60984.294 ops/s

Boon wins by over 2x

i.g.j.b.MainBoonBenchmark.small thrpt 16 6 1 14608691.917 4637278.405 ops/s
i.g.j.b.MainJacksonObjectBenchmark.small thrpt 16 6 1 3843711.836 237716.513 ops/s

Boon wins by 4x.

i.g.j.b.MainBoonBenchmark.webxml thrpt 16 6 1 503674.722 30799.444 ops/s
i.g.j.b.MainJacksonObjectBenchmark.webxml thrpt 16 6 1 225047.361 31226.580 ops/s

Boon wins by over 2x.

i.g.j.b.MainBoonBenchmark.widget thrpt 16 6 1 1465630.533 38963.901 ops/s
i.g.j.b.MainJacksonObjectBenchmark.widget thrpt 16 6 1 718188.167 76972.830 ops/s

Boon wins by 2x.


Parse From String array

Boon and Jackson can use a reader to read a JSON stream.

 java -jar target/microbenchmarks.jar ".*string.*"  -wi 3 -i 3 -f 2 -t 16

String

Benchmark                                          Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.s.MainBoonBenchmark.actionLabel thrpt 16 6 1 1431392.169 193758.838 ops/s
i.g.j.s.MainJacksonObjectBenchmark.actionLabel thrpt 16 6 1 564775.347 17677.459 ops/s

Boon is 3x faster than Jackson


i.g.j.s.MainBoonBenchmark.citmCatalog thrpt 16 6 1 1531.678 304.146 ops/s
i.g.j.s.MainJacksonObjectBenchmark.citmCatalog thrpt 16 6 1 423.206 54.482 ops/s


Boon is over 3x faster than Jackson

i.g.j.s.MainBoonBenchmark.medium thrpt 16 6 1 1162388.483 58546.058 ops/s
i.g.j.s.MainJacksonObjectBenchmark.medium thrpt 16 6 1 366282.108 20279.569 ops/s


Boon is 3.5x faster than Jackson

i.g.j.s.MainBoonBenchmark.menu thrpt 16 6 1 4606554.667 390199.354 ops/s
i.g.j.s.MainJacksonObjectBenchmark.menu thrpt 16 6 1 1591678.644 99655.195 ops/s

i.g.j.s.MainBoonBenchmark.sgml thrpt 16 6 1 2930363.631 928801.932 ops/s
i.g.j.s.MainJacksonObjectBenchmark.sgml thrpt 16 6 1 1007490.344 100771.781 ops/s


Boon is almost 3x faster than Jackson

i.g.j.s.MainBoonBenchmark.webxml thrpt 16 6 1 703755.139 16904.302 ops/s
i.g.j.s.MainJacksonObjectBenchmark.webxml thrpt 16 6 1 197618.389 9224.057 ops/s


Boon is almost 4x faster than Jackson

i.g.j.s.MainBoonBenchmark.widget thrpt 16 6 1 2039753.711 164417.077 ops/s
i.g.j.s.MainJacksonObjectBenchmark.widget thrpt 16 6 1 640933.911 30993.768 ops/s


Boon is almost 4x faster than Jackson


i.g.j.s.MainBoonBenchmark.small thrpt 16 6 1 26063946.758 1320123.707 ops/s
i.g.j.s.MainJacksonObjectBenchmark.small thrpt 16 6 1 4328478.361 369396.951 ops/s


Boon is 5x faster than Jackson


Boon vs. Jackson String parsing JSON

Boon wins 7 out of 7 with very wide margins.





Object Serialization

Boon and Jackson can use a reader to read a JSON stream.
 java -jar target/microbenchmarks.jar ".*serialization.*"  -wi 3 -i 3 -f 2 -t 16
Jackson and Boon can do Object serialization. Boon uses this in SlumberDB to provide a fast key/value JSON store for Java using LevelDB, RocksDB, and MySQL.
Benchmark                                        Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.s.MainBoonSerializer.roundTriper thrpt 16 6 1 327384.050 7562.980 ops/s
i.g.j.s.MainJacksonSerializer.roundTriper thrpt 16 6 1 210291.358 15730.171 ops/s

Boon 30% faster.

i.g.j.s.MainBoonSerializer.serializeSmall thrpt 16 6 1 913616.875 89584.750 ops/s
i.g.j.s.MainJacksonSerializer.serializeSmall thrpt 16 6 1 703988.967 190145.965 ops/s

Boon 20% faster.

Weird Stuff serialization

Boon and Jackson can use a reader to read a JSON stream.
 java -jar target/microbenchmarks.jar ".*serializerTests.*"  -wi 3 -i 3 -f 2 -t 16
Benchmark                                                            Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.serializerTests.Test.complexTestJackson thrpt 16 6 1 69923.967 2382.158 ops/s
i.g.j.serializerTests.Test.complextTestBoon thrpt 16 6 1 65746.303 7178.469 ops/s

Faster.

i.g.j.serializerTests.Test.mediumTestBoon thrpt 16 6 1 1323458.047 55113.844 ops/s
i.g.j.serializerTests.Test.mediumTestJackson thrpt 16 6 1 848145.458 109868.000 ops/s

Boon almost 2x faster.

i.g.j.serializerTests.Test.simpleTestBoon thrpt 16 6 1 5762613.228 119177.393 ops/s
i.g.j.serializerTests.Test.simpleTestJackson thrpt 16 6 1 2611158.339 411964.607 ops/s

Boon 2x faster.

Beyond this is just boiler plate Boon advertising. :)

Thoughts

Thoughts? Write me at richard high tower AT g mail dot c-o-m (Rick Hightower).

Further Reading:

If you are new to boon start here:

Why Boon?

Easily read in files into lines or a giant string with one method call. Works with files, URLs, class-path, etc. Boon IO support will surprise you how easy it is. Boon has Slice notation for dealing with Strings, Lists, primitive arrays, Tree Maps, etc. If you are from Groovy land, Ruby land, Python land, or whatever land, and you have to use Java then Boon might give you some relief from API bloat. If you are like me, and you like to use Java, then Boon is for you too. Boon lets Java be Java, but adds the missing productive APIs from Python, Ruby, and Groovy. Boon may not be Ruby or Groovy, but its a real Boon to Java development.

Core Boon Philosophy

Core Boon will never have any dependencies. It will always be able to run as a single jar. This is not just NIH, but it is partly. My view of what Java needs is more inline with what Python, Ruby and Groovy provide. Boon is an addition on top of the JVM to make up the difference between the harder to use APIs that come with Java and the types of utilities that are built into Ruby, Python, PHP, Groovy etc. Boon is a Java centric view of those libs. The vision of Boon and the current implementation is really far apart.

Contact Info

Viewing all articles
Browse latest Browse all 217

Trending Articles