A Closer Look at Transit | Hacker News Reader

A Closer Look at Transit

swannodette.github.io

122 points_halgari36 comments

Why do they say JSON is faster than MessagePack and then link to a comparison for only Javascript? Most servers are not running Javascript, and most data goes its entire lifetime untouched by Javascript.

There's some reason to believe that MessagePack is noticeably faster than JSON on every language but Javascript.

Edit: s/entirely/entire/

No one is saying MessagePack is slower than JSON across the board. In fact for languages that have fast MessagePack implementations, server programs can speak to each other using the Transit MessagePack encoding. But having a good story to and from JavaScript was a big design constraint and Transit delivers this as well.

I thought that the big advantage of MessagePack was that it was smaller and quicker over networks. Network effects are the biggest cause of delay when transferring data to remote machines and are much bigger than serialization performance.

The localhost testing done in the article will not show this advantage of MessagePack.

Transit JSON payloads benefit from standard gzipping techniques for the cases where the latencies are quite large as is the case when communicating with client browsers. For the cases where latencies are not quite so big we have yet to see evidence that Transit JSON suffers dramatically.

It is faster for most languages, and could be made faster in server-side JS. However, this article is talking about performance for browser JS, where lack of robust string decoding destroys msgpack performance.

To what extent are legal EDN and Transit compatible? Are there types that appear in one and not the other? How does the extension mechanism compare?

Would it be possible to make a 2-way converter between them which would retain full fidelity of the Transit data when round-tripping, so that we could use EDN for actually inspecting/writing (e.g. config files), but then send Transit over the wire to a browser?

EDN and Transit are different formats but share a common philosophy towards types and extensions (as they originate from the same place). There are likely small differences in what types appear in both. The extension mechanisms are different but of equivalent expressive power.

I don't think it would be difficult to convert between EDN and Transit using existing readers and writers for both. The common meeting point is values in your language.

From the examples, it looks like JSON and EDN are both first-class "transport" formats for Transit:

https://github.com/cognitect/transit-format/tree/master/exam...

Hey Swannodette, I've currently got a Clojure backend talking edn with a browser running ClojureScript. Would it make any sense to switch to transit? More specifically, are there any advantages to use transit besides performance, and how big would the improvement in performance potentially be?

For Clojure & ClojureScript users Transit delivers little new in terms of feature set over EDN but the performance gains are quite significant. transit-cljs is up to 18-20X faster than cljs.reader/read-string on the EDN equivalent. Under V8 JS environments (Chrome, Opera, Node.js) I consistently see transit-cljs outperform JSON.parse on the equivalent data.

In addition transit-cljs is a great way to consume JSON endpoints from ClojureScript without the massive perf hit of cljs.core/js->clj. You can also trivially spit out JSON from ClojureScript via transit-cljs as long as your map keys are strings and not keywords using a json-verbose Transit writer.

Nice :)

The speed difference is largely a result of caching correct?

It's a part of it - the full story is more like: caching + array representation + array wrapping + preventing internal cache misses.

I can imagine using Transit as a format to communicate between servers, but I'm less sold on the JS story. If I'm exposing an API endpoint to the browser I'm also exposing it as an API endpoint. In this case I need a really good reason to not use JSON, which everyone understands and can parse. Adding a barrier to people using my API just doesn't make sense. It's fairly standard for single page apps to consume a public API, so I think this will be a fairly common objection.

Also, Computer Modern is so so so ugly (IMHO of course).

This is a winner-take-all narrative I simply cannot understand. I recall when all endpoints were XML and JSON.parse was called eval.

I can easily imagine a endpoint providing both Transit JSON and plain JSON. I know which one I'd rather consume from the client when building a sophisticated application.

Looking at http://jsperf.com/json-vs-transit/2 ...

Why does Transit performance take such a hit in Mobile Safari? (In desktop Safari on my Mac, performance seems to be about the same between JSON+hydrate vs. Transit. But the charts below show that especially on later versions of mobile Safari Transit is quite a bit slower.)

I just checked on my iPhone 5S running iOS 7.1.2 & Mobile Safari 7.0 - the transit-js results look good and now are reflected in the benchmarks. I have no idea what's up with the browsers showing up as Mobile Safari 7.1 & 7.2 (which are dev releases as far as I know).

What's with the predilection for unreadable fonts?

http://blog.cognitect.com/blog/2014/7/22/transit

http://swannodette.github.io/2014/07/23/a-closer-look-at-tra...

Not sure I understand, it looks perfectly fine on my box.

On FF in Windows, the fonts are extremely thin and difficult to read. I had to zoom in Nolen's blog four times to make it readable.

Huh. Here it is on a Mac, FYI: http://i.imgur.com/swOvBka.png

The main body of the text was extremely difficult to read. There's nothing wrong with the font itself, but the size is far too small and colour too faint. I had to turn off the font stylesheet to get to the text. It's too bad the same font as the sidebar section wasn't used throughout the body.

It's "possible to read" for me (FF on Debian), but it's not particularly "readable"... it seems to be using fonts that don't render very well for "traditional resolution" (~100dpi) displays, at an uncomfortably small size (maybe the author tests using a browser that tends to inflate text sizes?).

It would be nice to include the encoded Transit version, just to see what it looks like.

Edit: the tour linked at the bottom is basically what I was looking for. http://cognitect.github.io/transit-tour/

Have you thought about JSON-LD ? Having some semantic web meaning on top of it would be really cool !

Yes comparisons to JSON-LD has come up a few times and semantic web & HATEOAS are the reasons why the Transit spec includes the link type. JSON-LD looks interesting but it does not seem to address the basic performance concerns that Transit does.

Thanks for your answer. Can I create extensions for Schema.org types then ?

I don't see why not - I mean these are unlikely to get included in the official Transit spec, but sure, being able to build a common set of shared extensions seems mighty powerful to me - I had a bit of fun with this today when putting together the example for the blogpost https://github.com/swannodette/transit-js-example/blob/maste...

I haven't looked that closely at Transit yet, but one very small thing that irked me at first glance is that the tag for floats is 'd' and the tag for arbitrary precision decimals is 'f', which seems entirely backwards.

I think this likely has to do with how confusing the implementation of floats and doubles in programming languages has been over the years. A lot of languages will call a double precision decimal a float, but I don't know of any languages that call a single precision decimal a double. I think the general ambiguity around float is why they chose to use it as any precision float.

I'm curious how Transit handles cyclic object graphs for optimizing the payload size. Would be nice to see JSOG integration. https://github.com/jsog/jsog

I don't think Transit does or plans to handle cyclic object graphs.

The name generation from caching ought to allow it, though...

this jsperf revision is slightly more interesting to me http://jsperf.com/json-vs-transit/3

it's effectively the same input, but optimised for JSON.parse() rather than transit. This means it purely decodes into objects with fast properties (in V8 at least), so objects never have to undergo the -> hash table transition. It seems consistently faster, but you can argue that it's not doing quite the same thing.

These results aren't actually that different when you include all the other browsers. In fact when you look at Chrome 38 (Canary) the results are exactly the same for both versions of the benchmark.

As a general rule the optimizations in transit-js take all browsers into consideration - not any particular one.

Am I the only one who wants to know how transit data looks like (compared to JSON) over the wire?

Reply on news.ycombinator.com