Why Dart should learn JSON while it’s still young | Hacker News Reader

Why Dart should learn JSON while it’s still young

maxhorstmann.net

108 pointsMax_Horstmann54 comments

This is more complicated than it seems.

First, how does it work with minimization? In Dart, string constants are not minimizable (will be included as-is in the source code). Symbol constants will be minimized. This includes all method names, field names, and things like named parameters. You can see this in the arguments to noSuchMethod; the named parameters map has Symbol keys, not strings.

Probably something could be worked out, though. Maybe put an annotation on the customer class to tell the compiler to save the original field names?

But a second issue is how it works with inheritance, including mixins. If a class is serializable, does that make all the superclasses and subclasses serializable as well? JSON doesn't have a standard way to represent a type tag, so does that mean we should invent one? If we do that, what happens to JSON interop with other languages?

Also keep in mind that Dart supports types that don't interop with JavaScript like unlimited precision integers. (Well, okay, that only works with the Dart VM, not dart2js...)

It gets worse. What if your class has a field type like List<dynamic>, which could contain any object? Does that mean the Dart compiler now has to preserve all symbols because any class could be serialized? And keep in mind that with optional types, all lists should behave the same when not in checked mode.

Having seen the mess that resulted in GWT-RPC, I would recommend just saying no to default serialization. Serialization and object inheritance don't really mix well which is why specs like JSON and protobufs don't support it.

(And now you know why language designers don't throw things into a language just because it seems like a good idea at the time.)

does anything actually work with dart2js?

All I seem to hear is people complaining about it.

Given so little effort is given to it, I halfway think they should just give up on javascript and javascript interop completely and focus on making dart awesome.

Anyhow, what you say is totally true, but surely iterating recursively over an object and dumping values into a json block would be trivial.

The hard part is deserializing that back into objects, but maybe you dont need to? The python json package does a pretty good job. You could just have it return a dynamic or something.

Disclaimer: I'm the TLM of dart2js.

The dart2js team is one of the teams with the most developers on it. There is a huge effort going into it.

Dart2js generally does a good job, but a few features (like mirrors) that are still in beta-quality, are currently a constant source of grieve for the developers. We would have liked to delay their release until they were more stable, but several customers really, really needed them.

Things are getting better, though, and I hope that dart2js will fade into the background (as a simple working tool) soon.

Is there a plan for fixing the behavior of integers so they are the same on the VM and dart2js?

If there is one it's not public. Here's the bug to track: https://code.google.com/p/dart/issues/detail?id=1533

I understand you guys are working on it, and it's a tough problem, but for a flagship javascript drop in replacement isn't releasing something that griefs developers really bad?

Surely releasing a top quality library that works well is far more important than getting in early and releasing something that alienates developers and makes them try dart and them give up on it?

It's obviously your prerogative to do what you feel is best, but it's certainly not the choice I would have endorsed.

(I supposed the horse has bolted to a certain degree now, it's good to know that a lot of effort is being directed at dart2js I suppose, although from a developers persepective I don't really feel that it's particularly visible...)

> releasing something that griefs developers

Maybe you only see (and/or notice) people talking about corner cases that still need addressing (like azakai's sibling comment about the mapping of arbitrary precision integers to JS's number type)? For the vast majority of code, AFAIK, dart2js has no issues and emits good JS code.

I don't develop with dart, but I know a handful of developers that use dart2js to release their dart code on the web and it works just fine for them. mraleph just released IRHydra2[1], which is all Dart from top to bottom via dart2js and works really well (despite using the apparently still-nascent Dart port of Polymer).

If you have specifics, by all means, talk about them, but "All I seem to hear is people complaining about it" seems like really shaky ground from which to make comments about griefing and alienating developers.

[1] http://mrale.ph/blog/2014/01/28/prerelease-irhydra2.html

    - huge js file size (2MB+ easily)
    - obscure hard to use js interop facilities
    - very slow js interop
    - broken js minification (ignored tickets)
    - no older browser support
    - ridiculously long compile times

I honestly welcome anyone who uses dart in a production environment to link to a blog post or explain how they worked around these, because we found they made using dart2js such a nightmare we gave up on it.

I speak only for my own experience, perhaps others have different experience, but no one seems to have good things to say about the experience working with it that I can see...

(amusingly, from your link: 'Does not work in Firefox (neither Stable nor Nightly), but works in Chrome Dev and Safari.')

This isn't production, but I found dart:js to be pretty reasonable for interop with CodeMirror and a Chrome extension API. (I did find it better to send a batch to a wrapper function rather than making JavaScript calls in a tight loop, though.)

> (amusingly, from your link: 'Does not work in Firefox (neither Stable nor Nightly), but works in Chrome Dev and Safari.')

It's most likely problem with polyfills, not with dart2js compilation problem, I have not tried to debug it yet.

Fixed. It was purely my bug: I was relying on a certain ordering of propChanged() callbacks, but it does not seem like Polymer.dart guarantees any.

Now it works in Chrome/FF/Safari all the same.

This comment had me scratching my head. Right off the bat, if you need to run on older browsers, don't use Dart. It's not a secret. Use TypeScript instead. If you need to make a lot of calls into existing JavaScript libs to the point where speed reduction is noticable, don't use Dart. What's so hard to use about js interop? How is it obscure? Throwing out 2MB file size without stating the LOC of the original Dart source is meaningless. I've never noticed long compile times. I do my development in Dart, which runs almost instantaneously, then when all is well, I do a build which compiles and minifies which is reasonably fast. The only thing I can figure is that you're using Dart Polymer, which last I checked, was still in development. My single page app is approaching 10,000 LOC and minifies to about 295K, which is hardly too large. I'm using Dart as a straight replacement for JS and not trying to use the advanced language features, which my explain my more positive experience.

> releasing something that griefs developers

The belief that a programming language is persecuting you...this should be taken as a sign.

So, one thing that I should ask: have you ever designed and implemented a serialization format for an object-oriented language? There is a reason why JavaScript works so well with JSON.

Working with Smalltalk, Obj-C, Clojure, and many other languages besides, it's a fairly simple matter to use a library to do conversion. A few fun reflection tricks are usually enough to get you objects pretty quickly. Why this should be seen as a language issue is beyond me. It's a tool and library issue.

Can you talk more about your experience with GWT-RPC?

GWT-RPC was designed before I got there, but the basic idea was to make it work just like Java serialization. To do this, you need to calculate at compile time which types the client might possibly have to deserialize and include client-side code to instantiate those types. But we don't want to do this for every Serializable type because of code bloat; when a type is declared Serializable in Java it's often intended for server-side serialization, not because anyone intended that type to be usable in a browser.

This is pretty hard to do in a language that supports both inheritance and generic types. Suppose you have a field of type List<Foo>. What values could the server possibly send the client?

- All subtypes of List: ArrayList, LinkedList. Also, ImmutableList and a bunch of other subtypes if Guava is in the classpath.

- All the fields in Foo, its serializable supertypes, and all its subtypes. And then do the same calculation again for those types, recursively.

Or if Foo is a widely used marker interface, anything implementing that marker interface in any library could possibly be sent. At this point, probably nobody working on the app has a good idea on why a particular type is included in the app.

Of course you could tell people to tighten up their type declarations. For example, never use List in a DTO; instead use ArrayList. This doesn't work because it just takes one field declaration in one library to screw it up, so it's hardly worth doing.

And I haven't even gotten into the complexities of things like:

class Foo<T extends Something> extends Bar<T>.

I heard a rumor once that someone proved that doing the calculation "properly" was NP-complete. In practice, we use approximations and there are bugs, though mostly people don't run into them too often.

I think it makes more sense to generate serialization code from an IDL such as a protobuf declaration. Since there's no inheritance and no generics, it's clearer to everyone which fields can possibly be transmitted on the wire and what the semantics are for forwards and backwards compatibility. Or just write out the JSON serialization for each class; it's not that hard, just repetitive.

Just remembered this thread and came back. Thanks for the insight.

Wouldn't it be possible to add some kind of @json class annotation then, which cannot be combined with "extends" and dynamic types? I understand that serialization of inheritance trees and arbitrary data is a complicated problem. So why not allow a simple case first and continue from there?

I guess inbuilt json serialization of simple types would already cover alot of real world rest scenarios which most users care about.

I was the TL of the libraries for almost a year, and there are good reasons we didn't go this route:

- it adds overhead. Being able to write JSON.encode(customer) would mean that every class needs to retain its field names. Minification could still work if this information is stored on the side, but it would add overhead.

- it doesn't make sense to automatically encode elements that cannot be decoded. You should have: x =~= JSON.decode(JSON.encode(x)). (where x =~= y means that the two objects are Json-equivalent).

- The Dart-Editor already provides a quick-fix to write a toJson for you. To be honest I'm not really a fan of it, since I believe that serialization information should be outside the object, and not in the object (that's what "toEncodable" is for), but that's another story.

- It's extremely easy to get the behavior you want. Just use (or write, if none exists yet) a package that has your intended behavior. This is really a one-line change... This way you pay the price when you want to, and don't force it on all users.

Where is toJson in the Dart Editor? (I'm looking for it in 1.1.1, but couldn't figure it out.) And does it have a fromJson generator too?

It was recently added, but apparently removed again. (I don't know the exact reasons).

Edit. you can see the announcement here: https://plus.google.com/109866369054280216564/posts/XbjvN5QV...

Edit2. Removal of the feature: https://code.google.com/p/dart/issues/detail?id=16425

I certainly agree with the latter point if adding a default/out of the box JSON serialization does indeed add significant overhead, impacting all users including those which don’t require it. On the flipside, if a cost-effective solution can be found – e.g. by providing a default JSON serialization via class annotations – I feel that it should be part of Dart’s core and not up to individual developers to home-brew their own solution to a very common problem. In the particular case of DateTime, I’m pretty sure that adding a default JSON serialization now will increase interoperability between Dart-based services and sites in the future.

What's wrong with just using a library?

Scala learnt XML while it was still young, when XML was just as popular as JSON is now, yet incorporating XML into the language is now considered a mistake.

The problem with magic serializers is that they tend to be brittle in practice. This is because the assumptions they make don't work across languages.

For example, if you use a magic serializer, your properties might have to follow a different capitalization format than what's normal for your language.

Recently, I ported some Java code to C#. It was written with a magic JSON serializer. When I got to test my port, nothing worked! It turns out that Java's serializer used lowercase names in JSON, but my serializer used uppercase names! The magic, unfortunately, didn't work, because assumptions that were valid in Java did not carry over to C#.

I think it's best to think of magic serializers as rapid prototyping tools that work well when both ends are the same languages. Thus, while might be useful for Dart to include one very simple quick-and-dirty serializer to JSON, any production code of merit will most likely outgrow it.

Golang has great JSON support that I think could translate to Dart. Any overhead could be minimized through compiler support so we don't have to rely on the current mirror functionality. I think that would be a nice balance between what the author of this article is asking for and dart2js performance.

Go doesn't have inheritance and that makes things quite a bit easier; there's an obvious way to deserialize a map to a struct type.

You can see the seams in the limited support for interfaces; basically there's no way to deserialize to non-empty interface type. (How would that work?)

In Dart, since any class can be extended or used as an interface type, the issues are similar to interface types in Go. There's no bare struct type.

There might be an interesting compromise, but it's not a trivial design problem.

Is there still people willing to use Dart instead of Javascript for web now? (no irony)

Yes. For some use cases the speed of development and rapid prototyping is worth it.

...but there are unavoidable severe limitations like slow js interop, large file size and poor browser compatibility.

Its really only plausible for very specific project where these arent issues.

Dead code elimination and minification do a good job of reducing Dart2js file size. Dart converted to JavaScript works on IE9 and above and all other modern browsers. JsInterop is done through port send and recieve, so if you need to make a lot of those calls it is potentially slow, but if not, for example I'm making one call to PDF.JS, it works beautifully. Dart's real problem in my opinion is that many 10's (100's?) of nifty JavaScript libs are written everyday that won't work well with Dart. The JavaScript ecosystem just keeps getting better, while the Dart ecosystem plays catchup. Because of this I use TypeScript for some projects.

i was just downloading the sublime package to read Dart and was surprised to see the number of downloads going up as time goes by..

https://sublime.wbond.net/packages/Dart

Not any serious metric of course.. but a good thing anyway.. measuring technology adoption is hard and tend to give us imprecise metrics :/

It's the type data that makes this hard: JSON has no support for encoding new types into the serialization. This is a blessing and a curse: it makes JSON simple, but also inflexible.

Take this parting remark:

> So, here’s another thing I wanna add to my suggestion: consider adapting ISO 8601 for a default JSON serialization of DateTime, so the following just works:

> var dt = DateTime.parse("2014-01-26T11:38:17");

> print(JSON.encode(dt));

Ignoring the fact that it's a simple value you're attempting to encode to JSON, let's say this works. Do you get:

  "2014-01-26T11:38:17"

Now do to the reverse: what does that decode to? Either it decodes to a string, and you've lost the type information, or it decodes to a date, in which case, what does `JSON.encode("2014-01-26T11:38:17")` decode to? i.e., do strings sometimes decode to dates? What if someone uses what should be a string to cause the decoder to decode it as a date? Now anywhere you have a string, it might decode to a date!

YAML has support for this sort of thing, with a syntax like `!!date "2014-01-26T11:38:17"`. (If you want a literal `!!`, it's always `"!!"`.) Notably, the Python library for YAML can do what the author asks: encode and decode arbitrary objects. (but it turns out to be a security risk to do so on untrusted data.)

> The Dart editor should be able to display a JSON serialization of anything the mouse pointer touches.

If my class contains a file handle, what's the JSON representation of a file handle? Further, if I send that serialization across the wire, how do you deserialize it? I mention this because you can't, at least, not really. Python's `repr` is a good example here. When it can, it gives you close-to Python syntax for the object, such as strings (`"hello world"`), ints (`1`), decimals (`Decimal(3)`), etc. Perfect for debugging. For things it can't, you get something like `<file object "~/foo.txt" at 0xdeadbeef>` — which doesn't parse (on purpose) as Python.

I'd venture to say that what you really want is the ability to introspect. Take your customer example: if you could introspect it, and see what member variables were present, you could build a function that might output:

  {"type": "Customer", {"Id": 1, "Name": "Joe Bob"}}

Again, note the explicit serialization of the type here, in JSON. This is something I made up, of course, and not JSON. Of course, some language feature can make introspection interesting. (Can you even get a list of members? Some languages allow objects to make up members on-the-fly.)

Hopefully, I've convinced you language design is hard. :-)

With enough static typing, or some sort of schema, none of this is a problem: all you need to do is to specify the type you're deserializing to. That said, it sounds like Dart is in trouble here due to other problems with its design (inheritance).

Well, there is a language that's perfectly suited to JSON serialization, and it has inheritance but not static-typing.

The point here is that designing a language around the current fashion in serialization is a bad idea. The problem as laid out in the article is unsolvable. A serialization format can automatically capture the semantics of a given language's native data structures, or it can easily interoperate with other languages, but not both.

JSON hits a sweet spot on the web because it captures a large subset of Javascript's data structures, while being simple and inflexible enough that other languages can work with it via libraries.

You don't just want ISO 8601, but more exactly: RFC 3339 (a subset)

Ah, you're right. Thanks!

The lack of easy serialization is pretty strange. Even on stodgy old non-dynamic versions of C#, YamlDotNet does a really good job of translating arbitrary objects to and from YAML/JSON just using reflection.

Server-side languages have it easy. Code size isn't much of an issue so reflection works fine.

In a browser where code size matters, there are two ways to go:

1) In JavaScript, every library author needs to figure out how to crunch their library as much as possible and library authors brag about their small code size and minimal dependencies. We have debates about whether JQuery is worth it.

2) You implement tree-shaking so you only pay for the functions you use (GWT, Dart, and the Closure compiler's advanced mode). But tree-shaking doesn't play well with reflection, which is why dart2js reflection is a work in progress.

You can also go with code generation (essentially compile-time reflection) but that also increases compile times and adds code bloat due to the generated code.

Yes, it would be great if everything always just worked, and libraries always had your desired behavior as the default. Google should implement that right away.

I was recently writing a JSON un/serializing component in TypeScript, which is much nearer to JavaScript than Dart I believe, and even though I just had to make it work with a few certain types I knew in advance, it was still a PITA. I don't think there's a good way to JSON-to-anything-serializing, you always end up with some type related "tags" which shouldn't really be in the JSON data.

Thanks, was thinking of delving into Dart, but between the mediocre JS interop and this, I decided not to. Not to mention that ES6 brings tons of goodies to JS too, and the speed difference is not that great anyways.

Great JSON integreation is one of the things I enjoy most from Javascript (well, it makes sense given the J in JSON).

Personally I only care about JavaScript and am looking forward to ES6.

I don't care about transpilers and still looking for a use case worthwhile for Dart.

Unless Google pushes it, for example as Android language, ChromeOS or whatever might be the language killer application.

I know ES6 brings a lot of new shiny to the table, but JS is a poor language. It's riddled with traps and interesting stuff, and you have to put up with excessive differences between browsers. It's just stuff like

  Object.prototype.hasProperty.call(my_object, "property");

or not doing

  {
    foo: 1,
    bar: 2,
    baz: 3,
  }

or the truth table, things like `{} + []` and `[] + {}`, the insane implicit line joining… the list goes on and on.

I'm not sure Dart is the answer, but I welcome some competition. I think I'd be really interested to see a browser VM that languages could compile to, which would open the door to many languages. (I suppose asm.js might argue that they're this, and there is LLVM.)

Finally, I leave you with [1].

[1]: http://wtfjs.com/2012/08/11/Slashes!

> I know ES6 brings a lot of new shiny to the table, but JS is a poor language.

When started working professionally there was no web. So I do know JavaScript since the LiveScript days.

I never work alone, so I only care for languages ALL team members know, have proper support at the browser tooling[0] and are allowed to use by the customers.

For web development JavaScript is the only one that fullfills those requirements, so I don't care about anyhing else.

[0] good luck with source maps on mobile platforms

Reports of JS warts are greatly exaggerated. The two big remaining ones (post ES6) are type conversion by operators and silent failure when accessing object properties. Both are fixable with a nice type system with HM inference and structural types. I wish someone did JUST that, instead of trying to bring all the problems and warts of Java to JS while trying to fix it.

Roy looks very promising[1], and I'm hoping it will mature to the point where it can be considered a viable option for production projects

Typescript would've been ideal, but its kind of fake-open-source (no actual development can be seen on the repository, some of the code looks generated, no discussion about the internals or providing an API) and its moving towards having LESS type inference in the name of improving compiler speed, less type safety in the name of... "its too hard to do" I guess. And they still are not releasing anything which might help other vendors improve their tooling (e.g. Intellij IDEA, the second-best contender sucks at TypeScript). And the module system is a total disaster, especially if you're trying to do anything more complicated and CommonJS.

[1]: http://roy.brianmckenna.org/

Thanks for all the great feedback! I've updated the blog post with some of the key points from this discussion.

Learn JSON, but please for the love of Turing's ghost, teach it to use trailing commas.

So many needless syntax errors.

Dart should be JavaScript. Seriously, what's the point of Dart again?

I think they just mess it up with the marketing of it.. but the technology per-se is very good.. and if you try to see it not as a contender to any other technology like javascript.. but just as another player out there to create programs.. there's nothing wrong with it..

People tended to be very sentimental about it.. because they see as a threat to javascript, as something that came to replace it.. and thats what was bad in the image that came out, when they were launching dart..

but if you leave this wrong image behind and look only to the technology achievement with it, what's not to like?

Dart is basically a modern Smalltalk looking for a market.

And under modern Smalltalk do you imply modern Java?

No, because Java isn't able to provide the type of dynamic enviroment coupled with live coding as Smalltalk does.

Dart is actually trying to teach the web about static typing.

Static typing is not incompatible with JSON serialization, if done properly. It sounds like Dart's semantics are the problem here (and since they've reached 1.0 they can't fix it).

Dart effectively has very little static typing anyhow.

We should all support EDN

Reply on news.ycombinator.com