Hacker News Reader

Comment by Cthulhu_ | Hacker News Reader

When using parsers like e.g. Jackson or Gson for Java, this process is completely transparent and does not require any active thought from the developer - well, maybe if there's very specific formats that don't map 1:1 with the class that should be instantiated or generated from the json object.

It's a bit more tricky in JS, both client-side and node. You can't work with the json string there, but after that you work directly with the json object. They're not OOP languages, really. I wouldn't want to work with too much untyped / unstructured json in back-end land myself to be fair.

View on HN

I've never had good experiences with automated serialisation -- even though it sounds like other people do it with success. What's the secret?

To give you a flavour of the kind of poblem: In C# (or rather .net) json.net reads JSON and calls setters from the target class.

That means the setters have to be public, and you don't know what order they will be called in, and you have no real signal about when it is all done. The constructor is no longer enough to guarantee the object's invariants are met.

Most awkward.

Automated serialization has gotten much better than it was in the bad old days of RPC and COM!

I have nothing good to say about COM, but I'm seriously thinking about gRPC [0] to get away from the sloppy json endpoints we code around today, at work. Before I dive in I would love to hear, what it is that makes that architecture a bad one.

[0] http://www.grpc.io/

JSON.NET can use private setters. They just have to exist.

You do have to use thin constructors, but in JSON.NET there's a way to call a method post-deserialization.

Yes, the automatic serialization is not a solution for the most pressing problems presented by the article -- it's just the first thing from all the things that has to be done at the boundary.

You have some DTO class that is your system typed idea about the structure of the JSON -- this class is quite useful as an implicit documentation, but it really has to stay internal to the boundary. You will use an autodeserializer to such class and then you will continue by constructing real object from deserialized data that can be presented to the rest of the application. During such construction you can validate state and return errros.

This step can be eased by some validating attributes on the boundary DTO properties, but there is always some custom logic that describes what is acceptable and what is not.

In Java land with Jackson/Gson they can use the getters/setters or reflection and find the private fields. The only time it is not completely automatic is when some json object is mixed cased myField1 and my_field1. Even then, just adding an annotation fixes it. For any special formats, for example iso8601 dates, you can quickly define a serializer/deserializer and be done.

Is it really that hard in c#? It is not something I ever think about in Java.

Even beyond that, Jackson can use a private constructor if you use the @JsonCreator annotation on the constructor and @JsonProperty annotations on each parameter.

It sounds like you're parsing the JSON straight into your business objects, which is the source of the problem. You need an intermediate class which represents a strongly-typed version of the JSON message. So JSON.net goes from a JSON string into this message object, then you write your own code (or, if it works for you, use a tool like automapper), to go from that into your business classes.

This is what I settled on -- at least in the hard cases. And if I understand his acronyms, it's also what @mythz is recommending.

Perhaps I should have done it for the easy cases as well (where the business objects are struct-like enough that it doesn't matter) and just lived with the boilerplate.

But I see little advantage in this over just having a dictionary that I can inspect to initialise my real business object. True that is not strongly-typed, but the stage between the message-object and the business object can have validation errors anyway, so why not treat typechecking as part of that?

That's because you should be serializing purpose-specific DTOs or clean POCOs not Business Objects with behavior.

Automated serialization is the devil. Gson and Jackson require you to write EJB-style objects to get automatic serialization - default constructors, with getters and setters for each field - to achieve automatic serialization.

The problem with this approach is that you've completely abdicated the power of the type system to ensure that your objects are valid. What happens if a field is missing from the JSON? Well, that field just becomes null. So now you have one of two options:

1) Write highly defensive code with null-checks everywhere. This is a pain to write, a pain to read, and almost impossible to get right and actually prevent null pointer exceptions. This is a nightmare. Switching to a null-safe language like Kotlin doesn't really help you beyond making sure that you actually code in all the null checks - the code is still ugly and a pain to maintain.

2) Call a (potentially) expensive verification method at the beginning of each method call for your object. This is less error prone than having null checks everywhere, but it's not much of an improvement. Because verification happens not at object creation time but rather when it's used, you'll find yourself with a verification exception at the entrance to some business logic where the JSON was passed to your system a week ago, immediately stored in a schema-less ORM, retrieved now, so you kind of have an idea that you have some client which didn't populate the field, but you have no idea which of the many, myriad versions of the client is responsible. So now you're fucked, and you're doubly fucked if you're losing data because of it.

Or you could just take advantage of type safety and write immutable object factories which refuse to instantiate invalid objects. Then you can write clean code using objects which you know must be valid because of type system guarantees. Libraries like immutables.github.io make this a piece of cake.

Automatic serialization is not the devil, overly forgiving automatic serialization is the devil. I use JSON serialization libs all the time in Scala which properly support optional vs required fields. For Java devs, Gson has bad required/optional support [0] but Jackson does have it for creator properties [1]. It is important to qualify statements like your initial one to include the specific situation which it is bad instead of using a broad brush.

0 - https://github.com/google/gson/issues/61 1 - http://static.javadoc.io/com.fasterxml.jackson.core/jackson-...

Jackson allows you to create immutable object without any problems, with final fields initialized in constructor. It also supports numerous annotations that will throw an exception when field is missing, etc. You just have to know your tool and use it properly, that's all.

> Automated serialization is the devil. Gson and Jackson require you to write EJB-style objects to get automatic serialization - default constructors, with getters and setters for each field - to achieve automatic serialization.

I've used Gson fine with Scala case classes.

Although now I just use Scala JSON libraries, which do not suffer from the two problems you list at all.

It's a language/framework issue. C# supports first class properties with get/set semantics. In your method (actions) in the controller you would write something like this:

public List<CustomerModel> Get(SearchRequest request) { ..... return customers; }

Somewhere else in the (configurable) pipeline the framework can decide how to deserialize the SearchRequest and how to serialize the List<customer> based on the Accept-Header.

(CustomerModel/Request would not be business objects. They would only be used to on the API layer)

As far as validation. You could just put attributes on the properties of the Request like [Required] and they would automatically be validated before your Get method is called. Of course if the types don't match, the framework would send the appropriate error.

Gson does not require this. The following class will serialize and deserialze fine with Gson:

    class Example {
      private final int foo;
      private final String bar;
    
      private Example(final int foo, final String bar) {
        this.foo = foo;
        this.bar = bar;
      }
    }

More complicated cases will require custom serializers and deserializers, but any class that defines only basic data types (including collections) works just fine.

if you're just using python for simple scripting and a random failure now and again isn't going to ruin your day, it's fine to just use json.loads, IMO. I've written quite a few scripts where the time it would take to do it 'right' wouldn't be worth the effort.

> Gson and Jackson require you to write EJB-style objects to get automatic serialization

Not the case. I successfully used Jackson combined with Lombok to achieve some really nice DRY class definitions that Just Worked with Jackson. It took a little figuring out and a couple bugfixes to Jackson but it worked. That said, part of the hassle was that I insisted on being able to do this with @Wither so we could have the objects be immutable too.

Then you can write stuff roughly like

@Value public final class Thing { String name; int age; boolean boiling; }

Although IIRC I had to use some other random set of lombok annotations instead of @Value to get it to work right with Jackson (this was a while ago, don't recall details)

> The problem with this approach is that you've completely abdicated the power of the type system to ensure that your objects are valid.

Yeah, this was a big problem with my approach. The other choice would have been to use @Builders instead of @Withers. Then you get a little more boilerplate (having to type .build()), but you can guarantee the built objects meet consistency requirements. (In retrospect, I doubt I chose the right tradeoff there)

I think this might be where tools like Flow or TypeScript can become useful, since you can have typed javascript objects.

On the other hand, the fact that there's no runtime typecheck renders static analysis somewhat impotent when it comes to the result of a network call.

Reply on news.ycombinator.com