Hacker News Reader

Comment by sametmax | Hacker News Reader

Marshmallow is still the best lib in town. Indeed, most of the libs fall short when you start to use them in the real world. Where validation is more than a type, when fields are dependants on each others, when data is generated on the fly post validation and where you need all that to cascade down your nested, sometime recursive, data structure validation, which then should produce equally complex error messages.

A data validation framework is not a toy project.

View on HN

Pydantic is not just a toy, I built it having used and abused numerous other libraries and found them wanting in one way or another.

Because it reuses python's typing system it should have the most pythonic and flexible description of types possible.

I agree about the need for complex validation chains relating to numerous fields, that's already partially possible with pydantic (although not documented). I'll add support for this stuff as well as documentation over the next few weeks.

Having used both colander and marshmallow extensively - I prefer colander mainly because it has first class explicit handling of null, missing and required values and it's support of nested and inheritance is also much nicer than marshmallow.

I've recently been using good[0] , which also allows for minor data transformation. Looking at Marshmallow, it doesn't seem like it allows inline declaration of nested schemas - each level of the schema needs its own data type.

[0] https://github.com/kolypto/py-good

I agree. Marshmallow is the best game in town and I love using it. That said its not perfect and I'm eagerly awaiting a successor. I use it frequently and have run into a lot of cruft.

I would love to read about it.

Sure, my complaints are mostly around error handling. My comments might not be accurate since its off the top of my head.

When a field raises an error I've noticed the value gets replaced with a "None" type (or maybe its removed from the passed data object) when using @validates_schema. This is annoying because I have to pass the original data and see if the value is actually null or not. This can suck when using JSONAPI because you have to be super careful with your data extraction (eg. data.get('data', {}).get('relationships', {})...etc).

I would like more control over how validation is executed. The ordering of validation, the ability to stop validation or continue validation at arbitrary points. Maybe in my validator I could do something like "raise ValidationError(msg, stop_validation=True).

I would like more control over pre and post dump/load order. Sort of like a z-index in css. So when using marshmallow-jsonapi, I can specify pre_loads that access the data before and after the jsonapi pre_load formatting.

I would like a better way of using "class Meta". Right now its annoying to inherit from a base then define additional "class Meta". I think I ended up subclassing SchemaOpts and settings my defaults that way on my base schema.

I would like a way to replace error messages so instead of "Missing data for required field", it would say "Please specify data" (or whatever). I want to define this at the schema level too. I don't want to have to constantly define fields with the same behavior everywhere. It would be cool if Marshmallow had a dictionary of error codes and messages. So it would look like {'1': 'Missing data for required field.'} and I could override that error with "errors['1'] = msg".

If I had to sum it up. I like to write small functions and classes that have limited uses. A lot of times I feel like Marshmallow pushes me into more monolithic work so I can control the flow.

I believe marshmallow v3, which is not yet released, claims to solve the error handling / swallowing exception issue.

I use marshmallow all the time and love it - I agree that error handling is its weakest area, even using strict mode.

Thanks that was very interesting.

I wish the people who designed APIs would constrain themselves to libraries that could be validated as types. I rarely see insane data structures for APIs built in static languages, and I say this because my company has some absurd APIs which would not be so bad if the developers were constrained a bit.

You can't. You need permission validation, data integrity check, time sensitivity check, caching, etc. for any decent crud app nowaday.

What about good[0]?

[0] https://github.com/kolypto/py-good

Reply on news.ycombinator.com