Pydantic
github.com
github.com
PEP 563, PEP 649 and the future of pydantic and FastAPI - https://news.ycombinator.com/item?id=26826158 - April 2021 (150 comments)
Show HN: Pydantic – Data validation using Python 3.6 type hinting - https://news.ycombinator.com/item?id=14477222 - June 2017 (27 comments)
I love the tutorials, but an API reference is just as important; I may not want to check out long tutorials about things I already read, but check a detailed description of each function would be super helpful - other frameworks such as Sanic have it.
I think it's in the roadmap, kudos for that, and hope that it will get a few more maintainers to speed up the process
I'm hesitant, however, since marshmallow-sqlalchemy provides full integration with your SQLAlchemy models, but pydantic-sqlalchemy only is for generating Pydantic models based on SQLAlchemy models, and it seems as if it's still experimental (Why does it have more stars? :thinking:)
Otherwise, just between pydantic and marshmallow for straight up validations, it seems Pydantic is more legible and easier to use at first sight.
Will switch if pydantic had full integration with SQLAlchemy.
And for those of you looking down on me for using ORMs (yes, i know some of you exist), I use both raw SQL and SQLAlchemy.
I find it multitudes easier to build models and deal with migrations in SQLAlchemy than writing scripts.
It is a thin layer on top of Pydantic and SQLAlchemy. I haven't used it yet, so can't speak out of experience, but I think it is basically exactly what you describe.
At first sight it seems like you still have to write a "schema" for the SQLModel based on your SQLALchemy model - so basically, two sources of truth.
If you edit your SQLAlchemy model, you'll also have to edit your SQLModel.
sqlalchemy-marshmallow allows you to build your schema based on your SQLAlchemy model.
Other than that, I'm still somewhat intrigued.
Thanks for the suggestion.
> That class Hero is a SQLModel model... But at the same time, it is a SQLAlchemy model... And at the same time, it is also a Pydantic model
looks good.
As I understood you don't, as SQLModel inherits from sqlalchemy ORM base classes. From the user guide, this is an example how to define the model and generate the table.
class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None
engine = create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)looks good.
In my experience, the bottleneck is either: - JSON parsing and dumping; the solution for me is ORJSON, fantastic wrapper to use fast JSON serialisation for most common fields, and also datetime. - Validation - if you choose to validate your data, pydantic can indeed be slow... But it's not Pydantic the problem, but the validation that you apply to your data.
Indeed, this kind of validation is usually based on 'isinstance', which is really slow in Python, because you often need to call it many times. More than once, I doubled of tripled the throughput of some data pipelines (not microbenchmarks) just by replacing 'isinstance' calls with something else.
When you really need something like isinstance, type equality sometimes works and is much faster. For example, this works as a replacement for attrs-strict's type checking on a limited subset of types (non-generic classes, Any, Optional, Tuple, and Union): https://archive.softwareheritage.org/swh:1:cnt:7f4f1ea32eace... The downside is that you can't use subclasses of the specified types.
https://pydantic-docs.helpmanual.io/usage/models/#creating-m...
In the case of GeoJSON it might be worth using a custom data type with specific faster validation.
https://pydantic-docs.helpmanual.io/usage/types/#custom-data...
If you want to do data validation like this but for something with better performance while still retaining the benefits of a high-level GC'd language, then I'd try something like https://github.com/go-playground/validator for Go.
Maat is much faster then pydantic according to the benchmark of pydantic.
https://github.com/samuelcolvin/pydantic/discussions/3094
Maat readme also shares that benchmark. Its way faster.
“Way faster” is a bit hyperbolic isn’t it? 2.5 times faster? That’s no order of magnitude. What validation use cases are there where it makes a difference? Very few.
Here’s the thing: if people really want fast validation and transformation, they’re probably not going to use python. People use python for its developer ergonomics and experience. Dicts as a configuration DSL are inferior to classes and type hints for very simple reasons
1. One bad thing about python is that both single and double quotes are acceptable. Dicts are built with strings, so there’s an anxiety about having to standardize in one over the other that adds to cognitive overhead.
2. There are dict literals that get rid of one string, but again, just another “choice” people don’t care to make if they don’t have to
3. Curly brackets aren’t the easiest things to type relative to other characters.
4. Curly brackets are extremely difficult to pair if you don’t have an editor that does it automatically or if your code formatter puts many brackets on the same line.
That’s pretty much it right there.
In their own documentation they invite all other framework to send their results.
That said what pydantic team does is up to them, the Maat was made before pydantic was a option, it has filled that usecase. The benchmark was only added because of a internal discussion about which tool to use.
Engineering is about tradeoffs and each project will have own technical problems. Therefor there will never be the best solution, only the best solution for a particular problem within that context.