Master Hexagonal Architecture in Rust
howtocodeit.com
howtocodeit.com
Attempting these design patterns is a common part of getting over OOP when new to Rust. The result: over-abstracted, verbose, unmaintainable C++/Java written as Rust. Every layer of indirection ossifies the underlying concrete implementations. The abstractions inevitably leak, and project velocity declines.
I have seen the same types and logic literally copied into three different repositories in the name of separation of concerns.
Luckily people usually get over this phase of their Rust career after a couple of failures.
If you’d like to skip that part, here are a few rules:
1. Always start with concrete types. Don’t abstract until you have at least two, preferably three, concrete implementations.
2. Separation of concerns is a myth.
3. K.I.S.S.
The original code fits on one page, is readable from top to bottom, and doesn't contain any pointless abstractions that worry about 'future problems' that never actually come to pass in the real world anyway.
If the code no longer fits the requirements, no big deal, just throw away those 40 lines and rewrite them from scratch to fit the new requirements. That will most likely take much less time than understanding and modifying the refactored 'clean code', because there's a pretty good chance that the new requirements don't fit the abstractions in the refactored version either (and IME that's the typical scenario, requirement changes are pretty much always unpredictable and don't fit into the original design, no matter how well thought out and 'flexible' the original design was).
I guess part of the confusion came from how dependency injection is a form of inversion of control… the words are all very similar to dependency inversion.
With DI, you stop your business logic from knowing about your Postgres database.
With DInjF, not only does business logic still know about Postres, but now it knows about Spring too! (The upside is that it takes fewer lines of code to get from 0 to spaghetti)
Dependency injection and the Dependency inversion principle are not one and the same.
The principle makes a claim, that inversion is a good onto itself.
Injection is a tool not a claim.
1) Record of functions. Probably the most common, and pretty analogous to calling a constructor in OO. If you've heard "objects are a poor man's closures, and vice-versa", that's what this is referring to. You build a closure with as many dependencies as you can pass in at startup time (fields in OO), then later you can just call one of the functions inside it, with the remaining parameters (method call in OO).
2) final tagless. After you're comfortable with an Optional<Value> that might return a Value, and Future<Value> that will return a Value later, you can venture further down that path and work with a Writer<Value> which will return a value and do some logging, or a Reader<Env, Value> which will return a value while having read-access to some global config. But what if you want many of these at once? You end up with a ReaderT Env (Writer Value) or some nonsense. So instead you can write your code in terms of m<Value>, and then constrain m appropriately. A function which can access Git, do logging, and run commands on the CLI might have type:
(Git m, Logging m, Cli m) => m Value.
But that function does not know what m is (so therefore does not depend on it) You might like to have different m's for unit tests, integration tests, a live test environment and a live prod environment, for instance.3) Free Monad & interpreter pattern. A nifty way to build a DSL describing the computations you'd like to do, and then you can write different interpreters to execute those computations in different ways. For instance, you could write a network/consensus algorithm, and have one interpreter execute the whole thing in memory with pure functions, for rapid development, debugging and iterating, and another interpreter be the real networked code. It's fun, but someone wrote an article about how Free Monads are just a worse version of final tagless, so I haven't used this in a while.
class Git m where
clone :: Url -> m GitRepo
currentHead :: GitRepo -> m CommitHash
and Logging might be something like class Logging m where
log :: String -> m ()
which is very similar to defining data GitDict m = MkGitDict {
clone :: Url -> m GitRepo,
currentHead :: GitRepo -> m CommitHash
}
data LoggingDict m = MkLoggingDict {
log :: String -> m ()
}
class Git m where gitDict :: GitDict m
class Logging m where loggingDict :: LoggingDict m
So the "record of functions" style and the "final tagless" style are equivalent, except that the former passes operations manually and the latter passes operations implicitly. The former can be considered more cumbersome, but is more flexible.If you're interested in how my effect system Bluefin does this, you can have a look at https://hackage.haskell.org/package/bluefin-0.0.4.2/docs/Blu...
Incidentally, one thing it glosses over is the creation of the components, which may prevent completely inverting the dependency.
[1]: https://en.wikipedia.org/wiki/Go_(programming_language)
[2]: https://en.wikipedia.org/wiki/Rust_(programming_language)
I don't fully understand the quote from Djikstra where he first talked about this but I'm sure he didn't mean it as it's interpreted today: "draw invisible boundaries in random places because best practices."
(yes, this is indeed a generalization)
It's recommended to not split the low level details from. your business logic, in fact it's not just recommended the compiler slowly forces your hand.
If you write overly abstract code like the author recommends you will leave a large amount of performance on the table. Code like that doesn't play nicely with lifetimes, by trying to separate memory management from business logic you're left with only the least restrictive scheme owned heap allocated data.
The Rust type system teaches you not separate concerns, without giving up the ability to reason about your code.
There is a large body of content on why concepts discussed in the article are championed. And also a large body of content on how they are misused (and I agree that can be - even to a huge degree).
So while, I think it is fine to judge that ascribed benefits are not worth the cost (or are not even realizable in a typical development team). Or argue why the benefits of an architecture like this doesn't work for Rust in particular - which may be the case since many of these patterns are oriented towards design in the enterprise applications space. But ascribing the approach as being a "parody" is not at all constructive.
The patterns of hexagonal architecture isn't in any way coupled to OOP even if some of the terminology is highly aligned with languages in the OOP space. In fact the well regarded (at least by me) Mark Seeman has an article how Ports and Adapters, another name for Hexagonal Architecture, is inherently functional [1]. And this resonates with my experience.
I have seen the pattern implemented well across: Python, Typescript, Javascript, .Net, Scala, and Go. And while the systems languages such as: C, Rust, and beyond are quite distinct from the previously discussed languages. There is certainly space for debate on the viability of the application of these patterns.
[1] https://blog.ploeh.dk/2016/03/18/functional-architecture-is-...
You always have two concrete implementations: the production application and the testing application. Otherwise you yolo things into prod, or run only manual/integration tests. That can work for a while, but many people find it unsavory.
It is pretty easy and sometimes useful to make three implementations: http server, CLI, test. Maybe you want to use files in CLI and a db for a server.
It has always been a good idea to isolate persistence and transport concerns from the business logic and that doesn't change in Rust. Don't dependency drill SQLite up and down every call stack. If your application is small enough and will stay so, then separating it is more a question of habit than anything.
But you shouldn't abstract everything nor try to separate everything! It was a persistence layer before, in the hexagonal architecture it becomes adapter implementations of a port. Transport layer is similar. Separation of concerns in this case means that you have a concrete dependency (an http server, a db etc) that isn't part of your logic.
It's not intrinsically good design but it does improve unit testability (which sometimes has value and sometimes has zero value).
Don't start by saying how can I unit test this tiny bit of logic against several mocks, start with a simple integration test of your real routes.
As you add abstractions you are trading maintainable straightforward code for more granular testing. It's a hard trade off not a set of principles for good code.
What I mentioned was an adapter and port. If you can only run against a postgres database, your integration test is going to require setup and be slow. If you can easily swap the postgres adapter out for sqlite or in memory, the same test will be practically instantaneous and self contained. The same test can be occasionally run against a postgres database (like once a night), to ensure there are no postgres specific idiosyncracies.
Thus I started by saying how you can integration test quickly without mocks (sometimes it would be called a fake, but using a different db is something else) on real routes.
I would add - one of the symptoms of this type of over-abstraction is what you could call the "where tf" problem. Any time you need to do anything it's really hard to figure out where
1) ...the thing you're trying to fix actually happens so you can fix it
2) ...the new feature you're trying to add should actually be added
3) ...it's going wrong when it's going slow/not scaling/stalling somehow
...because in reality the answer to the question "where" is always "all over the place". And that means you typically need to make several small changes in various places to do anything. So you've papered over the intrinsic complexity of the system and you have a really nice looking whiteboard but the complexity is now distributed in a bunch of places so it's conceptually complex and much harder for a dev to actually get their arms around the whole system and understand it fully. And you now have a false sense of security because you have great test coverage but the kind of problem you now face isn't caught by tests. Because typically the type of problem you hit is you should have made 6 changes in different places to implement your feature but you've forgotten one and only implemented 5. So the system is now semantically broken in some way even though all the tests pass.
> If you ever change your HTTP serve
Yeah, that ain't happening. You also won't replace your database engine or queue either. If you do for some reason, it'll be a partial rewrite of you app no matter what. You can't abstract over these things in a useful way, the common denominator isn't rich enough for useful applications.
> You'd have the same problem if this code lived in a setup module
Moving code from file A to file B does nothing to it. This is the fallacy of assuming that the name of the thing changes the thing, and is in the same vein as having a "secure" network where it's secure because it is called that in a spreadsheet of subnets.
> our HTTP handler is orchestrating database transactions
Transactions are deeply linked to requests. If you try to abstract this away, you won't be able to read your code any more because you won't be able to see the control flow in... the controller.
> You cannot call this handler without a real, concrete instance of an sqlx Sqlite connection pool.
Faking a database is a fool's errand. It's a lot of work at best, and a subtle source of false negatives or positives in your tests at worst. Database engines have very complex behaviours such as concurrency, transactions, locking modes, type conversions, collation, and so on. Why try to emulate this!? Just use a local database file for testing!
A bigger concern with the repository pattern is that without eternal vigilance, it'll block the use of high-performance code.
For example, with the Author repository, retrieving authors is all-or-nothing. The blog author used sleight of hand to hide this by having a single "name" field along with a primary key. Okay, what if there are 287 fields, and a bunch of foreign keys? Now what? Do we read in the 1 name field along with 286 unrelated fields just to throw all that work away? That's 0.35% useful work performed per call!
Similarly, he returns single authors, one at a time. How do you returns collections in response to queries? As Iter? A Vec<Author>? What if it's an async streaming response!? If you try this, you'll quickly discover that there is no general portable pattern across different DB providers and every approach has some downside. That downside can be "OOM panic" or similar.
I've been doing a lot of work recently to clean up legacy ASP.NET apps and my #1 trick to directly invoke Entity Framework directly in the controllers (HTTP handlers). I select just the required columns, run the queries as async, and where possible/useful I stream back the results instead of trying to hold them in memory at once.
I've seen 5-10x speed ups compared to SOLID pattern code with everything broken out across dozens of interfaces, abstractions, and layers scattered across a bunch of projects. All this with a 30x (no joke) reduction in lines of code, dramatically faster builds, faster deployments, and readable code that can actually be maintained by one person. I reduced one project that had several thousand lines of code to one page, the same kind of thing as the "bad" everything-in-main example in this blog post.
Was I bad and wrong?
At $work we’re slowly ripping out a similar system in favour of handlers calling database functions directly, such that they can put transactions in the right place across more complicated sets of queries. Code is simpler, data integrity is better.
That's an easy scenario. General utilities should aim to make hard things possible and then minimize the overhead in easy scenarios, so the fact that this is more complex than the dumbest thing that could possibly work, is not by itself a good argument against.
By all means, write a test to make sure the queries actually work on the database. It will be slow, stateful and annoying, but still probably worth it.
But you don't want to bring that slow-statefulness over to every other upstream test in the entire system: want to test if the permission system (in some outer class) is allowing/blocking correctly? You'll need to start up a database to do so. Is your test making sure that an admin with permissions can create a user (without error?), well it's going to start failing due to USER_ALREADY_EXISTS if you run the test twice. To avoid that you'll need to reset and configure its state for every single invocation.
Good testing frameworks just do this for you.
I generally prefer focusing testing efforts around what you describe - spinning up the entire system under test - because that's how the system is used in real life. There's definitely times you want to test code in isolation statelessly, but I find that engineers often err on the side of that too much, and end up writing a bunch of isolated tests that can't actually tell you if an http call to their API endpoint performs the correct behavior, which is really what we want to know.
But you should totally do what you're doing until it breaks. You only start looking into better architecture when things are not working. Being aware of something like this means you'll have something to look up if/when you run into the problems that inevitably crop up in simple "db wrapper" type APIs.
I'm all for the unit testing argument, but in an active SaaS business I've never seen the hypothetical database change where a well architected app makes it smooth. I have certainly moved databases before, but the performance and semantics changes dwarf the call sites that need updating. Especially in Rust, where refactoring is quite straightforward due to the type system.
After many years of a lean team serving high scale traffic (> 1 million monthly active users per engineer), most abstractions between customer and data seem to turn into performance liabilities. Any major changes to either client behavior or data model are very likely to require changes to the other side.
There's a lot to be said for just embracing the DB and putting it front and center. A favorite system we built was basically just Client -> RPC -> SQL. One client screen == one sql query.
If your DBUserStore happens to know directly about SQL, that class is the wrong place to try to introduce flexibility around different DBs, ORMs, SQL dialects, etc. Just hard-code it to PostgresUserStore and be done with it.
Instead, put the interface one level up. Your PostgresUserStore is just a place to store and retrieve users, so it can implement a more general UserStore interface. Then UserStore could just be a HashMap or something for the purposes of unit tests.
Also, if you have some autowiring nonsense that's configured to assume "one service has one database", that's bad. Singletons used to be an anti-pattern and now they've been promoted to annotation and/or basic building block of Java microservices.
When it comes time to live-migrate between databases, your service will need to stand up connections to both at once - two configurations, not one, so architect accordingly.
Singleton was never the antipattern, it was the GoF implementation using the class to manage the singleton instance that everyone eventually ran from. Object lifecycles like Singleton are managed these days by a module system or a DI container.
We could do AIX, HP-UX, Solaris, Windows NT/2000, Red-Hat Linux, with Oracle, Informix, DB2, Sybase SQL Server, Microsoft SQL Server, Access (if you were feeling crazy, just for local dev).
It wasn't that the customers would switch database, or OS, rather the flexibility allowed us to easily adapt the product to customer needs, regardless of their setup.
Depending on the early history of your product that might be the same; or it might not. If you know from day one that you need to support two databases rather than one, that would be enough to cause design choices that you wouldn't otherwise make.
Also it wasn't like we developed all those OS and database backeds for version 1.0, and didn't do anything else afterwards.
Which OSes and RDMS to support grew with the customer base and added to be plugged into the product in some way or fashion.
I disagree (strongly in favour of of DI / ports-and-adapters / hexagonal).
I don't want my tax-calculation logic to know about one database, let alone two!
Bad design:
class TaxCalculator {
PGConnection conn;
TaxResult calculate(UserId userId) {..}
}
Hypothetical straw-man "future-proof" design: class TaxCalculator {
MagicalAbstractAnyDatabaseInterface conn;
TaxResult calculate(UserId userId) {..}
}
Actual better design: class TaxCalculator {
UserFetcher userFetcher;
PurchasesFetcher purchasesFetcher;
TaxResult calculate(UserId userId) {..}
}
I think a lot of commenters are looking at this interface stuff as writing more code paths to support more possible databases, per the middle example above. But I do the work to keep the database out of the TaxCalculator.That is, User is the domain object, which could be built from one or more database tables inside the Fetcher.
If it's really big data I imagine I'd be in some kind of scalable no-SQL situation.
If it's not so big, Postgres will comfortably handle my access patterns even without joins.
If it's in the sweet spot where I need Postgres JOINS but I don't need no-SQL, well then, refactoring will be a breeze. I'll turn:
class TaxCalculator {
UserFetcher userFetcher;
PurchasesFetcher purchasesFetcher;
TaxResult calculate(UserId userId) {..}
}
into: class TaxCalculator {
UserPurchasesFetcher userPurchasesFetcher;
TaxResult calculate(UserId userId) {..}
}
which is backed by JOINS inside. And I can do this refactoring in two mutually-independent steps. I can make my Postgres class implement UserPurchasesFetcher without thinking about TaxCalculator, and vice versa.And if it's about the data integrity that JOINs could notionally provide, I no longer believe in doing things that way. The universe doesn't begin and end within my Postgres instance. I need to be transacting across boundaries, using event sourcing, idempotency, eventual consistency and so forth.
Mixed feelings about this.
“Oh our http handler knows about the db”
Ok? Its job here is more or less “be a conduit to the db with some extra logic”.
It “knows about a lot of things” but it’s also exceedingly clear what’s going on.
The main function is fine. I’ll take a “fat” main function over something that obscures what it’s actually doing behind a dozen layers of “abstraction”. Nothing like trying to isolate a component when you’re fighting an outage and nobody can figure out which of the 16 abstract-adapters or service-abstracted launched the misbehaving task.
The original code might be “messy” but it’s at least _obvious_ what it’s doing, and it can pretty clearly be pulled apart into separate logic and IO components when we need to.
This all just feels a bit…over engineered, to say nothing of the insulting tone towards the other learning resource.
I uuuhh, I hate to tell you this, but uh, if you’re swapping Postgres to Mongo, and you think that hardcoded queries are going to be your migration pain points, I have some bad news for you. The difference in semantics (for any such change, not just db) will bite you first, and wil bite a lot harder.
This idea of “we can abstract over everything and then swap them out at our leisure” works less than we’d all like to imagine it does, and crucially building everything to accommodate this utopia, will leave you with mountains of code that will do nothing other than make your teammates hate you and obscure your actual logic.
> AuthorRepository
Oh hello C#. So instead of cursing everyone who has the misfortune of working on your C#-flavoured-rust codebase of having to write literal pages of boilerplate to inevitably add a new struct/type to the codebase, I suggest leaning into idiomatic Rust a bit more. Personally, I’d make read/delete/upset traits, whose methods take a handle to a connection pool. Logic for sql then lives inside these implementations, and can be brought into scope only when necessary. Something like `my_struct.upsert(&conn).await?`. We have locality of behaviour, we’ve separated IO details out, we have all the same advantages with about 50% less code-noise.
In C#, EF Core already implements a repository with the way its API is structured so most of the time writing another one on top of it is an anti-pattern.
Tell me the thing that's in the headline first. I'm not going to read your article if you don't do this. It's not that I'm not intellectually curious, it's that I don't like being messed around.
Never needed to. Premature optimization.
> We have the same issue with the database
What issue?? Cross that bridge when you get to it...
> To change your database client – not even to change the kind of database, just the code that calls it – you'd have to rip out this hard dependency from every corner of your application.
I really doubt this is such a big deal... Bit of ctrl+f for that module and paste the new one. All database libraries have a "connect()" and a "query()".
I'm so far convinced that this article is written for people who have too much time to waste on things that will probably never happen just so they get to feel smart and good about something that's not even visible.
Imagine if we built bridges this way: yes, right now we have settled on a stone masonry design, but what if we want to move to steel trusses in the future???
Why can software engineers not accept that it is unreasonable to expect one single human creation to last forever and be amendable to all of our future needs? Cross the bridge when you get to it. Don't try to solve future problems if you're not 100% certain you're going to have them. And if you _are_ certain, just do it right the first time, rather than leaving an opening for a future person to "easily" fix it up. And sometimes? A new bridge has to be built and the old one torn down.
There really are very different perspectives on what makes code easy to manage, and interfaces (especially ones used only for a single class) are a sure-fire way for me to make code unmanageable. But java enterprise coders disagree. That can clash hard.
Not sure java is bad for indirections, java is bad for boilerplate and lack of defaults, imho. But I usually don't use frameworks/stlib directly, so I create my own boilerplate that is reusable
So I'm not talking about having one or two sensible interfaces as a contract, but codebases where every class has at least one interface, for "decoupling". Pure hell.
Concrete example from work today - we have a trading application and there are many paths that lead to alerts of some kind. Alerts are usually raised inline with any business logic (as should be - they intrinsically coupled). Alerts however can be delivered differently - via SMS, other messaging systems and/or log messages. The different places where the alerts need to be generated do not _need_ to know how the alert is going to be physically delivered to its destination - they just need to generate it. Without an interface (or at least a type alias for a function) - it would make being able to say e.g. this alert is a direct phone message vs. a chat message in some channel because of the type it is - much harder.
In the system I worked on the shortcut did not work - too many options or some consequence of the dependency system, additional layers of indirection.
In real life, you just can't have it both ways. There's always a trade off. If the trade makes sense go for it I say, but I think people aren't always honest (to me or themselves) about the exact trade they are making.
Hah, no, they're not. Every production app is read many, many times by several different developers, mostly to search for bugs. Reading across adapters, services, and interfaces make that much harder. Especially, if the architectural pattern was implemented slightly differently every time by various authors.
Abstraction always comes with a price. Often, it is well worth to pay (or we would still code everything in C or assembly), but sometimes it's just a waste.
I think people here are devaluing the "kind of approach" taken in these over-abstracted-architectures. The snippets of "improved" code in TFA have a lot of new line-noise, a lot of boilerplate and half-a-dozen "new" pieces of terminology, all in the quest to essentially separate IO and core logic.
There's plenty of ways to perform this separation, many of them don't require replicating half the design-patterns from C# tutorial. I'm pretty partial to "functional-core, imperative-shell", I think that it achieves all the same advantages, with none of the new concepts, and far less line noise.
Now he's not here anymore, and posts in his linkedin about simplicity and how people overcomplicate things in software so much. This shows me that you should be careful when taking advice from other engineers, they learn and move on from what was previously a "best practice", while you might get stuck thinking it's worthy it because "that one very good engineer said it was how it should be done".
If you want to test, test at the integration layer using testcontainers[0] or something. Not everything has to resemble a Spring style dependency injection/inversion of control.
Hell is full of single-implementation abstractions.
Total nitpick: For `CreateAuthorError::Duplicate`, I would return a 409 Conflict, not a 422 Unprocessable Entity. When I see a 422 I think utf-8 encoding error or maybe some bad json, not a duplicate key in a database.
So it’s more “your request didn’t make logical sense” more than “your request was missing a closing bracket”. That’s just a 400.
[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/422
The two main use cases are routing frameworks and tests.
Axum already has this, but it is married to HTTP a bit too much.
What's the problem with constructor injection of the dependencies using traits?
Accessing something that might update it's internal cache needs to be mutable so i) this need for mutability is viral up the call chain ii) we can't share mutable references... so it's going to be a pain in the butt and need to sidestep compile guarantees somehow. Having an out of the box best solution for common scenarios like this would be nice to see at least.
Hexagonal architecture brings order to chaos and flexibility to fragile programs by making it easy to create modular applications where connections to the outside world always adhere to the most important API of all: your business domain.
okay.Maybe I’m being glib, but damn, a whole article, and boatloads of fancy new terminology, just to re-state what the author succinctly landed on in the _first_ paragraph:
> Create your application to work without either a UI or a database so you can <do a bunch of nice stuff and make your life easier>”.
- https://news.ycombinator.com/item?id=18043058 (Sep 2018, 127 comments)
- https://news.ycombinator.com/item?id=34860164 (Feb 2023, 39 comments)
Best explanation by far, worth the 5min watch
> Bullshit.
This arch really means use interfaces based on business use cases / domains. Call the User service/module and pass user ids into a billing service/module. Each service is over a defined interface (adaptor) that allows separation of concerns and separate data stores. You could use an in-memory port of the billing service and a real db for the user service service because both implementations leverage the same adapter code
That's pretty much it. Everything else flows from there.
To me, a hexagonal architecture essentially means creating an abstraction at the point between pure and impure functions.
The realization of this approach essentially means the core of your application should be entirely pure and as large as possible. And your impure adapters, at the application boundary, (e.g. a rest api, db client, file system, system clock, etc.) should be as small as possible and impure.
Doing this well essentially allows you to get the best of both worlds - highly coupled code (i.e.your pure functionality) and highly decoupled code (i.e. your pure-to-impure functionality).
Another good reason for leveraging "functional design" as an argument is that many of those skeptical of architectural patterns are ironically heavily onboard the "functional design" bandwagon. So it is a strong argument in a political sense also.
And what's the point again? To make it so that you can switch out Postgres for MongoDB later? To make your web app now accessible over XMPP? It feels like a lot of work to enable changes that just don't happen very often. And if you wrote your app with Postgres, the data access patterns will not look very idiomatic in Mongo.
I think X11 in *nix land is an interesting example of what I mean. X11's client-server architecture allows a window to be "powered" by a remote machine as well as local. But it's dog slow. Straight up streaming the entire desktop frame by frame over VNC is usually smoother than using X11 over the network. I think we just haven't reached a point yet where we can develop good apps without thinking about the database or the delivery mechanism. (I know X11 isn't exactly new, but even with decades of advancements in hardware, X11 over the internet still loses to VNC)