Hacker News Reader

Comment by corethree | Hacker News Reader

On hacker news someone told me a another name for this pattern: "Functional core, imperative shell". edit: I see other people have mentioned it.

One of the biggest issues with this pattern is if there's a ton of IO that's necessary for the application. If that's the case you'll see that the "bread" of the sandwich's is gigantic while the "meat" is very small and tiny.

This is the case with most web applications. The bread is sort of the main problem though but with this pattern we don't actually attempt to solve that problem, instead we just try to segregate the main problem away from pure code that intrinsically doesn't have that problem anyway. Don't get me wrong. The segregation is correct, but the problem of IO and mutating state is still unsolved.

View on HN

The IO doesn't usually involve business logic, almost by definition. A web service can usually make a pretty good sandwich if it's actually doing anything - you do a thin state fetch, a thick business computation on that state, and then a thin write / return to the user.

There are definitely cases where your business logic and your I/O are more tightly interleaved so that you can't really separate them the same way, and the post mentions that and links to https://blog.ploeh.dk/2017/07/10/pure-interactions/ as a deeper guide for those cases.

In web, IO is heavily intertwined with business logic and mutation. In fact most of the computation is done via the database. Database code is mutation heavy and offloaded as IO away from the web application. The web application is in fact just a thin routing layer.

It is not a one off case.

> In web, IO is heavily intertwined with business logic and mutation. In fact most of the computation is done via the database. Database code is mutation heavy and offloaded as IO away from the web application. The web application is in fact just a thin routing layer.

That's one way of writing a web service. It's not the only or even "normal" way - IME it's much more common to have the database act as a dumb CRUD key/value store, and put logic in the application layer (indeed many web services use a pure key/value store rather than a "database" as such).

You're right that if your logic is in the database, and it's an ACID-style "mutate the current state of the world" database then you can't really use pure functional techiques. The only solution to that is to not do that.

>That's one way of writing a web service. It's not the only or even "normal" way - IME it's much more common to have the database act as a dumb CRUD key/value store, and put logic in the application layer (indeed many web services use a pure key/value store rather than a "database" as such).

Wrong. The normal way is SQL. Web applications are suppose to be written in ways as nonblocking as possible. As much logic that can be pushed to the database as possible. That is the way to go. This reduces round trip time by a huge amount.

The pure key/value store is non-standard, but it is a way to do it. Not the traditional way and not the majority way either. Many of the major web application frameworks assume SQL as the underlying database not mongo or some other key value db.

>You're right that if your logic is in the database, and it's an ACID-style "mutate the current state of the world" database then you can't really use pure functional techniques. The only solution to that is to not do that.

Not only am I right. It's the main way things are done on the web. The functional layer is largely just code generation: Pure Functions with http requests as input and SQL code as output. This is the core model.

> Wrong. The normal way is SQL. The normal way is SQL. Web applications are suppose to be written in ways as nonblocking as possible. As much logic that can be pushed to the database as possible. That is the way to go. This reduces round trip time by a huge amount.

No it isn't. Most popular web frameworks expect to be used with an ORM and will autogenerate CRUD queries, while providing minimal if any support for more complex queries with logic in them.

> Many of the major web application frameworks assume SQL as the underlying database not mongo or some other key value db.

They do, but they use those SQL databases like a key value store - read/write by primary key most of the time, with occasional relationships but nothing fancier than that (e.g. no aggregations).

>No it isn't. Most popular web frameworks expect to be used with an ORM and will autogenerate CRUD queries, while providing minimal if any support for more complex queries with logic in them.

Or a query builder. Whether you use the ORM or query builder to build the queries or you manual build it is up to you. That's exactly my point. These queries aren't "autogenerated" you're writing a query in your web application with primitives decided by the framework and the ORM and then it's getting compiled into a SQL query with SQL primitives that's it. It's the same thing. I don't know what you're negating here, you're basically regurgitating exactly what I said.

>They do, but they use those SQL databases like a key value store - read/write by primary key most of the time, with occasional relationships but nothing fancier than that (e.g. no aggregations).

Wrong. A SQL database is more like an array. Things are stored in memory one after the other. That is the default. You can then choose an index, and have pointers to sections of the array that can make it havethe speed of your typical hash map or n-ary search tree. There is a huge amount of logic being thrown into IO.

> I don't know what you're negating here

You said "As much logic that can be pushed to the database as possible" and that's simply not true. The normal, mainstream way (like it or not) is to use the database as a dumb store and have the business logic in the application layer.

> A SQL database is more like an array. Things are stored in memory one after the other. That is the default. You can then choose an index, and have pointers to sections of the array that can make it havethe speed of your typical hash map or n-ary search tree. There is a huge amount of logic being thrown into IO.

None of that is business logic. It's implementation details at best.

>You said "As much logic that can be pushed to the database as possible" and that's simply not true. The normal, mainstream way (like it or not) is to use the database as a dumb store and have the business logic in the application layer.

False. What I said is true. As much as possible is 100% true. There are some things that have to live in the business layer. But as much as possible should be pushed to the database. All the "business layer" does is build the correct query.

What you're describing is multiple queries The web application has to fetch data, do "business logic" then fetch or update data again.

That is both slow and inefficient, but sometimes necessary. As much as possible the web application needs to build the query and do everything in a single IO call. The round trip time and even simply initializing multiple queries on the database side is magnitudes slower then one query that does the full operation. Business logic should be extremely thin.

This idea of a dumbstore is highly misguided. No framework works like this. The database is the workhorse of the basic web application, the application layer is suppose to be the thinnest layer. Your ORM, query builder or SQL code is where most of the logic real should lie.

>None of that is business logic. It's implementation details at best.

False again. The basic SQL query is this: SELECT <COLUMNS> FROM <TABLE> that is list retrieval at an API level and Not just implementation details. A key value store looks like this: VALUE = MAP[KEY] Understand?

There is no equivalent of key value fetching in SQL, you basically have to filter that list with SELECT * FROM TABLE WHERE id = ...

You can imitate a key value store both at the implementation level and at the API level if you use a hash index on a unique id like in the example query I just mentioned, but that is not the primary mode of the SQL api.

I don't think you get it.

> False. What I said is true. As much as possible is 100% true. There are some things that have to live in the business layer. But as much as possible should be pushed to the database. All the "business layer" does is build the correct query.

I think you are wrong, the other commenter is correct. In all my years of engineering, I rarely see a lot of business logic in the database.

Note that we are talking about what the situation _is_, not what it should be. It is an eternal debate as to whether it is better to push as much login into the Db as possible. My own view is that you should use the right tool for the job. Databases, excel at, well, managing data. Regular languages excel at logic.

> Business logic should be extremely thin.

I'm leading a team building a front end to statistical engines, storing metadata in the Db. Are you suggesting we should implement the higher level statistical functions in the Db?

Your talking about the multitudes of architectures that stem from the core model.

I'm talking about the core model. Which is crud.

The core model of a typical website is mainly updates to central state with little compute. Think nodejs server paired with a typical SQL database. The very reason nodejs exists is based on this core assumption of web: low compute and non blocking high io workflows.

You're talking about heavy compute... then that deviates from the core model. With heavy compute it turns more into a queue with workers. That is not the core architecture of the web. It's more of popular and important system that exists in conjunction with the core.

That is what your statistical function implies. If your stats function is simple and trivial, then you can indeed possibly achieve orders of magnitude of speed up if you offload that trivial calculation to the database.

> Business logic should be extremely thin.

> This idea of a dumbstore is highly misguided. No framework works like this. The database is the workhorse of the basic web application, the application layer is suppose to be the thinnest layer. Your ORM, query builder or SQL code is where most of the logic real should lie.

Like it or not, all the big mainstream web frameworks are oriented towards having the business logic in the application layer. Again, look at what kind of querying the ORM supports.

> The basic SQL query is this: SELECT <COLUMNS> FROM <TABLE> that is list retrieval at an API level and Not just implementation details. A key value store looks like this: VALUE = MAP[KEY] Understand?

> There is no equivalent of key value fetching in SQL, you basically have to filter that list with SELECT * FROM TABLE WHERE id = ...

If you look at the ORM APIs and the way mainstream web frameworks use them, fetching an object by primary key is very much the primary supported way of using them. MySQL even added the option of querying via the memcached protocol after their benchmarks showed that for typical web services it was taking 3x as long to parse the SQL as actually running the query.

I'm an expert on big mainstream web frameworks. It's not about what I like. It's about you being wrong. Orms are apis that are one to one with SQL queries. They do not turn SQL database into key value stores. It's not even a surface level change. On the surface the orm API is designed at it's core to allow you to construct a SQL query in the form of an oop object. And the SQL query is clearly not a key value store API.

Ask anyone. The primary usage of a SQL database is definitely not primary key queries to fetch one row at a time. Not at all not even close.

The primary key thing is just a unique Id that orms could use for a shortcut method that returns a singular row. But make no mistake primarily orms return lists and you have to filter those lists.

SQL is designed for relationships and sets. The mathematics behind it is all about creating a new set as a composition of other sets. Tables are sets, joins are relationships. The composition of tables results in a new table or aka set.

>If you look at the ORM APIs and the way mainstream web frameworks use them, fetching an object by primary key is very much the primary supported way of using them. MySQL even added the option of querying via the memcached protocol after their benchmarks showed that for typical web services it was taking 3x as long to parse the SQL as actually running the query.

It's one way of using SQL. It is not the primary way. You are utterly wrong. In your where clause "Id = 2" that would be what you're describing. The problem here is SQL obviously supports more complex where clauses. "Id > 2" "Id != 4" both of which will return lists and sets. And all orms should support features related to list filtering.

Don't understand how memcache lends any weight to your point. What does parsing and caching queries have to do with your wrong point of sql being a key value store. You can cache parsing in things unrelated to key value stores. Again I don't think you get it.

> Orms are apis that are one to one with SQL queries.

Lol. No they aren't. Not even close.

> The primary usage of a SQL database is definitely not primary key queries to fetch one row at a time. Not at all not even close.

It is, MySQL did the benchmarks. It may not be what they're designed for or what theorists want them to be used for, but it's how they end up being used in a typical web setting.

> In your where clause "Id = 2" that would be what you're describing. The problem here is SQL obviously supports more complex where clauses. "Id > 2" "Id != 4" both of which will return lists and sets. And all orms should support features related to list filtering.

Perhaps they "should". But in actuality they have more limited support the more complex your queries get; a basic column filter you can probably do, but as soon as you want to select something other than '*' most ORMs break down. And they're like that because the users don't need or care about those use cases.

> Don't understand how memcache lends any weight to your point.

Then maybe you should put a bit more effort into trying to understand rather than piling on the insults and attacks. You might learn something.

> You can cache parsing in things unrelated to key value stores.

It's not about caching, there's no caching involved. The point was to make it easy to fetch a whole row by primary key without having to parse an SQL query, because for typical MySQL use patterns, it was spending more time parsing the queries than actually executing them. So they enabled using a protocol from a key-value store, because the most common queries were using it like a key-value store.

>Then maybe you should put a bit more effort into trying to understand rather than piling on the insults and attacks. You might learn something.

I did not insult you. Read carefully. You should put more effort into your replys because nobody understands what you're talking about. Up to you, but clearly everything you're saying sounds ludicrous otherwise.

>MySQL did the benchmarks.

MySQL is a database and databases don't do benchmarks and benchmarks don't tell you how databases are used. You probably mean some researchers measured how databases were used. Show me this source otherwise I have to assume you're making it up. Most of your argument hinges on this.

>select something other than '*' most ORMs break down. And they're like that because the users don't need or care about those use cases.

Even if what you say is true, and it isnt with the orms I dealt with, filters are a basic feature of all orms and basic column filtering is 100 percent for sets. Orms are nowhere near turning SQL into key value stores.

>The point was to make it easy to fetch a whole row by primary key without having to parse an SQL query, because for typical MySQL use patterns, it was spending more time parsing the queries than actually executing them

I can see this optimization happening. But this is drastically different from MySQL being a big key value store.

The most frequent query may be a key value fetch but it doesn't make the databases primarily key value stores. For example on Google the most searched terms are for porn. Doesn't make Google a porn site and every user a porn enthusiast.

Basically every user who searches for porn on Google likely uses Google for other things too. It's just the aggregate most used queries happen to add up to porn.

Same with your basic website. The most frequent queries may be isomorphic to a key value fetch but clearly the websites utilize SQL databases for all the other features too. I find it highly unlikely that the majority of the web literally uses MySQL exclusively as a key value store. It is simply just the analytics showing that type of query as most frequent. There is a clear distinction here.

> I did not insult you. Read carefully. You should put more effort into your replys because nobody understands what you're talking about.

So if I don't understand what you wanted to convey that's my fault, and if you don't understand what I wanted to convey that's also my fault? Lol. Note the only third party to comment here came down on my side.

>So if I don't understand what you wanted to convey that's my fault, and if you don't understand what I wanted to convey that's also my fault?

I wanted to convey it's your fault for what? The only thing I wanted to convey was that you're wrong. If you have a real point then I'm not understanding it, that is your fault because it's a shitty explanation. Barring the shitty explanation if that's not the case then you're basically just wrong. But I'm open to the fact that you're just bad at explaining stuff.

Third party guy doesn't know what he's talking about. He's running some statistical algorithm in the web app. You know what statistical algorithms do right? They do massive compilations of data points. Huge averages and aggregations. He's calling that "business logic" lol and he's implying he puts it in the web app.

But that's the correct thing to do according to your misguided thinking.

Want to find the average price of all products being sold by an ecommerce website?

This is your genius architecture in the web app:

  //BUSINESS LOGIC!
  def handleRoute(request: Request) -> Response:
       result = 0
       for i in range(100):
          result += query(f"SELECT price FROM PRODUCTS WHERE PRODUCTS.ID = {i}")
       return Response(result)

It’s unsolved. The bread part is poker, and the meat part is chess. Yes, there is a ton of io, but the meat can pick the strategy.

Try this call no more than 3 times in 60ms, and get back to me. It’s complicated and frustrating and not always worth it, but even with lots of io, the logic, the strategy, can live in the pure compositional parts.

Your mileage may vary, but it’s often worth a little thought at least.

Of course. Nowadays though it's mostly metadata programming. Your haskell code is writing code for your database query. So you end up using functional code to do tons of IO and the data it sends to IO is just more code.

> One of the biggest issues with this pattern is if there's a ton of IO that's necessary for the application.

Yes, that's why patterns like this are necessary.

Do you stop doing dependency inversion when you have too many dependencies?

You're not getting it.

When there's so much IO, which is what happens in a lot of web applications, most of your code isn't going to be pure anyway.

Even your haskell code under do-notation looks very procedural because there's just so much IO. So you're basically back to programming regular code. It doesn't solve the core problem.

The pattern itself is good and is not a problem and I believe the pattern that should be followed.... but in the end practically speaking for the web it doesn't do too much to remove the main issue.

Reply on news.ycombinator.com