The Impedance Mismatch Test: Is Your Data Layer a Complex Mess? | Hacker News Reader

The Impedance Mismatch Test: Is Your Data Layer a Complex Mess?

thenewstack.io

24 pointsnode-bayarea21 comments

It looks like a rather oversimplified model: the examples assign the same overhead to any component, transformation, or feature, so anything monolithic or that can be grouped together is favoured, even if buggy, clunky, or poorly supported. Transformations may be awkward (lossy, or having to make assumptions and make up some data, and/or dealing with poorly specified formats), which makes them unequal as well, but it's often arguable which ones are simpler. Features may have different priorities, but even if they don't, I imagine them to be among requirements, not up to technical decisions; if they were, it'd be another way to hack the resulting value by adding useless ones. And ignoring all that, it just says that systems with fewer components are generally simpler.

It seems to be the case with most talks about simplicity: pretty much everyone agrees that it's good, but disagrees on what it means and how to estimate it.

I think the article is mainly attempting to provide you with a tool to calculate complexity. You can add additional columns to the calculator and make it your own for your needs. It's been used by some companies internally to compare complex systems or to migrate from an old system to a new system.

I don't really understand the need for developers to find some real world scenario or situation that they can use as analogy.

Impedance mismatch is a physical thing, it creates physical problems that are based on laws in nature. Having to map your database entities to a model that is different has nothing to do with any physical things, it's a mental construct.

If you have a mismatch of impedance, what do you do? You insert something between the two mismatched devices to match the impedance. Isn't that what all software does already anyways? Insert some mapping, converting, transforming layer between the two pieces?

Sometimes novel analogies lead to novel solutions.

Can you specify a single instance of this?

GUI as a desktop.

Well, I think that has been somewhat over-rated. Looking at my physical desktop now, I don't see any little "X" icons on the beer bottles littering it, or anything analogous (wish there was). And I hardly ever use the Windows desktop, and haven't for many years, even though I use Windows almost all the time.

Yeah, I agree - I'm really anti the over-use (and there is a lot of that) of bad analogies in the software development community - I've never seen the attraction myself. And I'm speaking as someone that has done a lot of teaching in this area.

But at least this one is not particularly harmful or obfuscatory - you have A and B and you need to frig things so they can talk to each other.

It kind of is?

When I have conversations about this, I encounter the term "object/relational impedance mismatch" a lot more often than I encounter people who really have a thorough understanding of what the term describes and why it exists.

It often seems to me like the way it functions in our profession is a lot like the way the word "quantum" functions in Deepak Chopra's vocabulary. It's not really a specific, definable thing, so much as an amorphous, esoteric cosmic principle that can be used as a universal logical connector for stitching together any arbitrary chain of reasoning.

Which is too bad. I don't think it's actually a particularly difficult concept, when you get down to it. But I do think that the fancy-sounding technical name for the concept tends to obfuscate its meaning by making it sound like a much bigger deal than it actually is.

Completely agree about the Object-relational mismatch. The general term "Impedance Mismatch" is used in several industries to mean the loss in efficiency due to increase in complexity.

"Object-Relational Impedance Mismatch"[1] is been around for decades. Martin Fowler[2] and others who are deeply technical (think Sr. architects, CTOs) use this all the time.

[1] https://en.wikipedia.org/wiki/Object%E2%80%93relational_impe... [2] https://youtu.be/qI_g07C_Q5I?t=98

> You have just one system ( Kafka + KSqlDB). Note that we are ignoring Zookeeper.

That's not really something you can ignore, unless someone else manages it for you. (And then, of course, you need to take cost into account.)

Really, the model is completely ignoring the cost (time or money) of operating the storage layer.

It's an interesting idea, but without taking the full picture into account, I don't think it's very useful.

Totally agree. Re: Zookeeper, Kafka are apparently going towards removing Zookeeper. The goal is to give you a calculator to measure complexity. You can totally add it to your calculation.

You can certainly overlay cost to the calculator by adding an additional column. Cost should not be just the storage layer cost, it should include HR cost, licensing cost, overhead cost, basically TCO for each system in the data layer.

But I think generally by simply listing all your systems, you'll already kind of know how complex or simple your system is.

I think you may be getting too bogged down in the details of the example. It was a purposefully simplified example to show how to calculate the IMS score for different setups.

The Impedance Mismatch Score (IMS) notion is worth exploring. Novel, right? From the hip, I don't recall a similar complexity measurement applied this way. Neat.

I like the potential that IMS potentially could be applied to more than ORMs.

Someone should probably link to Ted Neward's (?) Vietnam of Computer Science.

Happily, there's a resolution to this paradox. A time before ORMs, without an impedance mismatch. The key insight is realizing that client/server is the root cause of the imperative code-to-RDBMS impedance mismatch.

The root causes of other mismatches are also caused by suboptimal mental mentals (metaphors).

Teaser, since I'm using a pseudonym account. Maybe some day I'll get my shit together, polish and promote my FOSS projects.

It was interesting until it became a full time ad for Redis.

It's just a tool to calculate complexity. You don't need to use Redis or Kafka or any thing used in the article. If Kafka or CosmosDB or something else works better, that's great too.

How is adding features a reduction in impedance mismatch just because they are from the same vendor? For example: Redis timeseries is a separate setup - how does it reduce complexity?

Edit: fixed typo

Good question. Redis Timeseries is just a different data structure and works alongside other data structures so it wont add too much Impedance when compared to having different data in different systems. On the other hand, imagine you used Redis for just cache and used TimescaleDB instead. TimescaleDB is based on PostgreSQL, so you need to use a different thought process, different SDK, different commands (SQL), different hosting, different devOps, etc. And for performance purposes, say stored some data in Redis for caching, now you have additional overhead because you need to now manage data in two systems. And let's say TimeScaleDB needs some other adopters to sync the data to Kafka, and Kafka needs Zookeeper, now all of a sudden you have 4-5 systems.

The goal of the Impedance impedance mismatch test is to identify such scenarios using a simple calculator and avoid them. In 2021, there are many single systems like Kafka, Redis, Cosmos, etc which can do multiple things way more efficiently. Many of these are OSS projects, so you wont get vendor lock-in.

I hope credit was given to Krazam YouTube channel for that hero image.

If you scroll to the end you need not merely hope.

Reply on news.ycombinator.com