Hacker News Reader

Fewer pieces in the technology stack is better, because each piece is operationally expensive (it is its own set of work in maintenance, upgrades, migrations, etc). There are also the mentioned costs (you can't atomically do things across divergent pieces of infrastructure, at least not very easily at all).

Fighting the tendency of everyone wanting to draw in each new interesting bit of technology is important.

Every piece of complexity you incur should be proven as required, and you should build things to just somewhat larger than near term projected scale... and not expect that you are a temporarily embarrassed unicorn with loads of unexpected demand showing up.

If you run out of performance, you have some headroom from tuning; from that point, you can then choose whether it makes sense to retrofit to a higher scale technology or throw larger hardware at the problem (or both).

These days it's pretty trivial to have a cloud managed component e.g. Redis that is maintained, upgraded and supported.

And then you have a component that is designed for the job instead of trying to use a database as a poor man's queue.

> These days it's pretty trivial to have a cloud managed component e.g. Redis that is maintained, upgraded and supported.

It's still another moving part. Thing should be as simple as they can be, but no simpler.

> trying to use a database as a poor man's queue.

Of course, it's not really a "poor man's" queue-- it's got some superior capabilities. It just loses on top-end performance. (Of course, using those capabilities is dangerous, because it creates some degree of lock-in, so go into it open-eyed).

For as much as you accuse others of looking down their nose / not willing to seriously consider other technologies... you seem to be inclined that way yourself.

Redis is great. But if you have Postgres already, and modest to moderate queuing requirements, why add another piece to your stack? Postgres-by-default is not a bad technology sourcing strategy.

It is very much a bad technology sourcing strategy, and you will kill your business if you attempt it. Period.

It comes from a place of ignorance, and you're promoting ignorance. Learn why technologies exist and make an informed decision about tradeoffs, instead of being lazy and incompetent by blindly choosing technology based on what's a very locally maxima for you personally. It's selfish and damaging.

Edit since I'm rate limited: The folks landing on Postgres are not landing there after due consideration, they're landing on a technology they're familiar with because they don't know how to learn.

It is lazy, ignorant and selfish, and while of course people don't like having truth spoken to them, that's what it is; truth.

Edit 2: Making these poor technology choices will kill your business because you won't have the agility that using a specific message broker technology gives you. You won't have built-in solutions to common queueing problems, you won't have dedicated specific logging/metrics to monitor, you won't have the libraries in your preferred language to directly tackle your problem, you won't have the support community available to you (it will be much smaller), you won't be able to pivot onto related patterns as your needs change, your scaling will always be more complicated because fewer people are doing it and the tool you use isn't tailored for your use case; the list goes on. Your competitors will swallow you because they move faster than you do, and your business will die.

This is annoying to me because I've been in situations where I've had to maintain and write features against technology that was a poor fit for its use, but people like those in this comment sections bitched and moaned about a "new thing" existing in our stack. The reality was they didn't want to learn anything, and were fine pushing the hard work off onto the developers, so they could safely continue to do as little as they possibly could get away with.

Do your job, learn technology that actually fits your use case, and stop trying to push work off onto other people.

Bruh, folks are making an informed decision - and landing on PostgreSQL. Their scenario/conclusion is just different than yours.

It's not lazy, blind or selfish as you claim - and frankly when you disagree with authority & condescended tone - it's a turn off.

Wow, you're super jaded on this, do you sell queuing software for a living? Also wrong: how will this kill a business, at all?

Downvote away; but really “kill” your business!!?

So much love for this; 'Every piece of complexity you incur should be proven as required', plus 'Fewer pieces in the technology stack is better, because each piece is operationally expensive'. Also, this totally ignores (as these, imho are enough in themselves) the actual practical differences between using a single thing and two - synchronizing state; most of the cases here guarantee atomicnes individually, but not as a pair - you (the engineer) have to deal with that - which is non-trivial.

It's not bad-engineering at all; how many engineers have to operate at that level in your stack, ever. Close to zero, as normally they just use an abstraction on top of it - Rails for instance is ActiveJob. No one has to care what's behind it, except folks running production - and for them it's a trade off; it's simpler - one less moving part, but also more load on one single part. Is that worth it, or not? Depends.

I can't stress this enough; claiming anything is "vastly superior", especially with no examples, isn't useful - things aren't this black and white in reality. Most tech-choices like this are somewhere between trade-offs, preferences, ignorance or wrongly held opinions about something being vastly superior > something else.

Sorry, but this isn't one of those cases. A RDBMS is objectively inferior for queueing compared to dedicated message broker technologies. This is blatantly obvious if you've ever used both for queueing.

You're applying generic heuristics against a problem where folks have specific domain knowledge that contradicts those heuristics.

I cannot stress this enough; you are flat wrong if you think Postgres is appropriate to use for queueing.

An array can be appropriate for queuing for some things. You're wrong - as you are trying to state "a problem" has a specific solution, yet you don't know the problem. Yes, there are situations when using a dedicated message broker would be objectively better; there are many when that's totally wrong.

Nothing is likely be correct about your arguments if you don't know what you are trying to fix or the sitation, which should also be blatantly obvious. You're applying some unstated subjective situation in your head, then dumping out something you've heard / done before. LOL.

Also "if you've ever used both" - I have, which is why I know. It's gray; sometimes you should, sometimes you shouldn't. You're assuming I haven't as I have differing point of view to you.

No, I'm assuming you haven't otherwise you'd know that redis AND postgres are inferior message brokers, both of them, and the fact that you haven't corrected me means you're not well versed at all in this problem domain.

LOL; what problem domain? We've not talked about any specific situation - which is why you're wrong; you assume far too much. I was attempting to enlighten you a little, but after a few redeliveries, it seems it's gone to DLQ.

I believe you are missing the point a bit here. Postgres is not the optimal tool for queueing. No one is arguing that position. However, it's also true that for a very large number of use cases, it is entirely sufficient.

If you already have tooling and knowledge for managing Postgres, and Postgres is sufficient for your queueing needs, then why would you incur the cost of introducing a new service? Most situations don't require the best tool for the job, and in that case, you are well served to use a tool you already have.

If you are using an abstraction layer over your queues (you almost certainly are) then you should be able to easily change to a different queue implementation later, should the need arise.

This ^ x 100.

I'm not missing the point, I'm discounting the point, because the point that adding technology is additional complexity, and complexity is bad is a nonsense argument that actually stems from a fear of new things.

This has nothing to do with getting shit done, and everything to do with people who don't want to learn new things because they're afraid of not being smart enough to make the new thing work.

Every different technology added to an infrastructure adds potential failure points, raises the DevOps burden, and adds surface area subject to bugs and security attacks.

If a PG-based solution meets the needs then this reduces complexity in the large. And this has nothing to do with learning or not learning.

Sorry but your argument just does not hold up.

It only adds the burden to DevOps who can't figure out how to monitor/maintain a system, which is not rocket science, and if you're a DevOps person who doesn't know how to handle such a mainstream technology as, say, ZeroMQ or RabbitMQ or even Redis, then maybe you shouldn't be in the position you're in.

The only argument that doesn't hold up is "use one tool for everything". This ideology is for lazy people who want to anchor themselves in a period of time, and refuse to learn new things.

Nobody is saying use one tool for everything. In fact the argument is why not use PG until you grow out of it (most people won’t).

Agree on devops. Of course.

There’s still the security surface area argument and the overall complexity argument—complexity of the system of systems is NOT reduced by adding technologies.

By way of analogy, using military aircraft, take the F35 vs. the A-10. The F35 is insanely more complex than the nearly indestructible A10 in large part due to the massive number of subsystems, software, sensors, etc. Sure the F35 way more capable (this is NOT a head to head argument about the military capabilities or said aircraft) than the A10, but it also has orders of magnitude more failure modes. More parts. More complexity. More opportunities for failure. This is engineering fact.

People are literally saying to default to using PG for "everything". That's the attitude I'm arguing against.

What's frustrating about this conversation is that I'm literally, right now, supporting two different queueing systems based on PG and Redis, so I get on a very real level, the tradeoffs. I know in great detail the problems that come up, but HN is not conducive to talking at that level of detail. At this point every comment I make is flagged and downvoted, so why would I pay time into a system that has clearly decided my opinion isn't relevant?

I mean i think it’s mostly the blunt, binary right/wrong nature of your posts getting you this reaction. For most folks they’ve moved past that and almost all of us have non perfect world systems where we’ve made trade offs. It’s just not my (and I guess a lit of folks) reality to be so black/white on this.

Of course there are trade offs, of course we live in imperfect situations where we have to deal with shit that isn't ideal, but sitting around pretending like everything is alright is just lying to ourselves.

You're stuck in PG because your DevOps team is a bunch of incompetent dolts, so yeah, make the best out of the bad situation and go ahead and use something like Que.

Just don't develop Stockholm Syndrome while doing it.

People in this thread are protecting their own egos by expressing how great PG is as a message broker, but it's harmful to the industry to let that kind of attitude perpetuate.

But what about "box" products? We can't force clients have dedicated DevOps just to support our product. Less is better in this case. To have same code in Cloud and offline installs is even better.

Except that PG and other RDMS don't offer "queue" commands in driver because the queue is built on top of other basic commands, when you use a system designed for that you can leverage the API offered by the driver directly.

So if my language doesn't have a library like Que ( no thanks using Ruby ) I can't do anything with PG whereas using Redis or any other solution you can use "subscribe" / "publish" ect ...

Earlier you said something about making excuses for learning and now, claim something subjective is 100% true and blatently obvious -- and that could block your own learning.

Like many things in this space there are a number of good ways - each with minor trade offs

Others posters have detailed those trades very clearly.

So now we can all be more educated about which one of the six good ways we'd choose.

> I cannot stress this enough; fear of learning is anathema to software, and trying to hide it behind a veneer of caution is not only disingenuous but potentially malicious as well, maximizing exclusively for the benefit of the individual against the interests of the group.

This is wrong.

You are ultimately getting paid to solve the company's problems. If those problems actually require deployment of a new technology to satisfy the requirements, then it is fine. Note that I said satisfy requirements, not exceed them.

But more often than not, * that's not the case *. If your queue requirements can be served with PostgreSQL, and you already have it, then why not? To do otherwise would be over engineering.

Because you may know a new and fancy technology that would be objectively better. Cool. Now, do you have enough know-how in the company to use it effectively? Do you know what the best practices are? Can you deploy, monitor, audit and patch this in production? If you have answered NO to any of those questions, you'll either hire people or you shouldn't deploy. Period.

And if you have only one person answering YES to the previous questions, you still shouldn't deploy it. Because one day that person will leave, and you are now left the company in a worse situation than it was before. For no reason other than satisfying your engineering itch.

THAT is anathema to working, reliable production software. Toys, you can do whatever.

If you want to introduce something new, you can. But you need to do it responsibly. It needs a reason to exist - a valid reason, not "it's better". How does it being "better" help the company? Will it reduce costs? Maintenance? Future development work will happen faster? Will it make a faster user experience? Scale to the projected company growth?

It will have a cost that will have to be accounted for – including opportunity costs.

> By trying to use one tool for everything, you lose out on all kinds of optimizations, features, and performance enhancements that are specific to the problem you're trying to solve.

You only care about this if you have a reason to care about this. Otherwise it's all irrelevant.

Sorry, but what you call a "toy", I call a highly reliable technology that's used in production widely across the majority of Fortune 500 companies. No one here is suggesting anything remotely like what you're strawmanning, but it's clear the straw man is the only argument you can take down.

The reason you are completely wrong and a toxic member of your team is that your logic can be used to justify all kinds of terrible, short term wins that sacrifice any kind of long term/sustainable design. You'd rather make a bad technology play so your boss gets off your back, than think through your decision and make a long term investment into a sustainable, maintainable, improvable, and agile technology that's actually suited to solve the real problem you have.

OF COURSE you're there to accomplish the tasks your business needs to succeed. That's a given. What's not a given is that you need to slap together the shittiest piece of software that'll do that job right then and there.

Your attitude is what keeps companies from solving problems, and forces them to pay dollar after dollar to fix the same issue over and over again. You're selfish, you're greedy, and you cost your company insane amounts of money.

You should not be employed in this industry if what you wrote here is how you think, period.

You gotta relax your tone eh. Not sure why you’re so personally/emotionally invested in this topic but your aggressive tone makes your myriad comments grating.

This isn't an aggressive tone, except to people who have their own emotional investment into the topic.

I put out what is given.

Would you please stop posting flamewar comments and crossing into personal attack? This breaks the site guidelines and is not ok here.

https://news.ycombinator.com/newsguidelines.html

I'm not doing anything like that, but please, pretend that I am, seems like your go-to move here anyway.

We cut you much more slack than we give to most accounts who break the site guidelines, so I'm not really feeling your complaints.

Would you please pick one account to post with? Using multiple accounts to work around moderation restrictions is obviously abusive.

You know moderation is at an IP level, we've had multiple discussions about it, and I'm clearly still getting hit with the rate limit, so when you say "to work around restrictions" it's obviously disingenuous on your part. Someone/multiple people are auto-downvoting everything I comment on with my other account, so I switched to see if it kept happening. Unsurprisingly, it didn't until I mentioned it again, and it started up immediately after.

As for the other comment you made (can't reply directly, rate limit), it isn't a low information post, I'm showing a screenshot of the message I get when I hit the rate limit.

You're literally gas lighting right now, it's insane.

You also know if you actually do anything else beyond rate limiting I'll just go dark, which is even "worse" for you, so this so-called "leniency" is nonsense. Just stop.

Edit: You can't use multiple accounts to get around rate limits, as you and I have discussed multiple times, so I'm not "doing" anything that constitutes a ban, but feel free to ban at your leisure. Over the years you've established yourself firmly as an unreasonable person, so there's no point trying to treat you like one.

I'm afraid I don't really follow what you've written here.

Rate limits are meaningless if people can just use multiple accounts to get around them. If you keep doing that, we're going to have to ban at least one of your accounts—but more likely all of them. After several years of trying to persuade you to use HN as intended, I'm beginning to lose patience.

> You should not be employed in this industry if what you wrote here is how you think, period.

That seems pretty personal. Not that I'm hopping on dang's bandwagon, but you can't post that sort of attack and say you're not doing anything like that.

I've wound up seeing a lot of your recent posts and I can understand where a lot of your responders are coming from... you are very strongly opinionated and have no problem crossing the line from "I hold this opinion" to "you're wrong for not agreeing and therefore you <insert consequence>".

THAT's the behavior that's getting you the pushback. Be more open to discussing things and comparing experiences and maybe you'll see more cordial response back.

It's not a personal attack, it's my opinion. I've worked with people who think that way, and I'd prefer if they didn't exist in the industry. How would you prefer I shape that comment so it's not crossing some magical line?

I am aware, generally, how things are. My complaining here is not confusion, it's frustration. I know I can change my behavior to elicit a better response, but I'm frustrated that it's relevant. People should have thicker skin, and when people dish it out (as what was going on here), I should be allowed to give it back. Dang's not commenting on anyone else's posts here, though there were many other rule violators.

It's an unfair application of the rules.

Edit: This post is 0 minutes old, how does it have a downvote already? This is the shit I'm talking about...