Hacker News Reader

This whole comment reads so weird. Why do you or anyone else care how complicated someone else’s architecture is? Why is it only “OK” if it’s an educational exercise?

Are you going to be maintaining it? Are you going to even be using it? Why does it matter to you what someone else did in their spare time for their own amusement? Or even to make a commercial product out of it? Either way, it’s not your architecture to backseat architect, let people do what they want.

I think I already elaborated on why? A bunch of people take information they come across on the internet and think that's the only way of doing things, so if someone comes across the post, looks at the architecture they might think building it with so many components is a required thing, not just optional for really big projects.

I'm personally dealing with infrastructure at loads of different places, and the amount of engineers who over-complicate their infrastructure pains me, they all go for a distributed architecture before the products they are building even have any users, because they've read bunch of blog posts about building it that way.

It sucks to spend so much time and effort on maintaining and improving something when it's not really needed in the first place. Like using a Ferrari to deliver food in a traffic-heavy city, instead of just having a moped.

So basically, I want to save myself and others time.

Gotta love gate keeping architecture in the interest of “saving others time”.

If someone takes one look at one project and decides to copy it’s architecture word for word, then it’s on them. Not on the person who built an open source project, for whatever reason, and put it out there for the world.

It’s not your architecture to architect. If you want to build your own service to show best practices, then go for it. You weren’t there to make decisions, you don’t know of the reasoning behind them, and it seems ludicrous to backseat architect with context, let alone without any amount of it.

I don't know what your expertise is exactly, but if someone new in your subject area intentionally made things more complicated than necessary for the scope of the project and publish it as a public resource for others to take inspiration from and learn from, wouldn't you at least have some sort reaction against it?

See it as wanting to advance my own trade by providing opinion on the current "everything should be a serverless function" trends in the hope of hopefully be able to move the trade into a different direction.

> I don't know what your expertise is exactly, but if someone new in your subject area intentionally made things more complicated than necessary for the scope of the project and publish it as a public resource for others to take inspiration from and learn from, wouldn't you at least have some sort reaction against it?

If I have no affiliation, or plan to affiliate with a project, why would I give a shit about what they do? It’s their project to figure out.

Did they ask you for an architecture review? No. Then don’t backseat architect. It’s as simple as that. Especially when your critique is “you’re doing it all wrong and you should feel bad if it’s not an educational exercise”.

> If I have no affiliation, or plan to affiliate with a project, why would I give a shit about what they do?

We critique whatever gets shared on HN regardless of our affiliation with it.

In fact, why was this shared here, if not to ask for discussion and opinions?

Do you have any affiliation to me or plan to affiliate with me or my projects? Then why you give a shit about what I write?

I'm assuming we two have at least one thing in common, we're here on HN to discuss things we come across, both submission articles and comments, since we're having a conversation right now. That's how this works.

So I came across something that I have feeling and opinion about, and that led me to share my opinion about it, as I care about my trade. You're welcome to ignore things even if you have opinions about it, but to request others to do the same in a forum that is meant for discussion, feels a bit unproductive.

What exactly do you think is the purpose of a discussion forum like this one?

The OP states he wanted it to scale and cost-wise using AWS on-demand resources is much cheaper than running multiple EC2 instances which are over-provisioned to begin with. AWS functions and EventBridge is the same thing as CRON and a monolith. LOL why waste a full VM when you could pay pennies on the same system. Obviously vendor lock-in a bit, but who cares! lol ridiculous.

[EDIT]: Removed comment about user.

From the guidelines: https://news.ycombinator.com/newsguidelines.html

When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."

I am sorry. Fixed

It may be helpful to assume good faith. I think the GP is making the point that if this guy who is "a student" chose that diagram as his architecture for a hobby / learning project, then it's worrying that somewhere along the way we may have inadvertently created the expectation on newbies that this is The Way™

Said differently, it's worrying that people decide to overengineer architectures like that before there's any real need. It's a sysops version of the early optimization problem.

Ten years ago, the student would have probably built something much simpler with a couple cron jobs on a simple EC2 instance, maybe celery/rabbitmq as a message queue if he was feeling fancy and called it a day

I mean, they are very plainly not concerned about the OP or “state of architecting” but that this successful project coming from a student is going to inspire other students to copy the architecture shown here.

Explicitly,

> A bunch of people take information they come across on the internet and think that's the only way of doing things, so if someone comes across the post, looks at the architecture they might think building it with so many components is a required thing, not just optional for really big projects.

> So basically, I want to save myself and others time.

This seems like not only ridiculous critique, but also just plainly not the author’s problem what others choose to do with his open source project.

> Said differently, it's worrying that people decide to overengineer architectures like that before there's any real need. It's a sysops version of the early optimization problem

And why exactly is it yours or other commenter’s problem what people choose to do with their own time? How does it in any way shape or form impact you what this other person chose to do with their project that you will never maintain, contribute to, or likely even use?

Again, I think you're focused on this one instance and the GP was saying "if this guy chose this approach, countless more people are making similar decisions", which as a trend in the long run could make it bad "for everyone" (himself and others)

You may or may not disagree, but it's arguably a worthwhile comment. Possibly more so than your comment on the merits of someone else's comment... which just derails the conversation into this meta-discussion

> And why exactly is it yours or other commenter’s problem what people choose to do with their own time? How does it in any way shape or form impact you what this other person chose to do with their project that you will never maintain, contribute to, or likely even use?

Critiquing technology referenced in submitted posts is what we do for fun around here...

I am replying to a person’s comment so, yes, of course I’m “focused on this one instance” and replying to what was said. That is, literally, the purpose of replies.

I think the critique was not constructive, was dismissive about an entire project, backseat architected, and did all that with absolutely no context on the requirements, limitations, and decisions made.

In short, it’s a shit critique and I called it as such.

Any student’s production application can assumed to be educational. I think this is even true for a mid level engineer.

His comment was valuable and fostered discussion.

He's allowed an opinion especially one that's valid.

It was constructive and explained how he would want to see it structured.

He did it the right way.

Where as your comment is unconstructive. Doesn't foster discussion.

If the only point you're trying to make is to ask someone to stay quiet why not leave it to the mods or even silently downvote and move on?

He completely dismissed the entirety of their architecture as too bloated without any sort of context on the decisions that brought it here.

There was no constructive criticism, no “you should do this differently” it was “this is worryingly large and you should feel bad if this isn’t an educational project”.

If this is “the right way” then I don’t want to be right, because it’s a shit way that lacks empathy or tact.

> dismissed the entirety of their architecture

No he didn't. He had a caveat.

> no “you should do this differently”

That's not the case..he clearly says this is how I would do this differently.

None of this matters. Arguing about arguing is futile.

“Unless this is an educational project” is not an appropriate caveat. Especially about a project that was clearly not submitted as an educational one.

> Background

> This project started from two points: as part of my PhD, I had to become very familiar with working with NOAA forecast results (https://orcid.org/0000-0003-4725-3251). Separately, an old tablet set up as a "Magic Mirror,” and was using a weather module that relied on the Dark Sky API, as well as my Home Assistant setup. So when I heard that it was shutting down, I thought, "I wonder if I could do this.” Plus, I love learning new things (http://alexanderrey.ca/), and I had been looking for a project to learn Python on, so this seemed like the perfect opportunity! Spoiler alert, but it was way more difficult than I thought, but learned a lot throughout the process, and I think the end result turned out really well!

Seems perfectly fine to assume that this might be purely an exercise in order to learn as much as possible, rather than to build something that is trying to be as cost-effective and performant as possible.

If someone posts their architecture diagram on Hacker News, it's fair game for feedback? Anyone is welcome to ignore said feedback.

As the adage goes: Suppose you have a problem, and you say I know, I'll use microservices! Now you have N problems.

I kinda agree but it’s also not that complicated. It’s a critique of a craft many of us share. It’s natural for an engineer to have an opinion on the design and share his opinion with others. He’s also not being mean about it or anything.

I would venture as far as saying nearly every personal project built to scale is over engineered.

Yeah, 13 years ago I wrote a small signup app for tracking pick-up hockey skates I run regularly.

It's a pretty cool system, you have your roster of players and setup skates, and it automatically emails everyone x number of days before the skate. You can see who signs up, pay online, etc and if a goalie who's signed up cancels, the 'Skate Operator' is emailed immediately to take care of that (it's a big deal to not have two goalies). It also has an automatic waitlist, and if someone cancels before the skate, and you're the first person on the waitlist, it moves you to the end of the list and emails you to let them know you're in for the skate. It also sends out a reminder X hours before the skate, day of, etc.

After I started using it for my own skates, a few rink managers contacted me to discuss using it for their rink. I met with 2 different groups / municipalities (because most rinks are owned by the city they're in), and it never went any farther than those meetings. They didn't want to use a system by some guy that couldn't maintain it, wouldn't keep it going for more than a year or 3.

13 years later and I'm writing a new version and have run something like 11k skates in that time. There is one small, privately owned rink near my house, has used it the entire time without issue. I've got a few other friends/individuals that use it, too.

It is possible for a single person to create and maintain something purely because they need it.

How is it hosted?

AWS virtual machine (I have an old school windows service processing the emails; the rewrite will use a new approach).

This isn't a complicated architecture. It looks like a pretty small and simple cloud deployment. The EC2 looks a bit out of place, but I'm assuming there were technical reasons it couldn't be a lambda instead. Apart from EC2, all of the compute is consumption based so you're just paying for actual utilization. This is a far more efficient approach to building in the cloud than just throwing a monolith on a VM and calling it a day.

Agreed. Guy above doesn't know how to save money or use resources efficiently in the cloud...

I've built lots of things like this working at AWS. I probably would have used ECS instead of Fargate, S3 and S3 events instead of EFS, and CW Events instead of EventBridge for simplicity sake, but the "gist" of what he is trying to do is pretty sane and this is what a real-world combination of building blocks would look like. If it were me I'd also try to find a way to modify WGRIB2 to run entirely in Lambda and get the costs down.

Most architecture look complicated when include everything. He includes the load balancer. It would look simpler as VM but then all the complexity is the things running on VM.

Also, much of the complexity is from pulling from data sources. Lots of apps are just the web site and database, but weather app needs the complexity. I bet EventBridge has some important purpose.

Not sure what you think about this, but I'd wager that this fellow posted his detailed architecture diagram for transparency and maybe even to solicit some constructive criticism or feedback.

It sounds like you'd be well positioned to offer some feedback, but I'd recommend against that without your practicing some empathy for the stage he's in with his education.

I don't know if a person pursuing a PhD is a student

They're "graduate students", usually. Still have a student status at most universities.

I'm guessing they are studying? That would make them a student, as far as I know. But English is not my first language, maybe there is multiple meanings of student and not all of them fit someone who is "pursuing a PhD".

> To me it’s a bit worrying that someone who is still is a student, builds such a complicated and hard to maintain architecture

To me, it is kind of expected. “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away,” sure, but perfection – or even distantly approaching it – is the product of expertise and practice.

This reads as someone who has never designed a cloud architecture. If he wanted to use a VM he might as well use a VPS. The whole point seems to utilize the resources AWS provides... not some basic VM monolith. That's child's play.

[EDIT]: He's a Phd for chrissakes!

Yes, use a VPS or dedicated instance, one machine is usually enough. This is exactly what I'm saying :)

And yes, if the whole point is to utilize AWS resources, then of course a architecture like that is fine, as it's a learning exercise (to learn how to utilize resources). My comment includes that as well.

As far as I can tell, the author is studying at a university, that makes them a student in my mind. Maybe they do other things than just studying, I didn't look into it very far, hence the "maybes" and other caveats in my initial comment.

Someone quoted note where he says using AWS for scalability and reliability. Would single VPS have survived being on Hacker News? Would he need to make new VPS to handle the load? I bet AWS scaled up without problems. Also important, it will scale down when spike is done and will only cost what was used.

AWS is more reliable, and from this architecture, probably doesn't have to do anything. If the VPS dies, then the site is down until can build new VM. Or have to run two of them at twice the cost. More since he has to setup database or shared storage.

That might be overkill for this site, but for some upfront work, he gets something that needs less maintenance. If going to learn, then should do it the right way. He may be scaling for the future, but it is tradeoff to design now instead of later.