It's easy to say "scalability is underrated" if you've dealt with businesses built on hundreds of standalone PHP/Perl/Python/JS scripts with zero unifying architecture, where migrating a data schema is virtually impossible because of the state of technical anarchy.
Scaling is hard.
There was a brief moment after Oracle purchased Sun where I thought Oracle might try to engineer am Excel competitor on Open Office that had an easier migration to a real database (Oracle), but that dream died early.
Are there scaling issues? Version control issues? Absolutely! But again, that doesn’t mean that it’s not the best tool for the job.
It’s easy to mount the highest horse from a technical perspective, but as engineers it’s also our responsibility to be curious about what people are using voluntarily, and not just what we think they should be using.
A git integration in the editor, and a decent test runner. Some API hooks do Jetbrains can do something with it maybe.
The whole structure is called Open Packaging Conventions and it is implemented in `System.IO.Packaging`.
I think I was hit by everything. Easy stuff at first: Crlf issues, XML Apis, weird rpc apis.
Then, halfway through the project, the results had to change. Not only the order, datatype and headers (I actually overengineered the first version so those were configurable), but the format, duplicate on multiple columns (and empty fields counted as duplicate...). Worst job I've ever done. I'm also disappointed in myself tbh.
But now I'm a bit of an expert on excel format issue and limitations, and that already helped me.
IIRC he just optimized their sheets and set up a CRM for the things that had no business being stored in an Excel file and that was already very helpful.
Attempts at writing actual software to deal with the problem would have failed and everyone was acutely aware of that.
They eventually replaced me with a handful of their relatives, or at least it seemed. It was a lot of fun watching how many LOC one could "write" to request some trivial data from an endpoint. Luckily I was only golfing the latter half of that tenure so I have no regrets.
But the question is, are you trying to make your life miserable by scaling before exhausting other options?
Most applications can be optimised for performance by orders of magnitude, much easier than trying to scale them by orders of magnitude. Any problem is much easier to solve and faster to deliver when you can fit it on a single server.
Usually it is simple thinking that goes wrong. System is slow - add more hardware. But then it turns out developers did bad job and you could still run it from single small server only that someone would have to write code understanding big O notation.
Main point of big O notation is that there are things that implemented incorrectly will be slow regardless of how much hardware you throw at it.
I have a Java codebase that calls more constructors inside their constructor. So you have this massive god object that instantiates the whole project in side the constructor. If you want to run parts of it in separate threads you can't just take it apart. You first have to rewrite all the constructors.
“… Because the problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. “ —Joe Armstrong, creator of Erlang progamming language
I don't think this is a problem with object oriented languages. You could certainly do the same thing in any programming language. All you have to remember is to keep your constructors as simple as possible. Passing in dependencies through the constructor is often an easy solution so I don't get the hate for dependency injection frameworks. The original XML driven iteration of Spring was pointlessly overcomplicating it but nowadays you can just define your "beans" in code which basically means the DI Framework just helps you set up a dependency graph and nothing more.
The Silicon Valley world would do well to learn some things from finance. You’d never write a horizontally scaled order matching engine.
It wasn't successful....
Exactly. The author forgot the put the word "premature" before the word scalability.
That's why you use something like a Rails monolith....but oh wait, Ruby doesn't scale well!
Look at Github, for instance.
If you're doing any computation or highly concurrent workloads then you will discover the performance issues with Ruby well before you outgrow your persistence layer.
It took more CPU than if we'd rewritten the core of it in something else, but it let us iterate the rendering engine itself much faster, and most of the expensive computations were done by extensions in C anyway (e.g. GDAL and the like).
Of course you can find areas where it's still not viable, but my experience is that if you start by prototyping in whichever high level language - no matter how slow - that is most productive for you, you'll inevitably find you need to rewrite far less than you might think to get it fast enough. But more importantly: The parts you end up rewriting will very often be entirely different parts than what you expected, because being able to iterate your architecture quickly tends to make you end up in a very different place.
With a kafka queue, and worker nodes, compute-heavy jobs are easily distributed across many worker nodes.
For many parallel requests, you load-balance.
If the PostgreSQL or mysql table is the bottleneck (persistence layer), well this is actually a design decision thats orthogonal to the programming language (PostgreSQL won't scale better with a JDBC ORM).