Design patterns can definitely be useful, but only when used sparingly and if they really are the simplest solution to the problem. When applied just for the sake of "best practices" without really looking at the problem, all they do is create a new problem.
The other danger here is that especially the GoF patterns have often times succeeded their own usefulness with the introduction of closures, first-party functions, enums and option types to the languages. And some (like Singletons) are clearly harmful these days.
> The problem with design patterns is that it's far too easy to "apply existing solutions" to problems that don't exist.
If you use "something" (in this case design patterns) wrongly, it's not "something"'s fault. It's like saying there's something wrong with the car if you are drunk, speeding and crash it.
That's not a design patterns problem. At most, that's an abstraction layer problem/YAGNI.
> Design patterns can definitely be useful, but only when used sparingly and if they really are the simplest solution to the problem.
This statement reflects that the whole point of design patterns was completely missed. Design patterns are not pre-baked solutions. They are higher-level coding constructs that reflect specific concepts, and thus provide an ubiquitous language for developers to describe and discuss what they are doing without resorting to ad-hoc terms and definitions. Programming constructs, like callbacks, promises, futures, dependency injection, singletons, etc, don't lose their importance, usefulness or value if they are covered in a GoF-type book or if they are implemented by a library.
It's how some people tend to use them when they are not necessary or a simpler solution would have sufficed. This I believe was OP's point.
Yeah you are right, YAGNI is indeed not taken into account enough. However, blaming design patterns not only misses the whole point and shows a lack of understanding regarding what design patterns are and how/why they are used. In fact, I would go as far as claiming that complaining about design patterns is a symptom of limited technical knowledge and experience, and the bulk of the typical anti-design pattern cliches are just the defense mechanism kicking in within the spirit of the "fox and the grapes" fable.
What's the correct tool? The simplest tool that can get the job done, the Principle of Least Power[0]. We often hear "Favor Composition over Inheritence", but never "These are the instances where Composition does not work, and Inheritence needs to be used instead."[1] Rich Hickey's "Simple Made Easy" shows many more examples, but this principle is very wide reaching. Prefer expressions over code blocks. Prefer for over while. Prefer map over for. Prefer data over functions. Prefer functions over macros. Prefer monoliths over micro-services. Prefer unikernels over containers. Prefer smaller teams over bigger teams.
For the monolith vs micro-service, it's better to have a separate micro-service than a spaghetti monolith, but if you can un-spaghetti your monolith it will be simpler. It might not be easier. It might not be possible for your team based on the difficulty of the task. Likewise for team size, smaller is better, but only if that team size has all of the required skills. If they do not, grow the team or train your team.
The left side is simpler, and therefore composes better. If you need more power, by all means use it. If you are designing a system, always make the left easy, and the right ever so slightly annoying. Nudge people into the Pit of Success[2].
[0]: https://blog.codinghorror.com/the-principle-of-least-power/
[1]: I believe the answer is that composition always works if you have access to all of the code, and inheritance is a back-up if you are using a library you can't change. I might have a
[2]: https://blog.codinghorror.com/falling-into-the-pit-of-succes...