I often encounter situations where a day or two were spent creating a mess of classes and abstractions, yet the actual logic to solve the problem still hasn't materialised. A defense about "code quality" or "not wanting to write spaghetti code" is often made.
My recommendations for a problem you don't have a clear solution in mind for:
1) Hack some pseudo-code spaghetti together in a blank file until you think "you got it"
2) Write some theoretical test scenarios in another blank file (given state, steps, outcome) to verify that it's actually "solved". Discuss with a team member if possible.
3) then write out those functions from 1) inside the existing codebase
4) verify stuff is working
5) only then do abstractions / splitting up into multiple files
For a problem where the solution is already existing, but the implementation lacking (e.g. a refactoring)
1) design the new code (data structures, interfaces)
2) always propose to the team - no ninja-architecture/refactors
3) ensure the solution can be tested, document what should be tested
4) execute on the implementation
YMMV
To convert from a prototype to an end-product, you need to do the other steps.
Proper understanding of the problem is extremely important. Great advice.
My experience has taught me the exact opposite: no amount of Google Docs or whiteboarding meetings will tell you what the real problems are, you're only going to find them when you start implementing.
I got burned on my last project: we spent several weeks mostly talking and writing (English) about problems that turned out to be trivial, and got totally blindsided by problems that were not detectable until we actually connected to the firehose of incoming data (it's a streaming/analytical system) and then verified our work. There were lots of subtleties in the way a small portion of messages could be related over days and weeks. Nothing we would have been able to forsee with unit tests or manual inspection. We had to actually write the program to find out what would be hard about it.
When you hear them say: "I need a modern site", don't start until you're clear why they want it. For example "My customers told me my site doesn't look very trustworthy" will lead to a different solution than "My site is too slow" or "My site doesn't work on mobile devices".
Once you know the customer's why, you are also better able to make decisions like "It's okay to lose 1% of the data in case of failure in order to have a simpler system", or "We only look at the results once a day anyway, so using batching is acceptable".
But:
If you know multiple programming languages. Sometimes adjust for the toolset. Eg. Python or NodeJS for crawling websites. I always abstract it with an api. For eg. Try take a screenshot of a website in c# ( my most proficient language).
SOME tools do get you much faster to the end result with more options. Just make sure you tackle that problem with that programming language and try to abstract it away.
Edit: My solution for website as a screenshot, a c# wrapper for a nodejs api
The API in Nuget ( dot nets package manager): https://www.nuget.org/packages/WebsiteAsImageWebService.Api/
There is a lot to be gained by going along with the dominant groupthink on a topic, and there is a lot of risk involved in declining to do so. This is especially true as the groupthought picks up steam and gains powerful backers, who will be offended that little-old-you would dare to contradict someone of their high status.
The likelihood that your successes over the conventional/trendy method will be undeniably obvious is very low, especially when you consider that as larger interests and personalities invest in the groupthought, their PR machines will be working to discredit/counteract those who stand to imperil their investment. You will probably not get the punchy ending that you'd need to justify the burn you initiated by going your own way.
You could try to argue that the rest of the team is just trying to follow the zeitgeist instead of thinking about how to generate objective value, but this will hurt much more than just smiling and using the "Tech Flavor of the Week" on the project.
Team cohesion is more important for large-scale productivity than technical implementation details. It's easier to work around a buggy framework than it is to get people to cooperate and release their grudges.
If you want to be a "great programmer" in the sense that you have maximal self-satisfaction, then yeah, you should use whatever you think is best.
If you want to be a "great programmer" in the sense that you're a hot commodity on the market with a strong network full of people who want to give you money, you must recognize that your field of operation is not really computer code, but the delicate egos of the humans around you.
---
Along a less social track: the bigger the technology gets, the more likely that its kinks will get worked out and that it or something based on it will become a driving force in something that is objectively useful/valuable down the road, at which time having the investment in the platform makes it easier to adopt the better tech from both a technical (increased familiarity with tech-specific syntax and concepts) and personal perspective (don't have years of resistance that you've brainwashed into yourself, + accompanying reputation).
JavaScript is a good example of a technology that has been difficult to justify at a technical level in any case where it wasn't mandatory up until very recently. However, its widespread faddiness is now starting to cause actually useful new things to be JavaScript-first or JavaScript-based, even if they're doing that just so they can ride the hype wave themselves.