At that point, you lose the feeling that there are magic things out there that you will never understand and that for the most part everything is just regular old code that regular people wrote.
I don't really remember when I felt that external dependencies were magic, but thinking about this, it explains a lot of the behavior I see on some developers who are very negative about the more challenging parts of the job.
Some of them don't really believe the research stuff we do at work are even possible. They're constantly surprised when other devs finish those tasks. Some don't believe that other devs can code in C++ or Rust, or write parsers, database modules, implement IQueryable in C#, or develop novel algorithms for novel applications.
To them, if a package exists it must just work, and that package comes from another breed of developer that can't coexist with them. I see a similar thinking with AI: now with ChatGPT and GPT-4, there's a hubbub about there being "no reason for our AI team to exist anymore".
I'm not a big fan of working with those developers.
I agree. And it ties into something I often see that puts me on edge: programmers not taking responsibility for the code they put into their projects.
What I mean is that when you incorporate any code, from any source (library, framework, copypaste, etc), then you are responsible for that code and its proper behavior as much as for the code you actually wrote. So you're well-advised to understand it.
That's one of the reasons why I won't include code that I don't have the source code to. I need to understand it and be able to fix it.
The "out of sight, out of mind" approach doesn't really work for code you're actually responsible for.
To me the issue is they assuming everyone around them is like this. I'm totally fine with lack of experience or knowledge, but a co-worker constantly underestimating their peers is not alright.
Where it really stings is when I do something of my own initiative (like, for example, create a transparent API over memoizing and caching some expensive calculation, or refactor some of our common client customizations into their own set of classes so it’s easier to extend) and he ignores it or scoffs at it.
Only to then find a third party library that implements something similarly. THEN it’s presented to me as a “brilliant idea” that we can take advantage of.
At that point, when I say “we” already do this, he usually rephrases what his brilliant 3rd party library does, as though he can’t fathom that I would be capable of doing something like that myself, and clearly I’m just not understanding what he’s telling me.
I think it’s a defensive mechanism for his ego against one of his peers or employees being more capable than he is.
But it’s also not like he can’t learn this stuff himself, he just doesn’t ever put in the time or effort.
For, say, security implementations for authorization and access controls, or even low level HTTP request routing? Absolutely. The goal there is to adhere to something standard and battle tested by experts, and the third party libraries tend to be fewer in number, and of higher quality, with longer term support and clearly defined upgrade paths.
But that’s the lower level stuff, where your special custom needs are superseded by the primary goal of just “doing the one right thing”, or “adhere to the commonly agreed upon standard”.
For all the other things that make an app unique - things like CSS frameworks, UI components (beyond basic, accessibility-minded building blocks), chart drawing, report generation and caching - my experience has taught me otherwise, the hard way.
Being stuck using a 3rd party library that doesn’t do what the client or business needs it to do, having to juggle our own internal patches and bug fixes with updates to the library itself, all only to have the library abandoned or deprecated in favor of the author’s next pet project, really sucks and often comes with a high opportunity cost and a high development cost.
I now consider third party implementations of higher level features (and especially anything front-end) to be something that needs to be evaluated as equally costly as an internal implementation by default, and not favored just because somebody else wrote it.
Maybe I’ve just been unlucky in my experience, though. I also suspect ecosystem makes a difference. The PHP and JS ecosystems are full of poor libraries with snake oil sales pitches. I suspect this is different with, say, Rust.
I’ve mostly worked with Python and JVM languages, which probably explains why I’m less passionate about the counter-argument than you are. Ecosystem definitely matters a lot. VanillaJS is the only good JS framework IMO.
So the go to for every solution is to find a third party library from a “real” witch or wizard, and follow its basic tutorial. Maybe try to customize it a bit at most.
If something breaks, just start randomly moving things around or copying and pasting more code from forums until it works.
I can’t live like that. I need to know why something’s not working, AND why it IS working. I like stepping through my code with a debugger just to make sure things look right, even when they’re working.
I think the craziest part, though, is just how much people with this “software is magic” mindset can actually get just by brute force cobbling things together.
That mediocre dev is a straight shooter with middle management written all over them…
It's a matter of preference but IMO/IME it works better when a manager is actually good at it.
A manager that is supposed to be technical but only wants/knows how to do the management part, and can't make more than the basic stuff won't really fly. I've seen a couple of those getting fired during probation period.
On the other hand, the "bad developer" that thinks packages are magical will eventually settle as an expert beginner in a low-expectation environment. Which is fine.
If you highlight a method you call from external code and hit CTRL + F12, Visual Studio will automatically decompile it for you.
Dotpeek is integrated into Rider and is a world class decompiler. It also integrates into Visual Studio either standalone, or with ReSharper.
You can also integrate external source symbol servers into your IDE of choice as well that will let you debug into libraries seamlessly.
In the case of non-compiled languages, of course you don't even need this step since all your libs exist in source form already, so it was pretty simple for me to step through Ruby library code with a simple debugger and no fancy IDE.
I have a habit of sometimes debugging even horribly abstract framework (e.g. Spring) code when I don't understand what it's doing. That's maybe not the most efficient method, but it does usually make me understand why thing X is not working the way I expected it to work.
I think the real skill then is to learn to navigate big codebases in few days time than taking a few weeks and then feel dejected by the time spent and still unsure.
I often feel ambitious for such endeavors but navigating big codebases take time, any tips?
More importantly is to use an IDE with a good fast global search function and get comfortable with it. At least for me, 99% of navigating a large codebase is global search.
You can goto or peek definition in VSCode. And the find tool has the option to find all occurrences throughout your project.
Imagine buying a hammer, that hammer not working, the hammer's design being so complicated that it's impossible to understand or mend, and then having to design and build your own hammer, and then putting up with that situation over and over again and accepting that as the status quo. That would be the correct analogy.
We have do have tools like the hammer - there is one design, everyone more or less agrees on it. There is still high quality and low quality, but it has one job. We have tools like a bulldozer - complex, numerous parts, requires constant maintenance, closed source.
As the parent said - it is not uncommon to have to maintain old equipment, as well as design new tools as new requirements pop up.
Sure, our rust is a little bit different - time wears on software in a different way. Use wears on software differently. (Changing product requirements leading to a new tool is probably common.)
The maintenance may be trickier - but I'm sure changing components on a tool when a certain component is no longer available is not easy, thats where shim layer comes from!
A mine site usually fixes its own tools used to do their job, they usually fixed things they've purchased to use that are broken or become broken, is that not the same?
Very different situation
I've also found that the software industry obsesses over tools more than any other industry
On a mine site, the job is to mine and process minerals. That includes engineering a mine using products that eventually fail over time and need fixing. As part of that process an automotive electrician may need to fix the electronics on a vehicle that have become faulty or damaged.
Mechanics and engineers need to inspect, rebuild, and fix the tools they are given to do their job. The difference is that yours may not work up front, whereas these tools need maintenance over time.
The difference between most professions and programming is that we have open source. The equivalent of that in the real world is getting blueprints to build our tools and then building them ourselves, with no guarantees.
This was and is quite common in any blue-collar field where works don't always have the money or time for a brand new jawn every time something goes wrong.
I love estate/yard/garage/barn sales and going through the tools and reading the tales they tell from their wear patterns, field repairs, revisions, and hacks from their owner.
Case in point, here's a tool which has gotten multiple leases on life through modifications. https://youtube.com/watch?v=oTA513ttrbQ
For example, I have a graphics project where screen tearing suddenly started appearing although my code didn't change, and the tearing wasn't there before. Is the issue in Skia, OpenGL, the graphics driver, the Intel or NVidia GPU, or the OS? Or is it some latent issue in my code that started showing up because of a change in these dependencies? Or is it some other complex interaction between multiple dependencies? I couldn't possibly know, and I actually don't think there is anybody that actually knows. And there is zero chance I could ever figure it out. I mean, it at least appears it was a driver issue as after some updates and a reboot it just went away, but there is zero insight into why or what actually made it go away.
If I modify a plow because some part as originally designed was flaky and constantly broke, you usually know to a pretty good degree why your fix works.
In software, the abstractions are such that it is practically impossible atbtimes to understand.
Sure, welding back on a hardpoint that broke off a plow is concrete and obvious (maybe that's analogous to fixing a broken dependency that got renamed), but there are other fixes that definitely fall under "I don't know why it works but it works". I've had electrical gremlins and fixed them by grounding something that looked like it ought to be grounded, and was grounded when I tested for continuity, but nonetheless was acting floaty. It was probably a loose connection elsewhere in the system, but I didn't know that for sure, nor do I even know that it was the problem in the first place.
Software definitely reaches insane depths of complexity, but again, if you can dig down and understand the abstraction well enough to attempt a fix, and the problem goes away, isn't that good enough? The real difference is mechanical systems typically only have a few layers of abstraction, while software has dozens to thousands of layers of abstraction.
For React devs, this means learning how React actually works under the hood.
The other thing I find is that big libraries are either mostly dependency bloat (as implied above) or dealing with a hard domain problem. If it's the latter, what you're really struggling with is not the library, but the domain it's trying to represent.
But this doesn't change anything of the programmers problem. If I stumble on a bug in a physics libary I am using for a game, I cannot just jump right into there and go fixing things. I mean I can start doing it, but at the cost of not getting anything else done for quite some time.
There are lots of hard domains in programming. Cryptography is hard. Networking is hard. Fast rendering is hard. Efficient DBs are hard. OS are hard. Drivers are hard.
You can maybe fix a trivial error in such libaries, but everything else is usually a (big) project on its own.
Additionally, once you recognize that you can also recognize if you’re dealing with incidental complexity (e.g. poorly thought out/designed code) and inherent complexity (e.g. physics calculations). The former can be fixed, the latter cannot. Knowing the difference saves much pain.