Learning (needlessly) hard technology | Hacker News Reader

Learning (needlessly) hard technology

johndcook.com

73 pointspeterkrieg54 comments

>> If something really is unnecessarily complex, better alternatives are likely to arise, perhaps suddenly. (This assumes people are free to choose alternatives, not prohibited by law, for example.)

Not being free to pick alternatives is a common case. There are a lot of dev teams out there that have "approved technology lists". What gets on this list is decided by management and/or a lead architect. It can often be the case that simpler alternatives can't get used because they aren't approved.

This isn't necessarily a bad thing. I could see project management saying "no" to simpler solutions in the name of preserving the stability of a project. Besides, a sufficiently talented team should be able to work within such resource constraints and come up with creative solutions anyways.

I've had to face the most horrible people with the most horrible requests.One of them being a request to "Send a payload with a Get request" . I spent nearly 2 months of my life convincing them this was not possible.I understand maintaining a high quality of code.But a major part of the reason for using archaic/available-for-the-past-10-years technology is 2 fold: 1.The HPBs do not want to/cannot learn anything new lest they need to one day code in those languages 2.They do not want to risk 'wasting time' of the sub-ordinates on learning something new which would benificial compared to actually doing the work needed.

Another third situation that I would agree with: Clients sometimes really do not like to trust new technology.An age old anecdote is always brought ; "Why fix what is working?"

Or, "No one ever got fired for picking Microsoft or IBM."

> One of them being a request to "Send a payload with a Get request" . I spent nearly 2 months of my life convincing them this was not possible

I assume you know it really is possible, right? (in URL parameters - with certain limitations about size if I remember correctly... or did just IE choke at 2048 bytes? I forget...) I am not arguing you should use GET instead of what should clearly be a POST, but saying it is not possible is IMHO misleading. And there are sometimes valid business reasons to use (invalid) technical solutions.

I understand your point, but neither extreme (always staying with legacy / always rebuilding) is particularly good.

Maybe one could try to split up the payload into mutiple GET-request packets, then assemble them server-side. >:‑)

don't forget all that storage space in the cookies too!

But the AJAX spec says that if the the request type is GET,the body/data is set to null[1] or am I reading it wrong.

[1]:http://www.w3.org/TR/XMLHttpRequest/#dom-xmlhttprequest-send

Depends on what you meant by "Payload" in the first comment. I assumed it to mean "a bunch of bytes", so you could have just used URL parameters, which don't have a standardised maximum length, but you can generally rely up to about 2,000[0] or so. If that wasn't enough, you could split up the requests like the other comments said[1], or go full hack-mode and use the cookies[2]. (using a Set-Cookie response header to clear them after the request was made)

Edit: Of course this isn't recommended, but I think the time wasted trying to convince the (seemingly terrible) manager could have been better spent.

[0] http://www.boutell.com/newfaq/misc/urllength.html

[1] https://news.ycombinator.com/item?id=10649995

[2] https://news.ycombinator.com/item?id=10650110

Do the servers really throw away the POST data on GET requests, like the RFC say they do?

Because that would be an easy place to send data.

And I agree with you. I was not even talking about rebuilding.Its more of.Like imagine if you were asked to make up http handling code from scratch in every project.If you were asked to ignore things like Jquery Ajax or Angular or any other such libraries which are designed to make your work easier and no matter how much you do you can never test the robustness of your private library compared to these. My Point essentially is: I do not want to reinvent the wheel every time.There are a large number of wheels in todays FOSS world.When I can concentrate on the things that matter like stream lining the business logic better,Why should is spend time trying to make framework level code work.

To summarize: You do not need to always rebuild or always stay with legacy code.You need to adapt and determine where would you spend more of your development effort.I cannot spend 80% of my time solving problems that the world has already solved

Agreed. :)

It's unethical to inflict "needlessly hard" technologies on a client who isn't using them already. But an enormous fraction of the world's valuable, productive software is both needlessly complex and built using (arguably) obsolete tools and frameworks.

The are good career reasons to think long and hard before specializing in this stuff. After all, once you go too far down the rabbit hole, you may not be able to climb back out, and nobody wants to be a 35-year-old developer with a completely obsolete skill set.

But if you're a 55-year-old consultant, you may be planning to retire long before the systems you're working on disappear. In this case, I can't see anything wrong with specializing in ugly, valuable dinosaurs that nobody else wants to get near. Under the right circumstances, I've heard of this work paying $350/hour and up.

> The are good career reasons to think long and hard before specializing in this stuff. After all, once you go too far down the rabbit hole, you may not be able to climb back out, and nobody wants to be a 35-year-old developer with a completely obsolete skill set.

Exactly

Also you spent time learning something completely unusable beyond a certain niche, that usually does not transport anywhere back to "sanity land"

I'm strongly opposed learning something that will be good for nothing beyond actually making the wheels of bureaucracy and/or legacy turn

I often find people are a lot more focused on the technology than on actually solving problems. As well as having a flashy POC for the sales team I'm all for having a dead simple but working shell of app with no polish.

What he's missing here is that this is a great strategy for any tech that will be hard to get rid of. Overly complex tech strongly integrated with businesses internal apps and procedures rarely disappear when something better comes along. Name a major company supplying enterprise software and there's probably better stuff. You don't see everyone flocking to it.

The concept is called lock-in. One can build a career strategy on it if done right. Just ask all the people working 9a-5p getting paid a premium to do SAP, legacy Microsoft, Oracle, COBOL on IBM mainframes, pipelines between legacy data sources/consumers, and so on. Some of this gets hit by outsourcing but many jobs remain. And one can always start an outsourcing consultancy with in-country, priority support or services. ;)

So, it's a valid approach that's working out for many much better than alternatives which require dozens of constantly changing skills and high layoff risks due to being replaceable & easy to automate. I'm not saying it's the best or lowest risk approach. You'll probably hate the career unless you see work as a means to an end to have fun in spare time. It's got lots of potential though.

I have heard from a colleague that he has a friend who is an exceptional SharePoint expert and loathes it because of its complexity. Multiple times he flew to client locations around the US and changed a single line of code and got paid $$$ it.

All the oracle dba that would be jobless if oracle was not insane crap.

>> If something really is unnecessarily complex, better alternatives are likely to arise, perhaps suddenly.

I suspect that a good majority of corporate code assets trend toward becoming unnecessarily complex. After a certain level of complexity is reached, a company will often look for that better alternative and decide to rewrite from scratch. But in many/most cases of this that I know of, this attempt to recreate/rebuild is found to be intractable.

What seems to happen is that more and more developers are hired to maintain the complexity and I wonder if that's what most developers out there are doing, maintaining complexity, which always makes me think the same thing:

>> That sounds like an unpleasant way to earn a living.

>> I wonder if that's what most developers out there are doing, maintaining complexity

Oh they are. No need to wonder.

I long for the day PDF is finally expunged from the technical environment, in the meantime people still like to generate documents with it; and even with the variety of toolkits, frameworks and APIs, there's still a bit of knowledge needed to work with them efficiently.

I don't sell myself as a PDF expert, but my level of knowledge and experience is high enough that I can maintain and improve systems and workflows tightly bound to them. I keep it on my resume, but have no desire to mine that resource for fear of being the canary.

Good ol' accidental complexity and essential complexity.

There's a nontrivial amount of technologies that are complex and otherwise byzantine for no reason. Given the lack of alternatives, knowing the arcane tech inside and out can provide for a sustained stream of money, but that's hardly ethical.

To me the more interesting point is the high risk. Given that someone is using a needlessly complex technology, and given that they can't maintain it themselves, they actually do need someone who can handle the needlessly complex technology. So unless you're brownfielding, the practice isn't unethical. But it can still be high risk -- mainly, I guess, because of the high opportunity cost of coupling your expertise to suboptimal tech.

It's not always "no reason."

Back the 1990s, GNU autoconf looked like a miracle to me, and in fact, back then you got better results compiling from source than you got using what passed for a package manager.

These days people complain that autoconf is too complicated, being a bunch of shell scripts on top of shell scripts compiled with M4.

Then there is SAP, which stands alone in its ability to destroy value. SAP started out as a mainframe application, got ported to Unix in the 1990s (not many succeeded at that) and now they want you to run it on an in memory database for which you can't afford the hardware, never mind the software.

SAP is the perfect example to support learning hard tech for dollars. So many people get paid a premium to work on that stuff because nobody can get rid of it without risky moves.

Perhaps what is "needlessly" hard (whatever that really means) to one is the exact right solution for someone else?

Anyway, I can hardly think of a dumber way to spend my oh-so-valuable time then to spend it on learning a tech just because its "needlessly" hard (yes I know I'm overusing the parenthetical, for effect of course), but to each their own.

To be honest, learning it as a strategy to bump up one's rates could be considered a rather smart thing to do, but of course you had better understand the true economics of the move and know thy markets.

Aren't most of us interested in making (more) money?

It would be nice to know of what he is speaking of.

> Perhaps what is "needlessly" hard (whatever that really means) to one is the exact right solution for someone else?

Trying to be everything to everyone is one of the surest ways I know to generate byzantine monstrosities. On the other hand, it takes some real nerve and discipline to tell 'No, that is not really what our product is about' to a paying customer.

Yet, extensible interfaces are often successfully used to create stuff that are everything for everybody.

It takes good engineering, and you still won't be able to say yes for every request (for lack of time if no other reason), but it does not require your product to be needlessly hard to use.

It's also not for every product. I just wanted to point that exceptions exist.

I immediately thought about Oracle, then IBM, then SAP ... it seems his friend didn't think about this first !

Some people thrive doing consulting for things like that because they're satisfaction is largely driven by how much they're taking home.

While this is true to some degree, IBM/Oracle/SAP solutions are complex because the requirements are ridiculously complex. Could the requirements be streamlined? Probably, but that would require a wholesale restructuring of the way the company does business. Which, again, will probably happen, but usually not before the company ventures down the path of an ERP implementation and realizes how needlessly complex their business is. But there is a market demand for these types of products; they don't solely exist to fund the increasing size of Larry Ellison's yacht (though maybe in Larry's mind they do).

The main school of thought in business these days is still organizational and process focused. Only the newer guys think of business in terms of products, SOAs, SDGs and APIs, and it'll take decades yet for that kind of thinking to fully infect corporate America.

As exelius mentions, a large part of the demand involves companies that are a little schitzoid and dysfunctional, who have decided it's easier to throw money at the problem and use technology to paper over the cracks.

There's going to be big dysfunctional corporations for decades to come. There's no guarantees in life but investing a little time in a technology like that is probably going to pay for itself and much, much more. I wouldn't do it but I can see why someone would.

Made me think of INTERCAL: https://en.wikipedia.org/wiki/INTERCAL

Bet it was kdb

KDB is simplicity itself. It doesn't get much more elegant.

I guess his friend was talking about Java EE. ehehehe

Seems to work well for SAP. I also have a suspicion that for example Java is so complicated because it was created by a consulting company - complex tech means high paid consultants.

I don't see how Java is more complicated than any other programming language. It certainly is not one of the more difficult ones. Assembly is complicated. C is arguably complicated. Java being complicated? I disagree.

Assembly and C are dead simple compared to Java. The C reference manual is about 200 pages, if I dropped the Java reference manual(s) from a building it would create a sizeable crater.

Assembler is simple like lego is simple: the basics are tremendously easy to understand but the distance between the basic elements and the problems you're trying to solve is very large.

The more 'batteries included' something is the more time you'll spend learning about the eco-system and less about the language per-se.

> The more 'batteries included' something is the more time you'll spend learning about the eco-system and less about the language per-se.

Not necessary. As a young programmer who did not grow up in the java dominated world of 1990-2008.

I took one look at Java and noped the fuck out.

I found C to be more useful.

But its hard to beat the simplicity of java-script.

General problem of Java: It lacks decent onboarding.

I disliked java after school and was convinced I was hopeless so I didn't even bother to apply for Java jobs. Then I was recruited into a Java position and picked up the necessary skills to work during my first 4 days and improved steadily from there, but it only worked for me because I was on a good team.

Now it is me who helps people using Java more efficiently : )

> Assembly and C are dead simple compared to Java. The C reference manual is about 200 pages, if I dropped the Java reference manual(s) from a building it would create a sizeable crater.

I don't think that's a fair assessment of Java. The full specification for version 8 is actually 788 pages, so 500 more than C, but you could reasonably argue that of those at least 100 are dedicated to specify what in C is simply undefined behavior and probably another 100 are dedicated to the memory model. The other 300 hundred would be a reasonable overhead for specifying classes and class loading.

You can say a lot of bad things about Java, but it could very well be the best specified language around; the spec is actually very readable, leaves no room for doubt and actually treats that thing called multithreading that still has no citizenship in the C language. You can't say that of a lot of languages.

EDIT:

I took the time of checking the C11 standard PDF (actually the latest available draft, as the ISO asserts copyright on the standard and you can't get it for free). It's 701 pages. But at least C11 has <threads.h>.

I just picked up my fairly dog eared K&R for that reference it's 226 pages (first edition). Yes, the C spec grew over the years, but the Java docs were huge right when it started out. I remember buying the full set and it weighed more than I could easily lift up the stairs to my office. All in all it amounted to a few thousand pages. This included the docs on the VM and a bunch of other stuff that you might not need (but which I did need).

The C standard library was a relatively small affair reflective of the way UNIX was built, a couple of well thought out interfaces combined with a lot of freedom (and enough rope to hang yourself several times over).

The Java standard library was absolutely huge and extremely hard to make headway in because each and every part of it seemed to be designed by a committee rather than by an individual (or two).

It's interesting that you try to compare simplicity via manual length and say in the same breath that assembly is simple. Assembly language manuals easily dwarf most programming languages: x86-64 is a whopping 3439 pages (of which ~1500 pages is the instruction set reference). And before you say that's just because x86 is unnecessarily complex, ARM is 1138 pages, PowerPC 640 pages. By comparison, the JLS is only about 800 pages, C11 sans library about 450, and C++14 sans library about 500.

The bricks are simple but there are many of them. When I say that assembly language is simple I mean exactly that. The language is simple. If you know what one mnemonic does you'll be able to infer the existence and function of a whole bunch of others since they're usually organized around a matrix of operations and locations to be operated on. Of course a manufacturer would document each and every instruction separately but if you understand the logic behind an instruction set then you have a much easier time of it.

The x86 set, well, let's not go there, it's just too painful.

Assembly may not be simple, as in 'very little language'. Assembly can be thousands of different language elements. But assembly is simple as in 'very little variation'. There are maybe a dozen addressing modes. There may be 100 or more instructions. When you multiply them, you get large manuals.

And assembly is simple as in 'primitive'. The atoms of assembler are instructions, and even they may be clustered into only a few groups i.e. arithmetic, branch.

I find the resistance to learning assembler is mostly fear. When debugging I very often resort to assembler display, even for machines that I'm not familiar with. I can learn the gist of it in a few minutes just by looking at a little generated code. And there's nothing like the actual machine instructions to find what really caused an exception.

Assembly isn't complicated. The instruction list for most processors is small and easy to remember, because it's mostly variations on the same few operations.

It gets unwieldy if you try to build complex data structures, because you literally have to do everything the compiler would usually do. So if you're trying to build an incredibly complex machine with lots of objects and interfaces, it will take you a very long time.

Java is complicated because the culture is complicated. It's not just a language - it's a set of architectural practices and expectations, some of which are questionable, combined with a unfeasibly large ecosystem of libraries and add-ons.

So it's easy to learn the basics, but it's not so easy to get to the point where you can start producing high-quality maintainable code within a typical working context.

What do you mean by Sun having been a consulting company?

I am no expert on Sun, but I think consulting was a huge part of their business? I know they were also well known for their servers.

They only got into consulting at the end as their business was dying and they were trying to copy IBM. In 1997 they were a server and workstation company (selling hardware and software as an integrated package).

s/no expert/completely wrong

I don't think Java the language is particularly complicated - it has a much simpler core than C++, which it sort of superseded in a lot of areas. People went a little nuts with the XML and ObjectBuilderFactoryFactory nonsense and all of the "enterprise" stuff, but you could create similar levels of insanity in any programming language, if the fad storm hits.

I think a lot of those people, at least initially, were employed by Sun. They came up with verbose Java naming, class libraries and other idioms. Also, they were the one who wrote J2EE specs and such. So it would be reasonable to say that Core Java developers set the tone of Java community at large.

Java was created as simplification to other object and modular programming languages in the early 1990s - mainly C++ and ADA. Java added more features and class libraries as time went on.

Reply on news.ycombinator.com