Hacker News Reader

Comment by mark_l_watson | Hacker News Reader

I had a boss 25 years ago (smart guy, did compiler work, etc. for new computer architectures) who claimed that there was really a 100x factor difference from the best to the worse. I believe this: would you rather have one day of work from Daphne Koller, Peter Norvig, or Jeff Dean - or 100 days from an average developer? Well, the answer is that "it depends." There is a ton of work that can be effectively done by non-rock stars, and there is some work where a 100x person can do things that other people can't, no matter how much time they spend.

An example: I use TensorFlow, but looking at the concept and implementation, I don't think I could conceive of and write TensorFlow in 10 years of work.

View on HN

> I use TensorFlow, but looking at the concept and implementation, I don't think I could conceive of and write TensorFlow in 10 years of work.

And this is exactly why software is eating the world. Once written it stays written and all those people that could not write something like TensorFlow if their life depended on it are able to use it as if they could.

Abstraction is a super powerful thing, all this building on the knowledge and capability of those that came before is what makes the whole thing possible in the first place. Probably the people that wrote TensorFlow would not be able to design (each of them individually) the computers that it runs on, or the operating system that allows them to run TensorFlow in the first place.

I agree very much. Just came to say that this has always been the case with the know-how of starting fires, metal molding, forging tools and weapons, inventing microscopy, large scale industrial processes, computers... software is the extension of the exponential acceleration of abstraction and reuse that our society was built on.

I feel like there's a difference with software though. Maybe not in degree, but in the constant factor.

With e.g. metalworking, you can transfer the knowledge but training takes time. And building a new implementation of that knowledge takes time too.

But in software the knowledge and implementation is transferred effectively instantly. You can nab software off the internet (transfer knowledge) and start running it locally right now (implementation).

Software changes the complexity from O(2^N / T) to O(2^N).

Sure, but is this not the same as buying a hammer/fertilizer/metal press and using it for your own means? That's a quasi-instantaneous acquisition of the result of hundreds of thousands of years of innovation.

Electrons move a lot faster than molecules, you could invent X in the United States and a person in China could be using it tomorrow without any cumbersome shipping or cultural barriers to adaptation.

The hammer, fertilizer and the metal press are all part of our tool-making culture, ways to adopt nature to do our bidding and in the case of the hammer and the metal press to shape it by brute force. In contrast, something like TensorFlow is a tool of the mind which means you can encode it as bits and those will travel the world a lot faster than your average bulk goods or container ship every will.

So they are comparable at some level and totally incomparable at others.

Even if software didn't have tremendously higher cultural barriers for usage than manual equipment, the rapidity of exchange is a very small component of the exchange value for the means of software production (which is primarily hardware anyway). The value of both hardware and software is measurable by the amount of labor needed to produce them, so it's not even comparable--it's identical.

Metal molding seems to be eating the world better than software.

You can say the same of artists...there is certainly a huge difference between the best and the worst. That's taking to broad of a view IMHO.

You aren't, generally, going to hire a Picasso. Picasso will be off being Picasso.

You also aren't going to hire Greg, the guy who took one art class in middle school who just decided last month that he's going to be a artist.

What we are really talking about is the 'artist of the line'. Accessible professionals. Yes, there is still a spectrum there but it is more in the 10x territory. And some are 10x for very particular work and some for others.

What gets me with the whole 10x thing is the idea that a 10x developer in one particular circumstance can be dropped in another and be expected to perform at that same level. That's not how it works.

Google did hire Peter Norvig, if I'm not mistaken. I very much doubt his job is to write programs for them, though.

Chances are his job is to make those right "10x" decisions for teams of average programmers who do the actual work, to ensure that those average programmers are writing the right code.

I'd argue that if such a multiplier exists, it's infinite - there are completely non-productive engineers in the world.

And I don't just mean on the scale of a day (we've all had those) - there's a reason FizzBuzz was a thing; there are people with no practical coding ability nonetheless interviewing for engineering jobs. Somewhere, one of them got hired. Somewhere, they were able to BS - or download already prepared software that did what they were supposed to code - and keep that job.

> I don't think I could conceive of and write TensorFlow in 10 years of work.

Don't understimate the human ability to learn. 10 years are plenty of time to learn the math and to find good code structures.

> I don't think I could conceive of and write TensorFlow in 10 years of work

Google was using for years a tool for deep belief networks, then suddenly deep learning was a thing and they started TensorFlow based on what they learn of the deep belief networks tool. So you can track the origins of TensorFlow to 2010 or before, and for sure it wasn't developed for 1 person. It could start as a 5 person project and now probably it is more than a hundred (aiming very low). If you add on top of it all the research done where they have been collecting ideas, then you realize the amount of knowledge behind TensorFlow is huge. And the amount of people and time under it is also huge. Probably no one can understand it completely, even if he/she spend 10 years.

Well the worst (or even simply "kinda bad") are so much worse that it's not hard to believe and even bigger gap exists.

I once spoke to a guy that literally did not understand the lifetime scope of local variables nor how simple field assignment worked (a.x = y). I don't mean this in an esoteric, edge-case, way, but quite literally. I asked him why he didn't hire some devs to do the work for him, and his answer was "most devs are terrible and I've not found people that are as good as me".

The downward range goes very, very deep. And that's of people that actually do end up shipping something that "works", somehow.

I've personally rewritten an app in less than a week that had two devs on it for over 6 months. Their deliverable didn't really work at all, even after the 1+ minute loading time (Win32 app).

> I've personally rewritten an app in less than a week that had two devs on it for over 6 months. Their deliverable didn't really work at all, even after the 1+ minute loading time

To be fair, a lot of that six months was probably just nailing down the requirements. Your rewrite enjoyed having that design work already done.

I'd argue that building applications is 90% design, 10% code. The code part can balloon to a greater fraction if the developers are not competent enough of course.

The only time I've had improvements on that scale, it felt almost like cheating: the previous consulting company had spent ~8 months coding but the developers didn't understand functions or how to do a SQL WHERE constraint. Simply moving redundant code into an include file and getting rid off all of the cases where they were looping over some massive cross product looking for a record ID was good for ~4 orders of magnitude response time reduction and allowed completing about half the functionality for a net negative 90% lines of code.

Given how many clients that company had, the main conclusion I draw is that as an industry focusing on helping the .1x people improve is more likely to make a noticeable difference than chasing the other end of the spectrum.

Whatever people thing of flashy trends, there is true value in pair programming. The unstructured spread of not just knowledge, but how to think about programming.

The lack of true apprenticeship, fellowship, mentorship is an incredible disservice to our industry. The number of people who all need to rediscover the same lessons the hard way wastes so much programmer capital.

I completely agree, but I think it feels to business people like you're doubling their costs.

> Given how many clients that company had, the main conclusion I draw is that as an industry focusing on helping the .1x people improve is more likely to make a noticeable difference than chasing the other end of the spectrum.

Do you think that's possible? More often than not the .1x people don't seem to care either.

Not always but I do think there are organizational contributing factors: some people will see inefficiency and try to do something about it but others might just roll up their sleeves and do what they're told (maybe after unsuccessfully trying to change things a few times). That's the group I think could be helped given changes in support & incentives.

That's possible. But more likely, given they included over 100MB of system32 binaries with the app, they just didn't know what they were doing.

Was TensorFlow written by a single person?

Keep in mind there typically is a large team behind projects like this, and they probably had experience with previous designs to build on.

You're comparing a single person to a whole community of PhD's. Getting up to speed enough to be a useful member of such a community is more doable.

I'm not sure about this TensorFlow example. Theano was there in 2010. Same thing, only TensorFlow has a slightly better UI on top, and a mobile backend.

And in general, what one fool can do, another can.

I likewise had a boss 25 years ago. But for me the story ends there. I don't even remember his name!

in reality, I would want 100 days from the average developer.

Reply on news.ycombinator.com