Where are the “big problem” jobs?

...and traditional market forces have never been known to catalyze work on "big problems". The only good catalyst I can think of is war (think the nuclear weapons problem or the race to the moon fueled by the cold-war).

I'm not entirely sure I would classify Google Glass as a "big problem." Self-driving cars can address driver safety, fuel efficiency and route efficiency, among other things and therefore certainly rank up there. Google Glass, although I like the concept and very well might even purchase a unit when they become widely available, seems to address the same areas already covered by smart phones, just with a new interface. I think there are huge strides that can be made in the area of pervasive computing, but I'm not sure Google Glass is what will propel that.

(I would love to be proved wrong though.)

Of course the imprecision of 'big' and 'small' in defining problems is also an issue. You can have 'big' problems which exist not because of the science, but because of the structure. Today we were reminded that its a 'big' problem to build a house in South Florida that won't be eaten by a sink hole. Earlier we were reminded about building on eroding beaches. But those are actually "easy" in the sense that you could push piles into bedrock, 75' below the house. But we don't.

Bill Gates has made it a priority to solve some 'big' problems, sanitation, malaria, to name a couple. Other foundations have similar goals. So they are out there but as the other has seen working on them when they aren't self funded is hard to achieve.

(The site was down for me, but I assume this is the direction of the article, and this quote comes to mind).

Many of these problems are so-called collective action problems. If you are able to get enough funding and a strong team, that may not be enough: you need more than technical chops to move the system enough to make a difference!

P.S. I'm glad this topic is getting some air-time!

At very least, I wish these writers would preface with a few examples of what kind of "big problem" they think corporate R&D should be working on, but isn't. OP's example falls flat to me:

> Private businesses seem remarkably uninterested in tackling serious problems such as energy despite soaring prices and evident problems.

They may not be interested in solving energy problems in the way OP wants them to, but there is certainly massive industry R&D in the energy space (Siemens and Schlumberger come to mind, many others). Just look at the ever-increasing yields from fracking and (ever deeper) deep-sea drilling. That requires huge R&D investments (down-bore sonar and NMR, anyone?)

If you consider the sorts of problems that are presently somewhat tractable modulo engineering {to pick a few: energy, cancer, food, genetics} I would argue that there is probably a lot more industry R&D work than the author realizes. It may not happen in the software industry, but realistically, for many "big problems", software is auxiliary at-best.

If you are considering the sorts of problems that are presently only tractable modulo magic {fusion, life extension, interstellar travel} then of course corporate R&D is not working on those.

The point is that the big technical problems that need the most research muscle aren't necessarily that important, and you can't really know what the future needs ahead of time.

There is a threshold to capitalism. The problem is that no one has yet figured out how to hedge the bet, attempt to solve extremely risky borderline impossible problems, and guarantee a profit. It's much much safer to go after an obvious, well-described, minor problem that over 50% of the population experiences and would pay a minor sum to improve. It's a safe bet. And there are a lot of those!

The other problem is surely capital and education. They've diverged and are being pulled in opposite directions. Higher education saddles you with debt and makes you extremely risk averse. You can't ever get access to capital because you're always playing it safe. So the system has been inadvertently designed to lock out these "big problem" jobs, because, again, they are too risky for people to undertake.

And then the rich are also risk-averse these days. The focus is on ROI and increasing wealth—look at Warren Buffet, one of the world's richest men, entirely rewarded for being risk-averse and investing in solid, tried-and-true companies. And you're surprised that the big problems aren't being solved? People like Buffet aren't investing in them, at least not at a large scale. And they're rewarded for that behavior. The system is set up to reward this sort of behavior. The idea is that once you get rich, you invest "prudently" and grow your wealth. You don't bet it all on the big problem that's probably impossible.

Except sometimes you do. Some people are solving the big problems. Elon Musk is up there doing it (in space and on the ground). Bill Gates and Buffet too are putting up a lot to fight seemingly impossible big health problems. This is all excellent, and not to be ignored.

I think the incentives are not yet aligned correctly to get the result you're looking for. What you want is to reward risk taking, and capitalism just doesn't do that in the right way. There are too many hurdles, too many punishments for failure. And the right people to solve the problems aren't the same ones who have the capital to get the solving done, for so many reasons.

There's your big problem to solve—fix that system. Align the incentives. It's the meta-problem, and if you solved that, the world would probably be a better place. (Hint: there are already some good solutions, but the America won't vote for them)

That said, I'm one of those grad school dropouts who's stumbled into the whole trying to bootstrap a hard problems only business.

Its a lot of work! I'm lucky that my family is based in NYC so that I can reduce my living costs plus do hard problems only consulting part time while spending the rest of my time working on building my tech products @Wellposed.

There are uncountably many high value problems you can try to build an interesting business around. Just look at any Specialized / niche high value industry and ask "can we build better tools and how valuable would those be?". You should choose a domain where you'd enjoy talking with/getting to know potential customers personally. if there are other folks attacking the same question domain+combo, ideally they should be smart folks you respect who are solving the problem with a different combination of tradeoffs so that the products are complementary rather than substitutes. Oh, and you should be OK with building a satisfactory product taking at least a year of hard work, constant learning , and feeling stupid because you keep on seeing better ways to do every thing.

Or at least that's sort of the meta pattern behind how I'm operating at Wellposed. Its hard work, I get to feel stupid in the always learning more sort of way, and its the most fun I've ever had.

I'll find out towards the middle / end of this year if my ideas and process bear out on my first hard problem domain. I'm terrified and excited. Either I'll quickly be trying to figure out a sane hiring process, or I'll be job hunting with an interesting engineering and consulting portfolio accumulated over the preceding year, and I don't know which outcome is more likely!

That said, entrepreneurship is hard work and gets in the way of spending focused time on engineering. Good thing I'm a tad extroverted.

This is only a problem if the government is failing to fund research adequately. The point of government grants is to pay for riskier research. There are limits to what governments will fund, and unfortunately there are "research topics du jour," but I do not really expect companies to go wild with shareholder money. The free market is not going to cure cancer, and that is fine because the free market is not the only way to pay for research.

We live in a modern world with many advanced technologies. But in a way, we're not unlike the hunter gatherers who had to spend their hours worrying about survival. We spend the vast majority of our 24 hours commuting, and working for The Man instead, worrying about paying our bills, loans, mortgages, putting food in the table, college tuition, etc.

He omitted my basic rule: For getting venture funding for such a company, the rule is that some successful research promising as the core 'secret sauce' of a new business together with a dime won't cover a 10 cent cup of coffee, that some solid research for a great technological advantage with some traction is significantly less good than just the traction.

For more, such discussions miss some big points likely especially relevant now.

First, let's consider the 'big problem': Since we've already decided that we have to be in business, the 'big' we might as well take as promising to make a lot of money, e.g., build a company worth $100 million times some factors of 10.

Second, for building this company, we come up with two lists: The list in column A is for technology we have or might invent. Column B is for problems people have and are or at least will be quite willing to pay to get solutions (if there is a big question about 'product/market fit', then leave the problem on the cutting room floor -- we want no questions about eager customers). Yes, it can be that we deliver the solution for free on a Web site and then get paid by advertisers eager for the eyeballs of the users -- that's a small detail we all understand. We want no technology sold before its time, no newly invented technology looking for suitable real problems, no products without eager customers.

Then we pick a 'crucial core' technology from column A and a corresponding problem we will solve from column B.

Third, we already know how we will deliver the solution -- over the Internet -- and we already know how the solution will be generated for the users/customers -- via software on computers. Why do we know these things? Because of the astounding, historic improvements in price/performance of bandwidth and computing, and platform software, the fact that these improvements have not yet been fully exploited, and the fact that in the current economy information generated from software can be among the most valuable, and most profitable, results to bring to the market. Biotech? Have to wrestle with the FDA, at $100 million or so a round. Energy? It's an old field, has been slow to exploit nuclear energy which was and remains one of the greatest opportunities in the ascent of man, and, thus, really is stuck in the mud. Materials science? A sheet of carbon atoms is not yet an actual product that can be sold to solve a real problem. Flying cars? I won't comment!

Fourth, here's a secret (make a billion): When we look at column A of the technology, note carefully the phrase "or might invent". No joke, guys. It's too easy and common to laugh at original research. Most of the people laughing have done no original research. And it's too easy to notice that the guys in universities that do original research usually get maybe papers, professional reputation, and tenure but no business revenue. Why? Because heavily that research is funded for US national security or US health, and in either case the problem sponsors in the funding agencies don't want just 'research' to make good products to solve real problems but another E = mc^2 or actual good progress on a cure for a major disease, especially cancer. Considering that they are spending my tax money, I'm essentially pleased. NSF/NIH money is too short now to fund the best, good, far out, fundamental research; there's no hope for it funding good original research to make money. But such research can be done. Trust me!

Then with the research, old or original, from column A and problem from column B where the research provides a good solution to the problem, convert the research to the corresponding software, load it onto a server, put a Web site or an app in front of the server software, please users/customers, and make money.

In this, research just such as I described is crucial for much of the progress people are not seeing but have been expecting.

Silicon Valley VCs, listen up: You guys will have to start taking research, original and/or old, seriously or risk funding just more social, mobile, locational, sharing apps or trying to buy tickets on airplanes that have already left the ground and are not taking on any more passengers.

Entrepreneurs, listen up: Pick a pair from columns A and B, write the software, get the server up, and go live. The software? Mostly the time is just 'overhead' that you have already covered as part of your day job. The server might cost you less than $1000. The bandwidth might cost you less than $100 a month. Or just put your software in the cloud (at maybe some risk of your crucial, core research).

That's the opportunity.