Three of the most important scientific papers in modern history—uncovering the photoelectric effect, Brownian motion and the special theory of relativity—were written about a hundred years ago by a man who could only sign them "Mr." when first writing them. About 80 years before that, another physicist who had no formal schooling past the age of 14 was pioneering advances in humanity's understanding of electricity and magnetism.
Science isn't restricted to or based on credentialed people. If you look at the history of science, credentials and recognition often follow groundbreaking scientific work.
The PhD is essentially an apprenticeship to a practicing scientist.
Several people have insisted that I should go into a phd program, but when I looked into it I saw a lot of: learn how to do grant applications, publish this non-reproducible and barely significant result just so you have something published, work on a phd adviser's bad idea so he can take the credit, etc.
If you're working on an idea no one has had before, what kinds of support can others give you and do modern phd programs give that support?
A PhD isn't really about doing something useful, although that is typically a result. It's learning how to explore an idea and expand upon it.
Your advisor does not lead you as much as guide you. Hence the name.
While the dangers you mentioned are absolutely a threat, much of that can be mitigated by choosing a good advisor. Personally I think taking a tenured, well established prof is a better bet, because they aren't as worried about their own career advancement. But everyones mileage varies.
To expand on my original idea: imagine being given an opportunity to work closely with a recognized expert in your field. For years he will meet with you, advise you and honestly try to help you also become a world authority on this subject. Afterwards, he will always support your career and care for your future. That's what I have, and while I certainly feel fortunate, I would not call my situation atypical.
I'm happy to provide more info.
Unpromising in what sense? Unpromising for you personally, or for the greater good of humanity?
Research always comes with a risk. It is called "re-search" and not "re-find" for a reason. You can learn some heuristics that will help you explore, but it is certainly no guarantee for finding anything.
Agree. Those heuristics were precisely what I was alluding to. It is all about how you explore that space. The best researchers have an amazing ability to intuitively ascertain promising directions. Many smart folks I know constantly spin their wheels working on problems that are intractable.
To me, that ability is the principle value add of a senior PI (aside from funding, of course!)
Another interesting anecdote is that on the crowd sourcing platform innocentive, the majority of problems (which have stumped fortune 500 R&D departments) are solved by people outside the domain of the problem.
You are correct, but what other evidence do we have showing whether these people are scientists?
Gates has one published paper: Gates, William H. and Christos H. Papadimitriou. “Bounds for sorting by prefix reversal.” Discrete Mathematics 27 (1979): 47-57.
PDF: https://people.eecs.berkeley.edu/~christos/papers/GP79.pdf
Pancake sorting seems to attract Harvard people who decide to forego their planned science careers for other fields. It got Gates with normal pancake sorting, then 18 years David S. Cohen did burnt pancake sorting, before leaving science to change his name to David X. Cohen and write for The Simpsons and later Futurama:
Cohen, David S.; Blum, Manuel (1995). "On the problem of sorting burnt pancakes". Discrete Applied Mathematics. 61 (2): 105–20
I think with Jobs et al, for however long they work, it's likely they still only do a limited amount of focused creative work in that time. The rest is spent doing the every-day politicking, intuitive decision-making, and administrative stuff that goes along with being a CEO.
Having PhD or working on it is not a prerequisite for being a scientist. Musk is not in academics, but he is a CTO of a SpaceX, so he is in charge of scientific and technological issues within an aerospace company. He does more applied science than vast majority of scientists that I can think of.
Edit: Typo
Does he do it or does he manage a company and people who do it?
Gates' only research paper[0] is one on the Pancake Sorting algorithm[1] — which he wrote in his second year at Harvard.
[0] - https://people.eecs.berkeley.edu/~christos/papers/GP79.pdf