Sadly, Patrick Collison does not have relevant experience in these areas.
More importantly, though, the article I was being interviewed about is one I cowrote with Michael Nielsen, who does have a lot of first-hand experience: https://en.wikipedia.org/wiki/Michael_Nielsen. (And, as part of writing it, we spent time discussing the questions with many practicing scientists across a lot of different fields.)
There are plenty of ways to improve research productivity on the margin, but I don't think that's the point of the argument here. The argument isn't that research productivity is getting worse because of X, and to solve it we need to do Y. The argument is research productivity is getting worse because of X, and because X is 'the laws of physics', this slowdown is a feature of the universe, not a feature of our poor research methodologies. It's inherently unavoidable, to some extent.
The plain fact is that everything we do is limited by (1) the laws of physics and (2) the space of possible inventions. In the early days of the industrial revolution, we didn't really know the laws of physics and we hadn't mapped out the space of possible inventions. But today is qualitatively different. Broadly, research productivity is hitting diminishing returns. The key questions here are (1) how do we know this to be true? and (2) why is this happening?
(1) is touched on by a number of books and papers. You can look at progress in transportation. The gains of today are pitiful compared to the gains of 1800-1950, where we went from horses to steam locomotives to airplanes to rockets. You can look at progress in medicine. We made progress earlier by discovering that germs existed and that we should wash our hands. Equivalent gains today now take billions in drug R&D. Really the only sector where we've been doing well over the past few decades is IT, and now even that's maturing. Moore's law is dead, the gold rush for the web is over, and smartphone penetration is approaching its natural limit of 100%.
The two key readings I would recommend here are Robert Gordon's 'The Rise and Fall of American Growth' and Nicholas Bloom et al.'s 'Are ideas getting harder to find?'
On (2), I think it's worth emphasizing that electromagnetism was mostly figured out in the late 1800s. Quantum mechanics was mostly figured out in the early 1900s. The essentials of nuclear energy were figured our in the early to mid 1900s. Even though we know gaps in our physics knowledge remain, these gaps are not going to overturn what we already do know. And what we already do know really constrains the efficiency of our designs. We're not gonna make engines that beat the Carnot limit. We're not gonna make vehicles that are simultaneously (1) fast, (2) energy efficient, and (3) move through the atmosphere. We're not gonna find chemical fuels that are denser than hydrocarbons. We're never going to beat entropy in improving the energy efficiency of ore smelting. Etc.
Now, I don't mean to establish a straw man that progress is over, that innovation is impossible, etc. But I really do believe that design space is finite and there are diminishing returns in its exploration. Before you believe it, it sounds crazy and defeatist, but after you believe it sounds banal and obvious.
1. I think you could to some extent William Gibson this though. "The future is already here — it’s just not evenly distributed". Maybe we have made less progress, but it also looks far less because it isn't being obviously distributed. (I guess in regards to Moore's law you can remove "not evenly" from the quote).
2. I agree. I wonder if this doesn't have a lot to do with misguidance. Basically a lot of our ideas of the future are old. Therefor we want self-driving cars, even though we know cars are limited in the sense that they have non-specific applications. So a lot of what people see as the future is a non-starter from the beginning.
I will have to read some of that material. I do think that science could be getting harder. On the other hand I am not sure we are even making the effort. Space exploration is something that get really hard really quick. But there are certainly a bunch of things you could at least try to explore in our solar system. But I guess that progress can be hard to quantify as well.
In addition, from my perspective, research isn't just about what we find out in an isolated field, but also the interplay of discoveries and knowledge between fields. This latter view, 'integration', is largely unexplored at scale, though lip service is constantly being given to it. I think there is a tremendous amount we can do here, and it's not just marginal. Overall, I am more optimistic I think, perhaps to an unwarranted degree :)
I also think that language is important, especially in the service of scientific understanding and progress. In this respect, I think category theory is poised to play a major role as a language for the sciences, and particularly integrative science in the 21st century.
I've been working on this in my spare time: https://www.eonias.org/, which I definitely hesitate to link to here because it's unfinished, but I figure I might as well start putting it out there. I think there are lots of ways to improve the way we go about research, having been in the grind for a few years now. There are tons of different opinions on this, but I just wanted to do something.
In short, I personally am not concerned whether 'research productivity is getting worse because of X, and because X is "the laws of physics"' is actually true or not (this is a complicated question, and the relevant question is whether 'productivity' can go up again--I think it can). I believe there's plenty left to do, and we need far more people doing it than we have engaged now. After all, should we move the money from the budget of the NSF into tax incentives for football stadiums? From my perspective, I think the NEH should have a much larger budget, which in and of itself is an interesting topic of debate. I mean, where have the humanities gone?? I'm f@#C!ing tired of AI and robots already. ;)