Between 1970 and 2010 if you could design special purpose hardware that ran 10 times faster than the state of the art you would need to get it to market in volume in under 3 years.
If you took any longer the general purpose CPUs from Intel would by that point be within spitting distance of your superior architecture, at a fraction of the cost.
That's what happened to Symbolics, general purpose PC's could run their software faster than the dedicated machines they designed.
IMO, what killed them off were caches.
First because more than half their shtick was that they ran their VM interpreter in microcode in order to avoid the bus contention between the interpreter and the code it was trying to run. When the inner loop of an interpreter can fit into icache, that gain is moot. (This is the same reason why those silly memcpy and strcpy single instructions you see in just about every cisc exist. They weren't that silly at the time, about half your bandwdith would be ifetch otherwise).
And less importantly at the time, in a caching architecture their naive implementation with lots of pointer chasing is about worse case from a data flow perspective.
So the Lisp machines were targeting a fairly small segment of the market that might have exploded into the mainstream but didn't, so they had less income to reinvest and therefore struggled to maintain their lead over the general purpose/mass market hardware vendors.
Symbolics had C, Pascal and Fortran compilers written in Lisp running on their machines. Others developed an Ada system for these machines. Drawback for all those: expensive and running not that fast. The interesting part: they allowed interactive development in these languages and one could run some C software a bit more safely due to runtime type checking. Examples for non-Lisp programs running on them: TeX and the usual X11 server.
The explosion did not happen during their lifetime, because the whole package was expensive (RAM&Disk was very expensive), it was tied to higher-end hardware with lots of memory, it has never been ported to other architectures on the metal (an Alpha machine - expensive already - would have been fast enough to run a full Lisp OS - but all one had was an emulator for it ... there was not enough manpower to make it run natively) and because the OS was mostly limited to one person, one GUI and one Lisp per machine - it was no real server OS with securely (somehow) separated programs.
I think something like Lisp/Smalltalk/SomethingNewOS machines might have another shot here soon, especially with the loss of gains from Moore's law
Yes, but it's different now. Now our programming languages are closer to lisp than C. We use javascript, python, ruby, haskell and throw functions all over the place and they come with a cost of frames etc. Even "low level" languages like C++ and Java recently gained powerful capabilities to express lambdas, and C++ programmers use lambdas and std::function pretty often. Why not experiment with a machine that can handle function passing better Turing machines. It seems like if you use enough high-level structures in your code, even a "slower" lisp machine can run your faster than "fast" classical Turing-machine-like-CPU.
This is not even for general purpose CPUs. There are embedded devices people write python or javascript code on. Why not make those CPUs lisp machines?
Luckily, there’s a way out: have your lisp machine transpile to blub.
It is the mediocrities with delusions of grandeur that do.
Citation needed. This is the exact opposite of my experience in life so far. Anecdotal maybe, but I met quite a few people from >10 different countries.