They were not extemely slow compared to contemporary machines. Keep in mind: Lisp Machines were single-user workstations mainly for developers and were developed originally in the end 70s and commercialized in the 80s.
Commercial Lisp Machines (LMI, Symbolics, TI) were basically as fast as contemporary machines 1981-90. Especially for running Lisp. The first systems appeared end 70s and there basically almost no machines like them. LMI and Symbolics machines were basically the first or at least among the first GUI-based workstations in the early 80s. In the mid 80s a Lisp Machine was as fast as the then contemporary machine, but much more extensive in capabilities - which made the hardware very expensive.
In 84 Apple released a Mac with tiny b/w screen, an 8 Mhz 68000, 128kbytes memory, no virtual memory, a floppy drive, no extension cards, no development environment, ...
At that time the Lisp Machine from Symbolics used a 68000 as a frontend processor for its megapixel console, had ethernet, supported several megabytes of RAM, had a dozen of extension slots, had disk drives, tape drives, color graphics options, and an object-oriented OS running with 100+ virtual memory with garbage collection.
At the end of the 80s machines catched up - especially the Unix machines - but they were not dramatically faster for running Lisp. But they allowed machines with less RAM running C code/applications - while RAM was really expensive. Real faster machines appeared in the early 90s, when Lisp Machines were already abandoned and newer Lisp architectures based on Lisp RISC chips were not brought to the market - due to lack of customers interested in expensive niche technologies...
In the end 80s a 68030 wasn't running Lisp faster than a Lisp Machine - basically the same for the usual (micro-) benchmarks. The 68040 in the early 90s was slightly faster, then. Generally the during its lifetime 75-92 the Lisp Machine architectures were not designed for speed, but for other capabilities - they were basically enablers for the first development workstations with able to run large (at that time) software/datasets. A single Macsyma could thrash a timesharing mainframe end 70s, but on a Lisp Machine the user had Macsyma on his 'personal computer' with his/her own 1MB-4MB RAM and his/her own virtual memory.
SGI Machines in the mid-80s used Motorola processors - thus weren't 'extremely' faster either. The early MIPS processors weren't much faster either - around 1990 SGI introduced cheaper SGI machines with good graphics - but still not with 'extremely' fast CPUs...