I think it was the victim of changing tastes and fashions more than anything else. People used to appreciate that kind of wackiness and linguistic whimsy. I think these days people want more straightforward and simple tools
In 2000-ish, they announced Perl 6 with great fanfare. People started working on something called "Parrot" which was supposed to be awesome in every way. Quickly it came out that Larry Wall was going to do a serious redesign of the language. It wouldn't just be Perl 5 on a VM, but have a significantly different syntax.
A lot of people then became very reticent to start new big projects. Because Perl 5 was evidently about to be dead, and Perl 6 was some new incompatible thing, so why spend all this effort on writing code now that'll probably need rewriting later? Python suddenly became more interesting.
It turned out that project management was lacking. Perl 6 attracted lots of theorists and fans of experimentation and few people interested in getting out something usable in a timely manner. Multiple VMs and varieties came and went. It took until 2015 until something was released. By then, Perl was already moribund.
It quickly became even more clear than Perl 6 and Perl 5 were different things entirely. Perl 6 didn't share the architecture of the Perl 5 interpreter, so XS wasn't a thing, and XS was a huge part of many prominent modules which meant that a lot of the Perl 5 CPAN modules would be a lot of work to port. The fact that Perl 6 was presenting itself as the successor while Perl 5 still existed and was being worked on didn't help.
In the end Perl 6 was renamed to Raku to make this more explicit -- it's not Perl 5's direct successor, but a new language built from a similar philosophy but otherwise an entirely new creation. But I think by then it was already too late, it'd have been better to do the renaming way before 2015.
The 15 year long development cycle also didn't help -- it created all sorts of confusion and uncertainty about the future. Meanwhile, the upcoming Perl 6 project just wasn't suitable for almost anything other than working on the interpreter itself, and that excluded a lot of Perl programmers who were used to writing in Perl, and not working with the actual guts of the interpreter.
I do think before Python there was no real alternative. We tried C for web development but it was a huge pain. The Python CGI module changed everything for us (Reading about it in the German iX magazine)