1) the syntax is very finicky (slighttly more so than C, I'd say). Most software languages (thanks to more experience with parsers and compilers) have moved on from things like requiring semicolons, verilog has not.
2) writing tests is awful. Testbenches are crazy confusing. Much better would be some sort of unit testing system that does a better job of segregating of what constitutes "testing code" versus the "language of the gates". You would have a hard time doing something like, say, property testing using verilog.
3) there isn't a consistent build/import story with verilog. Once worked with an engineer that literally used perl as a verilog metaprogramming language. His codebase had a hard-to-find perl frankenbug which sometimes inserted about 10k lines of nonsense (which somehow still assembled a correct netlist!) but caused gate timings to severely miss and the footprint to be bloated. It took the other hardware developers one week to track down the error.
None of these things have anything to do with the fundamental difference between software and hardware development.
For chisel: at least to some degree, you can get some developer ergonomics from the Scala ecosystem, and do most of your unit, functional, in integration testing outside of verilog, in chisel, and last-minute autogenerate verilog and do a second round of testing to make sure chisel did everything right. It's the same reason why people do things like "use elm to develop frontend, compiling down to javascript" and it's a perfectly valid strategy.