I find plenty of other games were a lot harder for me, like Hades, Celeste, Meat Boy or Returnal for example. I think actually the genius about Souls games is that unlike Roguelikes or hard room based platformers, the difficulty is actually well balanced, it is approachable. You get better as you play, you learn the boss pattern, you level up and get better gear, and when you die the penalty isn't too harsh, getting back to the boss isn't like dying in a Roguelike where you gotta go through the whole run again, it's just harsh enough to make you really not want to die when you fight the boss though and feel some added stress from having to walk back to it to try again.
And if I think of casual AAA games likes FF7:Remake, Doom, Last of Us, none of those games I was able to beat at their hardest difficulty, yet I could beat all the souls games. And that's often because these games get hard by handicap or cranking the difficulty in cheap ways sometimes. Souls games are an easy kind of hard, they're approachable, that makes it more accessible in a way, and it's why I think they got popular.
Having said that, I still think they should add some accessibility features that can make the game easier. What they do in Mario Kart or Celeste I think is great. Instead of making difficulties which means some of them won't get proper design and balance. Focus on a single difficulty, but add accessibility features that can make it easier. I think it's important to do that so people who have handicaps, are younger, are too old, don't have the time and patience, can still experience and enjoy some of the games. They could choose to play some section normally, and when they get stuck, instead of quitting the game, they can try and give themselves some help to get over it.