Instead, I think the fascination with difficulty is exactly this -- the idea that some ideas are amorphous and difficult to express directly, and that authors have tried (successfully and less so) drastic measures to try and, in their own way, do exactly what you said in your comment.
And there is something, for some, inherently interesting in the different contortions of text that experimental writers have come up with.
Not necessarily. It's a lot easier to empathize with characters and a novel can capture something more ephemeral. Camus would write a novel first then turn it into an essay.
I have pages and pages of what I think are well-articulated ideas that would be relatively easy to turn into essays. My challenge is finding a narrative that ties them all (or at least some of them) together.
There's an interesting quote by Eliezer Yudkowsky, who is best known for writing the fan-fiction Harry Potter and the Methods of Rationality.
"Nonfiction conveys knowledge and fiction conveys experience. If you want to understand a proof of Bayes’s Rule, I can use diagrams. If I want you to feel what it is to use Bayesian reasoning, I have to write a story in which some character is doing that."
https://intelligence.org/2016/03/02/john-horgan-interviews-e...
It's interesting to note that he wrote HPMoR with a specific purpose in mind, namely to teach the specific skills of "rational" thinking he wrote about on the LessWrong site.
In his case it seems he saw fiction as something complementary to his essays, a means of conveying "experience", a simulation of living through situations in which the concepts could be used.
This is, of course, far from the only use of fiction, but I think it at least provides a specific example of a way in which fiction can be used to convey something in a way that cannot be (easily) done using dry prose (even if you are considering it purely from a practical standpoint without any reference to artistic merit).
You see this in software as accessible vs power users
Frameworks to simplify web development
Libraries do it gp’s example
Schools do it by simplifying the literature selection/ topic breadth
Etc
Anytime you have management without a minimum “gate”, the issue occurs, because the conflation exists but no one admits it, and with the goal of widespread accessibility, it gets further embedded each iteration.
In other words the quote “if you can’t explain it to a six year old, you don’t understand it” is misread to “if you can’t explain it in six minutes, you don’t understand it”, and then everything gets “simplified” right out of existence
I'm about a million miles from being an author, but I'm old enough that I have, what I would think, is a recognisable style that I'm comfortable with, and changing my approach would end up negatively affecting the quality of the writing because it comes less natural to me.
Authors will find their own level as much as readers will.