Ask HN: Why is A/B testing unacceptable for novels but encouraged for software?
Any insights on this?
That is, you can run this test in precisely two places: In early planning(what kinds of story do people want) and when adding finishing touches(alternative versions of a scene).
Everything else has to come from self-feedback, a structural plan for what the novel "does": Does it explore certain themes? Does it go deep into a character's psyche? Is it a way to build up a world? Once you've set that plan, you have the tool you need to rubric yourself and determine what MUST be written(as in when you see a MUST in RFC specs) to satisfy the plan and what can be cut before it ever hits the page. And so most of the process never needs another person in the loop, it's just grinding through the technical details of how to make the story work given that spec. Seeking external advice is more counterproductive than not when you get into these details, because, as with the innards of coding, it's a spaghetti of dependencies that you can't get many "quick fixes" for.
When we look at a software product, there's still a great(and probably underserved) need for that kind of structured rubric, but most of the work is biased towards delivering a high quality of service according to specific metrics, which is something that can be evaluated on a feature-by-feature basis. Often, in fact, the metrics get in the way of addressing the structural problem by creating conflicts of interest, especially when profit motive appears. That leads you towards, e.g. productivity tools that dark-pattern in some ads so that you misclick them.
How about just upgradable versions? Say an author puts out a novel, and then over the next year, does some further edits, better character development, whatever. Why not put out a version 2 of the novel?
Same question with music. Especially now, with so much music worked on in software rather than recorded to tape, it would be easy to spin out a version 2 of a single or an album.
We do see this in some other areas; textbooks, for example, routinely have multiple editions with various updates.
It seems really odd to me that we don't make updates to all sorts of media.
But then, lots of folks were perturbed at the updates to the original Star Wars movies...
Similar to software, you run a/b tests to figure out an optimal end state. Once you get the results you may not a/b test that part of the app ever again.
What many authors do is have a group of 'early readers' that they know and trust to give them actionable qualitative feedback ("I didn't really feel that decision by the character was consistent with what we know about them", or "I get you're trying to establish the setting, but the action and conflict doesn't start until chapter 3, you need to bring it forward, or at least do some foreshadowing"). This is much more like agile software development that revolves around close collaboration with a customer, and sometimes an author's agent is also an early reader.
That said, publishers also send out hundreds or even thousands of early (usually edited but not always copy-edited) copies of a novel to a much wider group to solicit feedback (and generate buzz). Such Advance Review Copies, or ARCs (and their electronic equivalents eARCs) are often sent out looking for bug reports or pain points ("Hey this unnecessary rape scene isn't cool and doesn't even advance the plot") and some indication as to what sort of readers will most like it, which is often really helpful when a work doesn't fall neatly into a particular subgenre or subverts a subgenres tropes and cliches (you can think of this as the publisher getting data on "people who like X also like Y" or "This is Space Opera for people who don't like Space Opera!"). This stage is more about priming (ARCs go to reviewers as well) and optimizing the top of the funnel than doing major iteration on the book itself, and collecting endorsements from other authors or celebrities to feature on the cover, but detailed feedback if provided won't be discarded, though it will be filtered by the editor.
So, you see, within the constraints of novel publishing and the long lead times to get books printed (12 months from when the editor accepts the manuscript!) Novels actually do incorporate a lot of feedback and iteration, just not the sort of micro optimization that is the hallmark of A/B tests.
Another thing is that I don't think you can isolate basic parts like you can with software since there needs to be lots of writing to convey an idea. The time needed to do it would increase by a large factor when you try to break up to test it.
I'd like to to see a/b testing for book outlines? Or book elevator pitches?
My other reply is that A/B testing might be said to be accomplished in consultation with an editor.
Perhaps a publisher might think so, but I would suggest the appeal is the unique POV and voice of the author and the choices they make.
Novelists frequently have editors and other people who comment on and refine drafts. I don’t know what the OP is really asking- of course authors get opinions and feedback on drafts. If they want it. But for a classic buy-the-novel-read-the-chapters, clicks won’t be clicks of engagement, or they won’t matter because you pay before you read, so A/B testing won’t help. I’ll pay based on a review, and if the version I read is not the one I heard was good, I’ll be disappointed in the author and whatever medium provided me info about the work.
Possibly the OP is thinking of a new literary form.