Greg Egan is often cited as one of the current greats when it comes to hard sci-fi. His novels explore some very far out ideas in terms of how the world may work, but he sticks to the consistency and really explores the politics and consequences of that universe.
Clarke also does this, but to a somewhat lesser extent. In many of his stories, the world and its rules are the main character, and the actual beings are the supporting characters.
In other words, I thought "hard sci-fi" means fiction that deals mostly with fictional facts in fields like physics, astronomy, geology, and biology, while "soft sci-fi" means fiction that deals mostly with fictional facts in fields like psychology, economics, and political science.
It’s not exactly about rules consistency either as stated by the GP, though that’s part of it. It’s more about strong consistent application of scientific principles even theoretical or untested principles.
This is in contrast to futuristic fantasy with no real focus on the science. But futuristic or space fantasy can be very consistent just like magical systems in fantasy can be very consistent. Hard scifi has to be constrained by plausible consistent science and that science is typically a main character in the story, or even THE main character.
https://en.wikipedia.org/wiki/Hard_science_fiction
> The complementary term soft science fiction, formed by analogy to the popular distinction between the "hard" (natural) and "soft" (social) sciences,[6] first appeared in the late 1970s. Though there are examples generally considered as "hard" science fiction such as Isaac Asimov's Foundation series, built on mathematical sociology,[7] science fiction critic Gary Westfahl argues that while neither term is part of a rigorous taxonomy, they are approximate ways of characterizing stories that reviewers and commentators have found useful.[8]
https://en.wikipedia.org/wiki/Soft_science_fiction
> The term soft science fiction was formed as the complement of the earlier term hard science fiction.
> The earliest known citation for the term is in "1975: The Year in Science Fiction" by Peter Nicholls, in Nebula Award Stories 11 (1976). He wrote "The same list reveals that an already established shift from hard sf (chemistry, physics, astronomy, technology) to soft sf (psychology, biology, anthropology, sociology, and even [...] linguistics) is continuing more strongly than ever."
This is how I think of it as well. I actually don't really consider it science fiction, it just uses the scenery. Although I equally wouldn't say anyone who calls it "science fiction" is wrong.
That brings up another thing: I think a story can be science fiction without involving anything futuristic or space-related at all.
Whereas something like Foundation is about the exploring the concept of psycho-history and galactic civilisations and the people are there to move that story forward.
https://www.imdb.com/title/tt0390384
"The Man from Earth"
I'd add "Her" and "Ex Machina"
It helped that Gene Roddenberry was in the Navy and that Trek was basically The Cold War In Space which is actually more of an interesting dynamic to me than Starfleet being basically invincible and everything running on space magic.
I'd also suggest a good look at Baxter's books and Robert L Forward's too.
I've jokingly called Forward's books science papers with a plot. Timemaster starts out with:
> There exist semieducated but obstinate people who have raised the concept of strict local causality to godhead, and attempt to use such words as "obviously" and "it only makes sense that ..." in an attempt to "prove" that their version of causality cannot be violated, and that any sort of time machine is logically impossible. From my reading of the scientific literature, they are wrong. If I receive a letter from this sort of person complaining about the "impossibility" of the time machines in this novel, I will throw the letter in the nearest wastebasket... unless the letter is accompanied by a reprint of a scientific paper published in Physical Review (or any other reputable, refereed scientific journal), written by the person writing the letter, which proves that the paper "Cauchy Problem in Spacetimes with Closed Timelike Curves" by Friedman, Morris, Novikov, Echeverria, Klinkhammer, Thorne, and Yurtsever, is erroneous.
My take on it is more of a "what is the focus of the story?" Is it the soft sciences? Psychology and sociology and politics ... and the Foundation.
Or is it one more of challenges met with the STEM disciplines of physics and astronomy and biology?
It's also not a "it is either a this or that." Some books can be both and I think that Clarke is a prime example of this. 2001 is as much of a story of psychology as it is about astronomical distances and the ship needed to accomplish that goal. The Songs of Distant Earth is about two cultures clashing ... with the real problems of engineering a ship to travel (it's a softer story than it is hard).
Protector by Niven likewise scores high on both the soft and hard scales.
Foster tends to higher on the 'soft' side of the scale, though sometimes it edges up there with some real biology and the limits of technology are sometimes real limits.
And so, it is "are the challenges that the characters surmount solved with tools developed from the soft or hard sciences?"
"Use the Force Luke" is not a hard science solution.
All the biggest scientific achievements always occur in Canada. His stuff's solidly in the fantasy category.
But the real dividing line in my mind isn't quite as stark as I made it sound. There's still a gray area. Unusual events that exist purely for plot purposes don't disqualify anything, for instance.
The differentiator I have in mind is more basic: if the story involves things that are simply not possible, it's not hard sci-fi. If it involves things that are very unlikely, but still within the realm of possibility, it can certainly still be "hard". Same if it involves things/effects that don't (as far as we know) exist, but wouldn't break the laws of physics if they did.
I think this is a vast oversimplification of how knowledge works. Like yes, no one knows with perfect certainty what the laws of physics or whatever else are, but that hardly means _anything_ is plausible or worth entertaining.
I amuse myself with the foundations of physics and I'm so sure that FTL is impossible that I find science fiction that uses the idea almost tragically silly, simply unwilling to grapple with the limitations imposed on us by the vastness of space.
I think you mean solar system. While nobody has traveled to Mars, it is reasonable to suggest that is only because we haven't tried hard enough yet. There are problems in the way, but they seem like engineering things that we will figure out if we try. It is debatable if we can make a self supporting colony on Mars, but it getting there seems perfectly possible in reach. Venus is harder (getting there is easier, but reaching the surface is questionable), but we could probably do it.
Getting just the farthest planet in our solar system though is getting close to a lifetime. The nearest star to earth is 4 light years away, we have no hope of reaching it in a lifetime with any technology we know works (there are nuclear options that seem promising but we don't know if they work)
The milky way is 100k lightyears. Not possible to cross without FTL.
Almost all my favorite books almost exclusively occupy settings with laws of physics that are internally consistent and believable and therefore have no FTL. See "The Expanse", "Three Body", "Blindsight", "The Sunflower Cycle", "Rendezvous with Rama" and too many others to name.
Generally I think the impact of instant communication and near-instant travel (or lack thereof) is not talked about enough. Most sci-fi stories derive from the present day, and keep this aspect of today (as do many fantasy stories, interestingly), but it's a narrative choice with large impacts.
Take FTL travel for instance. Sure, lightspeed is a hard limit, but we don't know yet whether wormholes are possible, which meanns warp drives are not impossible too.
But more than that, it's the effect that tech has on people that makes good sci-fi, not the tech itself. In this sense, even the technically impossible remains relevant. For instance, we'll (likely) never meet aliens, but most of us will one day be confronted with the themes of aliens stories, e.g. understanding people beyond our culture, dealing with the unfathomable, grieving our place in the universe, etc. "Hard"ness is just aesthetics.
Usually it also implies that it grapples with the nitty gritty details to "earn" the tech. Hohmann transfers vs. brachistochrone trajectories omg squee https://projectrho.com/public_html/rocket/torchships.php
https://www.amazon.com/Making-Atomic-Bomb-Richard-Rhodes/dp/...
A funny one is the Pern series, which starts out as a sword and dragon fantasy series, but then like 10 books in we find out it is actually hard sci-fi (!) with space ships, orbits, genetic engineering, computers, etc. But there is still that telepathic connection with the dragons…
It isn't about whether a technology is possible. "Soft SF" is SF where the actual scientific details don't matter. It is about people, their cultures, and their emotions. And really, just a metaphor for our own society. For example a story about how a future VR technology is addictive and causes people to interact less in real life could be just a metaphor for real life smartphone/Internet addiction. The story doesn't need to (and probably doesn't) explain how the VR tech works because it doesn't matter. "Hard SF" actually is based on actual science, and that is the focus of the story. It isn't that one is better than another, but they are different.