> I don't claim to have a crystal ball, but I can say which parts of the universe are hostile to science, and that can be summarised as 'that which we would struggle to measure'.
Are you sure that "struggle" is the right word here? We have struggled a lot to measure PET scans or the higgs boson, to take two random examples, and I suppose you would agree that neither of those are somehow "fundamentally hostile to science"? Somewhat hostile, sure, I guess you could say that, but obviously not sufficiently so to justify the conclusion that gaining knowledge is impossible!?
> Using your example of the person from the 10th century, whilst they would have almost certainly not been able to envision all of the things we would eventually come to measure, the principals of measuring the physical world were in place, and we would have known, if not precisely but intuitively, some of the key concepts behind the measurement of physical objects, such as speed, acceleration, inertia, friction, heat
But would they? I think inertia is a nice example for this: At the time, Aristotle's idea that the natural state of things is at rest and that things only continued moving if power was being applied to them was still prevailing. Inertia really was not a concept yet, not even a poorly understood one, people simply had a completely wrong idea about how things worked, even at this fundamental level.
Also, what about electric fields, say? Would you say they intuitively knew the key concepts behind measuring electric field strength?
> Whilst our understanding of the physical world would've been much cruder than today, at least we had a framework by which we could explore it (at the level of detail available to us before the telescope and microscope were invented).
Well, sure. But a lot of that framework that we have now was also still missing.
> Contrast this with the study of the metaphysical.
How? What is "the metaphysical"? How do you distinguish it from "the physical"?
> What concepts do we have that help us to measure the metaphysical?
Well, I think we first have to establish what the metaphysical even is before I could say how we could measure it.
> Perhaps some are around the corner, but at best all we can do at the moment is statistical analysis.
In other words: Science? What would be an example of scientific inquiry of what you would consider to be part of the physical that does not boil down to statistical analysis? (Mind you that for a useful comparison you have to start from the point of view of someone who doesn't have the scientific models that resulted from the inquiry yet--establishing those models was the result of the inquiry, not the starting point.)
> To clarify, whilst term 'metaphysics' could be used to describe anything that describes our world without necessarily being part of it.
I think something is missing from this sentence?!
> I gave the example before of society, which I still think is the most straightforward one to explore. Depending on your field of view, it's possible to say society doesn't exist.
Really, that is not a matter of view, but a matter of definition. I get the feeling that you are falling victim to essentialism here. Words don't have (intrinsic) meaning, they only have usages. That is to say, a word by itself does not somehow have any inherent semantics, the fact that words can be used to transmit ideas relies on a common mapping from words to concepts (aka "definitions") between communicating parties, not on inherent properties of words.
The word "society" happens to be used to refer to a whole lot of interrelated and overlapping but still distinct concepts, some of which happen to map to reality, some of which don't, that's all there is to it. If someone is using the word "society" to refer to some concept that doesn't map to reality (let's call this "society1" in the following), they can correctly state that "society doesn't exist", while someone else equally correctly can say "society does exist" by using a definition of "society" that does map to reality (let's call this "society2" in the following). You just have to be careful to not mix up the two concepts, even though they are being referred to using the same word. The fact that society1 does not exist does not say anything about society2's existence or vice versa--any conclusion of that sort would be an equivocation fallacy.
> For example, if a person lives a hermit lifestyle, are they still living within society, or does it take interacting with others to be part of a society?
Well, give me your definition of "society" and I can tell you whether they are. Without a definition, it's a meaningless question, as for some definitions, they are, for some, they are not, so there is no generally applicable answer.
As for the rest of what you wrote, I think it's a great example to try and use as the basis for the analogy I tried with a 10th century person, as I think that you are not appreciating the applicability of that analogy yet. Consider that same point that you are making, but try to replace the area that we are trying to understand with medicine instead of sociology, from the point of view of an educated person in the 10th century. What part of your argument couldn't equally be made by that person about medicine when basing it on the knowledge of their time?
You objected above that they knew the basics of physics, sort of. But even if we accept that premise: Were they even considering medicine to be a problem to be tackled with the tools of physics? Weren't demons a metaphysical thing that wasn't subject to physics? It might be just obvious to you now that the body is governed by biochemistry and mechanics, which thus in some way reduces it all to molecular dynamics, so it is obvious to you in hindsight that they maybe had the basic versions of the tools to investigate it all. But that was not their perspective, that is all hindsight. They were thinking in terms of humors. They were seeing overwhelming complexity. They were not aware that the ideas of speed and friction would "just" have to be generalized and applied to the human body to fix genetic diseases. The question is not what that person could have thought if they had combined the knowledge at the time in the right way (i.e., what you now know to be the right way), but what they would actually have thought, how they would have interpreted and applied what they knew.
As for the influence of a changing environment on scientific discovery, I think your perspective makes things unnecessarily difficult. You are conceptualizing a changing environment as a changing reality. But why not conceptualize it as a change in experiment? What I mean is: When society changes, and let's assume that it changes to a degree that existing knowledge about how society acts is not usefully applicable anymore, does that mean that reality has changed and therefore the knowledge is now wrong? Or isn't it rather that the knowledge is still perfectly valid, but its scope is limited to a form of society that just currently doesn't exist? When we replaced horses with steam engines, did that make our scientific knowledge about horses incorrect? Or wasn't it rather that our scientific knowledge about horses was simply not applicable to the source of power we used for carriages anymore, but still perfectly correct within its scope? The utility faded, because horses are not part of everyday life anymore, but did that change the correctness of the knowledge? Does that mean that it wasn't science?
Now, science obviously always strives to generalize and unify, so a theory of sociology that applies more broadly than what you describe certainly would be desirable. But that is something that applies to all areas of science and to all theories in existence, without exception. (The theory of) Electromagnetism wasn't always a thing. Electricity was one thing, magnetism another. Until people figured out about one and a half centuries ago that they could be modeled by a common framework, thus creating the model of electromagnetism that is more broadly applicable than either of them, and even more broadly applicable than the two simply combined. Neither the theory of electricity nor the theory of magnetism allows you to model antennas and radio transmission, but electromagnetism does. And further theorizing and investigation a century or so later led to further unification with the weak force to form the model of the electroweak interaction. The point is that there is no such thing as "the fundamental model", there is only always the current state of knowledge, and the history of science is filled with models that at some point seemed very fundamental but later were integrated with other ideas to form more broadly applicable theories of which the formerly seemingly fundamental models were just special cases. Experience tells us that considering some model fundamental is more often than not just myopic--all we can really tell is that it is more fundamental than what came before, but that does not mean that it's the end of the road. Therefore, I don't see how that could be a criterion for distinguishing the scientific from the non-scientific?!