The issue is that, ultimately, the only thing we can observe is electromagnetic waves[1], characterized by frequency, polarization and intensity, and originating from some point on a 2D sphere. Everything else is interpretation[2]. So cosmology today consists of dozens of layers of interpretation stacked upon one another and for me (as a mathematical physicist who once dabbled in cosmology but is ultimately an outsider to the field) it's hard to keep track of all the assumptions and the causal chains of observations and interpretations that are implicit in today's widely accepted results.
To give an example, people say that the universe is expanding and they point to redshift (~ (apparent) velocity) measurements such as those done by Hubble. Great, but how do you measure redshift as a function of distance if all you see is a 2D sky and you can't "see" the axis parallel to your line of sight? You introduce various distance measures like angular distance and luminosity distance which are based upon the notions of standard rulers and standard candles. But these are not exactly trivial to get right, either[3], and AFAIU rest upon an entire body of theories from astrophysics and plasma physics. Plus, there are countless of things that can happen to electromagnetic waves on their way to Earth (absorption, gravitational lensing, Sachs-Wolfe effect, …) and you must account for those, too.
Don't get me wrong. I'm not trying to cast doubt on any of the currently accepted theories. Not in the least. Nor is the particular example above not well-understood. I'm just saying that, for an outsider, it's not easy to follow the chain of implications sometimes and many books don't do a very good job of logical bookkeeping. A book painting the architecture of the Standard Model in broad but logically accurate strokes would go a long way.
Meanwhile, I notice reruns of "Through the Wormhole" on TV every other year with Morgan Freeman going on about hypotheticals like string theory, parallel universes and such, while completely foregoing the (well-established but IMO still exciting) "basics". It's almost physically painful to imagine what additional confusion this must cause the layman that knows barely anything about physics in the first place. Any sufficiently advanced physical theory is indistinguishable from magic comes to mind.
[00] See the conundrum there?
[0] https://en.wikipedia.org/wiki/Galaxy#Observation_history
[1] and some ionized radiation, and now of course gravitational waves.
[2] "Everything is interpretation" not in the esoteric sense but of course in the sense that we draw logically stringent conclusions based upon already well-established physical theories.
[3] https://en.wikipedia.org/wiki/Cosmic_distance_ladder#Problem...