Longer-range bomber flights also did it the same way ocean-faring ships have done in the past few hundred years: they had a roof window for taking celestial fixes with a sextant. Come to think of it, that's how the Apollo capsule also confirmed its location.
There's a lot to read about this. This is probably a good start since it branches to many other related articles: https://en.wikipedia.org/wiki/Oboe_(navigation)
Isn't that still how it's done? - When leaving earth orbit there isn't GPS or such, the guidance to the moon or further still uses star trackers.
And there is a quite-famous scene in Apollo 13 where they literally line-up the Moon itself in the reticule so that their engine burn puts them on exactly the correct trajectory.
How accurate a missile needs to be is a whole other dimension though. If the value of a missile is as retaliation to destroy a city (countervalue) then it can be a large warhead and "miss" by quite a margin but needs some form of credible survivability of an enemy first strike. If the missile is to be used to destroy enemy military installations (counterforce) then it needs to be a lot more accurate but usually the implication is as a first strike so less survivability is required.
When you have nuclear weapons that you can drop from manned aircraft, ICBMs from silos and SLBMs controlled by different military branches there is going to be a lot of politics over what the missile is for, which will determine its required accuracy, which will be a factor on if it needs star sighting.
I believe several of these were also used by civilian aviation before and after the war
TL;DR: both sides used systems with multiple ground antennas that allowed pilots to essentially triangulate their position or at least know they're in a given lane.
One thing i find interesting is the fact that by sending beacons ahead of the bombing party, they literally broadcasted their intentions to the enemy but that didn't seem to worry them too much.
[0] https://www.goodreads.com/book/show/1127842.Most_Secret_War
https://en.wikipedia.org/wiki/Ordnance_Survey_National_Grid
https://en.wikipedia.org/wiki/Military_Grid_Reference_System
Non-Directional Beacon (NDB) / Automatic Direction Finder (ADF) - Introduced 1930s-40s, standard in WWII. Ground station sends signal, aircraft needle points to it. Pretty simple but not super accurate. Still technically operational but barely used anymore. PPLs still learn it in training but honestly who uses this regularly? It's like knowing how to use a fax machine - technically still around but why would you?
Very High Frequency Omnidirectional Range (VOR) & Distance Measuring Equipment (DME) - These work together - VOR gives you direction from a station, DME tells you distance. Developed in the 1940s, and surprisingly still pretty common. Lots of GA pilots use these as primary or backup nav. Created all those "highways in the sky" that formed the airways system. VOR is definitely the most useful of the old-school nav aids, and you'll find these receivers in most cockpits.
Inertial Navigation (INS) - Commercial airliners got these in the 1960s. No external signals needed - just measures acceleration to track position changes from your starting point. Completely self-contained, works anywhere on Earth. Modern versions use laser gyros instead of mechanical parts. The drawback is drift - errors add up over time (1-2 miles per hour). Still used alongside GPS on long-haul flights.
US Postal Arrows: Also, i've not seen them myself - but the US in the 1920s before radio navigation, the US built a system of MASSIVE concrete arrows (up to 70 feet long!) across the country to guide airmail pilots. This was officially called the "Transcontinental Airway System." They were painted bright yellow and paired with beacons/lighthouses for night flying. The system eventually stretched from NY to SF.
Most got dismantled during WWII, but dozens of these concrete arrows still exist today scattered across the western US. It was literally a "follow the yellow brick road" situation for pilots. They'd fly from arrow to arrow during daylight, and at night there were gas powered lights showing them. More info here, https://www.core77.com/posts/25236/what-are-these-giant-conc...
It worked, but the usability of the tiny keyboard was crap in a bouncing, vibrating Cessna 152, forcing me to look down at my lap while flying. My instructor once said "you're going to kill yourself using that thing" so I eventually just gave up and bought the electronic E6B.
Do you still have the code? I'd love to see how you wrote it.
Try using SkyDemon or ForeFlight on an unmounted mobile phone, checking your way point and dropping it under your seat, whilst flying in congested airspace. It's a mistake you only make once :o