Here's a fantastic intro from Red Blob Games: https://www.redblobgames.com/pathfinding/a-star/introduction...

It just keeps exploring locations adjacent to locations it's already explored.

It chooses the next location by whatever one has the lowest "length of best known path to get there + estimate for distance to goal". The estimate for distance to goal is usually something like "distance as the crow flies".