You are paying extra for the service the gas station is providing which is that they shipped and stored the batteries in a location close to your home. The gas station is probably also open longer than the big box store.
A) they are closer to where you live. And B) in smaller unit packs. Maybe you only need one battery, but to get them cheaper (per unit) you need to buy an 8-pack.
If a one pack is $2, but an 8-pack is $0.50/ per battery. If you only need one, you’re still better off paying four times as much ($2 < $4).
It’s all about the convenience...
Did you mean "convenience makes the price higher"? No quarrel there! The issue is how much higher. 8x more expensive (as an example) is in the "ripping me off" territory, if you ask me :)
Wait until you find out the cost of producing most non-alcoholic beverages...