For some fun culture and history in Reykjavik, book a cooking class with Salt Eldhaus. Had a great experience there, the hosts talked about how much Iceland has changed from when they were young, got into the history of some of the dishes we made.
Hakarl is a tourist gotcha, it's really not bad to eat at all, even outside of Reykjavik. Not my first choice, but you can see why it got them through long winters. If you have time, stop in Selfoss for the Skyr Museum and yes, definitely eat all the Skyr you find, it is delicious.
Friðheimar, the big tomato greenhouse farm, is worth the drive. The food was good, walking around the grounds was super chill.
We went during Oct of 2023, so the weather was between 40-50F and slightly rainy, but nothing we were not used to being from the northern parts of the US. Pretty decent time to see some Aurora Borealis though. You often can't see it with the naked eye unless geomagnetic conditions are absolutely perfect, but the camera on your phone can pick up things your eyes cannot, so don't be afraid to just snap pics of a seemingly blank sky.