Can someone explain to me why weather data is seemingly hard to get/provide? Isn’t data from the US government (weather.gov) in the public domain and completely free? What did Dark Sky and similar provide on top?
It collects raw weather mode data and redistributes weather forecasts with simple APIs
Briefsky is also using it :)
The US provides free updated every 15 minute forecast for an absurd number of weather variables for every 3km square of the US in a nice GRIB file package. It is the gold standard on the free domain. Want a real api? Be prepared to pay real money. All Darksky did on top of this was interpolate that forecast data across time and the map.
Without an actual meteorologist on hand to guide the process, the forecast is probably going to be bad. NOAA and the weather community are just fine with the bar being high for entry. A proliferation of low effort/poor forecast web sites would just lead to them removing the realtime data from the public domain.
They make it all available at https://www.noaa.gov/nodd/datasets, but as noted - since the data is so incredibly large and NOAAs already running stats on it... you can't really outcompute them.
[There are very very few CS applications with both a) more computing power and b) more real world impact than predicting weather and hurricanes. Maybe cancer research.]
20TB source: https://www.nesdis.noaa.gov/news/data-dive-five-noaa-databas...
For the others, I tried to document the main limitations here: https://github.com/vsergeev/briefsky#unsupported-providers
Personally, I've found weather.gov APIs hard to work with. There's no geocoding included, and you can't even use lat, lon directly. But the bigger dealbreaker is that I have about a ~10-20% success rate querying any given endpoint, where sometimes I just get "An unexpected problem has occurred." And this is an automated twice-a-day query, so I don't think I'm hitting any rate limits...