Hacker News Reader

Comment by bisby | Hacker News Reader

"We need your location to give you accurate weather readings for where you are. We need internet access to fetch the weather data."

Weather apps also have plausible excuses for requesting permissions.

View on HN

Weather data is so tiny that there’s no good reason to not just fetch the whole weather point-map for your country and then select from it client side.

I just had a look at the per-app data usage on my phone. Out of a total of 5.7GB of data usage, the weather app used 12.1MB of data. Sure, that seems tiny. But I live in Canada where there are over 4000 weather stations. Multiply that by 12.1MB and you get 11.8GB of data usage for the weather app, dwarfing all other apps. Either the fidelity of weather data would have to be reduced, or the update frequency, or a smaller selection of weather stations would have to be downloaded from (those in my area), defeating the purpose of fetching the whole map (hiding my location info from the server).

I have a feeling that weather data compresses together well — especially if stations 1. are ordered along a space-filling curve geographically, with read-outs of geographically-neighbouring stations in sequence; and 2. are stored column-oriented where each future time is its own column. Basically the same optimizations as you would use to store any other IoT time-series sample-cloud data.

I suspect that, encoded this way, weather for 4000 locations wouldn’t be much bigger than weather for 1 location.

Also keep in mind that half the reason the Weather app needs to use as much bandwidth as it already does, is that whenever you move even slightly, it can no longer give you accurate info without grabbing the point-forecast again for your new location. It uses far less data if you just stay at home all day every day. If it had a pre-cache of forecasts for the entire local area, it wouldn’t need to do that; it could just refresh once every few hours. Maybe even wait for you to be on wi-fi before doing so, if it has modelled you as usually connecting to wi-fi several times daily.

I think the bigger issue is the update frequency. My weather app updates minute-by-minute, giving me essentially real-time precipitation information from my neighbourhood weather station. If each future time is a column, you're looking at 60 columns just for the next hour of forecast. It really adds up!

This level of detailed information can't be pre-cached, either. I believe they are feeding the real-time weather recordings from the weather station into their supercomputer model to generate minute-by-minute high resolution forecasts over the next 6 hours.

Look at the METAR data pilots use for an example of how easily weather data can be compressed.

KSEA 020153Z 34008KT 10SM FEW075 OVC090 18/14 A2997

Here's the local conditions at Sea-Tac at 7:15pm Pacific time. We have wind direction and speed, visibility, cloud conditions, temperature and dew point and barometric pressure in one small snippet.

The forecast data for the next 24 hours is given similarly. Radar data would be more complex but the basics for any weather app are only a handful of bytes.

I wonder how big METAR weather data and forecasts for the whole of the US would be to download?

I can't find the source but I believe there are about 4000 METAR sites. There are plenty more small airports that will issue a weather report but they are pulling from the closest site that actually records METAR data.

Are you sure that is 12.1MB of weather data, and not 40kB of json and 12.096 MB of ads and tracking data going back to them?

For example, a full month of hourly weather data for a single Canadian station is 131KB (pulling from the Climate Canada site), meaning that if you wanted an hours worth of data for the ENTIRE country you're looking at less than ~800kb.

But if you just limit it to a geographical subset, say the lower mainland of BC, or even just a single town, then you can eliminate pinpoint data about yourself. Knowing that a person is in Abbotsford is far less invasive than knowing that they went to the Shoppers Drug Mart at 8:46 PM after searching for cold medicine.

This isn't a black and white thing, I wish that I had more control over my iPhone. I would love to be able to tell it to report a general location (somewhere random within a 10km of my actual location) to some apps, and a precise location to others. I get that Wunderground has server bills to pay, and I'm not paying them, but I would imagine that there is still a viable ads based business model that doesn't get quite as invasive.

I use the built in weather app on my iPhone. I’ve never seen any ads in it, so I don’t know what “ads and tracking data” you’re referring to. The app does update on a minute-to-minute basis when it’s raining out, though. It looks like this [1] and it doesn’t just scroll left on a local timer. The magnitude of the predictions in the graph change on a minute to minute basis and can disappear entirely, giving a very nice prediction of when the rain will stop so that I can go out and expect not to get wet.

[1] https://iphonesoft.fr/images/2021/02/meteo-14.5-ios.jpg

> I wish that I had more control over my iPhone. I would love to be able to tell it to report a general location (somewhere random within a 10km of my actual location) to some apps, and a precise location to others.

The iPhone (actually iOS) allows you to do exactly this. You can choose precise or general location on a per-app basis.

Though I enjoy that apple at least let’s me give imprecise location to most maps. Would be nice if I could set it myself to X kilometres.

Using the built-in iOS app for nowcasting has been good enough for me.

I can look out the window and see what the weather is where I am now. Beyond that I am interested in the weather for my general area over the next couple of days, which is imprecise enough anyway that my exact location doesn't matter.

Can you tell whether it's going to be raining in 30 minutes? Can you tell whether it's going to be 10 or 22 degrees later today when you're up at 7am?

I definitely can't do either, and ive been wrong enough times to know that

Well no, but the daily forecast gives me a good enough approximation. If I know the temp in the AM when I get up and I know the forecasted high, I can get a pretty good guess of what midday temps will be. If a massive front is moving through that will lead to significant changes, well the forecast will capture that too in wind and precipitation forecasts.

As for the next 30 minutes, I have tried AccuWeather and DarkSky. Both get the timing wrong about as often as they get it right for my location.

Those daily forecasts are highly localised though. I live in a city with half a million people that has its own little microclimate because we're surrounded by large hills on 2 sides and ocean on the other two. A general forecast for even 20 miles around us would almost certainly be wrong.

> As for the next 30 minutes, I have tried AccuWeather and DarkSky. Both get the timing wrong about as often as they get it right for my location.

Anecdotally here the met office is right far more often than it's not.

I distinctly remember a time when all we had was tomorrow's forecast on TV after the evening news and/or today's forecast on the radio before the morning shower. And before that, it was mostly looking at the sky and listening to our intuition.

We were _fine_.

Im not going to claim that everyone needs live access forecasts all the time but to claim that there is no benefit to having live access to forecasts is just silly. Sure you were fine, but that doesn't mean that what we have now isn't better.

> 's forecast on TV after the evening news and/or today's forecast on the radio before the morning shower

I don't know about you but I certainly don't want to watch the news or listen to the news and traffic reports to get the weather. I'm _very_ glad we've moved past that

> And before that, it was mostly looking at the sky and listening to our intuition.

Really, your argument is that things were fine before we had instruments so we don't need them?

>I distinctly remember a time when all we had was

Sure, and I remember a time before internet (widely available public access) and smart phones. We were also fine.

Which doesn't mean these advances (including forecasts) aren't useful.

I use the 6 and 12 hour forecasts every single day, personally. Simple stuff like - is it going to rain while we go to the playground, what's the UV going to be while we're at that outdoor thing, how cold is it going to be after I go to bed and do I need to close some windows... that sort of thing.

Reply on news.ycombinator.com