A eulogy for Dark Sky, a data visualization masterpiece (2023)
nightingaledvs.com
nightingaledvs.com
Where I live (US East Coast), the weather can feel dramatically different depending on the humidity. Relative Humidity has always felt to me like a poor way of measuring how humid the weather will feel. For example, 50% RH at 84ºF will feel lightly humid and generally pleasant, whereas 50% RH at at 97ºF will feel like a swamp. The dew points at those respective points - 63ºF and 75ºF - do a much better job at immediately conveying how humid the air will feel.
Dark Sky used to show hourly dew point graphs that you could browse throughout the week and see when the humidity would break (or return). Apple Weather does show you the dew point, but only when you select a point on the RH graph. So to track the dew point over the coming week, you basically need to drag your finger over each day's graph and observe the changing numbers.
I think this is probably just due to the lack of general awareness about how dew point is a more elegant shorthand for "absolute humidity" than any other weather metric currently in use. I hope there will be more of us in the future!
- The dew point is independent from the temperature, so it is effectively a measure of "absolute humidity."
- While both measurements have their advantages, I find dew point a better indicator of the "mugginess." Also it is easier to estimate the relative humidity given the dew point vs. estimate the dew point given the relative humidity.
- You can compare both here: https://weather-sense.leftium.com (humidity plot disabled, but the value can be checked by hovering.)
(weather gov data is open/free, you can pull it down and plot anyway you want)
https://forecast.weather.gov/MapClick.php?FcstType=graphical...
Windy.com to the rescue for dewpoint lovers.
A very old version of their Android app had that view as a 3-day widget for the home screen. Right after it was bought they did a major upgrade and removed a ton of functionality, that widget included.
My web app plots hourly dew point for the next 24 hours, next 7 days, and past 2 days. (Still WIP.)
Inspired by https://merrysky.net (can also plot dew point), which was inspired by Dark Sky.
Also the toggle on hover for the day selection is not easy on desktop and very hard on mobile. I'd prefer a simple toggle.
I picked the Watercolor map style[1] because it looked pretty and the detail from other styles wasn't needed (like street names).
Then I picked The Weather Channel radar color scheme[2] because the green-to-red gradients seemed to contrast well with the Watercolor map style.
What map style/radar color scheme combination would you prefer? Perhaps I can make this configurable... (Is there any chance there simply was no precipitation on your map?)
---
I am not sure what you mean by "toggle on hover." Hover is meant to work without any toggling.
The yellow line indicates which stats are currently being shown.
Ideally, when the mouse cursor is over a timeline, it shows the stats for that time. Otherwise the stats for the current condition are shown.
Sometimes the hover is a little buggy; click + dragging on the timeline or click + dragging outside a timeline usually fixes it.
Usually it works better on mobile! If your finger is touching a timeline, that is considered hovering. Otherwise the current stats are shown.
The new one is way better. I'd like to see the streets and city names if possible because with just the watercolor I feel like I'm missing infos.
That should make the colored radar data easier to see!
Sadly, Naver stopped showing the past weather like that.
This was one of the main motivations for creating UltraWeather: https://github.com/Leftium/ultra-weather#readme
UltraWeather was missing some features like AQI and minutely rain predictions (coming soon!) so I made WeatherSense ^^
https://forecast.weather.gov/MapClick.php?lat=40.7482&lon=-7...
1. Go to https://www.weather.gov/
2. Put your zip code into the Local forecast box.
3. Click the Hourly Weather Forecast graph on the right side.
4. Bookmark it.
- The past three hours are shown in the 24 hour timeline
- The timelines for the past two days are shown
- (rendered slightly lighter/transparently)
Up to 92 past days can be shown, but this is currently hard-coded[1] and not configurable (yet).
[1]: https://github.com/Leftium/weather-sense/blob/55b505a7efae65...
EDIT: NVM, found it. You tap the temperature.
EDIT 2: It really ought to remember the units. 95% of the planet doesn't ever want to see Fahrenheit.
Eventually other unit types will also be implemented. URL params and cookies will also be added for persistence.
Right now, I chose to default to the most "sensible" units:
- Fahrenheit for temperature
- Millimeters per hour for precipitation
WeatherSense is still a work-in-progress!
---
Also: WeatherSense is designed to be used without numbers. Get a feel for the readings by looking at the shape of the plots and comparing to the current/past conditions.
Thanks for teaching me about dew point (and to the commenters below you for letting me know there's a free Dark Sky clone - merrysky.net - and a dew point graph on Weather Underground)!
For me, I've written myself a Google Apps script - to be more accurate, AI did, but with my interaction - that e-mails a "humidity forecast" as to how wet the next six days are going to feel to me in a simple way. I don't know who came up with the categories I've placed them in - it seemed to be common weather information.
https://github.com/mollyrealized/humid-fcast/blob/main/Code....
Carrot Weather has a mode that is almost a replacement for Dark Sky, btw, though the location-based alerts have gotten less and less reliable over the years (which I am completely unable to understand).
Also, FWIW this post is from 2023, and the Weather app has improved significantly since then.
For example, the post references a Reddit comment complaining about the lack of a precipitation map:
> there anything that has the precipitation graph similar to dark sky
This is available in Weather app. You can see the hourly graph, and you can also see the map with precipitation overlay.
The notification feature you ask for is also supported.
I repeat my original question: can anyone actually name a feature from DarkSky that isn’t present in the current Weather app?
I had turned mine off, probably from way back when I used Dark Sky and didn't want redundant notifications.
But, I don’t have any domain knowledge in this area. Perhaps, it is just more difficult due to climate change or other factors.
Things like the funding of NOAA or reallocation of funds within it can impact NOAA's ability to provide the data that is used to drive this (and other) weather and climate decisions. https://www.federaltimes.com/opinions/2024/04/30/noaas-budge...
I personally find, I’ve had to learn how to read a weather map to get a good understanding of forecasts
Is this done via some automated system, or do pilots report it manually?
I’d be curious to read more about this - do weather companies have partnerships with airlines where they pay them to carry sensors?
> do weather companies have partnerships with airlines where they pay them to carry sensors?
Nearly all of the US's forecasts come from the NOAA. Nearly every other "weather company" is simply reformatting this data.
I can't remember exactly what video I watched on the topic, but these two may be interesting:
As others have already mentioned, the fact that it displayed weather information in a clear and uncluttered way, and had a simple user interface. Apple Weather is a gaudy mess.
There was also a "Check out Cool Storms" feature that let you see the weather map for an interesting storm that was happening at the moment.
Notifications were also more configurable. You could set it to notify you if the precipitation chance were higher than a certain percentage or if the high were below or above a certain temp.
Easy. As the screenshots show, the DarkSky main page would _dynamically adjust_ the information it shows you: adding radar, precip level, combined precip/temperature forcecast, etc. The fact that those elements can be found individually by poking through three or four screens in the Weather app doesn't make them equivalent.
I sometimes think that the reason Dark Sky was so good is that it spent a lot of money on the data side of things and probably refreshed their data and models much more than a larger company would. They were probably burning their money quite fast. As a function of the acquisition Apple tried to reduce the spend on data/processing while keeping the functions (it didn't succeed). Weather app has the same problem.
All in all - seems like a well traveled road: nimble company trying to acquire new customers/market has better features until large companies take over and drive profit/revenue - diluting quality and pushing away customers. (different incentives)
The features may be there but they aren't well implemented.
Diluted for the masses.
Is there a meaningful difference between 25% cloud cover and 35%? Or is it better to just give names to the “conditions” at buckets of 25%, 50%, 75%, etc?
I've not seen other descriptors, and to view it changing over time its just on the timeline as a sun, sun+cloud & cloud emoji's
It's not useful other than as a binary 'is it cloudy' which in Bay Area weather it is a meaningful difference between 30% and 60% coverage
For example (and it always takes a bit for me to find it) https://www.yr.no/en/details/table/2-4887398/United%20States... (and https://www.yr.no/en/details/graph/2-4887398/United%20States... )
And here's their API - https://developer.yr.no/featured-products/forecast/
It breaks it down by overall, low, medium, and high clouds.
A 95% coverage day is a bit different than a 100% coverage day. Particularly, when it comes to rain and wind expectations.
Admittedly this information is not surfaced in any way like “other users say it’s raining.” And “report an issue” isn’t a great name. But it does say they incorporate user reports into their data.
I just tried in Apple Weather, and the process was:
1. Tap on the hourly forecast, or the day, to go into the graph screen
2. Tap on the dropdown icon
3. Tap "feels like"
4. Either drag your finger along the graph until the time indicator at the top indicates you're close to 3PM, then read the temperature, or you can try to read it directly off the graph, but the axes aren't labeled clearly enough to make this feasible
1. Scroll to the "Feels Like" tile
2. Tap it
While I understand the UI complaints others have, for the handful of things I normally want to see I find Weather fine enough vs Dark Sky. BUT, the accuracy took a noticeable downward trend.
I don't believe Weather or Dark Sky have had hyperlocal weather forecasting. I would absolutely love such a feature (since I live far from the center of town in an area with microclimates); I would say both were equivalently faulty on my weather forecasts.
I'd say there's about a 80% chance that it loads the forecast in under 5 seconds.
Loading the radar has far worse performance - it only works ~50% of the time and failures just show an empty map forever.
When it does manage to load the weather I've found the accuracy to be hot garbage.
The fact that Apple chose to shut down one of the best apps on the iPhone in order to promote Apple Weather is still really irritating. I keep Darksky in the app graveyard on my phone, next to Apollo.
It's included now, but still not as well as Dark Sky did it.
For an app not trying to reproduce Dark Sky, but doing a nice job with an overview of the day's weather, I've been using Overlook. But now I'm seeing that its app store listing is gone. https://apps.apple.com/app/overlook-weather/id1639571738
Ongoing customer value (hyper-local precipitation forecasts) that has ongoing costs for the developer (weather data) is unsustainable without ongoing revenue (subscriptions). This may have had a thing or two to do with Dark Sky ending in an acquihire.
While the radar view is nice, the outlook view is very nice.
From a screen shot of the screen shot in the App Store: https://imgur.com/a/U8o1DJw and my own outlook
The 'ring' data view at the top (and bottom of the second image) is a nice way to represent the data in a limited amount of space (phone / watch widgets).
> Data Used to Track You
> The following data may be used to track you across apps and websites owned by other companies:
> Location
> Identifiers
> Usage Data
Which view do you mean? I was a longtime Dark Sky user, and now I switch between Carrot and Apple Weather.
Besides the API, besides the hyper-local (and, in my experience, _always_ accurate) forecasts, besides the excellent visualizations, besides the nice web app, what I miss most was the history.
I loved the history. It was perhaps DarkSky's least appreciated feature.
You could search (at least within the US) for any date in the past few decades, and find the temperature and precipitation and whatnot for a given location.
You could see what the weather was like on those important dates in your life. You could see what the weather was like the day after those important dates in your life.
It's a little bit of memory that's been excised from the commons. It still makes me sad.
Any clue why? Did broadening the pool of reporting devices from Dark Sky users to the general iPhone population somehow break their models?
But IIRC, DarkSky stopped using phones altogether some time before Apple killed the service. I can't find a source on that given Apple also deleted DarkSky's blog, so, grain of salt.
I'd go as far to say that the absence of past weather data in most weather apps is symbolic of the wider state of design today. Anyone with any real design knowledge, any basic capacity to synthesize argument or think about problems, would find it obvious that showing past weather is useful because it provides context. So many statistical apps (so-called "dashboards") are such drek because of this too.
I’m using the weather app now, but am still longing for the spiritual successor to Dark Sky.
I had a similar feeling when Google bought and killed Sparrow. They should have simply replaced the Gmail app with Sparrow, instead, they killed the only email client I ever genuinely enjoyed using.
No excuse to not just buy Dark Sky and let them continue unburdened by corporate politics
Pirate Weather is the backend for the Dark Sky-like Merry Sky: https://merrysky.net/
[1] https://web.archive.org/web/20191210071310/https://blog.dark...
(For what it's worth, I never used the Dark Sky app directly. I've always consumed it via Carrot or a free API key that Dark Sky used to give out for individual developers.)
Where I am big thunderstorms are very common and it can be dry a quarter mile away. It’s a good indicator but I never found Dark Sky, Accuwhatever, or Apple Weather to be accurate with the rain forecast.
Many times it would say something like Rain stopping in 10 minutes. Then 8 minutes later… Rain stopping in 15 minutes. On and on as it steadily rained.
There have to be dozens of devs in apple who would love to be on the 1-2 person team it would take to maintain it. (It was a 2 person startup for years, don't come at me with how hard stuff is.) It could even be a reward for good service, "ok you successfully mucked around with weird EU privacy law in the health app for 2 years, instead of a sabbatical for therapy how about you get to work on Dark Sky for a year?"
How dare you want information!
And if that information does still exist in the (public) Apple Weather API, why hasn't anyone (not just some Apple Engineer) just created an app with the views people care about?
https://news.ycombinator.com/item?id=35268026
> Meteorologists seemed to feel that Dark Sky was a graphics processing tool, not a weather app.
---
“Any weather forecast beyond a couple of hours ... depend on supercomputer models that work according to the laws of physics ... But when we talk about Dark Sky, all it was doing was taking the visual input of the radar and extrapolating what was going to happen over the next couple of hours.”
I've read that is what the underlying precipitation "models" did as well, but obviously can't confirm.
The rain nowcasting feature that Dark Sky popularized is now table stakes in any consumer weather app. There's little value in making these types of forecasts any more complex (e.g. using AI or other contemporary techniques) because they still have egregious and noticeable failure modes. And it's so trivial to make this type of forecast that there is open source software you can easily run to do it [1].
The difference is that 15 minute rain forecasts don't need that to be usefully accurate.
A model of equivalent simplicity would be about as accurate as the farmer's almanac for a 10 day forecast.
Simpler models are fine for very short term forecasting which is what they were doing.
This is just weather modellers getting annoyed because someone else is showing them up - not with 10 day atmospheric pressure forecast accuracy, which only weather modellers care about - but with "is it going to rain in 2 hours" which normal people really care about.
The morning of the eclipse it was a question of "this town or that town or that other town at this time."
This may be an edge case, but there are certainly times where prediction for a certain time 24h or 48h in advance is useful.
It wasn't necessarily "I want to know the weather for this block" but "I want to know the weather for this square mile."
One of the features that I liked in Dark Sky of very old (it was removed well before Apple decided to buy them) was the Lines feature which showed the models that it used. Knowing how to get at and read the models themselves (and asking the weather geek who I work with about the strengths and weaknesses of each model - "well X is good, but it over predicts precipitation and cloud cover within 50 miles of a lake that can produce lake effects, but if you're not near a lake its spot on for...").
Dark Sky's hyper-local weather predictions weren't based on projecting hours into the future. They were based on very small-scale extrapolation and interpolation.
The meteorologists with their supercomputers aren't, to the best of my knowledge, even trying to do that. And Dark Sky wasn't trying to compete with their models, either. For that kind of long-term forecasting, it was simply passing along the forecasts that meteorological services used those models to produce.
These days I go to windy.com for my weather nerd needs but it's quite different from Dark Sky.
I check it a couple times a day, and it's good enough for my needs. My bar's pretty low, though. The weather is rather unpredictable where I live; IME there's really no such thing as an accurate hourly forecast more than ~12 hours out. On Sunday night, today's forecast showed heavy rain and thunderstorms. Currently, it's 85 and there isn't a cloud in the sky. No amount of good dataviz can make up for that.
If you lived somewhere where you might take weather forecast accuracy for granted, I can see why you'd hate Samsung's app. It does the 24-hour hourly forecast and the 10-day daily summary just fine, but that's it. It'll show you current air quality and humidity, but no forecast data for those. Also, if you tap anything to see more details, it just launches an AccuWeather web page.
which they seem to do for prolonged periods or won't load at all, no matter the hardware and internet connection (tested on: iPhone 8+, iPhone SE 2020, iPhone 15 Pro - makes no difference)
Most infuriating thing they do (while we're at it): load everything except the area i'm interested in. I can't even...
I'd do it but I know I don't care about the weather like this. I just ask my voice assistant in broad terms. So I would suck at building it. But you guys all care. Why is it non replicable?
None of this amazing innovation comes from big companies
Now it’s swallowed up, resting and vesting
It also has a ‘Dark Sky emulation’ mode for everyone who loved the now defunct app.
I use Foreca, which consistently ranks among top 3 services on forecastadvisor.com, and Pirate Weather & Open-Meteo, which are two awesome indie services that access and enrich the best public models.
Here is how it looks like: https://imgur.com/gallery/Rduy7qc
Is there no way to use the app (with free data sources) at your own location? Right now it appears to choose a random demo location. It’s a pity because I don’t use weather apps that much to justify spending money but I appreciate the app nonetheless.
It does sound like Dark Sky had one useful feature that has no match - analyzing weather radar and sending personalized rain alert push notifications based on your location. That's pretty awesome.
put your lat/lon on the end
(google HQ) https://forecast.weather.gov/MapClick.php?FcstType=graphical...
(amazon HQ) https://forecast.weather.gov/MapClick.php?FcstType=graphical...
The data is open/free, you can pull down the raw data and make it look however you want.
I just wish there was an easy website to get Google's new AI weather which supposedly is far more accurate (only on certain android builds?)
Wetter is more detailed and complicated to read as compared to Weather Strip but I like the information such as CAPE and pressure info.
They also have a web interface, if you want to check it out, though the app is better: https://www.meteoschweiz.admin.ch/lokalprognose/zuerich/8001...
- OpenWeather for a dense, one-page view of short-term upcoming conditions (temperature, precipitation) and the weekly forecast.
- Shadow Weather for extra details: cloud cover, wind directions, etc.
Discussion then: https://news.ycombinator.com/item?id=35263115
A eulogy for Dark Sky, a data visualization masterpiece - https://news.ycombinator.com/item?id=35263115 - March 2023 (251 comments)
Dark Sky is basically just fancy visualization for csv/json data from a weather API.
Merry Sky is a good homage but I've never evaluated it much for accuracy: https://merrysky.net/forecast/new%20york/us
I agree with you on Apple Weather - its predictions are nothing short of terrible. It's incorrect so often that it almost seems like a random prediction would be correct just as often.
I didn't live here when Dark Sky was still good, but Dark Sky was incredibly accurate when I lived in the Bay Area.
I'm kind of skeptical about that, given that the Bay Area has relatively poor radar coverage. The local NEXRAD site is near Mt Umunhum south of San Jose and is quite elevated, so the lowest scanning tilt has limited coverage below ~4,000 feet over much of the SF peninsula and into the Golden Gate. The consequence is that shallow maritime convection can be poorly observed by the radar, and you can frequently have low cloud decks that produce noticeable drizzle or light rain (although possibly not greatly accumulating) across the city and surrounding area wihtout seeing anything on radar. Since Dark Sky wasn't much more than re-packaged NEXRAD data, it has a GIGO problem - if the radars don't see rain, Dark Sky won't predict anything for you.
The exception are the large storm systems that come ashore a few times per year in the Winter and Spring. Those systems behave extremely linearly, so they were "easy" for the algorithms that Dark Sky used to process the radar data. That ease of analysis combined with infrequency probably skews the perception that Dark Sky performed well in the Bay Area.
For what it's worth, I was involved in a study that analyzed the performance of several consumer and enterprise products' performance for reporting and forecasting light rain specifically in the Bay Area, and Dark Sky was indistinguishable from other data products that very obviously used raw, unprocessed NEXRAD data.
I think my only response is that my bar for "incredibly accurate" has gotten very, very low, given the performance here where I'm living now.
I obviously haven't done any kind of quantitative analysis, but I wouldn't be surprised to find it's genuinely worse than a coin toss.
Oh, I should add that I was living in East Bay closer to Walnut Creek for much of my time there. If I understand your point about the radar coverage correctly, I expect the Oakland hills topology would interact with that limitation somehow?