Comment by gnicholas | Hacker News Reader

Comment by gnicholas | Hacker News Reader

Parent Full thread

This summer, I decided to build a better weather website with my 8 year old as an educational project. We have found that the weather forecasts in our area are terrible and decided we could build a more accurate prediction algorithm.

I haven't coded in 20 years (and never did any web dev), and I'm trying to figure out what tech stack to use so that my (1) it's comprehensible to my kid, (2) it's not too frustrating for someone as inexperienced as me, and (3) it allows us to get something off the ground relatively quickly.

Anyone have suggestions for what technologies to use, or if there are sample weather websites that we can look at and model after?

edit: just to clarify, I'm not talking about the tech stack for making predictions, just for the user-facing website. We have already developed an algorithm that predicts weather much more accurately than any weather website/app we've been able to find, in our local area. We are not looking outside our local area at this time.

i think you should probably start by reassessing your expectations. creating accurate weather forecasts is a huge industry worth billions of dollars, and is a very hard problem.

creating a better prediction algorithm is probably not something that's going to be achievable for an inexperienced programmer and their 8-year-old.

ECMWF is what you are after. It's usually better than GFS.

You could also let the user choose which model to use or show a comparison.

I'm doing something related and my next step is compare models vs current weather to see which I model is more accurate.

As models change and sometimes depend on current conditions this is something that you should never stop doing (comparing which model is better)

We'd like to have our website display comparisons, so people can see how much more accurate we are than other websites. Not sure whether there are IP rules that would stop us from displaying someone else's forecasts though. Don't want to land in hot water for some simple summer project!

You say you’re trying to build a more accurate prediction algorithm to beat meteorologists. Given that goal, you should probably look to acquire a supercomputer first; tech stack wouldn’t be a concern until then.

For our local area, it was actually trivial to develop a better weather prediction algorithm. We're not looking to build something for the whole world, which would be very difficult. At this point, I'm just trying to sort out the tech stack for the website, now that we have our prediction algorithm figured out. We'll of course optimize it over time, but even our v1 blows weather.com out of the water.

If you’re just building a website for the weather forecast of one place, you don’t really need a tech stack. Completely static HTML maybe with a couple SVG weather symbols updated hourly to daily will do. Every single programming language can do the simple HTML templating needed for that; at worst you just print some strings.

I was hoping to include a few neighboring towns, but I did wonder if I could use static HTML with a dropdown menu to select the proper town. I wouldn't need to update frequently, just a couple times a day. I'd love to find a template for something along these lines, since as I mentioned, I'm extremely rusty! I have found some "build a weather website" tutorials, and I was planning to start with one of those.

Im interested to know how weather forecasts can be trivialised or 'simple' in other words for a local area? It is still dependant on predicting what happens in a washing machine of pressure systems and weather patterns split between 2 hemispheres that are in different seasons.

Perhaps it's just that the existing weather websites are quite so bad! In recent weeks, the forecasted high has been off by as much as 15 degrees.

I had the same thoughts (that it would be easy to improve on my local area's forecasts) and considered toying with it too, just for fun.

Are there any resources you'd recommend looking at? Also, if you have any interest in sharing your algorithm I'd love to take a look.

Happy to chat — contact in profile! In terms of resources, I've just been looking at "build your first website" tutorials that happen to use a weather website as the example. We've been focused on gathering data to create/validate our algo up to now, so just turning to the tech stack as this new API is released. Perfect timing!

What technology did you use to implement your prediction algorithm? It might make sense to use the same language to have less moving parts. If you used Python, for example, you may consider using either Django or Flask for the web site.

Thanks, that's helpful! The algorithm is still in a spreadsheet at this point, so it's up for grabs right now.

Disagree, if he's only interested in producing a better forecast for his location.

Im interested to know how weather forecasts can be trivialised or 'simple' in other words for a local area. How is it easy to predict what happens in a washing machine of pressure systems and weather patterns split between 2 hemispheres that are in different seasons?

I'm not sure I understand the 2nd part of your comment, but:

from first principles, coming up with an all-purpose algorithm or generally-applicable model is going to be harder than making it work for a single "use case".

for weather patterns, some parts of the country (speaking of the US here since that's where I'm familiar with) have fairly predictable cold front movements from west to east. others have ocean interactions, lake effects, etc.

Well the weather by nature is a chaotic mess of pressure systems and weather patterns, a major cause being the junction between the hot and cold temperatures of the 2 hemispheres, no?

I say 'washing machine' as it is a swirling mass of unpredictability. Yes there are obvious patterns that follow the seasons, but forecasts are required to be much more accurate than predicting the weather by patterns of the season.

They need to be accurate to each day, predicting temp, rainfall, wind etc. etc. The only way to do that is to try and predict how this chaotic system will progress over a period of time. Thats why all these weather agency talk about their new 'supercomputers' able to create more accurate forecasts. The processing power required to build these models is enormous.

No matter if you are forecasting locally or across the multiple states or countries, it still depends on forecasting the outcome of this chaotic system.

Thanks for clarifying. And I agree with everything you've written. I'm (clearly!) not an expert. Would be fun to see this done as a challenge similar to the old Netflix recommendations ML challenges.

I just wanted to know if I was missing something, Maybe there is some way of simplify forecasts for local areas that I'm not aware of. I would be interested to know.

I expect there is if you constrain what we mean by "forecasts".

Like, for my purposes, I'd simply like to know the temperature trend for the next 3-5 days. Will each day's high be hotter or cooler than the previous?

Bonus points: what's the high temperature for the next 3 days (within say, 3°F).

Anecdotally, using these standards, this seems to be consistently wrong where I live for much of the summer. I don't pay attention in the other seasons as the weather is much milder.

Reply on news.ycombinator.com