Weather Service faces bandwidth shortage, proposes limiting key data
washingtonpost.com
washingtonpost.com
>>> The Weather Service held a public forum Tuesday to discuss the proposal and answer questions. When asked about the investment in computing infrastructure that would be required for these limits to not be necessary, agency officials said a one-time cost of about $1.5 million could avert rate limits. The NOAA budget for fiscal 2020 was $5.4 billion.
Buchanan, however, stated the actual cost to address the issue would be higher because the $1.5 million “would comprise just one component of what has to be a multifaceted solution.”
The officials at the forum also said that senior management at the Weather Service was aware of the relatively small cost of addressing the issue but that the agency faced “competing priorities.” <<<
So, the cost to fix would be peanuts, but it's "not a priority". Sounds like pretty typical corporate bureaucracy against improving infrastructure and efficiency until they're forced to.
Using their numbers, it'd still only something like .2% of their budget, if the cost was 10x the $1.5m figure.
But there are probably thousands of things that cost only peanuts and aren't a priority. If you fix them all they add up to not peanuts and then you don't have any money left and you didn't even fix any priority tasks!
These forecasts directly affect everything from air travel to crop planting/harvesting to sea travel. They are critical for our economic (and citizen's) ongoing health.
And the internet, for better or worse, is the primary means of dissemination for the NOAA. Failing in this is failing at one of their greatest duties, IMO.
It’s got nothing to do with government supporting it’s self and 100% to do with legislated corruption that’s dependent on public officials playing their role.
And, it's been available to the public via the internet for at least two decades I know of at nws.noaa.gov (which now redirects to weather.gov).
NOAA's data is a stupidly valuable resource, even discounting the resellers.
[1]: https://forecast.weather.gov/MapClick.php?lat=40.76031000000...
[2]: https://www.ncei.noaa.gov/data/global-forecast-system/access...
It’s the same issue where the IRS could in theory cheaply provide a user friendly tax prep software for ~1$ per taxpayer. However, tax prep software manufacturers have a massive incentive to avoid that. Further, they are going to lobby heavily for the issue which impacts themselves where most people don’t care about losing out in ~50$ per year.
Huh? Did you even look at the links I provided above? They are literally what your talking about. One is a free government run website that any person can use to get a weather forecast, the other is a free government run website where any person can download raw weather data and use any way they like. So no intermediates or companies required.
No. The second link isn’t something 99.9% of the general public will directly access. So in terms of directly communicating with the public without intermediaries it’s again useless to 99.9% of the population. The first link is useable but hardly designed to maximize views by the general public.
This isn’t some major expense, even 3% of their budget spent on communicating with the general public would easily cover several clean apps etc. Instead, it’s clear their largely a giant subsidy for private profit. Other government agencies charge for this kind of large scale data access by private companies to lower costs to taxpayers by removing subsidies.
So you admit that forecast.weather.gov is usable to the general public, but because it "doesn't maximize views" its some how invalidated? You said above that the public shouldn't need to use an intermediary to get a weather forecast and that's exactly what forecast.weather.gov is. And not only that but its faster and better than the for profit apps such as Accuweather, Weather.com, Weather Underground, etc.
> This isn’t some major expense, even 3% of their budget spent on communicating with the general public would easily cover several clean apps etc. Instead, it’s clear their largely a giant subsidy for private profit. Other government agencies charge for this kind of large scale data access by private companies to lower costs to taxpayers by removing subsidies.
3% of a multi billion dollar budget is a lot of money also why would they waste money on a "several clean apps" as you put it when they already have a working weather forecasting app and that money could be used to address issues like the bandwidth one described in the article above, or updating weather models? It sounds like you just want a fancy new app even though they already have a working one.
First I suggested these companies should be paying for API access. Alternatively, one app per device is multiple apps. It’s not about making weather.gov outcompete private companies, it’s about getting value for taxpayer money. People shouldn’t be watching adds or paying for apps that simply show them data they already paid for.
Hell their API’s are providing a longer forecast than the actual website. That’s silly.
forecast.weather.gov doesn't show longer than 7 day forecasts because forecasts beyond 7 days are not very accurate[1]. So why show something that has a 50% chance of being wrong?
And if your not able to use the raw data go to the NWS site which displays it in human friendly form: https://www.weather.gov/
On that note: thank you for providing those links!
I learned something new today, and I can think of some uses for that data.
Yeah probably true, I didn't post a source because I thought it was common knowledge which in retrospect I'm not sure why I thought that.
And I'm glad you enjoyed the links, when I first started looking into the raw NOAA data and PRISM[1] data it took a bit to figure out what to do with it. But I've found the GDAL library helpful for parsing various weather data formats, and there are a lot of them[2].
[1]: https://prism.oregonstate.edu/
[2]: https://gdal.org/
There's a name for this phenomenon! It's a cognitive bias called the "Curse of Knowledge": https://en.wikipedia.org/wiki/Curse_of_knowledge
In a nutshell, we instinctively assume others know what we know. This was probably true when we lived in small tribes on the Serengeti, but not so much in the global information age.
It takes practice, but you can grow the habit of always providing a little extra context without sounding patronizing, as well, at least in-person, a better sense for when people are following the conversation.
What helped me grow this skill was lots of pair-programming, because that daily pair rotation forces you to be really good at context-sharing on the quick.
Aside from point, weather station observations, most weather/climate data is gridded and you are right, GDAL provides lots of tools for processing and transforming gridded (raster) data products. DevelopmentSeed (no affiliation) creates some neat tools and I'd recommend following OSGEO[2] and the FOSS4G conference.
[0]: https://www.prism.oregonstate.edu/documents/pubs/1997appclim... [1]: https://github.com/developmentseed [2]: https://www.osgeo.org/
The majority of the data available to download is superfluous to most people hence why only a subset is shown on any weather app or website including forecast.weather.gov or Accuweather or whatever.
> It’s reasonable to separate providing data to providing data to the public
The NWS actually does provide visualizations for most of their data, and at this point I feel you don't know much about weather data/forecasting and are just looking to argue online. I provided a bunch of links below that allow curious people to look through American model data without having to use a company or having to program their own software. This list doesn't even include free to use apps/sites made by companies. American weather data is easy to access and view for free, Where as European model data is harder to access because you have to pay for it.
[1]: https://www.ready.noaa.gov/READY_animations.php
[2]: https://www.cpc.ncep.noaa.gov/
[4]: https://prism.oregonstate.edu/
My original interest was displaying each days forecast vs what it had been the day before, but it turned out less interesting than I had expected. However, it’s easy to get sucked into this stuff.
Stolen from a parallel thread: https://www.bloomberg.com/news/features/2018-06-14/trump-s-p...
Overall tho yes. Some large players take the data here , process it, and resell at larger value. Some might call it a value add. Accuweather is shit tho so dont ever go to them for anything
I do understand that the peanut gallery normally has very skewed, ignorant views of what complex agencies do. I also understand if you don't question, audit, and occasionally yell at them, they go bad and (at best) stop serving the people they're supposed to.
So of course it’s corporate bureaucracy. Like with the Post Office, corporate stooges are given control and run it like shit to have something to complain about.
Meanwhile, AccuWeather's old CEO has been working hard to slash the budget of the NWS https://weather.com/science/news/2018-02-13-trump-national-w...
Not the only time they got hacked: https://thenewdaily.com.au/news/national/2018/02/19/bureau-m...
This was called out a while ago: https://www.itnews.com.au/news/the-government-websites-that-...
The money shot is: "As part of this, BoM is planning to upgrade its web presence over the next two years as part of a $31 million digital channels platform deal with Accenture."
Two years later, they still don't have HTTPS on their main page. What kills me is that the main site URL is actually reachable over HTTPS.
Check this out: https://www.bom.gov.au/
You get an error message -- using HTTPS and a valid certificate -- saying: "The Bureau of Meteorology website does not currently support connections via HTTPS." and then you're redirected to HTTP!
Apparently $31 million and a year of effort is sufficient to buy a $50 SSL certificate, but doesn't stretch far enough to actually deploy it...
PS: In case you're wondering if this is some sort of infrastructure issue: Their site is served via Akamai CDN.
They've since back pedaled to the point where if you visit now you'll only see in house ads for their own services.
My latest effort is one that just deals with NWS/NOAA directly, and reading this article has me wondering if part of their bandwidth trouble is a product of the way their API works.
The API is a huge set of linked JSON documents. So it very often happens that a request for data is actually half-a-dozen requests through embedded URLs (all of which are made by the client one after the other). It's all very logical in some ways, but at the same time a bit byzantine. I often wonder which entity is served by the way it's set up. From my perspective, it's easily the hardest API to work with. Weather Underground was wonderful, and so was Dark Sky (which I played with for awhile).
I've had people over the years ask if one or another of my tools could be used for large scale or very rapid weather data collection, but all of the services I have worked with throttle connections exactly as NOAA is proposing. I suppose I'm at least relieved to hear that they're not restricting access by independent developers!
I hate wunderground - it is a giant, bloated mess of a website with a UI that jumps all over the place ...
However, I can't find anyone else to give me a 10 day forecast. It seems like wunderground is the only place that collates it like that, which I like very much.
Am I wrong ? Is there some other source for a proper, detailed, 10 day forecast ?
https://forecast.weather.gov/MapClick.php?lat=37.7771&lon=-1...
It was invaluable when I did outdoor work, everyone else was making guesses and extrapolations based on the shitty reports from different local news stations and radar and satellite maps, meanwhile I could tell exactly when a big storm would role through and if rain would be spotty or temperature swings with just a glance for multiple days ahead.
Look for an app which lets you download GRIB files. PocketGrib and LuckGrib are a couple to try. You can download GFS data (produced by NOAA) for the area/timescale/data you are interested in, drop a cursor anywhere on the map and view a meteogram.
This is the same data used by "value add" weather forecasting sites and services to produce their forecasts.
https://www.accuweather.com/en/us/boston/02108/march-weather...
will be upper 80s, low 90s. Chance of thunderstorms in afternoon.
Congrats, i have now predicted a typical 4th of July and will be correct more than normal. Damn i should have built a web company and sold my ideas for a bajillion dollars
But I still appreciate that the Dark Sky feature isn't artificially limited to a particular date. They have a nice error bar on the temperature that gives you an indication of how much it's unknown.
[1]: https://www.washingtonpost.com/news/capital-weather-gang/wp/... [2]: https://en.wikipedia.org/wiki/AccuWeather#Criticisms
Do you want it?
I also run a small REST API service (oikolab.com) that provides location-based forecast data out to 10 days using NCEP service. I started out using DarkSky and Wunderground a couple of years ago but couldn't really get the type & quality of data I needed so I ended up creating one from the original raw data.
It is a 're-packager' in a sense but what we bring to the table is the ability to query the data in a time-series manner efficiently for specific set of end-users. The way NCEP's GRIB data is packaged makes it very slow and painful to do this if you wanted weather parameters to feed into other analytical models (e.g. building energy simulation, time-series forecast etc).
Isn't that REST?
A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...
The article mentions rate limiting by connections:
> limit users to 60 connections per minute
You can make multiple requests over a single connection.
Right. That would be great, and actually, that's what the commercial services do.
With NWS it goes something like this: Say the user has a lat/long. You fire off an http request for a document that tells you what "quadrant" you are in and the URL for it. You then fire off a request for the quadrant document, that lists the products that are available (and their associated URLs). You then fire another request for the actual product for that quadrant . . .
This is not a real example, but this is basically how it works. Even when you can construct a single URL with the info you need, you still need to make multiple requests to have enough information to build the URL (if that makes sense).
> You can make multiple requests over a single connection.
You mean with some kind of http keep-alive thing? I'm not sure I can do that in these circumstance (with cURL).
(edits for grammar, minor issues of clarity)
What value add do companies such as AccuWeather and others provide over simply going to weather.gov? Is their UI simply better, or do they run their own weather models? Thanks
Most of the companies like AccuWeather et al provide, is better UI, and tailored weather. Not just there's going to be a severe thunderstorm, but "these truck routes are going to be affected by weather, so expect them to be 40% slower than normal" or similar.
[0]: https://www.af.mil/News/Article-Display/Article/1295678/supe... [1]: https://www.top500.org/system/178862/
Sounds like a HATEOAS API? https://en.wikipedia.org/wiki/HATEOAS
These things were all the rage as the theoretical perfect REST API as I recall.
That's an interesting quote coming from AccuWeather... https://citizentruth.org/the-politics-behind-the-movement-to...
"Richard Hirn, an employee who represents the National Weather Service’s Employee Organization, told Bloomberg, “We fear that he [Barry Lee Myers] wants to turn the weather service into a taxpayer-funded subsidiary of AccuWeather.”"
Also the problem isnt just weather.gov and weather API. Sites like accuweather are likely grabbing raw files from the FTP servers. Everytime a new forecast comes out (for say the GFS), all these people hit the server at same time to download gobs and gobs of data. Yall would be surprised at how much data in is weather prediction outputs!
https://preview-radar.weather.gov/ still seems to be getting assets off arcgis.com ; surely theres room for a public map server that can stand this traffic?
I haven't looked at the code; I suspect its amenable to hacking up one's own views.
On the new site, the base map tiles start to filter in after about the same amount of time, along with a few pieces of the first radar frame.
AGOL (ArcGIS Online) is only providing the basemap and layer overlays (state boundaries etc), those probably come from Esri's "public" data portal and I doubt are part of the bandwidth issue.
(Use four different CDNs.)
The political motivation is to portray the government as incapable of doing anything other than defense, so that the functions can be outsourced to the private sector.
Expect a solution to appear shortly after the regime changes.
Obama allocated funds specifically for satellites, but reduced the operating budget [1]. Trump actually increased funding for several areas as part of the stopgap funding in 2018, particularly for satellites [2].
There are other examples over the decades of course, but something that seems to be a trend is the specificity. Satellites are flashy and exciting and get funding from politicians. Servers? Meh... why don’t they just get a couple extra machines? My laptop is fast as hell and only cost like $600. Bandwidth? Why don’t they just get whatever internet I have at home? That costs like $100/m and the Por— I mean the CNBC videos start immediately.
1: https://www.washingtonpost.com/blogs/capital-weather-gang/po... 2: https://qz.com/1204034/noaa-trump-signed-a-bill-that-gives-m...
The Public Cloud provides a monthly expense that just becomes part of the operation budget.
It's exciting to think about the innovation to government services as an end-result.
Ops budgets aren't infinite. In this example, if the cloud egress costs kept rising due to demand then the weather service would still have to find ways to restrict usage.
Over the years they have attacked the NWS web properties in particular (Rick Santorum was their attack dog a few years back) as their business of selling graphics to TV stations has declined along with traditional TV.
Corruption at its finest.
And if the NOAA goes away, the government becomes smaller, and their taxes go down, right?
:/
EDIT: Dear downvoters - while this is not my opinion, this is how people think, what people believe. Dismiss it at your own risk.
The other problem here is that Accuweather charges huge fees for access to their forecasts and I'm not sure that I, as a private citizen, could afford that. So my choices go from pay a little bit of tax to get access to high-quality, reliable weather forecasts and warnings, to being totally unable to get access to this information without paying exorbitant fee. Or alternatively just not knowing these things and having to pay higher insurance costs for my ignorance.
This is a really good example of how making government smaller makes a lot of peoples' lives much crappier.
Although I agree I like NWS way more than Accuweather or weather.com, etc, and I would be sad if NWS were forced to take it down.
I used to work at a tv station and the Actual Meteorologists were very particular about not being confused with the "weather people."
There exists a certain group of people that think private sector and The Market™ should handle many/most/all things. Do you want to guess which political party those people tend to associate with?
Also, should we be more or less mad about this than someone trying to overturn fair elections like is happening now? Do you want to guess which political party is trying to do that? Do you want to guess if it's the same political party which believes in The Market™?
So you may not think it matters who people voted for, and that this should be a universal feeling… but that's not what reality is. Do you know what other things should have universal consensus?
* Taking a pandemic seriously.
* Climate change.
>There exists a certain group of people that think private sector and The Market™ should handle many/most/all things
I think the Accuweather problem is worse, much worse. What has happened is that NOAA & NWS had been happily supplying data to..whoever..when some people decided to create a commercial service with the same data. Then they hired disgraced Jesus-freak Santorum to play lobbyist and try to kill the NOAA/NWS public services. Note that Accuweather is not going to fly their own satelites, they just want you to be locked out.[1]
That's right: if a private company decides to commercialize something the government is already doing, using the same government services that that government function is using, then the government should stop doing the thing and let the private company have the market.[2][3] This is a continuation of the Republican principle of privatizing profits and socializing risk, as well as colonizing publicly-funded resources.
It's classic pulling-the-ladder-up.
1. https://en.wikipedia.org/wiki/National_Weather_Service_Dutie...
2. https://www.cnn.com/2017/10/14/politics/noaa-nominee-accuwea...
3. https://www.bloomberg.com/news/features/2018-06-14/trump-s-p...
The hiring is always a problem, even when there isn't a hiring freeze, but they don't fire enough lazy people, which demotivates hard workers who end up bailing for the private sector.
As taxpayers, we should be furious at the GOP for the Accuweather bullshit, but we also need to be furious at any government agency which tolerates the level of fraud, waste, and abuse that most of these agencies perpertrate.
As a person who happens to want universal healthcare like the NHS in Britain, it's infuriating to me that these agencies tolerate such constant mediocrity in their ranks (mixed in with the talented of course!!!) and are so bad with their spending efficiency.
Bezos had something to say about the Seattle city government last year:
"They don't have a revenue problem, they have a spending efficiency problem."
It seems like we always get political about this, with one side wanting to "starve the beast" (as if government is this evil beast), and the other side just always letting them get away with the excuse that they need more revenue.
Where are the people who BELIEVE in government that will demand that it be efficient?
Edit:
I love how people think that because corporations are inefficient, they assume that all organizations are equally ad inefficient as the US Federal Govt. The rate at which people are involuntarily terminated in the Fed Gov is far, far too low, because incompetence is routinely tolerated where it wouldn't otherwise be anywhere else. Why? Not because a person's supervisor in the Fed doesn't recognize the incompetence, but because the firing process is so difficult that they don't want to or can't expend the energy to do so. If you want government to have a bigger role in people's lives, but are so delusional as to not recognize how bad it is, then you aren't furthering your cause.
The FDA still hasn't approved the Pfizer vaccine, while the NHS already did. Pfizer is an American company. That's a national embarrassment.
Edit: Why is this question being downvoted?
I think that, like governments, companies are not all equal in their efficiency. I work with a several people who used to work on the factory robots at Ford, and the level of efficiency at Tesla dramatically exceeds that at Ford. As organizations age, they get more and more corrupted, unless reforms are periodically pursued. Think about what IBM used to be, vs what it is today.
I think the problem with the Federal Government agencies is that they started out very efficient, and filled with bad asses who were hungry. They are never actually reformed though. They just go through periods of politicians who starve them of money then to other politicians that give them plenty, and there's no consistency party wise of course. GOP throws money at their pet agencies, Dems at theirs.
But starving them of money and then throwing it at them again isn't any kind of reform. The result is they have remained stagnant, and filled with people who substitute meaningful work for work that generates great optics.
In late 2017, I was chosen to be on a team to go engage in a Hackathon for the Department of Health and Human Services that was focused on developing solutions to help combat the opioid crisis in the US. My company actually pushed to organize it, and therefore my team wasn't allowed to be considered for a prize, but we did participate.
The rules were that the teams would all get in the building, sign NDAs, and then be given data that wasn't available to the public, and have 24 hours to build a solution. We walk in, and they had changed the name to a "Codeathon", because high ups in HHS didn't like having the word "Hack" in the title. They had a massive room filled with suits, watching presentations about the opioid crisis and what HHS was doing to solve it, and pushed the actual competitors into side rooms that were incredibly uncomfortable, with card tables and folding metal chairs. We then ended up having to wait because they had delays in getting the USB drives with data out to the contestants. The person whose job it was to make the drives forgot to do it the week before. We waited for 5 hours for the drives to be distributed.
Eventually, we got the data, and started the work. Built some very cool solutions and presented them. The suits who had been sitting in the main room watching the presentations by HHS didn't show up to watch the presentations, because they were at 3PM on a Thursday, and we were told that the traffic in DC is bad, so most of them had bailed out back to Virginia and Maryland by then.
The hackathon (or codeathon) had no food provided. We had to periodically leave the building, buy food, and re-enter through the security checkpoints.
The sad part is that unlike me, the rest of my team had never seen the Federal Gov in action. They were all shocked at how incompetent everything seemed to be, and how many people in fancy suits seemed to do nothing. Hundreds of them, watching presentations. They had a long discussion on the flight back about how dismayed they were with the whole experience.
1. It sounds like you've mostly worked with government agencies, suggesting a selection bias in comparing public v private inefficiency.
2. The 2017 event you say was marred by incompetence occurred during the Trump administration, which was well known for being incompetent not at just governing, but at basic functions running an Easter egg hunt. https://www.nytimes.com/2017/04/11/us/politics/white-house-e...
Usually companies have better leadership and are more merciless with people. Unless they are unionized, problems get pushed out (unless they don’t). Big companies waste lots of money on other things.
Government agencies have by design a split between political/professional management. Sometimes that results in strange stuff. Also the programs in government are very meaningful — legislation limits how money is spent, one unit or function may be drowning in funds, another may be dumpster diving.
Companies waste money on things like exec compensation, marketing, etc.
When a company wastes their shareholders’ money, the shareholders have an easy path to sell their shares and stop being harmed by the waste. When a government wastes taxpayer money, there is not the same timeline nor precision in ability to stop directly supporting that waste so you reasonably see taxpayers seeking other avenues to attempt to curtail the waste.
https://news.ycombinator.com/item?id=25376087
And that bill you mentioned failed because the GOP did not support it. It was just the one congressman that wanted it. No one else agreed to join him. https://en.m.wikipedia.org/wiki/National_Weather_Service_Dut...
Apparently not getting all that NWS weather data for free and then being abl to use it in their commercial weather forecasts would put a bit of a dent in their business model. Actually, I think commercial organisations like AccuWeather probably be the main ones affected; ordinary end users aren't going to be requesting large quantities of data.
So basically, AccuWeather wants to have their cake and eat it too.
What's the nature of their lobbying?
Are they basically trying to defund any kind of front-end, consumer facing services from the NWS/NOAA, while keeping the back-end data sources available for them to feed their own consumer facing services?
Hilarious if that's the specific scenario.
Well that sounds incredibly reasonable in general, but it completely depends upon what type of interface they’re providing. Do you have to query all of those things separately for every locality you serve?
Honestly, this is why so many big companies still prefer things that HN would call “stupid” like an FTP drop of a file 5 times per day/hour/etc.
Every FTP implementation I’ve ever seen has included either checking the timestamp, timestamping the filename and filtering on that, or moving to an “archive” folder after processing. Most of the API implementations I see ignore the fact that HEAD requests exist, cache headers can and should be used, etc.
Just like that installer you just downloaded off the internet, did you check the md5/sha to make sure it wasn’t tampered with? Just because it was over HTTPS doesn’t really mean much.
GFS for example. IT has maybe 200 or so timesteps in each forecast, runs 4x a day. There are a shit load of configs to run for the GFS. And each config + output has maybe 300/400 variables in each file? So you download 200 files, each with 400 images in it, each image is pretty dense. Each file is ~200 megs, X 200 files. You do the math. Get that 4x a day. And that is just one forecast model. We have lots more, and many more that have higher resolution and denser data.
The biggest problem i think is they have a goddamn free-ass oldschool FTP server. They are allowing dimwit grad school scrip kiddies to code up whatever shit they want. Infinite loops to try and download gobs of data you dont understand? sure go ahead.
They dont even know how to trakc users. they want to keep an open server, and use IP Address as unique ID. I suggested to use fucking tokens, or username/pass combo at least haha
1) CDNs are a tried a true method for getting content closer to the end user, properly cached, and for less cost.
2) Expose more services that NOAA and the NWS delivers as APIs. If you are providing services for the public good where the consumers are just scraping Web pages to gather the necessary data and content, then you are doing it wrong. It sounds to me like they are serving more API type content over Web pages than they are Web content to Web consumers.
I don't want to second guess the NWS/NOAA technical architects, because I know they are probably doing the best they can. Someone simply needs to communicate these goals and solutions to the political appointees...
If you implement it yourself you still have to set up geographically distributed servers and a syncing system, as opposed to registering an account and changing a pointer in DNS.
Which is a shame; it's a light website serving a dense block of reasonably formatted data that used to be super snappy.
Or, shit, can cloudflare just go and donate this to the government? It would be pretty good publicity for them and probably not a huge cost on their scale.
What am I missing? This seems ridiculous that the USA cannot afford to provide this high-value public service. When did we become a two-bit country when I didn't notice?
I wonder if accuweather or someone got legislation passed to prohibit them from charging heavy users...
The situation that NWS is facing is basically that massive tax cuts coupled with growth in defense and social programs has left most other areas of the federal government hollowed out. The amount of money needed to fix the problem being discussed here is minimal but after decades of budget cuts across non-defense and non social-welfare programs there is simply little left to invest in improving services.
>>> The Weather Service held a public forum Tuesday to discuss the proposal and answer questions. When asked about the investment in computing infrastructure that would be required for these limits to not be necessary, agency officials said a one-time cost of about $1.5 million could avert rate limits. The NOAA budget for fiscal 2020 was $5.4 billion.
They do, in fact, appear to have the money.
$5.4 billion is indeed a huge number but gives no information as to the availability of the $1.5 million. Without knowing what the cost is of what they must accomplish with the $5.4 billion no conclusions can be drawn.
A government agency with bipartisan support and positive public image seems like a good entity for Cloudflare, Google, Amazon, Microsoft or someone else with infrastructure and bandwidth to be a white knight for.
Universities used to host a LOT of stuff. Can't they pick up some of the traffic again?
Offer data centers/hosting providers a tax incentive to help out?
I don't have all the answers, but "whatever Accuweather wants" is probably not the right decision.
Well then maybe they should be paying for more efficient feeds instead of querying more than once per second for a sustained duration? Really? In what world did anyone think a good solution to ANY problem was "query a website over 60 times per minute continuously"???
He also, in keeping with tradition for Trump appointees, was involved with various scandals.
https://www.bloomberg.com/news/features/2018-06-14/trump-s-p...
This is a stunt. They want money, and are using their most popular (key) data points as a threat.
Just like politicians. Money gets tight and they say they'll cut essential services like fire and police, but what they won't tell you is they really want their pet project funded, using deceit.
There is a solid case that citizens requesting weather data should have such requests fulfilled at no or minimal cost, similar to GPS services (a public good) or FOIA requests. The government does charge user fees for some commercial services (IRS tax transcripts for income verification, for example).
EDIT: Please take the time to submit a comment on this issue [1] (last page of the pdf, December 18th deadline).
[1] https://www.weather.gov/media/notification/pdf2/pns20-85ncep...
I totally agree that weather data should be provided for free as a public service. The NWS is one of the true gems of the US government.
That said, this view isn't universally shared. Many other countries' meteorology services charge for access to data - especially if you want access to the raw model output. Most famously, the US's GFS output is provided for free while the EU's ECMWF output requires a paid subscription.
If open access to weather data is something that's important to you, please make your voice heard!
This is one of the few areas I'll defend the US to the bitter end on. Content produced by a government can not be morally copyrighted. The funding comes from the people, and therefore the copyright belongs to them too. Another way of putting it is the public domain and government copyright are the same thing logically.