Hacker News Reader

Comment by cyclotron3k | Hacker News Reader

Would the data from this satellite be freely available to the public? I couldn't see anything obvious

As far as I can tell, they say: "Mission control and data distribution are managed by EUMETSAT." They have published their own blog post here: https://www.eumetsat.int/features/see-earths-atmosphere-neve...

There they say that: "Observations made by MTG-S1 will feed into data products that support national weather services …". So I guess there will be no simple, publicly available REST API or so... but if anybody finds anything, let us know here :)

https://user.eumetsat.int/api-definitions/data-store-downloa...

nice find. so you need a client_id to access the API

Read the data store user guide. You have to register.

For the datasets, I tried to access (like the full disc image in visible wavelength, MTG 0 degree), it is sufficient to register at eumetsat to get a username and password. The eumdac python tool is probably the easiest way to access the data:

https://pypi.org/project/eumdac/

(If you do not want to use python, the --debug option is quite useful to see exactly the request made. The output is either some JSON metadata or a large zip with the netcdf data)

Most weather data isn't generally available by easy to query REST API's (at least not at the original sources). One side project I had I wanted to use NOMADs data, and it was quite a grind downloading and processing the raw datasets into something usable at an application level (or viable to expose via an API).

That’s why you have service/products that have the sole purpose of taking all these region specific data sources and processing them in to a generic json api.

The government orgs probably do it intentionally so they don’t have ten million devices pinging their servers to update weather widgets.

Check out open-meteo. They’ve got pretty extensive historical and forecast weather apis in easy to consume formats. https://open-meteo.com/en/features#available_apis

EU citizens can get free access to it via Eumetcast DVB-S service for non-commercial use. A registration, an off-the-shelf DVB-S data receiver, a satellite dish and their decryption USB key is required. FOSS software like Satpy is available for processing those radiometric data. More info: https://user.eumetsat.int/resources/user-guides/eumet-cast-e...

Unlikely. EU countries are consistently restrictive about access to this kind of data. Even when it is available, it often has odd restrictive licensing. This is an area where the US, with its liberal data access policies, is far ahead of Europe.

Something else to keep in mind is that the data products are extremely large. It would be expensive to give the public access. I used to host these types of data sets for EU countries. The workload just from authorized users is resource intensive, it doesn't scale cheaply. (I once woke up to find a metaphorical smoking crater where my server racks were because an authorized user shared his credentials with a few friends overnight.)

Isn't EUMETSAT data usually under CC-by-SA 3.0? So all you have to do is to register with them and get your client ID for API access, or are there more hoops to jump through?

I don't know what you mean.

Data from the Copernicus program has always been fully available, served with a nice web UI, API for both near real time data and archives.

It's the best source of open satellite data by far.

As for the licensing, I never actually looked it up, so maybe you're right.

There are two aspects to this.

The licensing commonly restricts you to small hobbyist use cases. There are typically restrictions on use of data, the amount of data, and retention of data. I've never looked at Copernicus data before but it appears to have the same kinds of restrictions. This is the licensing equivalent of "source available" rather than true "open source". Hopefully they are improving on this front.

While the data may be available in theory, no one ever invests in the data infrastructure that would allow people to access it in practice. They always have a nice website and API but it is like trying to watch Youtube over a dial-up modem. Usable access is reserved for researchers with an approved use case.

The US government does an unusually good job at both of these in my experience. Even when US public data sets that are not readily available online, you have to contact someone, it is usually for good reason. For example, because they are multi-exabyte data sets sitting on tape somewhere that almost no one ever asks for.

Well it makes sense the public API has restrictions. They probably have separate enterprise licensing.

> restrictive about access to this kind of data

After all, we don't know if the weather consented to having its data displayed, or if it even allowed cookies.

Core data is CC-BY-4.0, "recommended" data is licensed with a fee for certain commercial uses.

https://user.eumetsat.int/resources/user-guides/data-registr...

As most EU projects yes. There was test data released last year to get you started.

https://user.eumetsat.int/resources/user-guides/getting-star...

Well, at least in my experience with EU projects, they tend to be much more restrictive with data sharing than equivalent US institutions: e.g. a lot of paid EUMET data has publicly available NOAA equivalents - though usually of worse quality.

Not sure why you're being down-voted. US weather models are free. EU models are not.

Depends on which model. Only really the ECMWF weather model is not fully free. The German, French, Dutch, ... models are all free (regional and global models). Of course, these global models are generally less accurate than ECMWF, still ECMWF has a lot of free data available too. US models are also freely available, and quite easy to work with (as opposed to some European ones).

You can see the most important charts from the ECMWF model for free on ecmwf.int. But you will not get the data behind them.

Yes! That government agencies data is PD is a nice feature of US law, we should implement that in EU.

Try to ask the NRO for their images and see how you go :)

Intelligence gathering data vs weather data. Yeah, that's the same thing.

Not Public Domain, TD - Taxpayers Domain. :)

Take a look at https://zenodo.org/communities/eu/

Yes, it's not everything, but it's a start.

It is not an EU project. It is an ESA and EUMETSAT project. Neither is an EU organisation. Both have multiple non-EU members, and I do not think all EU countries are members of either.

There was a good CCC talk on pulling images from weather sats (and data from other satellites) - https://youtu.be/fM5w7bFNvWI?si=Dq6S6nYOE_frAd7b

It's been done before, but this was a great talk imo.

Yes, it will be freely available to the public

EUMETSAT publishes data as CC-BY 4.0 past a timeliness of 1 hour https://www.eumetsat.int/data-policy/eumetsat-data-policy.pd...

Look for your dataset here https://data.eumetsat.int/ (Note: you need registration but it is free).

I guess you will be able to access the data with copernicus (usually thy even provide raw L0 data)

If they'll publish it through Copernicus, it'll probably show up here:

https://browser.dataspace.copernicus.eu/

Definitely not in anything like realtime, maybe an archive. There's a licence fee of 8000EUR/yr to access real-time EUMETSAT data. Welcome to Europe, where you pay for everything twice.

Hah! I don't believe this for a second. No, you need the 8k, a business entity (at the very least), five different licenses of some sort, and then some form of accreditation.

There's an 8k license for "recommended" (not "core", which is free under CC-BY-4.0 for all purposes) data if you are a service provider or broadcaster:

https://user.eumetsat.int/resources/user-guides/data-registr...

There are also fees in some other circumstances, but not for "personal, educational, research" use.

lame, with GOES-18 you can just download the latest full disk image in real time. Makes for a nifty desktop background when combined with a systemd user timer that fetches the current picture of the earth every 15 minutes.

https://www.goes-r.gov/multimedia/dataAndImageryImagesGoes-1...

Reply on news.ycombinator.com