JupyterGIS breaks through to the next level | Hacker News Reader

JupyterGIS breaks through to the next level

eo4society.esa.int

142 pointsarjxn-py31 comments

There's something wrong with the scrolling on this page, like something "puts the brakes on" (Chrome, macOS)

It seems like something called "nicescroll" hijacks the scrolling on the page, its compiled in the minified javascript so a bit hard to tell though.

Turning off javascript seems to fix it though.

Drag with middle mouse button is supposed to scroll, but actually does nothing on this page.

This looks great.

I do yearn for a day though when we're using something like Marimo over Jupyter as a default for these kinds of things. Particularly in GIS where there's more utility in being able to use a notebook-like interface for an executable routine (rather than an analysis or experiment, which is (and should probably remain) the primary use case for Jupyter).

You can try JupyterGIS live on this deployment powered by JupyterLite - https://jupytergis.readthedocs.io/en/latest/lite/lab/

Ok, you link to a deployment, and also the docs say this:

> Collaborative GIS Environment: Work together on geographic data projects in real-time.

How does that work? I can apparently make changes to the files, and even save them, but then my changes are gone when I reload it. Where were the changes saved? This is exactly what I'm wary of when using Jupyter...

> How does that work? I can apparently make changes to the files, and even save them, but then my changes are gone when I reload it. Where were the changes saved? This is exactly what I'm wary of when using Jupyter...

The GP linked to JupyterLite, which is browser-based Jupyter based on WebAssembly...

Then there is JupyterLab which runs on your own machine, or on a server somewhere (e.g. a K8S pod)...

And then there are options like JupyterHub and Kubeflow which start K8S pods for you dynamically to run JupyterLab in...

So "where is my data stored" all depends on how Jupyter is deployed... at the moment, when I use Jupyter, I'm mainly using it running inside a Docker container which in turn runs inside a K8S pod (weird way of deploying it, but I have my reasons)... and then the notebooks are stored in a volume (PVC) attached to the K8S pod (statefulset actually), but that's just temporary storage while I work on them, anything I want to keep permanently is put in Git and pushed to our corporate Git host... and then our actual datasets are mostly on S3 (or something else which speaks the S3 protocol)

I don't know! They said "collaborative" and they provided a link. I tried making some changes to test the collaboration and clicked the "save" button. It changed the "last changed" to "now" and upon reload my changes were gone.

It lied to me: my changes weren't saved! If I can't save a thing, I want to be explicitly told so!

I’m pretty sure it saves it using browser storage APIs… so rather than it lying to you, it could be your browser (or some extension even) lying to it

The Jupyter notebooks in ArcGIS pro are incredibly useful. Unfortunately it's in Arcgis Pro. I'm thrilled to have the similar setup that's not tied to a slow subsription software. Coding assistants also work pretty well at doing GIS in python.

Looks nice. Hopefully this is one more geospatial jenga piece removed from the current dominance of buggy, ancient closed source software like Google Earth and ESRI.

I don't see any advantage working on JupyterGIS over working in QGIS.

I see a few advantages. For my work in particular, I have to rely on creating desk study reports via exporting PDFs from QGIS - this depends on export DPI, page size etc. Following that I have to pull those plans into e.g. Word and it's a messy system.

A python notebook would be a nice way of generating reports of GIS data in an interactive way without being forced to use pages, PDFs, and embedded image files.

Where I work, I can't give anything is not a word document to anybody else in the company. A python notebook might help at creating the figures for example, but I can already do that with QGIS layouts.

If the working environment allows for checking/reviewing within the notebook, I guess this could help automatise things.

I can see it for programmers. Here you can use industry standard python libraries (shapely, geopandas etc.). Nobody really wants to learn PyQGIS (the python interface for qgis). So while qgis is much more full featured for "desktop" gis (designed to compete with esri arcgis) i can see the use case here for people who want to build their own extensions and port code from this to other python projects more easily.

Notebooks is a nice and soft way to bring a programming mindset to non-programmers.

Your actions are repeatable and can be stored in a centralized repository.

There are probably some macro abilities in QGIS (it is an amazing tool), but this means moving to "script first" from "click first".

If you're doing primarily analytics rather than making a map, I could see how JupyterGIS might be a better choice.

(but you can always just use proj/GDAL at the import/export layer...)

I'm beyond excited to see this as a refugee from esri's products. The only reason I still use anything esri is due to their layout tools. Nightmarish as they are, it's still the best way to print or export a map to pdf. I'd like to see some advances in that area though I would bet money that Jupyter's architecture isn't adequate for those use cases.

> One of the most significant updates is a new browser-based processing toolbox powered by a WebAssembly (WASM) build of the Geospatial Data Abstraction Library (GDAL).

> Available tools include:

> Buffer, Convex Hull, Dissolve, Bounding Boxes, Centroid, Concave Hull

Why would they want to calculate these from WASM in the browser instead of calling out to the Python kernel?

Since it was the ipython notebook I have only ever used jupyter locally. So my first instinct is to do it all on the backend too, but maybe they care about network latency? It looks like they're using this as a collaborative tool. So kind of like a multiplayer online video game, you don't want to waste time rendering graphics server side.

> Collaborative GIS Environment: Work together on geographic data projects in real-time.

What does this mean? How is it collaborative in real-time? (I don't even know how Jupyter is collaborative... as in, several people can open a Jupyter Notebook and make changes simultaneously, and things don't break for either of them?)

I guess things might get nasty when you try to _run_ simultaneously, unless they actually have implemented a CRDT for the whole notebook state (it would still be nasty from a usability POV, but at least it would be consistent). Just writing on the notebook would work just as any other collaborative editor out there, I expect.

JupyterLab does use CRDT via yjs (https://github.com/yjs/yjs), and the JupyterGIS collaboration system is based on the service provided by JupyterLab. So multiple users can edit the file concurrently.

Yes, Jupyterlab has a collaboration extension.

I've spent the past month writing a HTTP+WebSocket proxy to be used with Jupyterlab for an ESA-adjacent company, and spent a good deal of time trying to make the collaboration system work behind this proxy.

That said, I don't think my contribution is at all related to this post.

Looks great! What visualization backend is used? Is it GPU accelerated?

JupyterGIS uses OpenLayers as its visualization backend. It runs entirely in the browser and supports GPU-accelerated rendering through WebGL when available

Question from the ignorant, how does this relate to OpenStreetMap?

Osm is a database (in the Wikipedia sense not the Postgres sense). You will be able to use osm data in this tool.

This seems to be a Python based Jupiter notebook (style?) thing for collaboratively working with GIS data/visualizations.

OpenStreetMap is a project to "map the world". In the end, OpenStreetMap provides data (and map tiles) for other things to use.

Going out on a limb (since I haven't used it) but JupyterGIS can probably make use of OSM data, along with other data sources.

JGIS builds on OpenStreetMap and other open geospatial data sources as basemaps or overlays. JupyterGIS itself is a Jupyter-based interface for working with GIS data—so you can visualize, query, and process OSM or any other data directly from notebooks alongside Python tools like GeoPandas.

Openstreetmap is not mentioned anywhere in the JupyterGIS repo [1]. As is mentioned OSM data can be used anywhere.

[1] https://github.com/search?q=repo%253Ageojupyter%252Fjupyterg...

Reply on news.ycombinator.com