Hacker News Reader

Comment by tppiotrowski | Hacker News Reader

Ah. I hadn't even thought that I could do byte offsets on the fly. In general I would err on the side of over-sharing because until ChatGPT came along I struggled to understand what tools and options were to generate tiles as there wasn't much data I could find online.

One thing specific in my case: the LiDAR GeoTiffs are in imperial feet with a 32 bit floating point precision. If you take the elevation range from sea level to Everest in meters (8848) and pack it into an int16, you can get .2 meter precision. That's plenty for ShadeMap so converting from float32 to int16 should save half the cloud storage space in theory and more when taking PNG compression into account.

View on HN

This code is rough, and that's an understatement, but here's a basic gist showing how I'm doing what I'm doing in case it helps you in any way: https://gist.github.com/craigsc/fdb867f8971ff5b4ae42de4e0d7c...

I'm similarly using R2 as my static hosting backend and it's been pretty fast and seamless

note that i'm using the geotiff.js and fast-png libraries for the heavy-lifting behind the range-queries and the client-side png encoding. why reinvent the wheel, right?

Your browser has a very powerful image decoder built into it, offloading the PNG decoding into Javascript is very resource hungry.

Using maplibre (or any map viewer) you can load blobs of image data out of a tiff and use `Image` or `Canvas` to render the data onto a map.

Its even easier if the tiffs are already Cloud optimized as they perfectly align to a 1-to-1 map tile and they don't need to be rescaled, you can then just render the images onto the map. eg here is a viewer that loads webps out of a 15GB tiff and uses Canvas to render them onto a map [1]

Unless you are trying to layer all your maps together, you also could stop reprojecting them into webmercator, or if your goal is to layer them, then storing them in webmercator would save a ton of user's compute time.

There are a bunch of us that talk web maping and imagery in the #maplibre and #imagery slack channels in OSMUS's slack [2]

[1] https://blayne.chard.com/cogeotiff-web/index.html?view=cog&i...

[2] https://github.com/maplibre/maplibre-gl-js#getting-involved

Great points. Thank you for the links. The one trade off here is that uncompressed blobs will require longer downloads than PNG and I think usually the network transfer is slower than PNG decoding.

But maybe the sample gist takes a Tiff blob and encodes it to a PNG on the client and then maplibre decodes the PNG to canvas. That would be quite inefficient if that's what it's doing.

Those comments were more at pastmaps.

For elevation data, we store our DEM/DSM in S3 as LERC [1] COGS, LERC has a WASM bundle which I think can be used in the browser. We found LERC COGs to be one of the most space efficient ways of storing highresolution DEM/DSM data [2], If you wanted to you could fetch LERC tiles directly out of a remote COG and use that directly for the terrain heights.

I am more focused on storage/archiving/publishing of our LiDAR capture program [3] than web based visualizations of it though, so I am unsure if a LERC COG would even be better for you than a PNG TerrainRGB.

[1] https://www.npmjs.com/package/lerc

[2] https://github.com/linz/elevation/tree/master/docs/tiff-comp...

[3] https://linz.maps.arcgis.com/apps/MapSeries/index.html?appid...

WASM surely could be an improvement over js, especially for kind of BigData-ish/repetitive jobs, and where load on clients might become the next wall after we optimized the cloud/server part, or when we try to use js on cloud leafnodes.

amazing comments and callouts, thank you! I actually tried to load the raw image data blobs as a layer into MapLibre but couldn't figure out a way to do it and finally capitulated and did the "bad" move of re-encoding just to get the initial interactive map collection out the door for folks. It sounds like this is in fact possible but I missed something. I'll take a look at the Image and Canvas sources, thanks!

Re the webmercator reprojection - yea it's gnarly that I'm doing it clientside but it's exactly because I'm working towards the ability to layer them interactively on top of eachother (as well as on various basemaps). My projection code is also only half-working at the moment and it's where I'm currently spending my time next week. I'm trying to avoid building pipelines to re-encode the geotiffs as long as I can since there's 10+TB of them in my backend so this is why you're seeing me doing this clientside instead. This is a solo project so I need to be really picky where I spend my time so I can keep moving the ball forward

I'll join those 2 communities, thank you! Been crazy hard to find folks who are deep in this stuff so most of my learning has been through endless googling down deep dark corners of the web for the past 2 months

Reply on news.ycombinator.com