Self-hosted vector tiles | Hacker News Reader

Self-hosted vector tiles

openstreetmap.org

173 pointsaltilunium28 comments

I'm the author of a few of the tools mentioned in this post!

A convenient new development is instead of using tippecanoe -> go-pmtiles to create PMTiles archives, you can now output .pmtiles directly:

tippecanoe -o bks2.pmtiles mainroad.geojson ...

This is available in Tippecanoe (https://github.com/felt/tippecanoe) v2.17 and later.

Thanks to Felt (https://felt.com) for supporting this open source work.

This is great! I’ve used tippecanoe for a number of years but always hosted the tiles in the file system on a CDN. Fortunately I only needed to cover the area of a single city, but that was still 500,000 individual files! pmtiles should work nicely. I’m hoping MapLibre GL Native adds support for them sometime soon.

Reasonable base layers for maps are table stakes for geospatial apps, so it's great to see Protomaps getting some attention. Requiring an expensive MapBox subscription (pay per tile render) for popular apps just meant that it became prohibitively expensive to run side projects with maps.

If you're interested in cartography, Facebook created the Daylight Map Distribution [1] project. They analyzed satellite imagery, outlined buildings and roads, and published them for map makers. It significantly improves coverage in rural areas and non-Western countries.

[1] https://daylightmap.org

We've been building stadiamaps.com to help with those table-stakes--affordable, private, and independent mapping services are vital for everyone today.

That being said, projects like this will make it easier for folks to get the basics if they're comfortable with self-hosting. Overall, map tiles are losing their competitive edge and becoming commodities.

The details aren't very clear looking in from the outside, but that effort appears to be moving to https://overturemaps.org/ .

The other day, I was looking to embed a tile based world map without detailed zoom levels (up to z11-12) into an offline desktop application. I was surprised by the difficulty to achieve this. The source formats are really heavy (120GB+ for planet) and the tools either too slow and/or overly complex (for example requiring setting up a DB to import the data in). Most tools I found seemed to be for making maps for smaller areas. I ended giving up...

Maybe I've done it wrong. Would love some good pointers on this. Any simple ways to generate vector tiles for a world map up to zoom levels 11 or 12?

I built planetiler (https://github.com/onthegomap/planetiler) for this purpose. The output up to z14 is ~80gb and depending on how big of a machine you have it takes from 30 minutes up to a few hours - no DB required, just java or docker. If you are only going to z11-12, it should be quite a bit faster/smaller.

Brandon from Protomaps is also helping add pmtiles output natively to planetiler, so you won't need a conversion step afterwards!

Awesome, will check it out! Thanks for sharing & building this!

i tried to write my own map tile renderer, in go, for one airbnb-like project where using the google maps or any other provider would cost me an arm and a leg and after experiencing the tag system in OSM i'd rather cut said arm and said leg myself off...with a rusty saw.

I worked on a bootstrapped startup where we needed to render hundreds of layers of maps years ago. We didn't get a market fit, but the rendering pipeline worked well and included support for parallelising it out over multiple servers as we needed to be able to rapidly turn around customer-uploaded layers. It was massively painful to get there, but I'm tempted to dig out the code again and see if I can get it running and how it'd perform on a more modern machine (this was 10+ years ago).

the rendering itself was easy. the problem i had was actually the style of the rendering. because you have to use the node/way tags to determine what is what and how to render it(style). not only that but also order of rendering. that in itself is not an issue, but the tag system is. which everything(logic) is built on. hence why i gave up on it. i still have the code, it works, it renders roads and whatnot but it is impossible to write some useful style for it beyond utter basics. the only thing i struggled with for a while was tile edges where renderer would close the paths, making continuous lines broken on every edge. once i figured out to render a but further beyond the edge(which won't be actually rendered on the image) i was fine. it is really not hard to do, couple of hours at most. but again, the tag system makes this useless. it can be done but i realized that life is way too short for this and moved on. also i found there are other project that handle this that you can self-host nowadays so it lost it's financial potential anyway. i just prefer to use my own code.

If you think tags are the hard bit that suggests you didn't get as far as looking at multipolygons...

Are Protomaps going to result in drastically lower map costs from services like MapBox? Aside from competition, it should drastically lower their costs.

Edit: Removed paragraph about issues I had. See the author's reply below, sounds like it was just a configuration error on the homepage.

Hi, Protomaps author here (not author of the linked article) - can you link the area you're seeing the rendering issues? Most demos are built using OpenStreetMap data, which should be decent for NZ, so curious what the root cause is here.

(EDIT: if you're referring to the landing page demo, it only displays data to a limited zoom level (10), so I've fixed that to not display the map too far beyond that. Thanks!)

Yes that has fixed it! I edited my comment to avoid unnecessarily criticizing the project. I had assumed that was normal/expected behavior because it was on the homepage.

It is a very slick system, I am sure I will be using it in the future.

> because it was on the homepage

This was ‘only’ a diary page, of one of the 10 M OSM accounts.

I was having a discussion with the Protomaps author about the homepage of his project.

Sorry, misunderstood.

BTW this limitation made me believe that the data distributed with Protomaps is limited to that zoom level in general. Some wording related to "data" and "shopping" near that map quickly made me assume that deeper zoom levels will cost money.

Are you presenting a limitation of the product on the front page or is this just a limiting presentation of your product? Mind the difference.

The limitation is of the demo, which uses a publicly accessible asset on cloud storage.

The near-term goal for Protomaps to support z15, which is beyond what most OSM-based solutions provide; there's over a billion addressable tiles for just this one zoom level, and the Protomaps format (PMTiles) is designed with these use cases in mind.

I'll find a better way to communicate this on the site, thanks.

I just switched to protonmaps on cloudflare from Mapbox and went from $500-100 a month to abut $5 a month.

I think I am missing some text - is there anything between

"Then combine all those geojson file into one mbtile file by using tippecanoe."

and

"Then, convert that mbtile file to pmtile by using go-pmtiles"

??? I see only a grey box - looks like some code should be displayed there?

see this comment for a solution:

https://news.ycombinator.com/item?id=34542934

Woahhh this is epic. ProtoMaps is the future.

we're about to deploy a large, static, non-osm dataset with protomaps to serve up vector tiles. it's been super easy to work with.

Is it the most user friendly way to have own OSM map with vector tiles?

Checkout planetiler: https://github.com/onthegomap/planetiler. A super fast way to take an OSM data dump to create vector tiles.

I will mention Baremaps [1] and its OpenStreetMap demo [2]. Baremaps can be used to import data in Postgis and serve vector tiles efficiently. Its developer mode, featuring live reload capabilities, makes it easy to customize vector tiles and integrate third-party datasets into the base map.

[1] https://github.com/apache/baremaps

[2] https://demo.baremaps.com/

Reply on news.ycombinator.com