Vector Tiles are deployed on OpenStreetMap.org | Hacker News Reader

Vector Tiles are deployed on OpenStreetMap.org

blog.openstreetmap.org

280 pointsikawe72 comments

Background:

Mapping applications split up data into "tiles" so you can download only the data you are currently looking at. For example, you don't want to download the entire planet, just to look at your own neighborhood.

Historically, these tiles were literally images that the client application (i.e. web map) could "tile" side by side to cover the part of the map you were looking at. Now we refer to those images as "raster" tiles, to differentiate them with "vector" tiles.

Rather than a rendered image, Vector tiles contain the raw data that you could use to render such an image. Vector tiles allow a smaller file size and more flexibility. For example, with vector tiles you can crisply render at partial zoom levels, keeping lines sharp and avoid pixelating text. The client can also have customizable styles - hiding certain layers or accentuating others from the vector tiles.

Vector tiles are not new technology. For example, Google Maps started using them over a decade ago. So why has it taken so long for OpenStreetMap.org? One reason is no doubt a lack of engineering capacity. There were also concerns about older and less powerful clients hardware not being up to the task, but that concern has lessened over time.

OpenStreetMap also has some unique requirements. It is a community edited database, and users want to see their edits soon (immediately really). It's not feasible to dynamically generate every tile request from the latest data, so caching is essential. Still, to minimize the amount of time tiles will be stale, a lot of work went into being able to quickly regenerate stale tiles before the new vector tiles were rolled out.

>Vector tiles are not new technology.

Yeah, it really is not. Mapbox Vector Tiles spec came out in 2014, and they've been absolutely standard across all (non-government) web mapping for at least the last 5 years.

Indeed almost everyone except Apple's MapKit library for in-app native maps integration is using vector these days.

Could you explain why you say that? My understanding is that rasterized tiles are only a graceful degradation when necessary in MapKit, while vector is default, as the behavior and experience would indicate.

I was referring to using custom vector tiles like the ones in the post with MapKit, not Apple's tiles. The MKTileOverlay is the only way to do it and it only supports bitmap tiles.

+1. Google launched their (proprietary) vector tiles in 2013 as well, and I'd even argue that vector has been the standard well over 5 years now.

Tiles aren't just about data selection, they're also about caching. By turning a continuous domain (any part of the world at any scale) into a series of discrete requests (a grid of tiles at several fixed scales), maps become a series of cacheable requests.

For anyone who wants to actually try the new layer, it's called "Shortbread" and can be accessed under the layers selector. Or use this link: https://www.openstreetmap.org/#map&layers=S

The blog post could do a better job of surfacing that bit!

You can compare it to the existing vector tile layer created by MapTiler, which mimics the classic raster tile style: https://www.openstreetmap.org/#map&layers=V

Oh but that default style for Shortbread is so ugly! No contrast, missing details and lots of information lost.

Hopefully it's vector tiles and information are there (I checked in Maputik) so it's possible to create my own style (which I will definitely try).

There is also the MapTiler OMT style for OpenMapTiles, which uses Vector Tiles. It is a close copy of the Standard OSM map style. It appeared at the same time as Shortbread on the osm.org website.

I agree, it's super close to the classical OSM style, but some things are missing, e.g. roads that are currently planned or in the building phase.

Yes, buildings under construction aren't there yet either. OMT is OpenSource though, so you can make a pull request: https://github.com/openmaptiles/openmaptiles

Yeah. MapTiler has much better styles in general.

Yeah only motorways are highlighted everything else is faded in background.

The lack of boundaries below national level is off-putting. Both the style and the tile spec itself seem to need a lot of refinement... highways look like a patchwork at low zoom. I had also thought shortbread excluded a lot of POI types, but maybe I need to review again.

But that's not to diminish the accomplishment overall here for the OSM page itself– this is an awesome step forward!

OSM has always overdone boundaries in my opinion, the raster tiles show national/international waters and electoral districts, which are of limited use for most purposes.

In case you wonder how much time / resources it takes to generate vector tiles, I'm running benchmarks with Tilemaker here for https://cartes.app

https://github.com/systemed/tilemaker/issues/839

I am sorry but what is "300 Go RAM" 300 Go? I only know GB/GiB.

It's basically French for GB (Go = Gigaoctet)

OP is probably French or his native tongue is French. Byte is “Octet” in French and typically you’ll see Go = Gigaoctet which is the same as GB.

Also more technically correct as byte never "officially" meant 8 bits, but octet is unambiguous.

<troll>like measuring lengths in meters</troll>

Also worth checking out https://github.com/onthegomap/planetiler, which maybe a bit less mature, but is much faster than Tilemaker.

Is it ? The benchmark section says 42 m for a 128 Gb machine with 64 CPU.

Many important details are not visible anymore:

  - gates on private pathways 
  - shop names

Personally I'm quite happy that these tiles cut down on the clutter in the original OSM tiles. It made it very hard for me to actually use them for navigation because there was just so much stuff everywhere.

For example the old tiles displayed rail tracks extremely prominently, which just aren't relevant 99% of the time even when traveling by train. In the vector tiles they're much more muted and thinner.

Yes but this one only highlights motorways which are used usually just for long distance drives.

The nice thing is, that's just a style, not a property of the tiles. So if you want to alter the rendering, it's "just" a change to the style of that layer/attribute.

> I'm quite happy that these tiles cut down on the clutter in the original OSM tiles. It made it very hard for me to actually use them for navigation

The Openstreetmap.org tiles are a demo and a contributor quality assurance tool - they are not meant for end-user applications, hence the rather over-the-top selection of map objects.

The choices are tough, also in details: Which shops to show in busy areas at which zoom level?

But mind: openstreetmap.org is ment as a developer site, for developers to see their changes quickly. The official idea is that other people should take the data and do "nice" things elsewhere. But of course reality is that users want to use openstreetmap.org as alternative to Google maps ...

All the more reason to show rather more than less data.

Official? I've never heard this as official doctrine. Can you point me to anything official?

I'd love to see OpenStreetMap.org be more directly useful to more people, rather than only as a "developer site".

Well that's a bit of a balancing act, isn't it? "Why isn't this feature on the site, it would be more useful to me; also, that feature is annoying and useless to me" - says literally everyone.

Thus, we get a built-in editor (less friction, yay/too rudimentary, boo) and popups (OSM events, yay/Popups are now everywhere, boo). And endless discussions on "this should be more/less prominent."

I don't think there's any official edict, but the approach "I don't like this and I'll make my own version" is at the very least workable (as opposed to, say, Google Maps).

"developer" is a slightly weird way of putting it. osm.org is the contributor portal and demo site for OpenStreetMap, and yes it is not and never has been intended as an end user replacement for Google Maps and similar offerings.

The main purpose of the maps on the site is, besides showcasing some topical uses of OSM data, rapid feedback to contributors, which is something that required specific development so that can be provided with vector tiles too.

The 'late to the game' narrative seems to be a bit misplaced in any case given that OSM data has powered essentially all vector tile use outside of Google over more than a decade.

What happened to maps trying to make the names of most or all streets visible? It looks like the vector layer on the web viewer is sort of trying to show some street names, but it seems extremely buggy right now — names appear and then disappear again when making small zoom adjustments.

Not that Apple Maps or Google Maps are much better in this regard.

What's the impediment to not providing daily or weekly planet-wide mbtiles similar to the availability of OSM pbfs on AWS open data buckets[1]?

[1] https://registry.opendata.aws/osm/

I think mbtiles are being phased out for pmtiles because no DB required and can be served from static storage like R2/S3 (with a worker but hopefully the worker part goes away and they support byte offset requests soon)

You don't need a database to serve mbtiles. If you're deploying PMTiles somewhere that doesn't support byte offset requests, then they don't really have much advantage over mbtiles.

That depends what you mean by needing a database. MBtiles files are SQLite database files, so you need a SQLite process running somewhere to extract the requested tiles.

The worker isn’t required, having a z/x/y tile scheme just makes caching the tiles server side super easy.

Look at the network tab on https://pmtiles.io/#url=https%3A%2F%2Fdemo-bucket.protomaps....

An mbtiles file is just an SQLite db which can store image or vector tiles which can be very useful.

It's great for exploring tiles but it essentially doesn't support range queries/requests, which pmtiles does.

To answer the question: somebody needs to do the initial work and it's a further moving part that needs to be kept running (as the other responses point out such a distribution would likely use PMTiles as a container format). Given the current finances and staffing of the OSMF likely not top priority.

But they've done the work and are serving them now, just wondering why they don't just dump a copy on AWS. Thanks for answering!

Protomaps does this already. They do a planet build each day and offer a download for free:

https://docs.protomaps.com/basemaps/downloads

This is excellent. Thank you.

While vector tiles are often marketed as faster loading ones, and it is true in OSM case, I would like to see apples to apples comparison: vector tiles with same level of detail as original OSM raster tiles. Maybe someone already has such vector style built?

To everyone with complaints about the new "Shortbread" styling, I agree that it's not perfect but that's kinda missing the point. The real story is that vector tiles have the styling applied client-side so anyone can tweak the look with a little javascript.

The prior raster tiles have the style baked in; if you want a new look, you need to generate a new image. So each map publisher ends up running their own data and server infrastructure just to tweak the style.

The vector tile approach means a single (cacheable) asset can be used in many different maps. Huge win. If you don't like the style, you can make your own without having to literally download the planet.

Nice. I already use Vector Tiles on my backend. But it is nice to have other sources like this.

A feature goal for this deployment was that the tiles would update continuously, keeping up with changes that people are making to the OpenStreetMap database.

Providing that feedback is one of the main purposes of the site.

This comment is a reminder that OSM Vector Tiles is sponsored work. It is exciting to see the deployment of it — congrats to the team!

If you find value in it, consider donating any amount to the OpenStreetMap Foundation.

https://blog.openstreetmap.org/2024/02/11/2024-announcing-th...

I had no idea what vector tiles were and the page doesn't explain it.

https://en.wikipedia.org/wiki/Vector_tiles

Here's how I understand it: Previously, OpenStreetMap's tile endpoints would serve pre-rasterized PNG images, so zooming in on a tile could cause it to get blurry, until your client requests a new, zoomed in tile. Now, they can serve tiles in SVG format, which scale better

SVG tiles use less data too, and can be recoloured/restyled.

Not SVG

Basically that + vector tiles can store information about the tile, styles, objects or buildings etc

> so zooming in on a tile could cause it to get blurry, until your client requests a new, zoomed in tile. Now, they can serve tiles in SVG format, which scale better

They still are blurry, because openstreetmap.org uses a JS library that does not seem to support vector tiles :/

I just tested it and it's def vector here - Firefox Android. Looks great!

Make sure it's set to the "Shortbread" layer. Look for the "Layers" menu on the right.

Hmm I tried the shortbread layer and while it looks nice it still pixelates when zooming in until it loads the new zoom level

I guess you're right. I see the same on my desktop Firefox on Linux. But on my Android phone it's pretty smooth.

It seems to just not work in Firefox for me, it's the same for me on FF Android but in Chrome it's beautiful! Constant text size while zooming and it's super smooth.

Edit: It also weirdly doesn't work on Chrome on Linux for me either, only Chrome on android. Oh well, hopefully they can bring support everywhere eventually.

Not SVG, its Mapbox Vector tile format which is like Geojson coded in protobuf it is then rendered to raster in the browser using webgl typically.

https://github.com/mapbox/vector-tile-spec/tree/master/2.0

SVG is an XML based format that browsers render naitively, completely different.

https://en.wikipedia.org/wiki/SVG

Someone please correct me if I'm wrong here, but I think there's an additional benefit.

Traditional process is:

OSM Database -> PNGs -> Your screen

The first arrow decides what data to pull out and how to draw it.

The new process is:

OSM Database -> Vector Tiles -> Your Screen

The first arrow decides what data to pull out. The second arrow decides how to draw it. So given your vector tiles, you can choose and tweak the style that it's drawn as, deciding how (and if) to display certain things. And you can tweak that in your browser. That's useful for devs and users. "Night Mode", "Show bike lanes" (maybe?), etc.

Also relevant is that the vector tile is not only in a couple formats (pmtiles, mbtiles) but conforms to a couple different schemas (Shortbread, OpenMapTiles) which determines what kinds of data shows up. For instance (I'm just making up this example) one schema might have "big" "medium" and "small" roads. Another schema might just have "big" and "small". The transformation process will decide which kinds of roads in the OSM database map on to which type of road in the schema. (I think it turns out that you can't realistically just pull out all of the OSM database data, you have to pare it down). And then certain styles (Americana, etc) work for specific schemas, deciding things like "big roads are black", etc.

More like

OSM Database -> PNGs -> PNG Decoder in browser-> Your screen

OSM Database -> Vector Tiles -> MaplibreGL.js -> WebGL -> Your Screen

Could you please comment on any security implications of executing community-generated vector tile content, compared to classic PNG decoding?

I don't think anything's being executed here, in the same sense that both PDF (without JavaScript, at least) and JPEG are "data only", even though one uses vectors while the other only supports raster graphics.

If you care about security implications of reading untrusted data, you may be interested in trying Qubes OS, which isolates apps by running everything in VMs. My daily driver, can't recommend it enough.

The security surface area will be either the png decoder or webgl, both are pretty well scrutinized but if I had to pick I would the png decoder is less likely to have a security issue compared to webgl.

Does't really matter because both png or webgl are available to any website at anytime.

The enormous difference from an infrastructure standpoint is where the arrows are happening and how much effort it is.

> OSM Database -> PNGs

This is a tile rendering farm. It takes a lot of compute power and has to be redone every time a style changes or data changes.

> OSM Database -> Vector Tiles

This is a relatively cheap data extraction process. It has to be redone when data changes.

> PNGs -> Your screen

This is extremely simple.

> Vector Tiles -> Your Screen

This is pretty complex and hard to do fast. Mapbox GL JS is the leader here and they have put a lot of resources into doing it well and fast. Maplibre GL JS is the fork, which is decent, and there are also Leaflet and OpenLayers options.

Source: This is basically my life for the last 10 years.

Rendering of Vector Tiles is not that hard as all the geometry is already processed, polygons assembled and data sorted. If you have good source of vector tiles renderer itself can be relatively dumb.

I was able to implement relatively straightforward renderer in Nim language[0].

I have encountered only three hard parts:

- rendering map labels on paths (this is really hard!) - how to render labels not to be clipped by tile boundary (I had some ideas but did not implemented it yet) - collisions between labels and symbols

[0] - https://github.com/severak/lunarender3/

Fun fact: I wrote a lot of that article :)

https://en.wikipedia.org/w/index.php?title=Vector_tiles&oldi...

does a vector tile use more or less memory than a raster tile ? I mean, on average, in OSM.

Reply on news.ycombinator.com